Mercurial > repos > guerler > springsuite

comparison planemo/lib/python3.7/site-packages/cwltool/schemas/v1.0/Workflow.yml @ 0:d30785e31577 draft

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
"planemo upload commit 6eee67778febed82ddd413c3ca40b3183a3898f1"
author guerler
date Fri, 31 Jul 2020 00:18:57 -0400
parents
children
comparison
equal deleted inserted replaced
-1:000000000000 0:d30785e31577
1 $base: "https://w3id.org/cwl/cwl#"
2
3 $namespaces:
4 cwl: "https://w3id.org/cwl/cwl#"
5 rdfs: "http://www.w3.org/2000/01/rdf-schema#"
6
7 $graph:
8
9 - name: "WorkflowDoc"
10 type: documentation
11 doc:
12 - |
13 # Common Workflow Language (CWL) Workflow Description, v1.0.2
14
15 This version:
16 * https://w3id.org/cwl/v1.0/
17
18 Current version:
19 * https://w3id.org/cwl/
20 - "\n\n"
21 - {$include: contrib.md}
22 - "\n\n"
23 - |
24 # Abstract
25
26 One way to define a workflow is: an analysis task represented by a
27 directed graph describing a sequence of operations that transform an
28 input data set to output. This specification defines the Common Workflow
29 Language (CWL) Workflow description, a vendor-neutral standard for
30 representing workflows intended to be portable across a variety of
31 computing platforms.
32
33 - {$include: intro.md}
34
35 - |
36
37 ## Introduction to CWL Workflow standard v1.0.2
38
39 This specification represents the third stable release from the CWL
40 group. Since the initial v1.0 release, v1.0.2 introduces the following
41 updates to the CWL Command Line Tool standard. Documents should continue
42 to use `cwlVersion: v1.0` and existing v1.0 documents remain valid,
43 however CWL documents that relied on previously undefined or
44 underspecified behavior may have slightly different behavior in v1.0.2.
45
46 * 12 March 2017:
47 * Mark `default` as not required for link checking.
48 * Add note that recursive subworkflows is not allowed.
49 * Fix mistake in discussion of extracting field names from workflow step ids.
50 * 23 July 2017: (v1.0.1)
51 * Add clarification about scattering over empty arrays.
52 * Clarify interpretation of `secondaryFiles` on inputs.
53 * Expanded discussion of semantics of `File` and `Directory` types
54 * Fixed typo "EMACScript" to "ECMAScript"
55 * Clarified application of input parameter default values when the input is `null` or undefined.
56 * 10 August 2017: (v1.0.2)
57 * Clarify behavior resolving expressions in `secondaryFile`
58
59 Since draft-3, v1.0 introduces the following changes and additions
60 to the CWL Workflow standard:
61
62 * The `inputs` and `outputs` fields have been renamed `in` and `out`.
63 * Syntax simplifcations: denoted by the `map<>` syntax. Example: `in`
64 contains a list of items, each with an id. Now one can specify
65 a mapping of that identifier to the corresponding
66 `InputParameter`.
67 ```
68 in:
69 - id: one
70 type: string
71 doc: First input parameter
72 - id: two
73 type: int
74 doc: Second input parameter
75 ```
76 can be
77 ```
78 in:
79 one:
80 type: string
81 doc: First input parameter
82 two:
83 type: int
84 doc: Second input parameter
85 ```
86 * The common field `description` has been renamed to `doc`.
87
88 ## Purpose
89
90 The Common Workflow Language Command Line Tool Description express
91 workflows for data-intensive science, such as Bioinformatics, Chemistry,
92 Physics, and Astronomy. This specification is intended to define a data
93 and execution model for Workflows that can be implemented on top of a
94 variety of computing platforms, ranging from an individual workstation to
95 cluster, grid, cloud, and high performance computing systems.
96
97 - {$include: concepts.md}
98
99 - name: ExpressionToolOutputParameter
100 type: record
101 extends: OutputParameter
102 fields:
103 - name: type
104 type:
105 - "null"
106 - "#CWLType"
107 - "#OutputRecordSchema"
108 - "#OutputEnumSchema"
109 - "#OutputArraySchema"
110 - string
111 - type: array
112 items:
113 - "#CWLType"
114 - "#OutputRecordSchema"
115 - "#OutputEnumSchema"
116 - "#OutputArraySchema"
117 - string
118 jsonldPredicate:
119 "_id": "sld:type"
120 "_type": "@vocab"
121 refScope: 2
122 typeDSL: True
123 doc: |
124 Specify valid types of data that may be assigned to this parameter.
125
126 - type: record
127 name: ExpressionTool
128 extends: Process
129 specialize:
130 - specializeFrom: "#OutputParameter"
131 specializeTo: "#ExpressionToolOutputParameter"
132 documentRoot: true
133 doc: |
134 Execute an expression as a Workflow step.
135 fields:
136 - name: "class"
137 jsonldPredicate:
138 "_id": "@type"
139 "_type": "@vocab"
140 type: string
141 - name: expression
142 type: [string, Expression]
143 doc: |
144 The expression to execute. The expression must return a JSON object which
145 matches the output parameters of the ExpressionTool.
146
147 - name: LinkMergeMethod
148 type: enum
149 docParent: "#WorkflowStepInput"
150 doc: The input link merge method, described in [WorkflowStepInput](#WorkflowStepInput).
151 symbols:
152 - merge_nested
153 - merge_flattened
154
155
156 - name: WorkflowOutputParameter
157 type: record
158 extends: OutputParameter
159 docParent: "#Workflow"
160 doc: |
161 Describe an output parameter of a workflow. The parameter must be
162 connected to one or more parameters defined in the workflow that will
163 provide the value of the output parameter.
164 fields:
165 - name: outputSource
166 doc: |
167 Specifies one or more workflow parameters that supply the value of to
168 the output parameter.
169 jsonldPredicate:
170 "_id": "cwl:outputSource"
171 "_type": "@id"
172 refScope: 0
173 type:
174 - string?
175 - string[]?
176 - name: linkMerge
177 type: ["null", "#LinkMergeMethod"]
178 jsonldPredicate: "cwl:linkMerge"
179 doc: |
180 The method to use to merge multiple sources into a single array.
181 If not specified, the default method is "merge_nested".
182 - name: type
183 type:
184 - "null"
185 - "#CWLType"
186 - "#OutputRecordSchema"
187 - "#OutputEnumSchema"
188 - "#OutputArraySchema"
189 - string
190 - type: array
191 items:
192 - "#CWLType"
193 - "#OutputRecordSchema"
194 - "#OutputEnumSchema"
195 - "#OutputArraySchema"
196 - string
197 jsonldPredicate:
198 "_id": "sld:type"
199 "_type": "@vocab"
200 refScope: 2
201 typeDSL: True
202 doc: |
203 Specify valid types of data that may be assigned to this parameter.
204
205
206 - name: Sink
207 type: record
208 abstract: true
209 fields:
210 - name: source
211 doc: |
212 Specifies one or more workflow parameters that will provide input to
213 the underlying step parameter.
214 jsonldPredicate:
215 "_id": "cwl:source"
216 "_type": "@id"
217 refScope: 2
218 type:
219 - string?
220 - string[]?
221 - name: linkMerge
222 type: LinkMergeMethod?
223 jsonldPredicate: "cwl:linkMerge"
224 doc: |
225 The method to use to merge multiple inbound links into a single array.
226 If not specified, the default method is "merge_nested".
227
228
229 - type: record
230 name: WorkflowStepInput
231 extends: Sink
232 docParent: "#WorkflowStep"
233 doc: |
234 The input of a workflow step connects an upstream parameter (from the
235 workflow inputs, or the outputs of other workflows steps) with the input
236 parameters of the underlying step.
237
238 ## Input object
239
240 A WorkflowStepInput object must contain an `id` field in the form
241 `#fieldname` or `#prefix/fieldname`. When the `id` field contains a slash
242 `/` the field name consists of the characters following the final slash
243 (the prefix portion may contain one or more slashes to indicate scope).
244 This defines a field of the workflow step input object with the value of
245 the `source` parameter(s).
246
247 ## Merging
248
249 To merge multiple inbound data links,
250 [MultipleInputFeatureRequirement](#MultipleInputFeatureRequirement) must be specified
251 in the workflow or workflow step requirements.
252
253 If the sink parameter is an array, or named in a [workflow
254 scatter](#WorkflowStep) operation, there may be multiple inbound data links
255 listed in the `source` field. The values from the input links are merged
256 depending on the method specified in the `linkMerge` field. If not
257 specified, the default method is "merge_nested".
258
259 * **merge_nested**
260
261 The input must be an array consisting of exactly one entry for each
262 input link. If "merge_nested" is specified with a single link, the value
263 from the link must be wrapped in a single-item list.
264
265 * **merge_flattened**
266
267 1. The source and sink parameters must be compatible types, or the source
268 type must be compatible with single element from the "items" type of
269 the destination array parameter.
270 2. Source parameters which are arrays are concatenated.
271 Source parameters which are single element types are appended as
272 single elements.
273
274 fields:
275 - name: id
276 type: string
277 jsonldPredicate: "@id"
278 doc: "A unique identifier for this workflow input parameter."
279 - name: default
280 type: ["null", Any]
281 doc: |
282 The default value for this parameter to use if either there is no
283 `source` field, or the value produced by the `source` is `null`. The
284 default must be applied prior to scattering or evaluating `valueFrom`.
285 jsonldPredicate:
286 _id: "cwl:default"
287 noLinkCheck: true
288 - name: valueFrom
289 type:
290 - "null"
291 - "string"
292 - "#Expression"
293 jsonldPredicate: "cwl:valueFrom"
294 doc: |
295 To use valueFrom, [StepInputExpressionRequirement](#StepInputExpressionRequirement) must
296 be specified in the workflow or workflow step requirements.
297
298 If `valueFrom` is a constant string value, use this as the value for
299 this input parameter.
300
301 If `valueFrom` is a parameter reference or expression, it must be
302 evaluated to yield the actual value to be assiged to the input field.
303
304 The `self` value of in the parameter reference or expression must be
305 the value of the parameter(s) specified in the `source` field, or
306 null if there is no `source` field.
307
308 The value of `inputs` in the parameter reference or expression must be
309 the input object to the workflow step after assigning the `source`
310 values, applying `default`, and then scattering. The order of
311 evaluating `valueFrom` among step input parameters is undefined and the
312 result of evaluating `valueFrom` on a parameter must not be visible to
313 evaluation of `valueFrom` on other parameters.
314
315
316 - type: record
317 name: WorkflowStepOutput
318 docParent: "#WorkflowStep"
319 doc: |
320 Associate an output parameter of the underlying process with a workflow
321 parameter. The workflow parameter (given in the `id` field) be may be used
322 as a `source` to connect with input parameters of other workflow steps, or
323 with an output parameter of the process.
324 fields:
325 - name: id
326 type: string
327 jsonldPredicate: "@id"
328 doc: |
329 A unique identifier for this workflow output parameter. This is the
330 identifier to use in the `source` field of `WorkflowStepInput` to
331 connect the output value to downstream parameters.
332
333
334 - name: ScatterMethod
335 type: enum
336 docParent: "#WorkflowStep"
337 doc: The scatter method, as described in [workflow step scatter](#WorkflowStep).
338 symbols:
339 - dotproduct
340 - nested_crossproduct
341 - flat_crossproduct
342
343
344 - name: WorkflowStep
345 type: record
346 docParent: "#Workflow"
347 doc: |
348 A workflow step is an executable element of a workflow. It specifies the
349 underlying process implementation (such as `CommandLineTool` or another
350 `Workflow`) in the `run` field and connects the input and output parameters
351 of the underlying process to workflow parameters.
352
353 # Scatter/gather
354
355 To use scatter/gather,
356 [ScatterFeatureRequirement](#ScatterFeatureRequirement) must be specified
357 in the workflow or workflow step requirements.
358
359 A "scatter" operation specifies that the associated workflow step or
360 subworkflow should execute separately over a list of input elements. Each
361 job making up a scatter operation is independent and may be executed
362 concurrently.
363
364 The `scatter` field specifies one or more input parameters which will be
365 scattered. An input parameter may be listed more than once. The declared
366 type of each input parameter is implicitly becomes an array of items of the
367 input parameter type. If a parameter is listed more than once, it becomes
368 a nested array. As a result, upstream parameters which are connected to
369 scattered parameters must be arrays.
370
371 All output parameter types are also implicitly wrapped in arrays. Each job
372 in the scatter results in an entry in the output array.
373
374 If any scattered parameter runtime value is an empty array, all outputs are
375 set to empty arrays and no work is done for the step, according to
376 applicable scattering rules.
377
378 If `scatter` declares more than one input parameter, `scatterMethod`
379 describes how to decompose the input into a discrete set of jobs.
380
381 * **dotproduct** specifies that each of the input arrays are aligned and one
382 element taken from each array to construct each job. It is an error
383 if all input arrays are not the same length.
384
385 * **nested_crossproduct** specifies the Cartesian product of the inputs,
386 producing a job for every combination of the scattered inputs. The
387 output must be nested arrays for each level of scattering, in the
388 order that the input arrays are listed in the `scatter` field.
389
390 * **flat_crossproduct** specifies the Cartesian product of the inputs,
391 producing a job for every combination of the scattered inputs. The
392 output arrays must be flattened to a single level, but otherwise listed in the
393 order that the input arrays are listed in the `scatter` field.
394
395 # Subworkflows
396
397 To specify a nested workflow as part of a workflow step,
398 [SubworkflowFeatureRequirement](#SubworkflowFeatureRequirement) must be
399 specified in the workflow or workflow step requirements.
400
401 It is a fatal error if a workflow directly or indirectly invokes itself as
402 a subworkflow (recursive workflows are not allowed).
403
404 fields:
405 - name: id
406 type: string
407 jsonldPredicate: "@id"
408 doc: "The unique identifier for this workflow step."
409 - name: in
410 type: WorkflowStepInput[]
411 jsonldPredicate:
412 _id: "cwl:in"
413 mapSubject: id
414 mapPredicate: source
415 doc: |
416 Defines the input parameters of the workflow step. The process is ready to
417 run when all required input parameters are associated with concrete
418 values. Input parameters include a schema for each parameter which is
419 used to validate the input object. It may also be used build a user
420 interface for constructing the input object.
421 - name: out
422 type:
423 - type: array
424 items: [string, WorkflowStepOutput]
425 jsonldPredicate:
426 _id: "cwl:out"
427 _type: "@id"
428 identity: true
429 doc: |
430 Defines the parameters representing the output of the process. May be
431 used to generate and/or validate the output object.
432 - name: requirements
433 type: ProcessRequirement[]?
434 jsonldPredicate:
435 _id: "cwl:requirements"
436 mapSubject: class
437 doc: |
438 Declares requirements that apply to either the runtime environment or the
439 workflow engine that must be met in order to execute this workflow step. If
440 an implementation cannot satisfy all requirements, or a requirement is
441 listed which is not recognized by the implementation, it is a fatal
442 error and the implementation must not attempt to run the process,
443 unless overridden at user option.
444 - name: hints
445 type: Any[]?
446 jsonldPredicate:
447 _id: "cwl:hints"
448 noLinkCheck: true
449 mapSubject: class
450 doc: |
451 Declares hints applying to either the runtime environment or the
452 workflow engine that may be helpful in executing this workflow step. It is
453 not an error if an implementation cannot satisfy all hints, however
454 the implementation may report a warning.
455 - name: label
456 type: string?
457 jsonldPredicate: "rdfs:label"
458 doc: "A short, human-readable label of this process object."
459 - name: doc
460 type: string?
461 jsonldPredicate: "rdfs:comment"
462 doc: "A long, human-readable description of this process object."
463 - name: run
464 type: [string, Process]
465 jsonldPredicate:
466 "_id": "cwl:run"
467 "_type": "@id"
468 doc: |
469 Specifies the process to run.
470 - name: scatter
471 type:
472 - string?
473 - string[]?
474 jsonldPredicate:
475 "_id": "cwl:scatter"
476 "_type": "@id"
477 "_container": "@list"
478 refScope: 0
479 - name: scatterMethod
480 doc: |
481 Required if `scatter` is an array of more than one element.
482 type: ScatterMethod?
483 jsonldPredicate:
484 "_id": "cwl:scatterMethod"
485 "_type": "@vocab"
486
487
488 - name: Workflow
489 type: record
490 extends: "#Process"
491 documentRoot: true
492 specialize:
493 - specializeFrom: "#OutputParameter"
494 specializeTo: "#WorkflowOutputParameter"
495 doc: |
496 A workflow describes a set of **steps** and the **dependencies** between
497 those steps. When a step produces output that will be consumed by a
498 second step, the first step is a dependency of the second step.
499
500 When there is a dependency, the workflow engine must execute the preceeding
501 step and wait for it to successfully produce output before executing the
502 dependent step. If two steps are defined in the workflow graph that
503 are not directly or indirectly dependent, these steps are **independent**,
504 and may execute in any order or execute concurrently. A workflow is
505 complete when all steps have been executed.
506
507 Dependencies between parameters are expressed using the `source` field on
508 [workflow step input parameters](#WorkflowStepInput) and [workflow output
509 parameters](#WorkflowOutputParameter).
510
511 The `source` field expresses the dependency of one parameter on another
512 such that when a value is associated with the parameter specified by
513 `source`, that value is propagated to the destination parameter. When all
514 data links inbound to a given step are fufilled, the step is ready to
515 execute.
516
517 ## Workflow success and failure
518
519 A completed step must result in one of `success`, `temporaryFailure` or
520 `permanentFailure` states. An implementation may choose to retry a step
521 execution which resulted in `temporaryFailure`. An implementation may
522 choose to either continue running other steps of a workflow, or terminate
523 immediately upon `permanentFailure`.
524
525 * If any step of a workflow execution results in `permanentFailure`, then
526 the workflow status is `permanentFailure`.
527
528 * If one or more steps result in `temporaryFailure` and all other steps
529 complete `success` or are not executed, then the workflow status is
530 `temporaryFailure`.
531
532 * If all workflow steps are executed and complete with `success`, then the
533 workflow status is `success`.
534
535 # Extensions
536
537 [ScatterFeatureRequirement](#ScatterFeatureRequirement) and
538 [SubworkflowFeatureRequirement](#SubworkflowFeatureRequirement) are
539 available as standard [extensions](#Extensions_and_Metadata) to core
540 workflow semantics.
541
542 fields:
543 - name: "class"
544 jsonldPredicate:
545 "_id": "@type"
546 "_type": "@vocab"
547 type: string
548 - name: steps
549 doc: |
550 The individual steps that make up the workflow. Each step is executed when all of its
551 input data links are fufilled. An implementation may choose to execute
552 the steps in a different order than listed and/or execute steps
553 concurrently, provided that dependencies between steps are met.
554 type:
555 - type: array
556 items: "#WorkflowStep"
557 jsonldPredicate:
558 mapSubject: id
559
560
561 - type: record
562 name: SubworkflowFeatureRequirement
563 extends: ProcessRequirement
564 doc: |
565 Indicates that the workflow platform must support nested workflows in
566 the `run` field of [WorkflowStep](#WorkflowStep).
567 fields:
568 - name: "class"
569 type: "string"
570 doc: "Always 'SubworkflowFeatureRequirement'"
571 jsonldPredicate:
572 "_id": "@type"
573 "_type": "@vocab"
574
575 - name: ScatterFeatureRequirement
576 type: record
577 extends: ProcessRequirement
578 doc: |
579 Indicates that the workflow platform must support the `scatter` and
580 `scatterMethod` fields of [WorkflowStep](#WorkflowStep).
581 fields:
582 - name: "class"
583 type: "string"
584 doc: "Always 'ScatterFeatureRequirement'"
585 jsonldPredicate:
586 "_id": "@type"
587 "_type": "@vocab"
588
589 - name: MultipleInputFeatureRequirement
590 type: record
591 extends: ProcessRequirement
592 doc: |
593 Indicates that the workflow platform must support multiple inbound data links
594 listed in the `source` field of [WorkflowStepInput](#WorkflowStepInput).
595 fields:
596 - name: "class"
597 type: "string"
598 doc: "Always 'MultipleInputFeatureRequirement'"
599 jsonldPredicate:
600 "_id": "@type"
601 "_type": "@vocab"
602
603 - type: record
604 name: StepInputExpressionRequirement
605 extends: ProcessRequirement
606 doc: |
607 Indicate that the workflow platform must support the `valueFrom` field
608 of [WorkflowStepInput](#WorkflowStepInput).
609 fields:
610 - name: "class"
611 type: "string"
612 doc: "Always 'StepInputExpressionRequirement'"
613 jsonldPredicate:
614 "_id": "@type"
615 "_type": "@vocab"