Mercurial > repos > shellac > guppy_basecaller

comparison env/lib/python3.7/site-packages/cwltool/schemas/v1.1/CommandLineTool.yml @ 2:6af9afd405e9 draft

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
"planemo upload commit 0a63dd5f4d38a1f6944587f52a8cd79874177fc1"
author shellac
date Thu, 14 May 2020 14:56:58 -0400
parents 26e78fe6e8c4
children
comparison
equal deleted inserted replaced
1:75ca89e9b81c 2:6af9afd405e9
1 saladVersion: v1.1
2 $base: "https://w3id.org/cwl/cwl#"
3
4 $namespaces:
5 cwl: "https://w3id.org/cwl/cwl#"
6
7 $graph:
8
9 - name: CommandLineToolDoc
10 type: documentation
11 doc:
12 - |
13 # Common Workflow Language (CWL) Command Line Tool Description, v1.1
14
15 This version:
16 * https://w3id.org/cwl/v1.1/
17
18 Current version:
19 * https://w3id.org/cwl/
20 - "\n\n"
21 - {$include: contrib.md}
22 - "\n\n"
23 - |
24 # Abstract
25
26 A Command Line Tool is a non-interactive executable program that reads
27 some input, performs a computation, and terminates after producing some
28 output. Command line programs are a flexible unit of code sharing and
29 reuse, unfortunately the syntax and input/output semantics among command
30 line programs is extremely heterogeneous. A common layer for describing
31 the syntax and semantics of programs can reduce this incidental
32 complexity by providing a consistent way to connect programs together.
33 This specification defines the Common Workflow Language (CWL) Command
34 Line Tool Description, a vendor-neutral standard for describing the
35 syntax and input/output semantics of command line programs.
36
37 - {$include: intro.md}
38
39 - |
40 ## Introduction to the CWL Command Line Tool standard v1.1
41
42 This specification represents the latest development release from the
43 CWL group. Since the v1.0 release, v1.1 introduces the
44 following updates to the CWL Command Line Tool standard.
45 Documents should use `cwlVersion: v1.1` to make use of new
46 syntax and features introduced in v1.1. Existing v1.0 documents
47 should be trivially updatable by changing `cwlVersion`, however
48 CWL documents that relied on previously undefined or
49 underspecified behavior may have slightly different behavior in
50 v1.1.
51
52 ## Changelog
53
54 * Clarify behavior around `ENTRYPOINT` and `CMD` in containers.
55 * Clarify documentation around `valueFrom` and `null` inputs.
56 * Default values for some fields are now expressed in the schema.
57 * When defining record types with `CommandInputRecordSchema`, fields of
58 type `File` may now include `format`, `loadContents`,
59 `secondaryFiles` and `streamable`.
60 * `CommandInputRecordSchema`, `CommandOutputRecordSchema`,
61 `CommandInputEnumSchema and `CommandInputArraySchema` now have an optional
62 `doc` field.
63 * `inputBinding` has been added as an optional field for
64 `CommandInputRecordSchema` (was previously in CWL `draft-3` but
65 disappeared in `v1.0`).
66 * Any `doc` field may be an array of strings in addition to the
67 previously allowed single string.
68 * Addition of `stdin` type shortcut for
69 [`CommandInputParameter`](#CommandInputParameter).
70 * Clarify that the designated output directory should be empty
71 except for files or directories specified using
72 [InitialWorkDirRequirement](#InitialWorkDirRequirement).
73 * Clarify semantics of `shellQuote`.
74 * Expressions are now allowed to evaluate to `null` or `Dirent` in
75 [InitialWorkDirRequirement.listing](#InitialWorkDirRequirement).
76 * Items in [InitialWorkDirRequirement.listing](#InitialWorkDirRequirement)
77 are now allowed to be `null` and `array<File | Directory>`.
78 * Clarify behavior of secondaryFiles on output.
79 * [Addition](#Requirements_and_hints) of `cwl:requirements` field to
80 input object documents.
81 * Clarify behavior of `glob` for absolute paths and symlinks.
82 * Clarify behavior of `glob` to include directories.
83 * `secondaryFiles` can now be explicitly marked as `required` or not.
84 * Clarify `CommandLineTool.arguments` documentation.
85 * Clarify that `runtime.outdir` and `runtime.tmpdir` must be distinct
86 directories.
87 * Clarify that unspecified details related to execution are left open to
88 the platform.
89 * Added `InputParameter.loadContents` field. Use of `loadContents` in
90 `InputBinding` is deprecated; it is preserved for v1.0 backwards
91 compatability and will be removed in CWL v2.0.
92 * [Added](#ToolTimeLimit) `ToolTimeLimit` feature, allows setting
93 an upper limit on the execution time of a CommandLineTool.
94 * [Added](#WorkReuse) `WorkReuse` feature, allowing to enable or disable the reuse
95 behavior for a particular tool or step for implementations that
96 support reusing output from past work.
97 * [Added](#NetworkAccess) `NetworkAccess` feature, allowing to indicate whether a
98 process requires outgoing network access.
99 * [Added](#InplaceUpdateRequirement) `InplaceUpdateRequirement` feature, allowing tools to directly
100 update files with `writable: true` in `InitialWorkDirRequirement`.
101 * [Added](#LoadListingRequirement) `LoadListingRequirement`
102 and [loadListing](#LoadContents) to control whether and how
103 `Directory` listings should be loaded for use in expressions.
104 * The position field of the [CommandLineBinding](#CommandLineBinding) can
105 now be calculated from a CWL Expression.
106 * The exit code of a CommandLineTool invocation is now
107 available to expressions in `outputEval` as `runtime.exitCode`
108 * [Better explain](#map) the `map<…>` notation that has existed since v1.0.
109 * Fixed schema error where the `type` field inside the `inputs` and
110 `outputs` field was incorrectly listed as optional.
111 * For multi-Process CWL documents, if no particular process is named then
112 the process with the `id` of `#main` is chosen.
113 * Added explicit defaults for [ResourceRequirement](#ResourceRequirement)s.
114 * [Made explicit](#Generic_execution_process) that the `$namespaces` of
115 the top level CWL document are also available to the input object as
116 well.
117 * Fields with type "Expression" no longer include "string" in
118 the type union unless a string is semantically valid. Where
119 a bare string is not semantically valid, these fields must
120 contain either a non-string value accepted by the field
121 type, or a valid expression which returns a value having one
122 of the the other types defined for that field.
123
124 See also the [CWL Workflow Description, v1.1 changelog](Workflow.html#Changelog).
125
126 ## Purpose
127
128 Standalone programs are a flexible and interoperable form of code reuse.
129 Unlike monolithic applications, applications and analysis workflows which
130 are composed of multiple separate programs can be written in multiple
131 languages and execute concurrently on multiple hosts. However, POSIX
132 does not dictate computer-readable grammar or semantics for program input
133 and output, resulting in extremely heterogeneous command line grammar and
134 input/output semantics among program. This is a particular problem in
135 distributed computing (multi-node compute clusters) and virtualized
136 environments (such as Docker containers) where it is often necessary to
137 provision resources such as input files before executing the program.
138
139 Often this gap is filled by hard coding program invocation and
140 implicitly assuming requirements will be met, or abstracting program
141 invocation with wrapper scripts or descriptor documents. Unfortunately,
142 where these approaches are application or platform specific it creates a
143 significant barrier to reproducibility and portability, as methods
144 developed for one platform must be manually ported to be used on new
145 platforms. Similarly it creates redundant work, as wrappers for popular
146 tools must be rewritten for each application or platform in use.
147
148 The Common Workflow Language Command Line Tool Description is designed to
149 provide a common standard description of grammar and semantics for
150 invoking programs used in data-intensive fields such as Bioinformatics,
151 Chemistry, Physics, Astronomy, and Statistics. This specification
152 attempts to define a precise data and execution model for Command Line Tools that
153 can be implemented on a variety of computing platforms, ranging from a
154 single workstation to cluster, grid, cloud, and high performance
155 computing platforms. Details related to execution of these programs not
156 laid out in this specification are open to interpretation by the computing
157 platform implementing this specification.
158
159 - {$include: concepts.md}
160 - {$include: invocation.md}
161
162
163 - type: record
164 name: EnvironmentDef
165 doc: |
166 Define an environment variable that will be set in the runtime environment
167 by the workflow platform when executing the command line tool. May be the
168 result of executing an expression, such as getting a parameter from input.
169 fields:
170 - name: envName
171 type: string
172 doc: The environment variable name
173 - name: envValue
174 type: [string, Expression]
175 doc: The environment variable value
176
177 - type: record
178 name: CommandLineBinding
179 extends: InputBinding
180 docParent: "#CommandInputParameter"
181 doc: |
182
183 When listed under `inputBinding` in the input schema, the term
184 "value" refers to the the corresponding value in the input object. For
185 binding objects listed in `CommandLineTool.arguments`, the term "value"
186 refers to the effective value after evaluating `valueFrom`.
187
188 The binding behavior when building the command line depends on the data
189 type of the value. If there is a mismatch between the type described by
190 the input schema and the effective value, such as resulting from an
191 expression evaluation, an implementation must use the data type of the
192 effective value.
193
194 - **string**: Add `prefix` and the string to the command line.
195
196 - **number**: Add `prefix` and decimal representation to command line.
197
198 - **boolean**: If true, add `prefix` to the command line. If false, add
199 nothing.
200
201 - **File**: Add `prefix` and the value of
202 [`File.path`](#File) to the command line.
203
204 - **Directory**: Add `prefix` and the value of
205 [`Directory.path`](#Directory) to the command line.
206
207 - **array**: If `itemSeparator` is specified, add `prefix` and the join
208 the array into a single string with `itemSeparator` separating the
209 items. Otherwise first add `prefix`, then recursively process
210 individual elements.
211 If the array is empty, it does not add anything to command line.
212
213 - **object**: Add `prefix` only, and recursively add object fields for
214 which `inputBinding` is specified.
215
216 - **null**: Add nothing.
217
218 fields:
219 - name: position
220 type: [ "null", int, Expression ]
221 doc: |
222 The sorting key. Default position is 0. If the inputBinding is
223 associated with an input parameter, then the value of `self` in the
224 expression will be the value of the input parameter. Input parameter
225 defaults (as specified by the `InputParameter.default` field) must be
226 applied before evaluating the expression. Expressions must return a
227 single value of type int or a null.
228 - name: prefix
229 type: string?
230 doc: "Command line prefix to add before the value."
231 - name: separate
232 type: boolean?
233 default: true
234 doc: |
235 If true (default), then the prefix and value must be added as separate
236 command line arguments; if false, prefix and value must be concatenated
237 into a single command line argument.
238 - name: itemSeparator
239 type: string?
240 doc: |
241 Join the array elements into a single string with the elements
242 separated by by `itemSeparator`.
243 - name: valueFrom
244 type:
245 - "null"
246 - string
247 - Expression
248 jsonldPredicate: "cwl:valueFrom"
249 doc: |
250 If `valueFrom` is a constant string value, use this as the value and
251 apply the binding rules above.
252
253 If `valueFrom` is an expression, evaluate the expression to yield the
254 actual value to use to build the command line and apply the binding
255 rules above. If the inputBinding is associated with an input
256 parameter, the value of `self` in the expression will be the value of
257 the input parameter. Input parameter defaults (as specified by the
258 `InputParameter.default` field) must be applied before evaluating the
259 expression.
260
261 If the value of the associated input parameter is `null`, `valueFrom` is
262 not evaluated and nothing is added to the command line.
263
264 When a binding is part of the `CommandLineTool.arguments` field,
265 the `valueFrom` field is required.
266
267 - name: shellQuote
268 type: boolean?
269 default: true
270 doc: |
271 If `ShellCommandRequirement` is in the requirements for the current command,
272 this controls whether the value is quoted on the command line (default is true).
273 Use `shellQuote: false` to inject metacharacters for operations such as pipes.
274
275 If `shellQuote` is true or not provided, the implementation must not
276 permit interpretation of any shell metacharacters or directives.
277
278
279 - type: record
280 name: CommandOutputBinding
281 extends: LoadContents
282 doc: |
283 Describes how to generate an output parameter based on the files produced
284 by a CommandLineTool.
285
286 The output parameter value is generated by applying these operations in the
287 following order:
288
289 - glob
290 - loadContents
291 - outputEval
292 - secondaryFiles
293 fields:
294 - name: glob
295 type:
296 - "null"
297 - string
298 - Expression
299 - type: array
300 items: string
301 doc: |
302 Find files or directories relative to the output directory, using POSIX
303 glob(3) pathname matching. If an array is provided, find files or
304 directories that match any pattern in the array. If an expression is
305 provided, the expression must return a string or an array of strings,
306 which will then be evaluated as one or more glob patterns. Must only
307 match and return files/directories which actually exist.
308
309 If the value of glob is a relative path pattern (does not
310 begin with a slash '/') then it is resolved relative to the
311 output directory. If the value of the glob is an absolute
312 path pattern (it does begin with a slash '/') then it must
313 refer to a path within the output directory. It is an error
314 if any glob resolves to a path outside the output directory.
315 Specifically this means globs with relative paths containing
316 '..' or absolute paths that refer outside the output directory
317 are illegal.
318
319 A glob may match a path within the output directory which is
320 actually a symlink to another file. In this case, the
321 expected behavior is for the resulting File/Directory object to take the
322 `basename` (and corresponding `nameroot` and `nameext`) of the
323 symlink. The `location` of the File/Directory is implementation
324 dependent, but logically the File/Directory should have the same content
325 as the symlink target. Platforms may stage output files/directories to
326 cloud storage that lack the concept of a symlink. In
327 this case file content and directories may be duplicated, or (to avoid
328 duplication) the File/Directory `location` may refer to the symlink
329 target.
330
331 It is an error if a symlink in the output directory (or any
332 symlink in a chain of links) refers to any file or directory
333 that is not under an input or output directory.
334
335 Implementations may shut down a container before globbing
336 output, so globs and expressions must not assume access to the
337 container filesystem except for declared input and output.
338
339 - name: outputEval
340 type: Expression?
341 doc: |
342 Evaluate an expression to generate the output value. If
343 `glob` was specified, the value of `self` must be an array
344 containing file objects that were matched. If no files were
345 matched, `self` must be a zero length array; if a single file
346 was matched, the value of `self` is an array of a single
347 element. Additionally, if `loadContents` is `true`, the File
348 objects must include up to the first 64 KiB of file contents
349 in the `contents` field. The exit code of the process is
350 available in the expression as `runtime.exitCode`.
351
352 - name: CommandLineBindable
353 type: record
354 fields:
355 inputBinding:
356 type: CommandLineBinding?
357 jsonldPredicate: "cwl:inputBinding"
358 doc: Describes how to turn this object into command line arguments.
359
360 - name: CommandInputRecordField
361 type: record
362 extends: [InputRecordField, CommandLineBindable]
363 specialize:
364 - specializeFrom: InputRecordSchema
365 specializeTo: CommandInputRecordSchema
366 - specializeFrom: InputEnumSchema
367 specializeTo: CommandInputEnumSchema
368 - specializeFrom: InputArraySchema
369 specializeTo: CommandInputArraySchema
370 - specializeFrom: InputBinding
371 specializeTo: CommandLineBinding
372
373
374 - name: CommandInputRecordSchema
375 type: record
376 extends: [InputRecordSchema, CommandInputSchema, CommandLineBindable]
377 specialize:
378 - specializeFrom: InputRecordField
379 specializeTo: CommandInputRecordField
380 - specializeFrom: InputBinding
381 specializeTo: CommandLineBinding
382
383
384 - name: CommandInputEnumSchema
385 type: record
386 extends: [InputEnumSchema, CommandInputSchema, CommandLineBindable]
387 specialize:
388 - specializeFrom: InputBinding
389 specializeTo: CommandLineBinding
390
391
392 - name: CommandInputArraySchema
393 type: record
394 extends: [InputArraySchema, CommandInputSchema, CommandLineBindable]
395 specialize:
396 - specializeFrom: InputRecordSchema
397 specializeTo: CommandInputRecordSchema
398 - specializeFrom: InputEnumSchema
399 specializeTo: CommandInputEnumSchema
400 - specializeFrom: InputArraySchema
401 specializeTo: CommandInputArraySchema
402 - specializeFrom: InputBinding
403 specializeTo: CommandLineBinding
404
405
406 - name: CommandOutputRecordField
407 type: record
408 extends: OutputRecordField
409 specialize:
410 - specializeFrom: OutputRecordSchema
411 specializeTo: CommandOutputRecordSchema
412 - specializeFrom: OutputEnumSchema
413 specializeTo: CommandOutputEnumSchema
414 - specializeFrom: OutputArraySchema
415 specializeTo: CommandOutputArraySchema
416 fields:
417 - name: outputBinding
418 type: CommandOutputBinding?
419 jsonldPredicate: "cwl:outputBinding"
420 doc: |
421 Describes how to generate this output object based on the files
422 produced by a CommandLineTool
423
424
425 - name: CommandOutputRecordSchema
426 type: record
427 extends: OutputRecordSchema
428 specialize:
429 - specializeFrom: OutputRecordField
430 specializeTo: CommandOutputRecordField
431
432
433 - name: CommandOutputEnumSchema
434 type: record
435 extends: OutputEnumSchema
436 specialize:
437 - specializeFrom: OutputRecordSchema
438 specializeTo: CommandOutputRecordSchema
439 - specializeFrom: OutputEnumSchema
440 specializeTo: CommandOutputEnumSchema
441 - specializeFrom: OutputArraySchema
442 specializeTo: CommandOutputArraySchema
443
444
445 - name: CommandOutputArraySchema
446 type: record
447 extends: OutputArraySchema
448 specialize:
449 - specializeFrom: OutputRecordSchema
450 specializeTo: CommandOutputRecordSchema
451 - specializeFrom: OutputEnumSchema
452 specializeTo: CommandOutputEnumSchema
453 - specializeFrom: OutputArraySchema
454 specializeTo: CommandOutputArraySchema
455
456
457 - type: record
458 name: CommandInputParameter
459 extends: InputParameter
460 doc: An input parameter for a CommandLineTool.
461 fields:
462 - name: type
463 type:
464 - CWLType
465 - stdin
466 - CommandInputRecordSchema
467 - CommandInputEnumSchema
468 - CommandInputArraySchema
469 - string
470 - type: array
471 items:
472 - CWLType
473 - CommandInputRecordSchema
474 - CommandInputEnumSchema
475 - CommandInputArraySchema
476 - string
477 jsonldPredicate:
478 "_id": "sld:type"
479 "_type": "@vocab"
480 refScope: 2
481 typeDSL: True
482 doc: |
483 Specify valid types of data that may be assigned to this parameter.
484 - name: inputBinding
485 type: CommandLineBinding?
486 doc: |
487 Describes how to turns the input parameters of a process into
488 command line arguments.
489 jsonldPredicate: "cwl:inputBinding"
490
491 - type: record
492 name: CommandOutputParameter
493 extends: OutputParameter
494 doc: An output parameter for a CommandLineTool.
495 fields:
496 - name: type
497 type:
498 - CWLType
499 - stdout
500 - stderr
501 - CommandOutputRecordSchema
502 - CommandOutputEnumSchema
503 - CommandOutputArraySchema
504 - string
505 - type: array
506 items:
507 - CWLType
508 - CommandOutputRecordSchema
509 - CommandOutputEnumSchema
510 - CommandOutputArraySchema
511 - string
512 jsonldPredicate:
513 "_id": "sld:type"
514 "_type": "@vocab"
515 refScope: 2
516 typeDSL: True
517 doc: |
518 Specify valid types of data that may be assigned to this parameter.
519 - name: outputBinding
520 type: CommandOutputBinding?
521 jsonldPredicate: "cwl:outputBinding"
522 doc: Describes how to generate this output object based on the files
523 produced by a CommandLineTool
524
525 - name: stdin
526 type: enum
527 symbols: [ "cwl:stdin" ]
528 docParent: "#CommandOutputParameter"
529 doc: |
530 Only valid as a `type` for a `CommandLineTool` input with no
531 `inputBinding` set. `stdin` must not be specified at the `CommandLineTool`
532 level.
533
534 The following
535 ```
536 inputs:
537 an_input_name:
538 type: stdin
539 ```
540 is equivalent to
541 ```
542 inputs:
543 an_input_name:
544 type: File
545 streamable: true
546
547 stdin: ${inputs.an_input_name.path}
548 ```
549
550 - name: stdout
551 type: enum
552 symbols: [ "cwl:stdout" ]
553 docParent: "#CommandOutputParameter"
554 doc: |
555 Only valid as a `type` for a `CommandLineTool` output with no
556 `outputBinding` set.
557
558 The following
559 ```
560 outputs:
561 an_output_name:
562 type: stdout
563
564 stdout: a_stdout_file
565 ```
566 is equivalent to
567 ```
568 outputs:
569 an_output_name:
570 type: File
571 streamable: true
572 outputBinding:
573 glob: a_stdout_file
574
575 stdout: a_stdout_file
576 ```
577
578 If there is no `stdout` name provided, a random filename will be created.
579 For example, the following
580 ```
581 outputs:
582 an_output_name:
583 type: stdout
584 ```
585 is equivalent to
586 ```
587 outputs:
588 an_output_name:
589 type: File
590 streamable: true
591 outputBinding:
592 glob: random_stdout_filenameABCDEFG
593
594 stdout: random_stdout_filenameABCDEFG
595 ```
596
597
598 - name: stderr
599 type: enum
600 symbols: [ "cwl:stderr" ]
601 docParent: "#CommandOutputParameter"
602 doc: |
603 Only valid as a `type` for a `CommandLineTool` output with no
604 `outputBinding` set.
605
606 The following
607 ```
608 outputs:
609 an_output_name:
610 type: stderr
611
612 stderr: a_stderr_file
613 ```
614 is equivalent to
615 ```
616 outputs:
617 an_output_name:
618 type: File
619 streamable: true
620 outputBinding:
621 glob: a_stderr_file
622
623 stderr: a_stderr_file
624 ```
625
626 If there is no `stderr` name provided, a random filename will be created.
627 For example, the following
628 ```
629 outputs:
630 an_output_name:
631 type: stderr
632 ```
633 is equivalent to
634 ```
635 outputs:
636 an_output_name:
637 type: File
638 streamable: true
639 outputBinding:
640 glob: random_stderr_filenameABCDEFG
641
642 stderr: random_stderr_filenameABCDEFG
643 ```
644
645
646 - type: record
647 name: CommandLineTool
648 extends: Process
649 documentRoot: true
650 specialize:
651 - specializeFrom: InputParameter
652 specializeTo: CommandInputParameter
653 - specializeFrom: OutputParameter
654 specializeTo: CommandOutputParameter
655 doc: |
656 This defines the schema of the CWL Command Line Tool Description document.
657
658 fields:
659 - name: class
660 jsonldPredicate:
661 "_id": "@type"
662 "_type": "@vocab"
663 type: string
664 - name: baseCommand
665 doc: |
666 Specifies the program to execute. If an array, the first element of
667 the array is the command to execute, and subsequent elements are
668 mandatory command line arguments. The elements in `baseCommand` must
669 appear before any command line bindings from `inputBinding` or
670 `arguments`.
671
672 If `baseCommand` is not provided or is an empty array, the first
673 element of the command line produced after processing `inputBinding` or
674 `arguments` must be used as the program to execute.
675
676 If the program includes a path separator character it must
677 be an absolute path, otherwise it is an error. If the program does not
678 include a path separator, search the `$PATH` variable in the runtime
679 environment of the workflow runner find the absolute path of the
680 executable.
681 type:
682 - string?
683 - string[]?
684 jsonldPredicate:
685 "_id": "cwl:baseCommand"
686 "_container": "@list"
687 - name: arguments
688 doc: |
689 Command line bindings which are not directly associated with input
690 parameters. If the value is a string, it is used as a string literal
691 argument. If it is an Expression, the result of the evaluation is used
692 as an argument.
693 type:
694 - "null"
695 - type: array
696 items: [string, Expression, CommandLineBinding]
697 jsonldPredicate:
698 "_id": "cwl:arguments"
699 "_container": "@list"
700 - name: stdin
701 type: ["null", string, Expression]
702 jsonldPredicate: "https://w3id.org/cwl/cwl#stdin"
703 doc: |
704 A path to a file whose contents must be piped into the command's
705 standard input stream.
706 - name: stderr
707 type: ["null", string, Expression]
708 jsonldPredicate: "https://w3id.org/cwl/cwl#stderr"
709 doc: |
710 Capture the command's standard error stream to a file written to
711 the designated output directory.
712
713 If `stderr` is a string, it specifies the file name to use.
714
715 If `stderr` is an expression, the expression is evaluated and must
716 return a string with the file name to use to capture stderr. If the
717 return value is not a string, or the resulting path contains illegal
718 characters (such as the path separator `/`) it is an error.
719 - name: stdout
720 type: ["null", string, Expression]
721 jsonldPredicate: "https://w3id.org/cwl/cwl#stdout"
722 doc: |
723 Capture the command's standard output stream to a file written to
724 the designated output directory.
725
726 If `stdout` is a string, it specifies the file name to use.
727
728 If `stdout` is an expression, the expression is evaluated and must
729 return a string with the file name to use to capture stdout. If the
730 return value is not a string, or the resulting path contains illegal
731 characters (such as the path separator `/`) it is an error.
732 - name: successCodes
733 type: int[]?
734 doc: |
735 Exit codes that indicate the process completed successfully.
736
737 - name: temporaryFailCodes
738 type: int[]?
739 doc: |
740 Exit codes that indicate the process failed due to a possibly
741 temporary condition, where executing the process with the same
742 runtime environment and inputs may produce different results.
743
744 - name: permanentFailCodes
745 type: int[]?
746 doc:
747 Exit codes that indicate the process failed due to a permanent logic
748 error, where executing the process with the same runtime environment and
749 same inputs is expected to always fail.
750
751
752 - type: record
753 name: DockerRequirement
754 extends: ProcessRequirement
755 doc: |
756 Indicates that a workflow component should be run in a
757 [Docker](http://docker.com) or Docker-compatible (such as
758 [Singularity](https://www.sylabs.io/) and [udocker](https://github.com/indigo-dc/udocker)) container environment and
759 specifies how to fetch or build the image.
760
761 If a CommandLineTool lists `DockerRequirement` under
762 `hints` (or `requirements`), it may (or must) be run in the specified Docker
763 container.
764
765 The platform must first acquire or install the correct Docker image as
766 specified by `dockerPull`, `dockerImport`, `dockerLoad` or `dockerFile`.
767
768 The platform must execute the tool in the container using `docker run` with
769 the appropriate Docker image and tool command line.
770
771 The workflow platform may provide input files and the designated output
772 directory through the use of volume bind mounts. The platform should rewrite
773 file paths in the input object to correspond to the Docker bind mounted
774 locations. That is, the platform should rewrite values in the parameter context
775 such as `runtime.outdir`, `runtime.tmpdir` and others to be valid paths
776 within the container. The platform must ensure that `runtime.outdir` and
777 `runtime.tmpdir` are distinct directories.
778
779 When running a tool contained in Docker, the workflow platform must not
780 assume anything about the contents of the Docker container, such as the
781 presence or absence of specific software, except to assume that the
782 generated command line represents a valid command within the runtime
783 environment of the container.
784
785 A container image may specify an
786 [ENTRYPOINT](https://docs.docker.com/engine/reference/builder/#entrypoint)
787 and/or
788 [CMD](https://docs.docker.com/engine/reference/builder/#cmd).
789 Command line arguments will be appended after all elements of
790 ENTRYPOINT, and will override all elements specified using CMD (in
791 other words, CMD is only used when the CommandLineTool definition
792 produces an empty command line).
793
794 Use of implicit ENTRYPOINT or CMD are discouraged due to reproducibility
795 concerns of the implicit hidden execution point (For further discussion, see
796 [https://doi.org/10.12688/f1000research.15140.1](https://doi.org/10.12688/f1000research.15140.1)). Portable
797 CommandLineTool wrappers in which use of a container is optional must not rely on ENTRYPOINT or CMD.
798 CommandLineTools which do rely on ENTRYPOINT or CMD must list `DockerRequirement` in the
799 `requirements` section.
800
801 ## Interaction with other requirements
802
803 If [EnvVarRequirement](#EnvVarRequirement) is specified alongside a
804 DockerRequirement, the environment variables must be provided to Docker
805 using `--env` or `--env-file` and interact with the container's preexisting
806 environment as defined by Docker.
807
808 fields:
809 - name: class
810 type: string
811 doc: "Always 'DockerRequirement'"
812 jsonldPredicate:
813 "_id": "@type"
814 "_type": "@vocab"
815 - name: dockerPull
816 type: string?
817 doc: "Specify a Docker image to retrieve using `docker pull`."
818 - name: dockerLoad
819 type: string?
820 doc: "Specify a HTTP URL from which to download a Docker image using `docker load`."
821 - name: dockerFile
822 type: string?
823 doc: "Supply the contents of a Dockerfile which will be built using `docker build`."
824 - name: dockerImport
825 type: string?
826 doc: "Provide HTTP URL to download and gunzip a Docker images using `docker import."
827 - name: dockerImageId
828 type: string?
829 doc: |
830 The image id that will be used for `docker run`. May be a
831 human-readable image name or the image identifier hash. May be skipped
832 if `dockerPull` is specified, in which case the `dockerPull` image id
833 must be used.
834 - name: dockerOutputDirectory
835 type: string?
836 doc: |
837 Set the designated output directory to a specific location inside the
838 Docker container.
839
840
841 - type: record
842 name: SoftwareRequirement
843 extends: ProcessRequirement
844 doc: |
845 A list of software packages that should be configured in the environment of
846 the defined process.
847 fields:
848 - name: class
849 type: string
850 doc: "Always 'SoftwareRequirement'"
851 jsonldPredicate:
852 "_id": "@type"
853 "_type": "@vocab"
854 - name: packages
855 type: SoftwarePackage[]
856 doc: "The list of software to be configured."
857 jsonldPredicate:
858 mapSubject: package
859 mapPredicate: specs
860
861 - name: SoftwarePackage
862 type: record
863 fields:
864 - name: package
865 type: string
866 doc: |
867 The name of the software to be made available. If the name is
868 common, inconsistent, or otherwise ambiguous it should be combined with
869 one or more identifiers in the `specs` field.
870 - name: version
871 type: string[]?
872 doc: |
873 The (optional) versions of the software that are known to be
874 compatible.
875 - name: specs
876 type: string[]?
877 jsonldPredicate: {_type: "@id", noLinkCheck: true}
878 doc: |
879 One or more [IRI](https://en.wikipedia.org/wiki/Internationalized_Resource_Identifier)s
880 identifying resources for installing or enabling the software named in
881 the `package` field. Implementations may provide resolvers which map
882 these software identifer IRIs to some configuration action; or they can
883 use only the name from the `package` field on a best effort basis.
884
885 For example, the IRI https://packages.debian.org/bowtie could
886 be resolved with `apt-get install bowtie`. The IRI
887 https://anaconda.org/bioconda/bowtie could be resolved with `conda
888 install -c bioconda bowtie`.
889
890 IRIs can also be system independent and used to map to a specific
891 software installation or selection mechanism.
892 Using [RRID](https://www.identifiers.org/rrid/) as an example:
893 https://identifiers.org/rrid/RRID:SCR_005476
894 could be fulfilled using the above mentioned Debian or bioconda
895 package, a local installation managed by [Environement Modules](http://modules.sourceforge.net/),
896 or any other mechanism the platform chooses. IRIs can also be from
897 identifer sources that are discipline specific yet still system
898 independent. As an example, the equivalent [ELIXIR Tools and Data
899 Service Registry](https://bio.tools) IRI to the previous RRID example is
900 https://bio.tools/tool/bowtie2/version/2.2.8.
901 If supported by a given registry, implementations are encouraged to
902 query these system independent sofware identifier IRIs directly for
903 links to packaging systems.
904
905 A site specific IRI can be listed as well. For example, an academic
906 computing cluster using Environement Modules could list the IRI
907 `https://hpc.example.edu/modules/bowtie-tbb/1.22` to indicate that
908 `module load bowtie-tbb/1.1.2` should be executed to make available
909 `bowtie` version 1.1.2 compiled with the TBB library prior to running
910 the accompanying Workflow or CommandLineTool. Note that the example IRI
911 is specific to a particular institution and computing environment as
912 the Environment Modules system does not have a common namespace or
913 standardized naming convention.
914
915 This last example is the least portable and should only be used if
916 mechanisms based off of the `package` field or more generic IRIs are
917 unavailable or unsuitable. While harmless to other sites, site specific
918 software IRIs should be left out of shared CWL descriptions to avoid
919 clutter.
920
921 - name: Dirent
922 type: record
923 doc: |
924 Define a file or subdirectory that must be placed in the designated output
925 directory prior to executing the command line tool. May be the result of
926 executing an expression, such as building a configuration file from a
927 template.
928 fields:
929 - name: entryname
930 type: ["null", string, Expression]
931 jsonldPredicate:
932 _id: cwl:entryname
933 doc: |
934 The name of the file or subdirectory to create in the output directory.
935 If `entry` is a File or Directory, the `entryname` field overrides the value
936 of `basename` of the File or Directory object. Optional.
937 - name: entry
938 type: [string, Expression]
939 jsonldPredicate:
940 _id: cwl:entry
941 doc: |
942 If the value is a string literal or an expression which evaluates to a
943 string, a new file must be created with the string as the file contents.
944
945 If the value is an expression that evaluates to a `File` object, this
946 indicates the referenced file should be added to the designated output
947 directory prior to executing the tool.
948
949 If the value is an expression that evaluates to a `Dirent` object, this
950 indicates that the File or Directory in `entry` should be added to the
951 designated output directory with the name in `entryname`.
952
953 If `writable` is false, the file may be made available using a bind
954 mount or file system link to avoid unnecessary copying of the input
955 file.
956 - name: writable
957 type: boolean?
958 default: false
959 doc: |
960 If true, the file or directory must be writable by the tool. Changes
961 to the file or directory must be isolated and not visible by any other
962 CommandLineTool process. This may be implemented by making a copy of
963 the original file or directory. Default false (files and directories
964 read-only by default).
965
966 A directory marked as `writable: true` implies that all files and
967 subdirectories are recursively writable as well.
968
969
970 - name: InitialWorkDirRequirement
971 type: record
972 extends: ProcessRequirement
973 doc:
974 Define a list of files and subdirectories that must be created by the
975 workflow platform in the designated output directory prior to executing the
976 command line tool.
977 fields:
978 - name: class
979 type: string
980 doc: InitialWorkDirRequirement
981 jsonldPredicate:
982 "_id": "@type"
983 "_type": "@vocab"
984 - name: listing
985 type:
986 - type: array
987 items:
988 - "null"
989 - File
990 - type: array
991 items:
992 - File
993 - Directory
994 - Directory
995 - Dirent
996 - Expression
997 - Expression
998 jsonldPredicate:
999 _id: "cwl:listing"
1000 doc: |
1001 The list of files or subdirectories that must be placed in the
1002 designated output directory prior to executing the command line tool.
1003
1004 May be an expression. If so, the expression return value must validate as
1005 `{type: array, items: ["null", File, File[], Directory, Directory[], Dirent]}`.
1006
1007 Files or Directories which are listed in the input parameters and
1008 appear in the `InitialWorkDirRequirement` listing must have their
1009 `path` set to their staged location in the designated output directory.
1010 If the same File or Directory appears more than once in the
1011 `InitialWorkDirRequirement` listing, the implementation must choose
1012 exactly one value for `path`; how this value is chosen is undefined.
1013
1014
1015 - name: EnvVarRequirement
1016 type: record
1017 extends: ProcessRequirement
1018 doc: |
1019 Define a list of environment variables which will be set in the
1020 execution environment of the tool. See `EnvironmentDef` for details.
1021 fields:
1022 - name: class
1023 type: string
1024 doc: "Always 'EnvVarRequirement'"
1025 jsonldPredicate:
1026 "_id": "@type"
1027 "_type": "@vocab"
1028 - name: envDef
1029 type: EnvironmentDef[]
1030 doc: The list of environment variables.
1031 jsonldPredicate:
1032 mapSubject: envName
1033 mapPredicate: envValue
1034
1035
1036 - type: record
1037 name: ShellCommandRequirement
1038 extends: ProcessRequirement
1039 doc: |
1040 Modify the behavior of CommandLineTool to generate a single string
1041 containing a shell command line. Each item in the argument list must be
1042 joined into a string separated by single spaces and quoted to prevent
1043 intepretation by the shell, unless `CommandLineBinding` for that argument
1044 contains `shellQuote: false`. If `shellQuote: false` is specified, the
1045 argument is joined into the command string without quoting, which allows
1046 the use of shell metacharacters such as `|` for pipes.
1047 fields:
1048 - name: class
1049 type: string
1050 doc: "Always 'ShellCommandRequirement'"
1051 jsonldPredicate:
1052 "_id": "@type"
1053 "_type": "@vocab"
1054
1055
1056 - type: record
1057 name: ResourceRequirement
1058 extends: ProcessRequirement
1059 doc: |
1060 Specify basic hardware resource requirements.
1061
1062 "min" is the minimum amount of a resource that must be reserved to schedule
1063 a job. If "min" cannot be satisfied, the job should not be run.
1064
1065 "max" is the maximum amount of a resource that the job shall be permitted
1066 to use. If a node has sufficient resources, multiple jobs may be scheduled
1067 on a single node provided each job's "max" resource requirements are
1068 met. If a job attempts to exceed its "max" resource allocation, an
1069 implementation may deny additional resources, which may result in job
1070 failure.
1071
1072 If "min" is specified but "max" is not, then "max" == "min"
1073 If "max" is specified by "min" is not, then "min" == "max".
1074
1075 It is an error if max < min.
1076
1077 It is an error if the value of any of these fields is negative.
1078
1079 If neither "min" nor "max" is specified for a resource, use the default values below.
1080
1081 fields:
1082 - name: class
1083 type: string
1084 doc: "Always 'ResourceRequirement'"
1085 jsonldPredicate:
1086 "_id": "@type"
1087 "_type": "@vocab"
1088 - name: coresMin
1089 type: ["null", long, Expression]
1090 doc: Minimum reserved number of CPU cores (default is 1)
1091
1092 - name: coresMax
1093 type: ["null", int, Expression]
1094 doc: Maximum reserved number of CPU cores
1095
1096 - name: ramMin
1097 type: ["null", long, Expression]
1098 doc: Minimum reserved RAM in mebibytes (2**20) (default is 256)
1099
1100 - name: ramMax
1101 type: ["null", long, Expression]
1102 doc: Maximum reserved RAM in mebibytes (2**20)
1103
1104 - name: tmpdirMin
1105 type: ["null", long, Expression]
1106 doc: Minimum reserved filesystem based storage for the designated temporary directory, in mebibytes (2**20) (default is 1024)
1107
1108 - name: tmpdirMax
1109 type: ["null", long, Expression]
1110 doc: Maximum reserved filesystem based storage for the designated temporary directory, in mebibytes (2**20)
1111
1112 - name: outdirMin
1113 type: ["null", long, Expression]
1114 doc: Minimum reserved filesystem based storage for the designated output directory, in mebibytes (2**20) (default is 1024)
1115
1116 - name: outdirMax
1117 type: ["null", long, Expression]
1118 doc: Maximum reserved filesystem based storage for the designated output directory, in mebibytes (2**20)
1119
1120
1121 - type: record
1122 name: WorkReuse
1123 extends: ProcessRequirement
1124 doc: |
1125 For implementations that support reusing output from past work (on
1126 the assumption that same code and same input produce same
1127 results), control whether to enable or disable the reuse behavior
1128 for a particular tool or step (to accomodate situations where that
1129 assumption is incorrect). A reused step is not executed but
1130 instead returns the same output as the original execution.
1131
1132 If `enableReuse` is not specified, correct tools should assume it
1133 is enabled by default.
1134 fields:
1135 - name: class
1136 type: string
1137 doc: "Always 'WorkReuse'"
1138 jsonldPredicate:
1139 "_id": "@type"
1140 "_type": "@vocab"
1141 - name: enableReuse
1142 type: [boolean, Expression]
1143 default: true
1144
1145
1146 - type: record
1147 name: NetworkAccess
1148 extends: ProcessRequirement
1149 doc: |
1150 Indicate whether a process requires outgoing IPv4/IPv6 network
1151 access. Choice of IPv4 or IPv6 is implementation and site
1152 specific, correct tools must support both.
1153
1154 If `networkAccess` is false or not specified, tools must not
1155 assume network access, except for localhost (the loopback device).
1156
1157 If `networkAccess` is true, the tool must be able to make outgoing
1158 connections to network resources. Resources may be on a private
1159 subnet or the public Internet. However, implementations and sites
1160 may apply their own security policies to restrict what is
1161 accessible by the tool.
1162
1163 Enabling network access does not imply a publically routable IP
1164 address or the ability to accept inbound connections.
1165
1166 fields:
1167 - name: class
1168 type: string
1169 doc: "Always 'NetworkAccess'"
1170 jsonldPredicate:
1171 "_id": "@type"
1172 "_type": "@vocab"
1173 - name: networkAccess
1174 type: [boolean, Expression]
1175
1176 - name: InplaceUpdateRequirement
1177 type: record
1178 extends: cwl:ProcessRequirement
1179 doc: |
1180
1181 If `inplaceUpdate` is true, then an implementation supporting this
1182 feature may permit tools to directly update files with `writable:
1183 true` in InitialWorkDirRequirement. That is, as an optimization,
1184 files may be destructively modified in place as opposed to copied
1185 and updated.
1186
1187 An implementation must ensure that only one workflow step may
1188 access a writable file at a time. It is an error if a file which
1189 is writable by one workflow step file is accessed (for reading or
1190 writing) by any other workflow step running independently.
1191 However, a file which has been updated in a previous completed
1192 step may be used as input to multiple steps, provided it is
1193 read-only in every step.
1194
1195 Workflow steps which modify a file must produce the modified file
1196 as output. Downstream steps which futher process the file must
1197 use the output of previous steps, and not refer to a common input
1198 (this is necessary for both ordering and correctness).
1199
1200 Workflow authors should provide this in the `hints` section. The
1201 intent of this feature is that workflows produce the same results
1202 whether or not InplaceUpdateRequirement is supported by the
1203 implementation, and this feature is primarily available as an
1204 optimization for particular environments.
1205
1206 Users and implementers should be aware that workflows that
1207 destructively modify inputs may not be repeatable or reproducible.
1208 In particular, enabling this feature implies that WorkReuse should
1209 not be enabled.
1210
1211 fields:
1212 class:
1213 type: string
1214 doc: "Always 'InplaceUpdateRequirement'"
1215 jsonldPredicate:
1216 "_id": "@type"
1217 "_type": "@vocab"
1218 inplaceUpdate:
1219 type: boolean
1220
1221 - type: record
1222 name: ToolTimeLimit
1223 extends: ProcessRequirement
1224 doc: |
1225 Set an upper limit on the execution time of a CommandLineTool.
1226 A CommandLineTool whose execution duration exceeds the time
1227 limit may be preemptively terminated and considered failed.
1228 May also be used by batch systems to make scheduling decisions.
1229 The execution duration excludes external operations, such as
1230 staging of files, pulling a docker image etc, and only counts
1231 wall-time for the execution of the command line itself.
1232 fields:
1233 - name: class
1234 type: string
1235 doc: "Always 'ToolTimeLimit'"
1236 jsonldPredicate:
1237 "_id": "@type"
1238 "_type": "@vocab"
1239 - name: timelimit
1240 type: [long, Expression]
1241 doc: |
1242 The time limit, in seconds. A time limit of zero means no
1243 time limit. Negative time limits are an error.