Mercurial > repos > davidmurphy > codonlogo

comparison corebio/_future/subprocess.py @ 7:8d676bbd1f2d

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Uploaded
author davidmurphy
date Mon, 16 Jan 2012 07:03:36 -0500
parents c55bdc2fb9fa
children
comparison
equal deleted inserted replaced
6:4a4aca3d57c9 7:8d676bbd1f2d
1 # subprocess - Subprocesses with accessible I/O streams
2 #
3 # For more information about this module, see PEP 324.
4 #
5 # Copyright (c) 2003-2004 by Peter Astrand <astrand@lysator.liu.se>
6 #
7 # By obtaining, using, and/or copying this software and/or its
8 # associated documentation, you agree that you have read, understood,
9 # and will comply with the following terms and conditions:
10 #
11 # Permission to use, copy, modify, and distribute this software and
12 # its associated documentation for any purpose and without fee is
13 # hereby granted, provided that the above copyright notice appears in
14 # all copies, and that both that copyright notice and this permission
15 # notice appear in supporting documentation, and that the name of the
16 # author not be used in advertising or publicity pertaining to
17 # distribution of the software without specific, written prior
18 # permission.
19 #
20 # THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE,
21 # INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS.
22 # IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, INDIRECT OR
23 # CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS
24 # OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
25 # NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
26 # WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
27
28 r"""subprocess - Subprocesses with accessible I/O streams
29
30 This module allows you to spawn processes, connect to their
31 input/output/error pipes, and obtain their return codes. This module
32 intends to replace several other, older modules and functions, like:
33
34 os.system
35 os.spawn*
36 os.popen*
37 popen2.*
38 commands.*
39
40 Information about how the subprocess module can be used to replace these
41 modules and functions can be found below.
42
43
44
45 Using the subprocess module
46 ===========================
47 This module defines one class called Popen:
48
49 class Popen(args, bufsize=0, executable=None,
50 stdin=None, stdout=None, stderr=None,
51 preexec_fn=None, close_fds=False, shell=False,
52 cwd=None, env=None, universal_newlines=False,
53 startupinfo=None, creationflags=0):
54
55
56 Arguments are:
57
58 args should be a string, or a sequence of program arguments. The
59 program to execute is normally the first item in the args sequence or
60 string, but can be explicitly set by using the executable argument.
61
62 On UNIX, with shell=False (default): In this case, the Popen class
63 uses os.execvp() to execute the child program. args should normally
64 be a sequence. A string will be treated as a sequence with the string
65 as the only item (the program to execute).
66
67 On UNIX, with shell=True: If args is a string, it specifies the
68 command string to execute through the shell. If args is a sequence,
69 the first item specifies the command string, and any additional items
70 will be treated as additional shell arguments.
71
72 On Windows: the Popen class uses CreateProcess() to execute the child
73 program, which operates on strings. If args is a sequence, it will be
74 converted to a string using the list2cmdline method. Please note that
75 not all MS Windows applications interpret the command line the same
76 way: The list2cmdline is designed for applications using the same
77 rules as the MS C runtime.
78
79 bufsize, if given, has the same meaning as the corresponding argument
80 to the built-in open() function: 0 means unbuffered, 1 means line
81 buffered, any other positive value means use a buffer of
82 (approximately) that size. A negative bufsize means to use the system
83 default, which usually means fully buffered. The default value for
84 bufsize is 0 (unbuffered).
85
86 stdin, stdout and stderr specify the executed programs' standard
87 input, standard output and standard error file handles, respectively.
88 Valid values are PIPE, an existing file descriptor (a positive
89 integer), an existing file object, and None. PIPE indicates that a
90 new pipe to the child should be created. With None, no redirection
91 will occur; the child's file handles will be inherited from the
92 parent. Additionally, stderr can be STDOUT, which indicates that the
93 stderr data from the applications should be captured into the same
94 file handle as for stdout.
95
96 If preexec_fn is set to a callable object, this object will be called
97 in the child process just before the child is executed.
98
99 If close_fds is true, all file descriptors except 0, 1 and 2 will be
100 closed before the child process is executed.
101
102 if shell is true, the specified command will be executed through the
103 shell.
104
105 If cwd is not None, the current directory will be changed to cwd
106 before the child is executed.
107
108 If env is not None, it defines the environment variables for the new
109 process.
110
111 If universal_newlines is true, the file objects stdout and stderr are
112 opened as a text files, but lines may be terminated by any of '\n',
113 the Unix end-of-line convention, '\r', the Macintosh convention or
114 '\r\n', the Windows convention. All of these external representations
115 are seen as '\n' by the Python program. Note: This feature is only
116 available if Python is built with universal newline support (the
117 default). Also, the newlines attribute of the file objects stdout,
118 stdin and stderr are not updated by the communicate() method.
119
120 The startupinfo and creationflags, if given, will be passed to the
121 underlying CreateProcess() function. They can specify things such as
122 appearance of the main window and priority for the new process.
123 (Windows only)
124
125
126 This module also defines two shortcut functions:
127
128 call(*args, **kwargs):
129 Run command with arguments. Wait for command to complete, then
130 return the returncode attribute.
131
132 The arguments are the same as for the Popen constructor. Example:
133
134 retcode = call(["ls", "-l"])
135
136
137 Exceptions
138 ----------
139 Exceptions raised in the child process, before the new program has
140 started to execute, will be re-raised in the parent. Additionally,
141 the exception object will have one extra attribute called
142 'child_traceback', which is a string containing traceback information
143 from the childs point of view.
144
145 The most common exception raised is OSError. This occurs, for
146 example, when trying to execute a non-existent file. Applications
147 should prepare for OSErrors.
148
149 A ValueError will be raised if Popen is called with invalid arguments.
150
151
152 Security
153 --------
154 Unlike some other popen functions, this implementation will never call
155 /bin/sh implicitly. This means that all characters, including shell
156 metacharacters, can safely be passed to child processes.
157
158
159 Popen objects
160 =============
161 Instances of the Popen class have the following methods:
162
163 poll()
164 Check if child process has terminated. Returns returncode
165 attribute.
166
167 wait()
168 Wait for child process to terminate. Returns returncode attribute.
169
170 communicate(input=None)
171 Interact with process: Send data to stdin. Read data from stdout
172 and stderr, until end-of-file is reached. Wait for process to
173 terminate. The optional stdin argument should be a string to be
174 sent to the child process, or None, if no data should be sent to
175 the child.
176
177 communicate() returns a tuple (stdout, stderr).
178
179 Note: The data read is buffered in memory, so do not use this
180 method if the data size is large or unlimited.
181
182 The following attributes are also available:
183
184 stdin
185 If the stdin argument is PIPE, this attribute is a file object
186 that provides input to the child process. Otherwise, it is None.
187
188 stdout
189 If the stdout argument is PIPE, this attribute is a file object
190 that provides output from the child process. Otherwise, it is
191 None.
192
193 stderr
194 If the stderr argument is PIPE, this attribute is file object that
195 provides error output from the child process. Otherwise, it is
196 None.
197
198 pid
199 The process ID of the child process.
200
201 returncode
202 The child return code. A None value indicates that the process
203 hasn't terminated yet. A negative value -N indicates that the
204 child was terminated by signal N (UNIX only).
205
206
207 Replacing older functions with the subprocess module
208 ====================================================
209 In this section, "a ==> b" means that b can be used as a replacement
210 for a.
211
212 Note: All functions in this section fail (more or less) silently if
213 the executed program cannot be found; this module raises an OSError
214 exception.
215
216 In the following examples, we assume that the subprocess module is
217 imported with "from subprocess import *".
218
219
220 Replacing /bin/sh shell backquote
221 ---------------------------------
222 output=`mycmd myarg`
223 ==>
224 output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0]
225
226
227 Replacing shell pipe line
228 -------------------------
229 output=`dmesg | grep hda`
230 ==>
231 p1 = Popen(["dmesg"], stdout=PIPE)
232 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
233 output = p2.communicate()[0]
234
235
236 Replacing os.system()
237 ---------------------
238 sts = os.system("mycmd" + " myarg")
239 ==>
240 p = Popen("mycmd" + " myarg", shell=True)
241 sts = os.waitpid(p.pid, 0)
242
243 Note:
244
245 * Calling the program through the shell is usually not required.
246
247 * It's easier to look at the returncode attribute than the
248 exitstatus.
249
250 A more real-world example would look like this:
251
252 try:
253 retcode = call("mycmd" + " myarg", shell=True)
254 if retcode < 0:
255 print >>sys.stderr, "Child was terminated by signal", -retcode
256 else:
257 print >>sys.stderr, "Child returned", retcode
258 except OSError, e:
259 print >>sys.stderr, "Execution failed:", e
260
261
262 Replacing os.spawn*
263 -------------------
264 P_NOWAIT example:
265
266 pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
267 ==>
268 pid = Popen(["/bin/mycmd", "myarg"]).pid
269
270
271 P_WAIT example:
272
273 retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
274 ==>
275 retcode = call(["/bin/mycmd", "myarg"])
276
277
278 Vector example:
279
280 os.spawnvp(os.P_NOWAIT, path, args)
281 ==>
282 Popen([path] + args[1:])
283
284
285 Environment example:
286
287 os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
288 ==>
289 Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
290
291
292 Replacing os.popen*
293 -------------------
294 pipe = os.popen(cmd, mode='r', bufsize)
295 ==>
296 pipe = Popen(cmd, shell=True, bufsize=bufsize, stdout=PIPE).stdout
297
298 pipe = os.popen(cmd, mode='w', bufsize)
299 ==>
300 pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin
301
302
303 (child_stdin, child_stdout) = os.popen2(cmd, mode, bufsize)
304 ==>
305 p = Popen(cmd, shell=True, bufsize=bufsize,
306 stdin=PIPE, stdout=PIPE, close_fds=True)
307 (child_stdin, child_stdout) = (p.stdin, p.stdout)
308
309
310 (child_stdin,
311 child_stdout,
312 child_stderr) = os.popen3(cmd, mode, bufsize)
313 ==>
314 p = Popen(cmd, shell=True, bufsize=bufsize,
315 stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
316 (child_stdin,
317 child_stdout,
318 child_stderr) = (p.stdin, p.stdout, p.stderr)
319
320
321 (child_stdin, child_stdout_and_stderr) = os.popen4(cmd, mode, bufsize)
322 ==>
323 p = Popen(cmd, shell=True, bufsize=bufsize,
324 stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
325 (child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout)
326
327
328 Replacing popen2.*
329 ------------------
330 Note: If the cmd argument to popen2 functions is a string, the command
331 is executed through /bin/sh. If it is a list, the command is directly
332 executed.
333
334 (child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode)
335 ==>
336 p = Popen(["somestring"], shell=True, bufsize=bufsize
337 stdin=PIPE, stdout=PIPE, close_fds=True)
338 (child_stdout, child_stdin) = (p.stdout, p.stdin)
339
340
341 (child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize, mode)
342 ==>
343 p = Popen(["mycmd", "myarg"], bufsize=bufsize,
344 stdin=PIPE, stdout=PIPE, close_fds=True)
345 (child_stdout, child_stdin) = (p.stdout, p.stdin)
346
347 The popen2.Popen3 and popen3.Popen4 basically works as subprocess.Popen,
348 except that:
349
350 * subprocess.Popen raises an exception if the execution fails
351 * the capturestderr argument is replaced with the stderr argument.
352 * stdin=PIPE and stdout=PIPE must be specified.
353 * popen2 closes all filedescriptors by default, but you have to specify
354 close_fds=True with subprocess.Popen.
355
356
357 """
358
359 import sys
360 mswindows = (sys.platform == "win32")
361
362 import os
363 import types
364 import traceback
365
366 if mswindows:
367 import threading
368 import msvcrt
369 if 0: # <-- change this to use pywin32 instead of the _subprocess driver
370 import pywintypes
371 from win32api import GetStdHandle, STD_INPUT_HANDLE, \
372 STD_OUTPUT_HANDLE, STD_ERROR_HANDLE
373 from win32api import GetCurrentProcess, DuplicateHandle, \
374 GetModuleFileName, GetVersion
375 from win32con import DUPLICATE_SAME_ACCESS, SW_HIDE
376 from win32pipe import CreatePipe
377 from win32process import CreateProcess, STARTUPINFO, \
378 GetExitCodeProcess, STARTF_USESTDHANDLES, \
379 STARTF_USESHOWWINDOW, CREATE_NEW_CONSOLE
380 from win32event import WaitForSingleObject, INFINITE, WAIT_OBJECT_0
381 else:
382 from _subprocess import *
383 class STARTUPINFO:
384 dwFlags = 0
385 hStdInput = None
386 hStdOutput = None
387 hStdError = None
388 class pywintypes:
389 error = IOError
390 else:
391 import select
392 import errno
393 import fcntl
394 import pickle
395
396 __all__ = ["Popen", "PIPE", "STDOUT", "call"]
397
398 try:
399 MAXFD = os.sysconf("SC_OPEN_MAX")
400 except:
401 MAXFD = 256
402
403 # True/False does not exist on 2.2.0
404 try:
405 False
406 except NameError:
407 False = 0
408 True = 1
409
410 _active = []
411
412 def _cleanup():
413 for inst in _active[:]:
414 inst.poll()
415
416 PIPE = -1
417 STDOUT = -2
418
419
420 def call(*args, **kwargs):
421 """Run command with arguments. Wait for command to complete, then
422 return the returncode attribute.
423
424 The arguments are the same as for the Popen constructor. Example:
425
426 retcode = call(["ls", "-l"])
427 """
428 return Popen(*args, **kwargs).wait()
429
430
431 def list2cmdline(seq):
432 """
433 Translate a sequence of arguments into a command line
434 string, using the same rules as the MS C runtime:
435
436 1) Arguments are delimited by white space, which is either a
437 space or a tab.
438
439 2) A string surrounded by double quotation marks is
440 interpreted as a single argument, regardless of white space
441 contained within. A quoted string can be embedded in an
442 argument.
443
444 3) A double quotation mark preceded by a backslash is
445 interpreted as a literal double quotation mark.
446
447 4) Backslashes are interpreted literally, unless they
448 immediately precede a double quotation mark.
449
450 5) If backslashes immediately precede a double quotation mark,
451 every pair of backslashes is interpreted as a literal
452 backslash. If the number of backslashes is odd, the last
453 backslash escapes the next double quotation mark as
454 described in rule 3.
455 """
456
457 # See
458 # http://msdn.microsoft.com/library/en-us/vccelng/htm/progs_12.asp
459 result = []
460 needquote = False
461 for arg in seq:
462 bs_buf = []
463
464 # Add a space to separate this argument from the others
465 if result:
466 result.append(' ')
467
468 needquote = (" " in arg) or ("\t" in arg)
469 if needquote:
470 result.append('"')
471
472 for c in arg:
473 if c == '\\':
474 # Don't know if we need to double yet.
475 bs_buf.append(c)
476 elif c == '"':
477 # Double backspaces.
478 result.append('\\' * len(bs_buf)*2)
479 bs_buf = []
480 result.append('\\"')
481 else:
482 # Normal char
483 if bs_buf:
484 result.extend(bs_buf)
485 bs_buf = []
486 result.append(c)
487
488 # Add remaining backspaces, if any.
489 if bs_buf:
490 result.extend(bs_buf)
491
492 if needquote:
493 result.extend(bs_buf)
494 result.append('"')
495
496 return ''.join(result)
497
498
499 class Popen(object):
500 def __init__(self, args, bufsize=0, executable=None,
501 stdin=None, stdout=None, stderr=None,
502 preexec_fn=None, close_fds=False, shell=False,
503 cwd=None, env=None, universal_newlines=False,
504 startupinfo=None, creationflags=0):
505 """Create new Popen instance."""
506 _cleanup()
507
508 if not isinstance(bufsize, (int, long)):
509 raise TypeError("bufsize must be an integer")
510
511 if mswindows:
512 if preexec_fn is not None:
513 raise ValueError("preexec_fn is not supported on Windows "
514 "platforms")
515 if close_fds:
516 raise ValueError("close_fds is not supported on Windows "
517 "platforms")
518 else:
519 # POSIX
520 if startupinfo is not None:
521 raise ValueError("startupinfo is only supported on Windows "
522 "platforms")
523 if creationflags != 0:
524 raise ValueError("creationflags is only supported on Windows "
525 "platforms")
526
527 self.stdin = None
528 self.stdout = None
529 self.stderr = None
530 self.pid = None
531 self.returncode = None
532 self.universal_newlines = universal_newlines
533
534 # Input and output objects. The general principle is like
535 # this:
536 #
537 # Parent Child
538 # ------ -----
539 # p2cwrite ---stdin---> p2cread
540 # c2pread <--stdout--- c2pwrite
541 # errread <--stderr--- errwrite
542 #
543 # On POSIX, the child objects are file descriptors. On
544 # Windows, these are Windows file handles. The parent objects
545 # are file descriptors on both platforms. The parent objects
546 # are None when not using PIPEs. The child objects are None
547 # when not redirecting.
548
549 (p2cread, p2cwrite,
550 c2pread, c2pwrite,
551 errread, errwrite) = self._get_handles(stdin, stdout, stderr)
552
553 self._execute_child(args, executable, preexec_fn, close_fds,
554 cwd, env, universal_newlines,
555 startupinfo, creationflags, shell,
556 p2cread, p2cwrite,
557 c2pread, c2pwrite,
558 errread, errwrite)
559
560 if p2cwrite:
561 self.stdin = os.fdopen(p2cwrite, 'wb', bufsize)
562 if c2pread:
563 if universal_newlines:
564 self.stdout = os.fdopen(c2pread, 'rU', bufsize)
565 else:
566 self.stdout = os.fdopen(c2pread, 'rb', bufsize)
567 if errread:
568 if universal_newlines:
569 self.stderr = os.fdopen(errread, 'rU', bufsize)
570 else:
571 self.stderr = os.fdopen(errread, 'rb', bufsize)
572
573 _active.append(self)
574
575
576 def _translate_newlines(self, data):
577 data = data.replace("\r\n", "\n")
578 data = data.replace("\r", "\n")
579 return data
580
581
582 if mswindows:
583 #
584 # Windows methods
585 #
586 def _get_handles(self, stdin, stdout, stderr):
587 """Construct and return tupel with IO objects:
588 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
589 """
590 if stdin == None and stdout == None and stderr == None:
591 return (None, None, None, None, None, None)
592
593 p2cread, p2cwrite = None, None
594 c2pread, c2pwrite = None, None
595 errread, errwrite = None, None
596
597 if stdin == None:
598 p2cread = GetStdHandle(STD_INPUT_HANDLE)
599 elif stdin == PIPE:
600 p2cread, p2cwrite = CreatePipe(None, 0)
601 # Detach and turn into fd
602 p2cwrite = p2cwrite.Detach()
603 p2cwrite = msvcrt.open_osfhandle(p2cwrite, 0)
604 elif type(stdin) == types.IntType:
605 p2cread = msvcrt.get_osfhandle(stdin)
606 else:
607 # Assuming file-like object
608 p2cread = msvcrt.get_osfhandle(stdin.fileno())
609 p2cread = self._make_inheritable(p2cread)
610
611 if stdout == None:
612 c2pwrite = GetStdHandle(STD_OUTPUT_HANDLE)
613 elif stdout == PIPE:
614 c2pread, c2pwrite = CreatePipe(None, 0)
615 # Detach and turn into fd
616 c2pread = c2pread.Detach()
617 c2pread = msvcrt.open_osfhandle(c2pread, 0)
618 elif type(stdout) == types.IntType:
619 c2pwrite = msvcrt.get_osfhandle(stdout)
620 else:
621 # Assuming file-like object
622 c2pwrite = msvcrt.get_osfhandle(stdout.fileno())
623 c2pwrite = self._make_inheritable(c2pwrite)
624
625 if stderr == None:
626 errwrite = GetStdHandle(STD_ERROR_HANDLE)
627 elif stderr == PIPE:
628 errread, errwrite = CreatePipe(None, 0)
629 # Detach and turn into fd
630 errread = errread.Detach()
631 errread = msvcrt.open_osfhandle(errread, 0)
632 elif stderr == STDOUT:
633 errwrite = c2pwrite
634 elif type(stderr) == types.IntType:
635 errwrite = msvcrt.get_osfhandle(stderr)
636 else:
637 # Assuming file-like object
638 errwrite = msvcrt.get_osfhandle(stderr.fileno())
639 errwrite = self._make_inheritable(errwrite)
640
641 return (p2cread, p2cwrite,
642 c2pread, c2pwrite,
643 errread, errwrite)
644
645
646 def _make_inheritable(self, handle):
647 """Return a duplicate of handle, which is inheritable"""
648 return DuplicateHandle(GetCurrentProcess(), handle,
649 GetCurrentProcess(), 0, 1,
650 DUPLICATE_SAME_ACCESS)
651
652
653 def _find_w9xpopen(self):
654 """Find and return absolut path to w9xpopen.exe"""
655 w9xpopen = os.path.join(os.path.dirname(GetModuleFileName(0)),
656 "w9xpopen.exe")
657 if not os.path.exists(w9xpopen):
658 # Eeek - file-not-found - possibly an embedding
659 # situation - see if we can locate it in sys.exec_prefix
660 w9xpopen = os.path.join(os.path.dirname(sys.exec_prefix),
661 "w9xpopen.exe")
662 if not os.path.exists(w9xpopen):
663 raise RuntimeError("Cannot locate w9xpopen.exe, which is "
664 "needed for Popen to work with your "
665 "shell or platform.")
666 return w9xpopen
667
668
669 def _execute_child(self, args, executable, preexec_fn, close_fds,
670 cwd, env, universal_newlines,
671 startupinfo, creationflags, shell,
672 p2cread, p2cwrite,
673 c2pread, c2pwrite,
674 errread, errwrite):
675 """Execute program (MS Windows version)"""
676
677 if not isinstance(args, types.StringTypes):
678 args = list2cmdline(args)
679
680 # Process startup details
681 default_startupinfo = STARTUPINFO()
682 if startupinfo == None:
683 startupinfo = default_startupinfo
684 if not None in (p2cread, c2pwrite, errwrite):
685 startupinfo.dwFlags |= STARTF_USESTDHANDLES
686 startupinfo.hStdInput = p2cread
687 startupinfo.hStdOutput = c2pwrite
688 startupinfo.hStdError = errwrite
689
690 if shell:
691 default_startupinfo.dwFlags |= STARTF_USESHOWWINDOW
692 default_startupinfo.wShowWindow = SW_HIDE
693 comspec = os.environ.get("COMSPEC", "cmd.exe")
694 args = comspec + " /c " + args
695 if (GetVersion() >= 0x80000000L or
696 os.path.basename(comspec).lower() == "command.com"):
697 # Win9x, or using command.com on NT. We need to
698 # use the w9xpopen intermediate program. For more
699 # information, see KB Q150956
700 # (http://web.archive.org/web/20011105084002/http://support.microsoft.com/support/kb/articles/Q150/9/56.asp)
701 w9xpopen = self._find_w9xpopen()
702 args = '"%s" %s' % (w9xpopen, args)
703 # Not passing CREATE_NEW_CONSOLE has been known to
704 # cause random failures on win9x. Specifically a
705 # dialog: "Your program accessed mem currently in
706 # use at xxx" and a hopeful warning about the
707 # stability of your system. Cost is Ctrl+C wont
708 # kill children.
709 creationflags |= CREATE_NEW_CONSOLE
710
711 # Start the process
712 try:
713 hp, ht, pid, tid = CreateProcess(executable, args,
714 # no special security
715 None, None,
716 # must inherit handles to pass std
717 # handles
718 1,
719 creationflags,
720 env,
721 cwd,
722 startupinfo)
723 except pywintypes.error, e:
724 # Translate pywintypes.error to WindowsError, which is
725 # a subclass of OSError. FIXME: We should really
726 # translate errno using _sys_errlist (or simliar), but
727 # how can this be done from Python?
728 raise WindowsError(*e.args)
729
730 # Retain the process handle, but close the thread handle
731 self._handle = hp
732 self.pid = pid
733 ht.Close()
734
735 # Child is launched. Close the parent's copy of those pipe
736 # handles that only the child should have open. You need
737 # to make sure that no handles to the write end of the
738 # output pipe are maintained in this process or else the
739 # pipe will not close when the child process exits and the
740 # ReadFile will hang.
741 if p2cread != None:
742 p2cread.Close()
743 if c2pwrite != None:
744 c2pwrite.Close()
745 if errwrite != None:
746 errwrite.Close()
747
748
749 def poll(self):
750 """Check if child process has terminated. Returns returncode
751 attribute."""
752 if self.returncode == None:
753 if WaitForSingleObject(self._handle, 0) == WAIT_OBJECT_0:
754 self.returncode = GetExitCodeProcess(self._handle)
755 _active.remove(self)
756 return self.returncode
757
758
759 def wait(self):
760 """Wait for child process to terminate. Returns returncode
761 attribute."""
762 if self.returncode == None:
763 obj = WaitForSingleObject(self._handle, INFINITE)
764 self.returncode = GetExitCodeProcess(self._handle)
765 _active.remove(self)
766 return self.returncode
767
768
769 def _readerthread(self, fh, buffer):
770 buffer.append(fh.read())
771
772
773 def communicate(self, input=None):
774 """Interact with process: Send data to stdin. Read data from
775 stdout and stderr, until end-of-file is reached. Wait for
776 process to terminate. The optional input argument should be a
777 string to be sent to the child process, or None, if no data
778 should be sent to the child.
779
780 communicate() returns a tuple (stdout, stderr)."""
781 stdout = None # Return
782 stderr = None # Return
783
784 if self.stdout:
785 stdout = []
786 stdout_thread = threading.Thread(target=self._readerthread,
787 args=(self.stdout, stdout))
788 stdout_thread.setDaemon(True)
789 stdout_thread.start()
790 if self.stderr:
791 stderr = []
792 stderr_thread = threading.Thread(target=self._readerthread,
793 args=(self.stderr, stderr))
794 stderr_thread.setDaemon(True)
795 stderr_thread.start()
796
797 if self.stdin:
798 if input != None:
799 self.stdin.write(input)
800 self.stdin.close()
801
802 if self.stdout:
803 stdout_thread.join()
804 if self.stderr:
805 stderr_thread.join()
806
807 # All data exchanged. Translate lists into strings.
808 if stdout != None:
809 stdout = stdout[0]
810 if stderr != None:
811 stderr = stderr[0]
812
813 # Translate newlines, if requested. We cannot let the file
814 # object do the translation: It is based on stdio, which is
815 # impossible to combine with select (unless forcing no
816 # buffering).
817 if self.universal_newlines and hasattr(open, 'newlines'):
818 if stdout:
819 stdout = self._translate_newlines(stdout)
820 if stderr:
821 stderr = self._translate_newlines(stderr)
822
823 self.wait()
824 return (stdout, stderr)
825
826 else:
827 #
828 # POSIX methods
829 #
830 def _get_handles(self, stdin, stdout, stderr):
831 """Construct and return tupel with IO objects:
832 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
833 """
834 p2cread, p2cwrite = None, None
835 c2pread, c2pwrite = None, None
836 errread, errwrite = None, None
837
838 if stdin == None:
839 pass
840 elif stdin == PIPE:
841 p2cread, p2cwrite = os.pipe()
842 elif type(stdin) == types.IntType:
843 p2cread = stdin
844 else:
845 # Assuming file-like object
846 p2cread = stdin.fileno()
847
848 if stdout == None:
849 pass
850 elif stdout == PIPE:
851 c2pread, c2pwrite = os.pipe()
852 elif type(stdout) == types.IntType:
853 c2pwrite = stdout
854 else:
855 # Assuming file-like object
856 c2pwrite = stdout.fileno()
857
858 if stderr == None:
859 pass
860 elif stderr == PIPE:
861 errread, errwrite = os.pipe()
862 elif stderr == STDOUT:
863 errwrite = c2pwrite
864 elif type(stderr) == types.IntType:
865 errwrite = stderr
866 else:
867 # Assuming file-like object
868 errwrite = stderr.fileno()
869
870 return (p2cread, p2cwrite,
871 c2pread, c2pwrite,
872 errread, errwrite)
873
874
875 def _set_cloexec_flag(self, fd):
876 try:
877 cloexec_flag = fcntl.FD_CLOEXEC
878 except AttributeError:
879 cloexec_flag = 1
880
881 old = fcntl.fcntl(fd, fcntl.F_GETFD)
882 fcntl.fcntl(fd, fcntl.F_SETFD, old | cloexec_flag)
883
884
885 def _close_fds(self, but):
886 for i in range(3, MAXFD):
887 if i == but:
888 continue
889 try:
890 os.close(i)
891 except:
892 pass
893
894
895 def _execute_child(self, args, executable, preexec_fn, close_fds,
896 cwd, env, universal_newlines,
897 startupinfo, creationflags, shell,
898 p2cread, p2cwrite,
899 c2pread, c2pwrite,
900 errread, errwrite):
901 """Execute program (POSIX version)"""
902
903 if isinstance(args, types.StringTypes):
904 args = [args]
905
906 if shell:
907 args = ["/bin/sh", "-c"] + args
908
909 if executable == None:
910 executable = args[0]
911
912 # For transferring possible exec failure from child to parent
913 # The first char specifies the exception type: 0 means
914 # OSError, 1 means some other error.
915 errpipe_read, errpipe_write = os.pipe()
916 self._set_cloexec_flag(errpipe_write)
917
918 self.pid = os.fork()
919 if self.pid == 0:
920 # Child
921 try:
922 # Close parent's pipe ends
923 if p2cwrite:
924 os.close(p2cwrite)
925 if c2pread:
926 os.close(c2pread)
927 if errread:
928 os.close(errread)
929 os.close(errpipe_read)
930
931 # Dup fds for child
932 if p2cread:
933 os.dup2(p2cread, 0)
934 if c2pwrite:
935 os.dup2(c2pwrite, 1)
936 if errwrite:
937 os.dup2(errwrite, 2)
938
939 # Close pipe fds. Make sure we doesn't close the same
940 # fd more than once.
941 if p2cread:
942 os.close(p2cread)
943 if c2pwrite and c2pwrite not in (p2cread,):
944 os.close(c2pwrite)
945 if errwrite and errwrite not in (p2cread, c2pwrite):
946 os.close(errwrite)
947
948 # Close all other fds, if asked for
949 if close_fds:
950 self._close_fds(but=errpipe_write)
951
952 if cwd != None:
953 os.chdir(cwd)
954
955 if preexec_fn:
956 apply(preexec_fn)
957
958 if env == None:
959 os.execvp(executable, args)
960 else:
961 os.execvpe(executable, args, env)
962
963 except:
964 exc_type, exc_value, tb = sys.exc_info()
965 # Save the traceback and attach it to the exception object
966 exc_lines = traceback.format_exception(exc_type,
967 exc_value,
968 tb)
969 exc_value.child_traceback = ''.join(exc_lines)
970 os.write(errpipe_write, pickle.dumps(exc_value))
971
972 # This exitcode won't be reported to applications, so it
973 # really doesn't matter what we return.
974 os._exit(255)
975
976 # Parent
977 os.close(errpipe_write)
978 if p2cread and p2cwrite:
979 os.close(p2cread)
980 if c2pwrite and c2pread:
981 os.close(c2pwrite)
982 if errwrite and errread:
983 os.close(errwrite)
984
985 # Wait for exec to fail or succeed; possibly raising exception
986 data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB
987 os.close(errpipe_read)
988 if data != "":
989 os.waitpid(self.pid, 0)
990 child_exception = pickle.loads(data)
991 raise child_exception
992
993
994 def _handle_exitstatus(self, sts):
995 if os.WIFSIGNALED(sts):
996 self.returncode = -os.WTERMSIG(sts)
997 elif os.WIFEXITED(sts):
998 self.returncode = os.WEXITSTATUS(sts)
999 else:
1000 # Should never happen
1001 raise RuntimeError("Unknown child exit status!")
1002
1003 _active.remove(self)
1004
1005
1006 def poll(self):
1007 """Check if child process has terminated. Returns returncode
1008 attribute."""
1009 if self.returncode == None:
1010 try:
1011 pid, sts = os.waitpid(self.pid, os.WNOHANG)
1012 if pid == self.pid:
1013 self._handle_exitstatus(sts)
1014 except os.error:
1015 pass
1016 return self.returncode
1017
1018
1019 def wait(self):
1020 """Wait for child process to terminate. Returns returncode
1021 attribute."""
1022 if self.returncode == None:
1023 pid, sts = os.waitpid(self.pid, 0)
1024 self._handle_exitstatus(sts)
1025 return self.returncode
1026
1027
1028 def communicate(self, input=None):
1029 """Interact with process: Send data to stdin. Read data from
1030 stdout and stderr, until end-of-file is reached. Wait for
1031 process to terminate. The optional input argument should be a
1032 string to be sent to the child process, or None, if no data
1033 should be sent to the child.
1034
1035 communicate() returns a tuple (stdout, stderr)."""
1036 read_set = []
1037 write_set = []
1038 stdout = None # Return
1039 stderr = None # Return
1040
1041 if self.stdin:
1042 # Flush stdio buffer. This might block, if the user has
1043 # been writing to .stdin in an uncontrolled fashion.
1044 self.stdin.flush()
1045 if input:
1046 write_set.append(self.stdin)
1047 else:
1048 self.stdin.close()
1049 if self.stdout:
1050 read_set.append(self.stdout)
1051 stdout = []
1052 if self.stderr:
1053 read_set.append(self.stderr)
1054 stderr = []
1055
1056 while read_set or write_set:
1057 rlist, wlist, xlist = select.select(read_set, write_set, [])
1058
1059 if self.stdin in wlist:
1060 # When select has indicated that the file is writable,
1061 # we can write up to PIPE_BUF bytes without risk
1062 # blocking. POSIX defines PIPE_BUF >= 512
1063 bytes_written = os.write(self.stdin.fileno(), input[:512])
1064 input = input[bytes_written:]
1065 if not input:
1066 self.stdin.close()
1067 write_set.remove(self.stdin)
1068
1069 if self.stdout in rlist:
1070 data = os.read(self.stdout.fileno(), 1024)
1071 if data == "":
1072 self.stdout.close()
1073 read_set.remove(self.stdout)
1074 stdout.append(data)
1075
1076 if self.stderr in rlist:
1077 data = os.read(self.stderr.fileno(), 1024)
1078 if data == "":
1079 self.stderr.close()
1080 read_set.remove(self.stderr)
1081 stderr.append(data)
1082
1083 # All data exchanged. Translate lists into strings.
1084 if stdout != None:
1085 stdout = ''.join(stdout)
1086 if stderr != None:
1087 stderr = ''.join(stderr)
1088
1089 # Translate newlines, if requested. We cannot let the file
1090 # object do the translation: It is based on stdio, which is
1091 # impossible to combine with select (unless forcing no
1092 # buffering).
1093 if self.universal_newlines and hasattr(open, 'newlines'):
1094 if stdout:
1095 stdout = self._translate_newlines(stdout)
1096 if stderr:
1097 stderr = self._translate_newlines(stderr)
1098
1099 self.wait()
1100 return (stdout, stderr)
1101
1102
1103 def _demo_posix():
1104 #
1105 # Example 1: Simple redirection: Get process list
1106 #
1107 plist = Popen(["ps"], stdout=PIPE).communicate()[0]
1108 print "Process list:"
1109 print plist
1110
1111 #
1112 # Example 2: Change uid before executing child
1113 #
1114 if os.getuid() == 0:
1115 p = Popen(["id"], preexec_fn=lambda: os.setuid(100))
1116 p.wait()
1117
1118 #
1119 # Example 3: Connecting several subprocesses
1120 #
1121 print "Looking for 'hda'..."
1122 p1 = Popen(["dmesg"], stdout=PIPE)
1123 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
1124 print repr(p2.communicate()[0])
1125
1126 #
1127 # Example 4: Catch execution error
1128 #
1129 print
1130 print "Trying a weird file..."
1131 try:
1132 print Popen(["/this/path/does/not/exist"]).communicate()
1133 except OSError, e:
1134 if e.errno == errno.ENOENT:
1135 print "The file didn't exist. I thought so..."
1136 print "Child traceback:"
1137 print e.child_traceback
1138 else:
1139 print "Error", e.errno
1140 else:
1141 print >>sys.stderr, "Gosh. No error."
1142
1143
1144 def _demo_windows():
1145 #
1146 # Example 1: Connecting several subprocesses
1147 #
1148 print "Looking for 'PROMPT' in set output..."
1149 p1 = Popen("set", stdout=PIPE, shell=True)
1150 p2 = Popen('find "PROMPT"', stdin=p1.stdout, stdout=PIPE)
1151 print repr(p2.communicate()[0])
1152
1153 #
1154 # Example 2: Simple execution of program
1155 #
1156 print "Executing calc..."
1157 p = Popen("calc")
1158 p.wait()
1159
1160
1161 if __name__ == "__main__":
1162 if mswindows:
1163 _demo_windows()
1164 else:
1165 _demo_posix()