asyncio 系列二、asyncio 子进程

asyncio的子进程

官网连接:https://docs.python.org/zh-cn/3.7/library/asyncio-subprocess.html

 

官网例子：

 

import asyncio

async def run(cmd):
    proc = await asyncio.create_subprocess_shell(
        cmd,
        stdout=asyncio.subprocess.PIPE,
        stderr=asyncio.subprocess.PIPE)

    stdout, stderr = await proc.communicate()

    print(f'[{cmd!r} exited with {proc.returncode}]')
    if stdout:
        print(f'[stdout]\n{stdout.decode()}')
    if stderr:
        print(f'[stderr]\n{stderr.decode()}')

asyncio.run(run('ls /zzz'))

运行结果：

 

['ls /zzz' exited with 1]
[stderr]
ls: /zzz: No such file or directory

一、创建子进程

coroutine asyncio.create_subprocess_exec(program, *args, stdin=None, stdout=None, stderr=None, loop=None, limit=None, **kwds)

创建一个子进程。

The limit argument sets the buffer limit for StreamReader wrappers for Process.stdout and Process.stderr (if subprocess.PIPE is passed to stdout and stderr arguments).

Return a Process instance.

See the documentation of loop.subprocess_exec() for other parameters.

返回Process实例。

 

coroutine asyncio.create_subprocess_shell(cmd, stdin=None, stdout=None, stderr=None, loop=None, limit=None, **kwds)

Run the cmd shell command.

The limit argument sets the buffer limit for StreamReader wrappers for Process.stdout and Process.stderr (if subprocess.PIPE is passed to stdout and stderr arguments).

Return a Process instance.

See the documentation of loop.subprocess_shell() for other parameters.

 

重要

 

It is the application's responsibility to ensure that all whitespace and special characters are quoted appropriately to avoid shell injection vulnerabilities. The shlex.quote() function can be used to properly escape whitespace and special shell characters in strings that are going to be used to construct shell commands.

 

注解

 

The default asyncio event loop implementation on Windows does not support subprocesses. Subprocesses are available for Windows if a ProactorEventLoop is used. See Subprocess Support on Windows for details.

windows系统实现的是不同的模块。

 

参见

 

asyncio also has the following low-level APIs to work with subprocesses: loop.subprocess_exec(),loop.subprocess_shell(), loop.connect_read_pipe(), loop.connect_write_pipe(), as well as the Subprocess Transports and Subprocess Protocols.

 

属性：

asyncio.subprocess.PIPE

Can be passed to the stdin, stdout or stderr parameters.

If PIPE is passed to stdin argument, the Process.stdin attribute will point to a StreamWriter instance.

If PIPE is passed to stdout or stderr arguments, the Process.stdout and Process.stderr attributes will point to StreamReader instances.

如果这个属性传给stdin参数，Process.stdin属性将指向StreamWriter实例。

 

asyncio.subprocess.STDOUT

Special value that can be used as the stderr argument and indicates that standard error should be redirected into standard output.

可以作为stderr的参数，此时stderr将会定向到stdout。

 

asyncio.subprocess.DEVNULL

Special value that can be used as the stdin, stdout or stderr argument to process creation functions. It indicates that the special file os.devnull will be used for the corresponding subprocess stream.

可以被用来作为stdin，stdout，stderr，此时os.devnull将被调用给对应的流。

 

二、Subprocesses

create_subprocess_exec() and create_subprocess_shell() 将返回subprocesses的实例

 

coroutine wait()

Wait for the child process to terminate.

Set and return the returncode attribute.

等待自子进程结束，返回returncode。

注解

 

This method can deadlock when using stdout=PIPE or stderr=PIPE and the child process generates so much output that it blocks waiting for the OS pipe buffer to accept more data. Use thecommunicate() method when using pipes to avoid this condition.

 

在stdout=PIPE 或者 stderr=PIPE 并且子进程产生了大量的数据阻塞了os管道的缓存时将会产生死锁。使用communicate（）方法可以避免。

 

coroutine communicate(input=None)

Interact with process:

1. send data to stdin (if input is not None);
2. read data from stdout and stderr, until EOF is reached;
3. wait for process to terminate.

The optional input argument is the data (bytes object) that will be sent to the child process.

Return a tuple (stdout_data, stderr_data).

If either BrokenPipeError or ConnectionResetError exception is raised when writing input into stdin, the exception is ignored. This condition occurs when the process exits before all data are written into stdin.

If it is desired to send data to the process' stdin, the process needs to be created with stdin=PIPE. Similarly, to get anything other than None in the result tuple, the process has to be created with stdout=PIPE and/or stderr=PIPE arguments.

Note, that the data read is buffered in memory, so do not use this method if the data size is large or unlimited.

功能：发送数据给输入，从输出读，等待进程结束。

返回：（标准输出，标准错误输出）

BrokenPipeError 和 ConnectResetError 出现时说是因为进程退出。

缓存在内存里，如果数据量过大不建议用。

 

send_signal(signal)

Sends the signal signal to the child process.

 

terminate()

Stop the child process.

On POSIX systems this method sends signal.SIGTERM to the child process.

 

kill()

Kill the child.

On POSIX systems this method sends SIGKILL to the child process.

 

stdin

Standard input stream (StreamWriter) or None if the process was created with stdin=None.

 

stdout

Standard output stream (StreamReader) or None if the process was created with stdout=None.

 

stderr

Standard error stream (StreamReader) or None if the process was created with stderr=None.

 

pid

Process identification number (PID).

Note that for processes created by the create_subprocess_shell() function, this attribute is the PID of the spawned shell.

返回PID。

 

returncode

Return code of the process when it exits.

A None value indicates that the process has not terminated yet.

一个负值 -N 表示子进程被信号 N 中断 (仅 POSIX).

返回退出码，如果没有退出返回None。

 

三、Subprocess and Threads

Standard asyncio event loop supports running subprocesses from different threads, but there are limitations:

• An event loop must run in the main thread.
• The child watcher must be instantiated in the main thread before executing subprocesses from other threads. Call the get_child_watcher() function in the main thread to instantiate the child watcher.

asyncio事件循环可以用于不同的线程。但是有两个限制。

1，必须有一个事件循环跑在主线程。

2，子线程的监视器必须在子线程执行之前于主线程实例化。可以用get_child_watcher方法。

参见

The Concurrency and multithreading in asyncio section.

 

官网例子：

 

import asyncio
import sys

async def get_date():
    code = 'import datetime; print(datetime.datetime.now())'

    # Create the subprocess; redirect the standard output
    # into a pipe.
    proc = await asyncio.create_subprocess_exec(
        sys.executable, '-c', code,
        stdout=asyncio.subprocess.PIPE)

    # Read one line of output.
    data = await proc.stdout.readline()
    line = data.decode('ascii').rstrip()

    # Wait for the subprocess exit.
    await proc.wait()
    return line

if sys.platform == "win32":
    asyncio.set_event_loop_policy(
        asyncio.WindowsProactorEventLoopPolicy())

date = asyncio.run(get_date())
print(f"Current date: {date}")