Concurrency In Python 简明教程

Processes Intercommunication

进程间通信是指进程之间的数据交换。对于并行应用程序的开发来说，必须在进程之间交换数据。下图显示了用于多个子进程之间同步的各种通信机制 −

Process intercommunication means the exchange of data between processes. It is necessary to exchange the data between processes for the development of parallel application. Following diagram shows the various communication mechanisms for synchronization between multiple sub processes −

intercommunication

Various Communication Mechanisms

在本节中，我们将学习各种通信机制。机制如下所述 −

In this section, we will learn about the various communication mechanisms. The mechanisms are described below −

Queues

队列可与多进程程序一起使用。 multiprocessing 模块的 Queue 类类似于 Queue.Queue 类。因此，可以使用相同的 API。 Multiprocessing .Queue 为我们提供了线程和进程安全的进程之间通信的 FIFO（先进先出）机制。

Queues can be used with multi-process programs. The Queue class of multiprocessing module is similar to the Queue.Queue class. Hence, the same API can be used. Multiprocessing.Queue provides us a thread and process safe FIFO (first-in first-out) mechanism of communication between processes.

Example

以下是一个摘自 Python 官方多进程文档的简单示例，用于理解多进程的 Queue 类概念。

Following is a simple example taken from python official docs on multiprocessing to understand the concept of Queue class of multiprocessing.

from multiprocessing import Process, Queue
import queue
import random
def f(q):
   q.put([42, None, 'hello'])
def main():
   q = Queue()
   p = Process(target = f, args = (q,))
   p.start()
   print (q.get())
if __name__ == '__main__':
   main()

Output

[42, None, 'hello']

Pipes

它是一种数据结构，用于在多进程程序中的进程之间进行通信。Pipe() 函数返回一对通过管道连接的连接对象，该管道默认情况下是双工的（双向的）。其工作方式如下 −

It is a data structure, which is used to communicate between processes in multi-process programs. The Pipe() function returns a pair of connection objects connected by a pipe which by default is duplex(two way). It works in the following manner −

  1. It returns a pair of connection objects that represent the two ends of pipe.

  2. Every object has two methods – send() and recv(), to communicate between processes.

Example

以下是一个从 Python 官方文档中关于多处理中获取的简单示例，以了解多处理的 Pipe() 函数的概念。

Following is a simple example taken from python official docs on multiprocessing to understand the concept of Pipe() function of multiprocessing.

from multiprocessing import Process, Pipe

def f(conn):
   conn.send([42, None, 'hello'])
   conn.close()

if __name__ == '__main__':
   parent_conn, child_conn = Pipe()
   p = Process(target = f, args = (child_conn,))
   p.start()
   print (parent_conn.recv())
   p.join()

Output

[42, None, 'hello']

Manager

Manager 是多处理模块的一个类，可提供一种方法来协调所有用户之间的共享信息。管理对象控制着服务器进程，该进程管理着共享对象并允许其他进程操纵它们。换句话说，管理器提供了一种创建可在不同进程之间共享的数据的方法。以下是管理器对象的各种属性——

Manager is a class of multiprocessing module that provides a way to coordinate shared information between all its users. A manager object controls a server process, which manages shared objects and allows other processes to manipulate them. In other words, managers provide a way to create data that can be shared between different processes. Following are the different properties of manager object −

  1. The main property of manager is to control a server process, which manages the shared objects.

  2. Another important property is to update all the shared objects when any process modifies it.

Example

以下是一个示例，该示例使用管理器对象在服务器进程中创建列表记录，然后在该列表中添加一个新记录。

Following is an example which uses the manager object for creating a list record in server process and then adding a new record in that list.

import multiprocessing

def print_records(records):
   for record in records:
      print("Name: {0}\nScore: {1}\n".format(record[0], record[1]))

def insert_record(record, records):
   records.append(record)
      print("A New record is added\n")

if __name__ == '__main__':
   with multiprocessing.Manager() as manager:

      records = manager.list([('Computers', 1), ('Histoty', 5), ('Hindi',9)])
      new_record = ('English', 3)

      p1 = multiprocessing.Process(target = insert_record, args = (new_record, records))
      p2 = multiprocessing.Process(target = print_records, args = (records,))
	  p1.start()
      p1.join()
      p2.start()
      p2.join()

Output

A New record is added

Name: Computers
Score: 1

Name: Histoty
Score: 5

Name: Hindi
Score: 9

Name: English
Score: 3

Concept of Namespaces in Manager

管理器类具有命名空间的概念，它是一种快速的方法，用于在多个进程之间共享多个属性。命名空间没有任何可以被调用的公共方法，但它们拥有可写属性。

Manager Class comes with the concept of namespaces, which is a quick way method for sharing several attributes across multiple processes. Namespaces do not feature any public method, which can be called, but they have writable attributes.

Example

以下 Python 脚本示例帮助我们利用命名空间在主进程和子进程之间共享数据——

The following Python script example helps us utilize namespaces for sharing data across main process and child process −

import multiprocessing

def Mng_NaSp(using_ns):

   using_ns.x +=5
   using_ns.y *= 10

if __name__ == '__main__':
   manager = multiprocessing.Manager()
   using_ns = manager.Namespace()
   using_ns.x = 1
   using_ns.y = 1

   print ('before', using_ns)
   p = multiprocessing.Process(target = Mng_NaSp, args = (using_ns,))
   p.start()
   p.join()
   print ('after', using_ns)

Output

before Namespace(x = 1, y = 1)
after Namespace(x = 6, y = 10)

Ctypes-Array and Value

多处理模块提供 Array 和 Value 对象，用于将数据存储在共享内存映射中。 Array 是从共享内存中分配的 ctypes 数组， Value 是从共享内存中分配的 ctypes 对象。

Multiprocessing module provides Array and Value objects for storing the data in a shared memory map. Array is a ctypes array allocated from shared memory and Value is a ctypes object allocated from shared memory.

首先从多处理中导入 Process、Value、Array。

To being with, import Process, Value, Array from multiprocessing.

Example

以下 Python 脚本是取自 Python 文档的一个示例，用于利用 Ctypes Array 和 Value 在进程之间共享一些数据。

Following Python script is an example taken from python docs to utilize Ctypes Array and Value for sharing some data between processes.

def f(n, a):
   n.value = 3.1415927
   for i in range(len(a)):
   a[i] = -a[i]

if __name__ == '__main__':
   num = Value('d', 0.0)
   arr = Array('i', range(10))

   p = Process(target = f, args = (num, arr))
   p.start()
   p.join()
   print (num.value)
   print (arr[:])

Output

3.1415927
[0, -1, -2, -3, -4, -5, -6, -7, -8, -9]

Communicating Sequential Processes (CSP)

CSP 用于说明带有并发模型的其他系统与其系统之间的交互。CSP 是通过消息传递编写并发或程序的框架，因此它有效地描述了并发性。

CSP is used to illustrate the interaction of systems with other systems featuring concurrent models. CSP is a framework for writing concurrent or program via message passing and hence it is effective for describing concurrency.

Python library – PyCSP

为了实现 CSP 中找到的核心原语，Python 有一个称为 PyCSP 的库。它保持实现非常简短且可读，以便可以非常容易地理解它。以下是 PyCSP 的基本进程网络——

For implementing core primitives found in CSP, Python has a library called PyCSP. It keeps the implementation very short and readable so that it can be understood very easily. Following is the basic process network of PyCSP −

pycsp

在上述 PyCSP 进程网络中，有两个进程 - Process1 和 Process 2。这些进程通过两个通道 - 通道 1 和通道 2 传递消息进行通信。

In the above PyCSP process network, there are two processes – Process1 and Process 2. These processes communicate by passing messages through two channels – channel 1 and channel 2.

Installing PyCSP

借助以下命令，我们可以安装 Python 库 PyCSP——

With the help of following command, we can install Python library PyCSP −

pip install PyCSP

Example

以下 Python 脚本是一个简单示例，用于并行运行两个进程。这是借助 PyCSP Python 许可证完成的——

Following Python script is a simple example for running two processes in parallel to each other. It is done with the help of the PyCSP python libabary −

from pycsp.parallel import *
import time
@process
def P1():
   time.sleep(1)
   print('P1 exiting')
@process
def P2():
   time.sleep(1)
   print('P2 exiting')
def main():
   Parallel(P1(), P2())
   print('Terminating')
if __name__ == '__main__':
   main()

在上述脚本中，创建了两个函数，即 P1 和 P2 ，然后用 @process 对它们进行装饰，以将它们转换为进程。

In the above script, two functions namely P1 and P2 have been created and then decorated with @process for converting them into processes.

Output

P2 exiting
P1 exiting
Terminating