Python 简明教程

Python - Thread Pools

线程池是一种机制，它可以自动高效地管理多个线程，从而允许并发执行任务。Python 不通过 threading 模块直接提供线程池。

A thread pool is a mechanism that automatically manages multiple threads efficiently, allowing tasks to be executed concurrently. Python does not provide thread pooling directly through the threading module.

相反，它通过 multiprocessing.dummy 模块和 * concurrent.futures* 模块提供基于线程的池化。这些模块为创建和管理线程池提供了便捷的接口，从而更轻松地执行并发任务执行。

Instead, it offers thread-based pooling through the multiprocessing.dummy module and the concurrent.futures module. These modules provide convenient interfaces for creating and managing thread pools, making it easier to perform concurrent task execution.

What is a Thread Pool?

线程池是由池管理的一组线程。池中的每个线程称为工作线程或工作者线程。这些线程可以被重用来执行多个任务，这减少了重复创建和销毁线程的负担。

A thread pool is a collection of threads that are managed by a pool. Each thread in the pool is called a worker or a worker thread. These threads can be reused to perform multiple tasks, which reduces the burden of creating and destroying threads repeatedly.

线程池控制线程的创建及其生命周期，使其更有效地处理大量的任务。

Thread pools control the creation of threads and their life cycle, making them more efficient for handling large numbers of tasks.

我们可以在 Python 中使用以下类实现线程池：

We can implement thread-pools in Python using the following classes −

Python ThreadPool Class
Python ThreadPoolExecutor Class

Using Python ThreadPool Class

multiprocessing.pool.ThreadPool 类在 multiprocessing 模块中提供了线程池接口。它管理一个工作线程池，可以向其中提交作业以进行并发执行。

The multiprocessing.pool.ThreadPool class provides a thread pool interface within the multiprocessing module. It manages a pool of worker threads to which jobs can be submitted for concurrent execution.

ThreadPool 对象通过处理任务在工作线程之间的创建和分配，简化了多个线程的管理。它与 Pool 类共享一个接口，该类最初设计用于进程，但已被调整为也可用于线程。

A ThreadPool object simplifies the management of multiple threads by handling the creation and distribution of tasks among the worker threads. It shares an interface with the Pool class, originally designed for processes, but has been adjusted to work with threads too.

ThreadPool 实例与 Pool 实例完全接口兼容，并且应该作为上下文管理器管理，或手动调用 close() 和 terminate()。

ThreadPool instances are fully interface-compatible with Pool instances and should be managed either as a context manager or by calling close() and terminate() manually.

Example

此示例演示了使用 Python thread pool 在数字列表上并行执行方和立方函数，其中每个函数都并发地应用到数字上，最多使用 3 个线程，每个线程在执行之间延迟 1 秒。

This example demonstrates the parallel execution of the square and cube functions on the list of numbers using the Python thread pool, where each function is applied to the numbers concurrently with up to 3 threads, each with a delay of 1 second between executions.

from multiprocessing.dummy import Pool as ThreadPool
import time

def square(number):
   sqr = number * number
   time.sleep(1)
   print("Number:{} Square:{}".format(number, sqr))

def cube(number):
   cub = number*number*number
   time.sleep(1)
   print("Number:{} Cube:{}".format(number, cub))

numbers = [1, 2, 3, 4, 5]
pool = ThreadPool(3)
pool.map(square, numbers)
pool.map(cube, numbers)

pool.close()

Output

执行上述代码时，您将获得以下输出 -

On executing the above code you will get the following output −

Number:2 Square:4
Number:1 Square:1
Number:3 Square:9
Number:4 Square:16
Number:5 Square:25
Number:1 Cube:1
Number:2 Cube:8
Number:3 Cube:27
Number:4 Cube:64
Number:5 Cube:125

Using Python ThreadPoolExecutor Class

Python 的 ThreadPoolExecutor 模块中的 * concurrent.futures* 类为使用线程异步执行函数提供了高级界面。concurrent.futures 模块包括 Future 类和两个 Executor 类 - ThreadPoolExecutor 和 ProcessPoolExecutor 。

The ThreadPoolExecutor class of the Python the concurrent.futures module provides a high-level interface for asynchronously executing functions using threads. The concurrent.futures module includes Future class and two Executor classes − ThreadPoolExecutor and ProcessPoolExecutor.

The Future Class

concurrent.futures.Future 类负责处理任何可调用的异步执行，比如函数。要获取 Future 对象，您应在任何 Executor 对象上调用 submit() 方法。不应通过其构造函数直接创建它。

The concurrent.futures.Future class is responsible for handling asynchronous execution of any callable such as a function. To obtain a Future object, you should call the submit() method on any Executor object. It should not be created directly by its constructor.

Future 类中的重要方法是：

Important methods in the Future class are −

result(timeout=None): This method returns the value returned by the call. If the call hasn’t yet completed, then this method will wait up to timeout seconds. If the call hasn’t completed in timeout seconds, then a TimeoutError will be raised. If timeout is not specified, there is no limit to the wait time.
cancel(): This method, attempt to cancel the call. If the call is currently being executed or finished running and cannot be cancelled then the method will return a boolean value False. Otherwise the call will be cancelled and the method returns True.
cancelled(): Returns True if the call was successfully cancelled.
running(): Returns True if the call is currently being executed and cannot be cancelled.
done(): Returns True if the call was successfully cancelled or finished running.

The ThreadPoolExecutor Class

此类代表一个指定数量的最大工作线程池，以异步执行调用。

This class represents a pool of specified number maximum worker threads to execute calls asynchronously.

concurrent.futures.ThreadPoolExecutor(max_threads)

Example

以下是一个使用 concurrent.futures.ThreadPoolExecutor 类在 Python 中管理和异步执行任务的示例。具体来说，它展示了如何向线程池提交多个任务以及如何检查其执行状态。

Here is an example that uses the concurrent.futures.ThreadPoolExecutor class to manage and execute tasks asynchronously in Python. Specifically, it shows how to submit multiple tasks to a thread pool and how to check their execution status.

from concurrent.futures import ThreadPoolExecutor
from time import sleep
def square(numbers):
   for val in numbers:
      ret = val*val
      sleep(1)
      print("Number:{} Square:{}".format(val, ret))
def cube(numbers):
   for val in numbers:
      ret = val*val*val
      sleep(1)
      print("Number:{} Cube:{}".format(val, ret))
if __name__ == '__main__':
   numbers = [1,2,3,4,5]
   executor = ThreadPoolExecutor(4)
   thread1 = executor.submit(square, (numbers))
   thread2 = executor.submit(cube, (numbers))
   print("Thread 1 executed ? :",thread1.done())
   print("Thread 2 executed ? :",thread2.done())
   sleep(2)
   print("Thread 1 executed ? :",thread1.done())
   print("Thread 2 executed ? :",thread2.done())

它将生成以下 output −

It will produce the following output −

Thread 1 executed ? : False
Thread 2 executed ? : False
Number:1 Square:1
Number:1 Cube:1
Thread 1 executed ? : False
Thread 2 executed ? : False
Number:2 Square:4
Number:2 Cube:8
Number:3 Square:9
Number:3 Cube:27
Number:4 Square:16
Number:4 Cube:64
Number:5 Square:25
Number:5 Cube:125