Python 简明教程

Python - Diagnosing and Fixing Memory Leaks

当程序错误管理内存分配时，会导致 Memory leaks 发生，这会减少可用内存并可能导致程序变慢或崩溃。

Memory leaks occur when a program incorrectly manages memory allocations which resulting in reduced available memory and potentially causing the program to slow down or crash.

在 Python 中， memory management 通常由 interpreter 处理，但 memory leaks 仍然可能发生，尤其是在长期运行的应用程序中。 Diagnosing and fixing memory leaks 在 Python 中涉及理解如何分配内存、识别问题区域和应用适当的解决方案。

In Python, memory management is generally handled by the interpreter but memory leaks can still happen especially in long-running applications. Diagnosing and fixing memory leaks in Python involves understanding how memory is allocated, identifying problematic areas and applying appropriate solutions.

Causes of Memory Leaks in Python

Memory leaks 中的内存泄漏可能由若干个因素引起，主要与对象如何被引用和管理有关。以下是 Python 中内存泄漏的一些常见原因−

Memory leaks in Python can arise from several causes, primarily revolving around how objects are referenced and managed. Here are some common causes of memory leaks in Python −

1. Unreleased References

当某个对象不再需要，但代码某个地方仍引用它时，它就不会被取消分配，从而导致内存泄漏。以下是一个示例−

When objects are no longer needed but still referenced somewhere in the code then they are not de-allocated which leads to memory leaks. Here is the example of it −

def create_list():
   my_list = [1] * (10**6)
   return my_list

my_list = create_list()
# If my_list is not cleared or reassigned, it continues to consume memory.
print(my_list)

Output

[1, 1, 1, 1,
............
............
1, 1, 1, 1]

2. Circular References

Python 中的循环引用未经妥善管理可能会导致内存泄漏，但 Python 的循环垃圾收集器可以自动处理许多情况。

Circular references in Python can lead to memory leaks if not managed properly but Python’s cyclic garbage collector can handle many cases automatically.

要了解如何检测和打破循环引用，我们可以使用 gc 和 weakref 模块等工具。这些工具对于复杂 Python 应用程序中的高效内存管理至关重要。以下是循环引用的示例−

For understanding how to detect and break circular references we can use the tools such as the gc and weakref modules. These tools are crucial for efficient memory management in complex Python applications. Following is the example of circular references −

class Node:
   def __init__(self, value):
      self.value = value
      self.next = None

a = Node(1)
b = Node(2)
a.next = b
b.next = a
# 'a' and 'b' reference each other, creating a circular reference.

3. Global Variables

全局作用域中声明的变量将保留整个程序的生命周期，若未经妥善管理，则可能会导致内存泄漏。以下是一个示例：−

Variables declared at the global scope persist for the lifetime of the program which potentially causing memory leaks if not managed properly. Below is the example of it −

large_data = [1] * (10**6)

def process_data():
   global large_data
   # Use large_data
   pass

# large_data remains in memory as long as the program runs.

4. Long-Lived Objects

如果应用程序中的对象长期存在，随着时间的推移，它们可能会导致内存问题。以下是一个示例−

Objects that persist for the lifetime of the application can cause memory issues if they accumulate over time. Here is the example −

cache = {}

def cache_data(key, value):
   cache[key] = value

# Cached data remains in memory until explicitly cleared.

5. Improper Use of Closures

捕捉和保留对大型对象的引用的闭包也有可能无意间导致内存泄漏。以下是一个示例：−

Closures that capture and retain references to large objects can inadvertently cause memory leaks. Below is the example of it −

def create_closure():
   large_object = [1] * (10**6)
   def closure():
      return large_object
   return closure

my_closure = create_closure()
# The large_object is retained by the closure, causing a memory leak.

Tools for Diagnosing Memory Leaks

Diagnosing memory leaks in Python 可能有挑战性，但有许多工具和技术可以帮助识别和解决这些问题。以下是诊断 Python 中内存泄漏的一些最有效的工具和方法：−

Diagnosing memory leaks in Python can be challenging but there are several tools and techniques available to help identify and resolve these issues. Here are some of the most effective tools and methods for diagnosing memory leaks in Python −

1. Using the "gc" Module

gc module 可以帮助识别垃圾收集器未收集的对象。以下是如何使用 gc 模块诊断内存泄漏的示例：−

The gc module can help in identifying objects that are not being collected by the garbage collector. Following is the example of diagnosing the memory leaks using the gc module −

import gc

# Enable automatic garbage collection
gc.enable()

# Collect garbage and return unreachable objects
unreachable_objects = gc.collect()
print(f"Unreachable objects: {unreachable_objects}")

# Get a list of all objects tracked by the garbage collector
all_objects = gc.get_objects()
print(f"Number of tracked objects: {len(all_objects)}")

Output

Unreachable objects: 51
Number of tracked objects: 6117

2. Using "tracemalloc"

tracemalloc 模块用于跟踪 Python 中的内存分配。它有助于跟踪内存使用情况并识别内存分配的位置。以下是如何使用 tracemalloc 模块诊断内存泄漏的示例：−

The tracemalloc module is used to trace memory allocations in Python. It is helpful for tracking memory usage and identifying where memory is being allocated. Following is the example of diagnosing the memory leaks using the tracemalloc module −

import tracemalloc

# Start tracing memory allocations
tracemalloc.start()

# our code here
a = 10
b = 20
c = a+b
# Take a snapshot of current memory usage
snapshot = tracemalloc.take_snapshot()

# Display the top 10 memory-consuming lines
top_stats = snapshot.statistics('lineno')
for stat in top_stats[:10]:
   print(stat)

Output

C:\Users\Niharikaa\Desktop\sample.py:7: size=400 B, count=1, average=400 B

3. Using "memory_profiler"

memory_profiler 是一个用于监视 Python 程序中内存使用情况的模块。它提供了一个用于分析函数并用于逐行内存使用情况分析的命令行工具。在下面的示例中，我们将使用 memory_profiler 模块诊断内存泄漏−

The memory_profiler is a module for monitoring memory usage of a Python program. It provides a decorator to profile functions and a command-line tool for line-by-line memory usage analysis. In the below example we are diagnosing the memory leaks using the memory_profiler module −

from memory_profiler import profile

@profile
def my_function():
   # our code here
   a = 10
   b = 20
   c = a+b

if __name__ == "__main__":
    my_function()

Output

Line #      Mem   usage    Increment  Occurrences   Line
======================================================================
     3     49.1   MiB      49.1 MiB         1       @profile
     4                                              def my_function():
     5                                              # Your code here
     6     49.1   MiB      0.0 MiB          1       a = 10
     7     49.1   MiB      0.0 MiB          1       b = 20
     8     49.1   MiB      0.0 MiB          1       c = a+b

Fixing Memory Leaks

一旦识别出内存泄漏，我们就可以修复它，其中涉及找到并消除对对象的无用引用。

Once a memory leak is identified we can fix the memory leaks,, which involves locating and eliminating unnecessary references to objects.

Eliminate Global Variables: Avoid using global variables unless and untill absolutely necessary. Instead we can use local variables or pass objects as arguments to functions.
Break Circular References: Use weak references to break cycles where possible. The weakref module allows us to create weak references that do not prevent garbage collection.
Manual Cleanup: Explicitly delete objects or remove references when they are no longer needed.
Use Context Managers: Ensure resources that are properly cleaned up using context managers i.e. with statement.
Optimize Data Structures Use appropriate data structures that do not unnecessarily hold onto references.

最后，我们可以得出结论， Diagnosing and fixing memory leaks 中涉及使用 gc、memory_profiler 和 tracemalloc 等工具来跟踪内存使用情况并实施修复（例如删除无用引用并打破循环引用）来识别存在的引用。

Finally we can conclude Diagnosing and fixing memory leaks in Python involves identifying lingering references by using tools like gc, memory_profiler and tracemalloc etc to track memory usage and implementing fixes such as removing unnecessary references and breaking circular references.

通过遵循这些步骤，我们能够确保 Python 程序有效地使用内存并避免内存泄漏。

By following these steps, we can ensure our Python programs use memory efficiently and avoid memory leaks.