How to Resolve AttributeError When Using Multiprocessing with Custom Functions?

Multiprocessing Issue: Resolving the AttributeError

When attempting to implement multiprocessing in code using the example provided in the official documentation, some users may encounter the following error:

AttributeError: can't get attribute 'f' on <module '__main__' (built-in)>

This error stems from a design feature specific to multiprocessing.Pool, where it may encounter difficulties working with functions defined within the main Python script.

Solution:

To resolve this issue, it is necessary to separate the function definition into a separate file and import the module containing the function.

1. Create a new file, e.g., defs.py, and define the function in that file:

<code class="python"># defs.py
def f(x):
    return x*x</code>

2. In the main script, import the module containing the function:

<code class="python"># run.py
from multiprocessing import Pool
import defs

if __name__ == '__main__':
    with Pool(5) as p:
        print(p.map(defs.f, [1, 2, 3]))</code>

Note:

If you had utilized a standard built-in function, such as print, instead of a user-defined function, the original example should have functioned properly. This indicates that the error arises from the design of multiprocessing.Pool in handling custom functions defined within the main script.

The above is the detailed content of How to Resolve AttributeError When Using Multiprocessing with Custom Functions?. For more information, please follow other related articles on the PHP Chinese website!