Different behaviors of Synchronized variables in different operating systems

SEO:

• Python Synchronized variable value doesn't change in global context
• Synchronized value not modified in main process
• Synchronized value not shared between processes

In Python's multiprocessing library, there is multiprocessing.sharedctypes.synchronized, a wrapper type used to share data between different processes. By default, a reentrant lock RLock is used to maintain data consistency. Before relearning Python this time, I was using Python 3.6. Although I had used some superficial knowledge of the mp library at that time, I had not considered in depth the different behaviors of multi-process programs on different operating systems and their possible impact.

While studying this time, I encountered a problem: when I create a multiprocessing.sharedctypes.Value variable in the global context using Python 3.12 on the macOS system, if I access this variable in a new process, its value Synchronization is not maintained between different processes. After discussing with teacher Eric Greene, I discovered that since Python 3.8, the way to create new processes in different operating systems has changed:

• Linux systems always use fork to create new processes. During fork, all resources of the parent process are inherited by the child process, so the synchronization variables defined globally in the parent process can also be accessed by the child process, so data consistency is retained
• Windows systems always use spawn, which is a method of creating a new Python interpreter process to implement multi-processing. It is equivalent to "opening more" Python, so there is more overhead and lower efficiency, but this is a limitation of the Windows system itself. In this case, the child process only inherits the resources required by the run() method of the parent process to start the new process.
• The macOS system used fork before Python 3.8, and then changed to spawn, so some multi-process behaviors have changed. (macOS can still be set to use fork, but Python is not officially recommended)

In my example, because the Synchronized variable is declared globally in the parent process, it will not be inherited by the child process on Windows and macOS systems, so the values ​​​​are different between different processes; on Linux systems , all resources of the parent process are inherited, so the child process can change its value.

This caused the phenomenon I saw: the global synchronization variables of my code on macOS did not change; while other people's Linux systems ran normally and the variables were synchronized among all processes.

Reference reading:

• https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods
• https://docs.python.org/3/library/multiprocessing.html#multiprocessing.sharedctypes.synchronized

The above is the detailed content of Different behaviors of Synchronized variables in different operating systems. For more information, please follow other related articles on the PHP Chinese website!