Detailed introduction to nodejs garbage collection

This article brings you a detailed introduction to nodejs garbage collection. It has certain reference value. Friends in need can refer to it. I hope it will be helpful to you.

The garbage collection mechanism of nodejs is automatically managed by the v8 engine.

Memory limit of nodejs

In general back-end languages ​​​​(php), there is no limit on the use of memory, but for nodejs It says that only part of the system can be used - 1.4G for 64-bit systems and 0.7G for 32-bit systems. At this time, if you want to process a 3G file for data analysis, even if the system memory is 8G, the nodejs process memory will still overflow.

The above problem is mainly caused by the fact that nodejs is based on v8, and nodejs manages memory through v8's own method. So why does v8 limit the size of heap memory? There are 2 reasons:

1. Superficial reason: v8 is designed for browsers and is unlikely to encounter large memory situations.

2. Deep reason: limitations of the v8 garbage collection mechanism. Taking 1.5G of heap memory as an example, v8 takes 50ms to do a small garbage collection and 1s to do a non-incremental garbage collection. Garbage collection will cause the js thread to pause. With such time spent, the application's performance and responsiveness will plummet.

The memory limit can be turned on:
--max-old-space-size (old generation)
--max-new-space-size (new generation)
v8 heap Memory size = old generation new generation

v8 garbage collection mechanism
v8 garbage collection is mainly based on generationalgarbage collection mechanism. The memory garbage collection is performed in different generations according to the survival time of the object, and different algorithms are performed on different generations of memory.

New generation--->Objects with shorter survival time
Old generation--->Objects with longer survival time or resident memory
As mentioned above, nodejs heap memory The size is the new generation memory space plus the old generation memory space.

New generation algorithm
The new generation mainly uses the scavenge algorithm for garbage collection.

This is a copying method to implement garbage collection, dividing the heap memory into two, and each space is called semispace. Among these two semispaces, only one is in use (called from space) and the other is free (called to space). When starting allocation, we first start from the from space. When garbage collection starts, we also start from the from space to check the surviving objects, copy the surviving objects to the to space, and the space occupied by the non-surviving objects will be released. After the copy is completed, the roles of the from space and the to space are reversed.
As you can see from the above process, the disadvantage of scavenge is that it only uses half of the heap memory, sacrificing space acquisition time.

The old generation passes the mark-sweep and mark-comopact algorithms.

mark-sweep Mark removal is divided into two stages: marking and clearing. mark-sweep first traverses all objects in the heap memory in the marking phase and marks the surviving objects; it clears unmarked objects in the clearing phase. It can be seen that scavenge only copies living objects, and mark-sweep only cleans up dead objects. Because the surviving objects in the new generation occupy a small part, and the dead objects in the old generation occupy a small part, this is the reason why these two algorithms are efficient.

Mark-compact mark organization, there will be a problem in mark-sweep. After recycling, the memory will be in a discontinuous state (memory fragmentation). Memory fragmentation will have an impact on subsequent memory allocation, because there will be a situation where a large memory needs to be allocated, but all memory fragments cannot complete the allocation. This will trigger garbage collection in advance, and this recycling is unnecessary. Mark-compact is evolved on the basis of Mark-sweep. Its main difference is that after the object is marked, all surviving objects will be moved to one end during the sorting process, and they will be cleared directly after the movement is completed.

Summary: During normal use, the memory limit of v8 is still sufficient, but the garbage collection and single thread of nodejs will still affect performance. If you want high performance, you need to keep garbage collection as small as possible. In actual development, old generation objects need to be used, such as when implementing a web service session (session), which is generally stored (array) in memory. When the amount of access is large, it will lead to a sharp increase in old generation objects, which may cause overflow. If we need to process large memory data, such as reading a 3G file, we will use the pipe() method of the readable stream, so that it will not be affected by v8 memory limitations and improve the robustness of the nodejs program.

The above is the detailed content of Detailed introduction to nodejs garbage collection. For more information, please follow other related articles on the PHP Chinese website!