What are memory leaks and memory overflows in java

Memory leak means that after the program applies for memory, it cannot release the allocated memory space. Memory overflow means that when the program applies for memory, there is not enough memory for the applicant to use; or it provides a storage space to store int data, but long data is stored, the result is that the memory is not enough, and an OOM error is reported. The accumulation of memory leaks will eventually lead to memory overflow.

The operating environment of this tutorial: windows7 system, java8 version, DELL G3 computer.

1. Memory leak:

means that after the program applies for memory, it cannot release the applied memory space. A memory leak does not seem to occur. It has a big impact, but the consequence of accumulation of memory leaks is memory overflow.

2. Memory overflow out of memory:

means that when the program applies for memory, there is not enough memory for the applicant to use, or in other words, it gives you a piece of storage int Type data storage space, but you store long type data, then the result is that the memory is not enough, and an OOM error will be reported, which is the so-called memory overflow.

3. The relationship between the two:

• The accumulation of memory leaks will eventually lead to memory overflow

• Memory overflow means that the memory space you want exceeds the space actually allocated to you by the system. At this time, the system is equivalent to being unable to meet your needs and will report a memory overflow error.

• Memory leak means that you apply to the system to allocate memory for use (new), but do not return it (delete) after using it. As a result, you also lose the memory you applied for. It can no longer be accessed (perhaps you lost its address), and the system cannot allocate it to the required program again. It is equivalent to renting a cabinet with a key. After you store your things and lock the cabinet, you lose the key or fail to return the key. The result is that the cabinet cannot be used by anyone and cannot be trashed. The recycler recycles because it cannot find any information about him.

• Memory overflow: A plate can only hold 4 fruits through various methods. You put 5 fruits, but it fell to the ground and could not be eaten. This is overflow. For example, if a stack is pushed when the stack is full, it will inevitably cause a space overflow, which is called an overflow. If the stack is empty, a space overflow will occur if the stack is empty, which is called an underflow. That is, the allocated memory is not enough to hold the sequence of data items, which is called a memory overflow. To put it bluntly, I can't bear that much, so I will report an error.

#4. Classification of memory leaks (classified by how they occur)

• Common memory leaks. Code with memory leaks will be executed multiple times, causing a memory leak every time it is executed.

• Occasional memory leaks. Code that causes memory leaks will only occur under certain circumstances or operations. Frequent and sporadic are relative. For certain circumstances, what is occasional may become common. So the testing environment and testing methods are crucial to detecting memory leaks.

• One-time memory leak. The code that causes a memory leak will only be executed once, or due to algorithmic flaws, there will always be only one and only one block of memory leaked. For example, if memory is allocated in the constructor of a class, but the memory is not released in the destructor, the memory leak will only occur once.

• Implicit memory leak. The program continuously allocates memory while it is running, but does not release the memory until the end. Strictly speaking, there is no memory leak here, because the program eventually releases all the requested memory. But for a server program that needs to run for days, weeks or even months, failure to release memory in time may also lead to the eventual exhaustion of all the system's memory. Therefore, we call this type of memory leak an implicit memory leak.

5. Causes and solutions of memory overflow:

(1) Cause of memory overflow:

• The amount of data loaded in the memory is too large, such as fetching too much data from the database at one time;

• There are references to objects in the collection class. It is not emptied afterwards, making it impossible for the JVM to recycle;

• There is an infinite loop in the code or the loop generates too many duplicate object entities;

• used BUG in third-party software;

• The startup parameter memory value is set too small

(2) Solution to memory overflow Solution:

The first step is to modify the JVM startup parameters and directly increase the memory. (Be sure to add the -Xms and -Xmx parameters.)

The second step is to check the error log to see if there are other exceptions or errors before the "OutOfMemory" error.

The third step is to walk through and analyze the code to find out where memory overflow may occur.

Focus on the following points:

• Check whether there is a query to obtain all the data in the database query. Generally speaking, if one hundred thousand records are fetched into the memory at one time, it may cause a memory overflow. This problem is relatively hidden. Before going online, there was less data in the database and it was less likely to cause problems. After going online, there was more data in the database, and a single query may cause memory overflow. Therefore, try to use paging for database queries.

• Check whether there are infinite loops or recursive calls in the code.

• Check whether there is a large loop that repeatedly generates new object entities.

• Check whether there is a query to obtain all data at once in the database query. Generally speaking, if one hundred thousand records are fetched into the memory at one time, it may cause a memory overflow. This problem is relatively hidden. Before going online, there was less data in the database and it was less likely to cause problems. After going online, there was more data in the database, and a single query may cause memory overflow. Therefore, try to use paging for database queries.

• Check whether collection objects such as List and MAP are not cleared after use. Collection objects such as List and MAP will always have references to the objects, making these objects unable to be recycled by GC.

The fourth step is to use the memory viewing tool to dynamically view memory usage

JVM8 memory model

##Ten scenarios of memory overflow

When the JVM is running, it first needs the class loader (classLoader) to load the bytecode file of the required class. After loading, it is handed over to the execution engine for execution. During the execution process, a period of space is needed to store data (analogous to CPU and main memory). The allocation and release process of this memory space is the runtime data area that we need to care about. Memory overflow occurs when the class loader is loading. Memory overflow is divided into two categories: OutOfMemoryError and StackOverflowError. The following lists 10 memory overflow situations and explains how memory overflow occurs through example code.

1.java heap memory overflow

When a java.lang.OutOfMemoryError:Java heap space exception occurs, the heap memory overflows.

1), problem description

• #The jvm memory set is too small, and the memory required for the object is too large. Space is allocated when creating the object. This exception will be thrown.

• Traffic/data peaks, there are certain limits for the application's own processing, such as a certain number of users or a certain amount of data. When the number of users or the amount of data suddenly surges and exceeds the expected threshold, then the operations that were running normally before the peak stop will stop and trigger java. lang.OutOfMemoryError: Java Heap Space Error

2), sample code

Compile the following code, and set the jvm parameter to -Xms20m -Xmx20m during execution

In the above example, if a request only If 5m of memory is allocated once, the request amount will be small and garbage collection will be normal and no errors will occur. However, once concurrency occurs, the maximum memory value will be exceeded and a memory overflow will be thrown.

3. Solution

First of all, if there is no problem with the code, you can appropriately adjust the two jvm parameters -Xms and -Xmx, and use the stress test to adjust these two parameters to achieve The optimal value.

Secondly, try to avoid applications for large objects, such as file uploads and obtaining large batches from the database. This needs to be avoided. Try to process it in chunks or batches, which will help the normal and stable execution of the system. .

Finally, try to increase the execution speed of a request. The earlier the garbage collection is, the better. Otherwise, when a large number of concurrency comes, new requests will not be able to allocate memory, which will easily cause an avalanche of the system.

2, java heap memory leak

1), problem description

Memory leaks in Java are some objects that are not It is used again by the application but cannot be recognized by garbage collection. Therefore, these unused objects still exist in the Java heap space indefinitely. The constant accumulation will eventually trigger java.lang.OutOfMemoryError.

2), Sample code

When executing the above code, you may expect it to run forever without any problems, assuming that the pure caching solution only maps the underlying Scales to 10,000 elements instead of all keys already in the HashMap. However, elements will actually continue to be added because the key class does not override its equals() method.

Over time, as the leaked code is constantly used, the "cached" results end up consuming a lot of Java heap space. When leaked memory fills all available memory in the heap area, garbage collection cannot clean it up, java.lang.OutOfMemoryError.

3), solution

The corresponding solution is relatively simple: just rewrite the equals method:

3. Garbage collection timeout memory overflow

1), Problem description: When the application exhausts all available memory, the GC overhead limit exceeds the error, and the GC fails to clear it multiple times, java.lang will be triggered. OutOfMemoryError. When the JVM spends a lot of time executing GC, with little effect, and once the entire GC process exceeds the limit, an error will be triggered (the default jvm configuration GC time exceeds 98%, and the recycled heap memory is less than 2%).

2), sample code

3), solution

To reduce the object life cycle, try to quickly perform garbage collection .

4. Metaspace memory overflow

1), problem description

If the metaspace overflows, the system will throw java .lang.OutOfMemoryError: Metaspace. The reason for this abnormal problem is that the system has a lot of code or refers to a lot of third-party packages, or dynamic code generation and class loading are used, which results in a large memory footprint in the metaspace.

2), Sample code

3), Solution

By default, the size of the metaspace is only limited by local memory. However, for the sake of the performance of the whole machine, this item should be set as much as possible to avoid causing the service shutdown of the whole machine.

• Optimize parameter configuration to avoid affecting other JVM processes

-XX:MetaspaceSize, initial space size, garbage collection will be triggered when this value is reached Perform type unloading, and the GC will adjust the value: if a large amount of space is released, the value will be appropriately lowered; if a small amount of space is released, the value will be appropriately increased if it does not exceed MaxMetaspaceSize.

-XX:MaxMetaspaceSize, the maximum space, there is no limit by default.

In addition to the above two size-specifying options, there are two GC-related attributes: -XX:MinMetaspaceFreeRatio. After GC, the percentage of the minimum Metaspace remaining space capacity is reduced to the allocated space. garbage collection. -XX:MaxMetaspaceFreeRatio, after GC, the percentage of the maximum Metaspace remaining space capacity is reduced to the garbage collection caused by freeing space.

• Refer to third-party packages carefully

For third-party packages, be sure to choose carefully and remove unnecessary packages. This will not only help improve the speed of compilation and packaging, but also help improve the speed of remote deployment.

• Focus on frameworks that dynamically generate classes

For frameworks that use a large number of dynamically generated classes, stress testing should be done to verify the dynamically generated classes If memory requirements are exceeded, an exception will be thrown.

5. Direct memory overflow

1) Problem description

When using allocateDirect() in ByteBuffer It will be used sometimes. Many javaNIO (like netty) frameworks are encapsulated as other methods. When this problem occurs, a java.lang.OutOfMemoryError: Direct buffer memory exception will be thrown.

Similar problems will occur if you directly or indirectly use the allocateDirect method in ByteBuffer without clearing it.

2), Sample code

3), Solution

If you often have similar operations, you can consider setting the parameters: -XX:MaxDirectMemorySize , and clear the memory in time.

6. Stack memory overflow

1), Problem description

When a thread executes a Java method, the JVM A new stack frame will be created and pushed to the top of the stack. At this time, the new stack frame becomes the current stack frame. When the method is executed, the stack frame is used to store parameters, local variables, intermediate instructions and other data.

When a method calls itself recursively, the data generated by the new method (which can also be understood as a new stack frame) will be pushed to the top of the stack. Each time the method calls itself, a copy will be made. The data of the current method is pushed to the stack. Therefore, each level of recursion requires the creation of a new stack frame. The result is that more and more memory in the stack will be consumed with recursive calls. If recursive calls themselves a million times, a million stack frames will be generated. This will cause stack memory overflow.

2), Sample code

3), Solution

If there are indeed recursive calls in the program and stack overflow occurs, you can increase the -Xss size can solve the problem of stack memory overflow. Recursive calls prevent the formation of an infinite loop, otherwise stack memory overflow will occur.

7. Memory overflow when creating a local thread

1) Problem description

The thread basically only occupies the memory area other than the heap, and also This error indicates that except for the heap, a memory area cannot be allocated for the thread. This is either because the memory itself is not enough, or the heap space is set too large, resulting in not much remaining memory, and due to the thread It takes up memory itself, so it is not enough.

2), sample code

3), solution

First check whether the operating system has a limit on the number of threads, use shell Threads cannot be created either. If this is the problem, you need to adjust the maximum number of files that the system can support.

In daily development, try to ensure that the maximum number of threads is controllable, and do not use the thread pool arbitrarily. It cannot grow without limit.

8. Memory overflow beyond the swap area

1) Problem description

During the startup process of the Java application, The required memory can be limited by -Xmx and other similar startup parameters. When the total memory requested by the JVM is greater than the available physical memory, the operating system begins to convert the content from memory to hard disk.

Generally speaking, the JVM will throw an Out of swap space error, which means that when the application fails to request the JVM native heap to allocate memory and the native heap is about to be exhausted, the error message contains the size of the allocation failure (in words section) and the reason for the request failure.

2), solution

Increase the size of the system swap area. I personally think that if the swap area is used, the performance will be greatly reduced. This method is not recommended and should be avoided in the production environment. The maximum memory exceeds the system's physical memory. Secondly, remove the system swap area and only use system memory to ensure application performance.

9. Array overflow memory overflow

1) Problem description Sometimes you will encounter this kind of memory overflow description Requested array size exceeds VM limit. Generally speaking, Java has a limit on the maximum size of an array that an application can allocate. However, the limit varies on different platforms, but it is usually between 1 and 2.1 billion elements. When the Requested array size exceeds VM limit error occurs, it means that the application is trying to allocate an array that is larger than the Java virtual machine can support. Before the JVM allocates memory for an array, it performs a platform-specific check: whether the allocated data structure is addressable on this platform.

2), sample code

The following is the code, that is, the array exceeds the maximum limit.

3), solution

Therefore, the array length must be within the length range allowed by the platform. However, this error is generally rare, mainly because the index of a Java array is of type int. The largest positive integer in Java is 2^31 - 1 = 2,147,483,647. And platform-specific limits can be very close to this number, for example: on my environment (64-bit macOS, running Jdk1.8) I can initialize arrays with lengths up to 2,147,483,645 (Integer.MAX_VALUE-2). If the length of the array is increased by 1 to reach nteger.MAX_VALUE-1, an OutOfMemoryError will occur.

10. System kills process memory overflow

1). Problem overview. Before describing the problem, first familiarize yourself with some knowledge of the operating system. : The operating system is built on the concept of processes. These processes work in the kernel. There is a very special process called "Out of memory killer". When the kernel detects that the system has insufficient memory, the OOM killer is activated, checks who currently occupies the most memory, and then kills the process.

Generally the Out of memory: Kill process or sacrifice child error will be triggered when the available virtual virtual memory (including swap space) is consumed to the point that the entire operating system is at risk. In this case, OOM Killer selects the "rogue process" and kills it.

2), Sample code

3), Solution

Although adding swap space can alleviate Java heap space exceptions, It is still recommended that the best solution is to upgrade the system memory so that the Java application has enough memory available, so this problem will not occur.

Summary: Through the above 10 types of memory overflow situations, everyone will know how to solve the problem when they actually encounter it. You should also remember the following in actual coding:

• Third-party jar packages should be introduced with caution, and useless jar packages should be removed resolutely to improve compilation speed and system memory usage.

• For large objects or a large number of memory applications, optimization must be carried out. Large objects must be processed in slices to improve processing performance and reduce the object life cycle.
  

• Try to fix the number of threads to ensure that the memory occupied by threads is controllable. When a large number of threads are needed, the maximum number of open connections of the operating system must be optimized.

• For recursive calls, the level of recursion must also be controlled, not too high and exceed the depth of the stack.

• The bigger the memory allocated to the stack is not, the better, because the larger the stack memory, the more threads there are, and there is not much space left for the heap, and it is easy to throw OOM. There is generally no problem with the JVM's default parameters (including recursion).

Recommended related video tutorials: Java video tutorial

The above is the detailed content of What are memory leaks and memory overflows in java. For more information, please follow other related articles on the PHP Chinese website!