java.lang.OutOfMemoryError: unable to create new native thread

Question

程序总是在两三天就报一次内存泄漏的异常，不知道是什么原因引起的，因为项目中有一个地方冲到队列，在队列中创建了很多的线程不知道是不是这个引起了。。。

public class GlobalVariables {
    public static ExecutorService pool = Executors.newFixedThreadPool(1000);
    public static BlockingQueue<Object> queue = new ArrayBlockingQueue<Object>(1000);
    public static Map<Integer, Integer> map = new HashMap<Integer, Integer>();
    
}

public class AddQueues implements Runnable{
    private Object obj;
    
    public AddQueues(Object obj) {
        this.obj = obj;
    }
    @Override
    public void run() {
        try {
            System.out.println("-------------"+obj);
            GlobalVariables.queue.put(obj);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}

@Override
public Map<String, Object> updateQueuePool(Map<String, Object> map) throws Exception {
        EntranceVisitCountDTO dto = new EntranceVisitCountDTO();
        dto.setAdminUserId(Integer.parseInt(map.get("adminUserId").toString()));
        GlobalVariables.pool.submit(new AddQueues(dto));

        Map<String,Object> param = new HashMap<String,Object>();
        param.put("result", "yes");
        
        return param;
    }

/**
* 定时将队列里的数据放到map中
*/
@Scheduled(cron = "0/3 * * * * ?")
public synchronized void updateVisitCountOfMap(){
    try{
        if(!GlobalVariables.queue.isEmpty()){        
            for(int i = 0; i < 100; i++){
                Object obj = GlobalVariables.queue.take();
                if(obj instanceof EntranceVisitCountDTO){
                    EntranceVisitCountDTO dto = (EntranceVisitCountDTO)obj;
                    Integer visitCount = GlobalVariables.map.get(dto.getAdminUserId());
                    if(visitCount != null && GlobalVariables.map.size() <= 1000){
                        GlobalVariables.map.put(dto.getAdminUserId(), visitCount +1);
                    }else if(GlobalVariables.map.size() > 1000){
                        visitDayCountService.updateEntranceShowCount();
                    }else{
                        GlobalVariables.map.put(dto.getAdminUserId(), 1);
                    }
                }
                if(GlobalVariables.queue.isEmpty()){
                    break;
                }
            }
        }
    } catch (Exception e) {
        e.printStackTrace();
    }
}

/**
     * 定时将map中的缓存数据同步入口
     */
@Scheduled(cron = "0 0/10 * * * ?")
public synchronized void updateEntranceShowCount(){
    try{
        visitDayCountService.updateEntranceShowCount();
    } catch (Exception e) {
        e.printStackTrace();
    }
}

@Transactional(rollbackFor = Exception.class)
@Override
public synchronized void updateEntranceShowCount() throws Exception {
        
    if(!GlobalVariables.map.isEmpty()){
        for(Map.Entry<Integer, Integer> entry : GlobalVariables.map.entrySet()){
                
            String day = TimeUtils.getSpecifyDate(0, "yyyy-MM-dd");
            VisitDayCount visitDayCount = (VisitDayCount)visitDayCountDao.getEntranceCountByEntranceInfoIdWithCreateDate(entry.getKey(), day, "VisitDayCount");
            if(null == visitDayCount){
                visitDayCount = new VisitDayCount();
                visitDayCount.setNum(1);
                visitDayCount.setAdminUserId(entry.getKey());
                visitDayCount.setVisitTime(day);
                visitDayCountDao.save(visitDayCount);
            }else{
                visitDayCountDao.updateEntranceVistCount(entry.getKey(), entry.getValue());
            }
                
            GlobalVariables.map.remove(entry.getKey());
        }
    }
}

Answer 1

1. The entry point of this code should be the updateQueuePool method. This method will generate DTO, and then create a new job. Let’s put the DTO into the queue, and the consumer of this queue is updateVisitCountOfMap. This method Consume 100 DTOs every 3 seconds. If the frequency of calling the updateQueuePooll method is greater than the frequency of consumption, OOM will happen sooner or later

2. GlobalVariables.map has a similar problem. It puts up to 100 objects every 3 seconds, but consumes it every 10 minutes.

3. Several fields of GlobalVariables are public, which means that any code can be used directly. This is also a factor that may cause problems.

4. If there is a memory leak, calling System.gc will not help.

This code has a lot of problems, but it is not a complete code, so it is impossible to determine where there will be a memory leak. You'd better let the code run for a while, then use jmap dump to download the jvm, and then use virtual vm to check Use this dump file to see whether it is the objects referenced by the queue or the objects in the map that are not recycled.

Answer 2

'unable to create new native thread'
This kind of OOM is rare, it must be caused by the constant new Thread() in the code.
Change it to a thread pool.

Answer 3

The thread pool size you set is 1000. There is no need to build so many. You can first check the number of cores and threads of the host. It is recommended to lower it. In addition, the size of the xxs allocation in the jvm affects how many threads you can open in the jvm. Checking the size of xmx.