Summary of the use of Java thread pool

In a recent project, I encountered some situations where I needed to frequently use SQL queries, write files, and split subtasks. To improve program throughput, I use a thread pool.用户 At the beginning, considering the number of users according to the quantity, because in the database query, Oracle has a limit on the amount of 1,000 queries on WHERE in (). task is handed over to the fix thread pool for processing.

But each task has two larger subtasks: querying coupons and querying the number of comments to be made. These two subtasks require a lot of SQL queries, so we considered encapsulating these two subtasks into tasks and handing them over to threads. Pool treatment. But because the execution of the main task depends on the result of the subtask, if the same thread pool is used, after the subtask is submitted, the task to be executed in the work queue is still the main task, and the CPU has been waiting for Future.get on the main task submitted first. () returns, the subtask cannot get CPU resources to execute, and has been waiting in the task queue, which will cause the program to wait for death. Therefore, the thread pool was changed from fix to cache, hoping that the submitted tasks would get CPU time and would not wait in the queue all the time. Later this also caused problems, which will be explained in the next article.

When the user's bill data is calculated, the template center needs to be called to generate the email, and the generated email needs to be saved to the disk. These are two different IO-intensive tasks, one is waiting on the socket, and the other is Wait on local io. Considering that if you continue to submit to the cache thread pool, the thread pool will generate a thread for each task, which will consume a lot of resources, and the CPU will frequently switch threads, reducing the throughput of the program. Therefore, create a new fix thread pool and submit these subsequent tasks to the fix thread pool for processing. The thread pool uses a fixed number of jobs to run, which allows the program to continue running under different blocking conditions and avoids the creation of a large number of thread resources. waste.

In fact, five thread pools were used in the final program. The reasons for using grading are as follows:

In order to improve the execution efficiency of the program, users can be classified according to language first, and then divided into slices according to 900 users. Each slice has tasks , it also involves two subtasks that frequently query the database. After each piece is processed, network IO is required to produce emails and disk IO is required to save emails. If the same thread pool is used, after the subtasks produced by the superior task are submitted to the fix thread pool, because the work queue is still occupied by the superior task, different types of IO operations will wait in the task queue, which cannot improve the system throughput. After submitting lower-level tasks to other thread pools, the lower-level tasks will have the opportunity to execute immediately and perform network IO and disk reading and writing, which improves operating efficiency.