javaweb内部查询1w~10w的数据，除了用循环，还有啥好点的方法?

Question

项目需要在内部查询一个可能有1W~10w的数据然后进行操作，目前是用循环每次查100条来实现的。现在就想讨教一下，各位在实现这种逻辑的时候，你们的做法会是什么。

Answer 1

1. First, use the method of writing SQL directly, instead of using java objects to assemble SQL like hibernate, or using Example to query like in MyBatis. It is recommended to write SQL directly.
2. Take a look at your current business logic to see if it involves transactional operations. If it involves transactional operations, it is recommended to process it in batches. You can control it at the controller layer and process the data in paging.

Answer 2

Is it just a simple query or do I need to modify it after querying it? For single query, I think you can play whatever you want, just control the memory. If you need to modify it, refer to the above.

Answer 3

If paging is possible, it is best to do paging. If there are still operations to be performed after the query, you can use the stream in jdk8.

Answer 4

It’s a bit similar to batch processing. It is best to find out the primary key at once and then process the data in batches with multiple threads