java - poi导入大量数据如何处理？

Question

我用的是poi框架去解析excel数据，大概有十万条。
我想先通过一行一行地读取，然后保存在一个集合里面list。
解析完成了，这个list就包含了十万个对象的信息了。
然后，再把这个list导入 数据库。

我的困惑是：十万条，一条有二十个字段，一下子保存在集合中，会不会内存不够，或者其他事故呢？如何存在，如何改变上面的思路，减少事故发生。

比如，我可不可以边解析边导入，导入后就删除list的呢？这个过程怎么实现？给个思路。

Answer 1

You can try the method of reading and writing at the same time, which is to read a row from POI, then write a row to the database and submit it. When the data is read from POI, the database writing is also completed. The flaw of this method is that if the data writing fails and the program is aborted, you need to restart reading and writing from the aborted row, so you may need to record the number of aborted rows so that you can continue the last progress if it fails. Start over.

Answer 2

@一书生VOID’s method can also be improved to read hundreds or thousands of lines at a time (the number depends on how much memory you have), and then write it, which will occupy less memory and reduce the number of database operations