python - 硬盘空间不足，如何释放已删除的mongodb集合

Question

菜鸟摸石头过河，使用mongodb有几个月，数据一直在不断累加，现在mongodb已经200G，我试图删除一些数据，但是mongodb并没有释放那些空间。
查资料显示需要使用 db.repairDatabase()，但是出现了一个错误：

"Cannot repair database db_wallpaper having size: 212447789056 (bytes) because free disk space is: 157012410368 (bytes)"。

硬盘空间不足以空出200G... 该怎么办...

mongo: 2.6.3

Answer 1

Slower method: You can try to add a replica-set to this database. After the newly added node is synchronized with the old data, you can stop the old database and delete the old data, and use the new node to provide external services. At this point the database space should have been organized and compressed to a minimum. This process is not controllable by itself. It is impossible to predict how long it will take to synchronize 200 GB, but it will not greatly affect mongodb's continued provision of services.

A faster method: use mongodump/mongoexport to export all data, which can be exported remotely, and then use mongorestore/mongoimport to restore the data. This operation will cause mongodb to lock the database and be unable to provide external services. If it is locked, it doesn't matter. You can use this method. It may take half a day for 200 GB, so you have to estimate it yourself.

In addition, db.repairDatabase() has a very limited effect and will cause the database to be locked for a long time, so it is better not to use it. A better way for mongodb to solve disk problems is to use auto-sharding to spread the data across multiple machines. However, this must be planned in advance. Once the database is very large, there is very little that can be done.

Answer 2

Official documentation states:

repairDatabase requires free disk space equal to the size of your current data set plus 2 gigabytes. If the volume that holds dbpath lacks sufficient space, you can mount a separate volume and use that for the repair. When mounting a separate volume for repairDatabase you must run repairDatabase from the command line and use the --repairpath switch to specify the folder in which to store temporary repair files.

If the current disk partition space is insufficient, you can try to use the --repairpath parameter to specify a partition path with sufficient space

Answer 3

When it reaches 100G, you should clean up
At least, when you know that the amount of data is large, you should prepare relevant knowledge

Answer 4

Personal test solution:
Before exporting, delete unnecessary data tables and content in mongo, and then perform the following operations.
Use Linux shell command:

mongodump -o /xxx/xxx

I exported all the databases. The original database took up more than 200G of space. After exporting, it took up 48G. It took about 3 days
Enter the mongodb database file directory, which is /var/lib/mongodb/ by default. You can check your directory address in /etc/mongod.conf, delete the entire folder, and then create a new folder. Be careful to change the folder permissions to the mongodb user. (chown command)

Using mongostore again, the space occupied after recovery is about 90G and it took 16 hours

mongorestore -d database_name /xxx/xxx 

The above operations are all completed on the remote server.
Also, thank you very much @huandu for your suggestion!!