我在MySQL incident_archive
中有一個包含數百萬筆記錄的大表,我想按created
列對行進行排序,並保留前X 行並刪除其餘行,最有效的方法是什麼。
到目前為止,我用 Python 提出了這個解決方案:
def do_delete_archive(rowsToKeep): if rowsToKeep > 0: db_name = find_environment_value('DB_NAME', False, "dbname") db_host = find_environment_value('DB_HOST', False, "host") db_user = find_environment_value('DB_USER', False, "username") db_pass = find_environment_value('DB_PASS', False, "password") db = MySQLdb.connect(host=db_host,user=db_user,passwd=db_pass,db=db_name) cursor = db.cursor() sql = f"""DELETE FROM `incident_archive` WHERE incident_id NOT IN ( SELECT incident_id FROM ( SELECT incident_id FROM `incident_archive` ORDER BY created DESC LIMIT {rowsToKeep}) foo) LIMIT 10000;""" try: rowcount = rowsToKeep+ 1 while rowcount > rowsToKeep: cursor.execute(sql) db.commit() rowcount = cursor.rowcount print(f"--- Affected Rows: {rowcount} ---") except: db.rollback()
我在這裡遇到的問題是,如果 rowsToKeep
的值大於或等於 10000
,則此方法將不起作用,此過程的更好方法是什麼?
**注意:rowsToKeep 值是動態的,這表示它可以更改。
P粉6271364502024-04-02 17:46:12
我想出了以下解決方案:
注意:閾值是包含我們希望在範例中保留 1000 個的最大記錄數的變數
sqlCreate = f"""CREATE TABLE new_incident_archive LIKE incident_archive;""" print(f"Running query is: {sqlCreate}") cursor.execute(sqlCreate) print(f"Done with: {sqlCreate}") sqlInsert = f"""INSERT INTO new_incident_archive SELECT * FROM `incident_archive` ORDER BY created DESC LIMIT {threshold}""" print(f"Running query is: {sqlInsert}") cursor.execute(sqlInsert) db.commit() print(f"Done with: {sqlInsert}") sqlDrop = f"""DROP TABLE incident_archive""" print(f"Running query is: {sqlDrop}") cursor.execute(sqlDrop) print(f"Done with: {sqlDrop}") sqlRename = f"""RENAME TABLE `new_incident_archive` TO `incident_archive`;""" print(f"Running query is: {sqlRename}") cursor.execute(sqlRename) print(f"Done with: {sqlRename}")