search
HomeDatabaseMysql TutorialMySQL 数据库中删除重复记录方法总结

本文章总结了关于在mysql数据库中各种删除重复记录的sql语句,下面我们用实例介绍了操作方法,大家可参考一下。

MYSQL数据库中,经常会遇到重复记录的情况,那么就需要SQL删除重复记录,下面为您列举了四种删除重复记录的方式,用于不同的情况,希望对您有所帮助。

1、查找表中多余的重复记录,重复记录是根据单个字段(peopleId)来判断

 代码如下 复制代码
select * from people  where peopleId in (select   peopleId from   people group by   peopleId having count(peopleId) > 1)    

 


2、SQL删除重复记录,重复记录是根据单个字段(peopleId)来判断,只留有rowid最小的记录

 代码如下 复制代码
delete from people  where peopleId in (select   peopleId from people group by   peopleId   having count(peopleId) > 1)  and rowid not in (select min(rowid) from   people group by peopleId having count(peopleId )>1)    

  
  

3、查找表中多余的重复记录(多个字段)

 代码如下 复制代码
select * from vitae a  where (a.peopleId,a.seq) in   (select peopleId,seq from vitae group by peopleId,seq having count(*) > 1)   

 

 
4、删除表中多余的重复记录(多个字段),只留有rowid最小的记录

 代码如下 复制代码

delete from vitae a  where (a.peopleId,a.seq) in   (select peopleId,seq from vitae group by peopleId,seq having count(*) > 1)  and rowid not in (select min(rowid) from vitae group by peopleId,seq having count(*)>1)


演示数据

表结构:

 

 代码如下 复制代码

mysql> desc demo;

+-------+------------------+------+-----+---------+----------------+

| Field | Type | Null | Key | Default | Extra |

+-------+------------------+------+-----+---------+----------------+

| id | int(11) unsigned | NO | PRI | NULL | auto_increment |

| site | varchar(100) | NO | MUL | | |

+-------+------------------+------+-----+---------+----------------+

2 rows in set (0.00 sec)


数据:


mysql> select * from demo order by id;

+----+------------------------+

| id | site |

+----+------------------------+

| 1 | http://www.111cn.netn |

| 2 | http://安卓主题_www.hzhuti.com |

| 3 | http://www.zhutiy.com |

| 4 | http://www.111cn.netn |

| 5 | http://www.zhutiy.com |

+----+------------------------+

5 rows in set (0.00 sec)


当没有创建表或创建索引权限的时候,可以用下面的方法:

如果你要删除较旧的重复记录,可以使用下面的语句:

 代码如下 复制代码

mysql> delete from a 

-> using demo as a, demo as b

-> where (a.id > b.id)

-> and (a.site = b.site);

Query OK, 2 rows affected (0.12 sec)

 

mysql> select * from demo order by id;

+----+------------------------+

| id | site |

+----+------------------------+

| 1 | http://www.111cn.netn |

| 2 | http://安卓主题_www.hzhuti.com |

| 3 | http://www.zhutiy.com |

+----+------------------------+

3 rows in set (0.00 sec)


如果你要删除较新的重复记录,可以使用下面的语句:

 代码如下 复制代码

mysql> delete from a 

-> using demo as a, demo as b

-> where (a.id

-> and (a.site = b.site);

Query OK, 2 rows affected (0.12 sec)

 

mysql> select * from demo order by id;

+----+------------------------+

| id | site |

+----+------------------------+

| 2 | http://安卓主题_www.hzhuti.com |

| 4 | http://www.111cn.netn |

| 5 | http://www.zhutiy.com |

+----+------------------------+

3 rows in set (0.00 sec)


你可以用下面的语句先确认将被删除的重复记录:

 

 代码如下 复制代码

mysql> SELECT a.* 

-> FROM demo a, demo b

-> WHERE a.id > b.id

-> AND (a.site = b.site);

+----+------------------------+

| id | site |

+----+------------------------+

| 1 | http://www.111cn.netn |

| 3 | http://www.zhutiy.com |

+----+------------------------+

2 rows in set (0.00 sec)

如果有创建索引的权限,可以用下面的方法:

在表上创建唯一键索引:

 代码如下 复制代码

mysql> alter ignore table demo add unique index ukey (site);

Query OK, 5 rows affected (0.46 sec)

Records: 5 Duplicates: 2 Warnings: 0

 

mysql> select * from demo order by id;

+----+------------------------+

| id | site |

+----+------------------------+

| 1 | http://www.111cn.netn |

| 2 | http://安卓主题_www.hzhuti.com |

| 3 | http://www.zhutiy.com |

+----+------------------------+

3 rows in set (0.00 sec)


 

重复记录被删除后,如果需要,可以删除索引:

 代码如下 复制代码

mysql> alter table demo drop index ukey;

Query OK, 3 rows affected (0.37 sec)

Records: 3 Duplicates: 0 Warnings: 0


如果有创建表的权限,可以用下面的方法:

创建一个新表,然后将原表中不重复的数据插入新表:

 代码如下 复制代码

mysql> create table demo_new as select * from demo group by site;

Query OK, 3 rows affected (0.19 sec)

Records: 3 Duplicates: 0 Warnings: 0

 

mysql> show tables;

+----------------+

| Tables_in_test |

+----------------+

| demo |

| demo_new |

+----------------+

2 rows in set (0.00 sec)

 

mysql> select * from demo order by id;

+----+------------------------+

| id | site |

+----+------------------------+

| 1 | http://www.111cn.netn |

| 2 | http://安卓主题_www.hzhuti.com |

| 3 | http://www.zhutiy.com |

| 4 | http://www.111cn.netn |

| 5 | http://www.zhutiy.com |

+----+------------------------+

5 rows in set (0.00 sec)

 

mysql> select * from demo_new order by id;

+----+------------------------+

| id | site |

+----+------------------------+

| 1 | http://www.111cn.netn |

| 2 | http://安卓主题_www.hzhuti.com |

| 3 | http://www.zhutiy.com |

+----+------------------------+

3 rows in set (0.00 sec)


然后将原表备份,将新表重命名为当前表:

 

 代码如下 复制代码

mysql> rename table demo to demo_old, demo_new to demo;

Query OK, 0 rows affected (0.04 sec)

mysql> show tables;

+----------------+

| Tables_in_test |

+----------------+

| demo |

| demo_old |

+----------------+

2 rows in set (0.00 sec)

 

mysql> select * from demo order by id;

+----+------------------------+

| id | site |

+----+------------------------+

| 1 | http://www.111cn.netn |

| 2 | http://安卓主题_www.hzhuti.com |

| 3 | http://www.zhutiy.com |

+----+------------------------+

3 rows in set (0.00 sec)


注意:使用这种方式创建的表会丢失原表的索引信息!

 代码如下 复制代码

mysql> desc demo;

+-------+------------------+------+-----+---------+-------+

| Field | Type | Null | Key | Default | Extra |

+-------+------------------+------+-----+---------+-------+

| id | int(11) unsigned | NO | | 0 | |

| site | varchar(100) | NO | | | |

+-------+------------------+------+-----+---------+-------+

2 rows in set (0.00 sec)


如果要保持和原表信息一致,你可以使用 show create table demo; 来查看原表的创建语句,然后使用原表的创建语句创建新表,接着使用 insert … select 语句插入数据,再重命名表即可。


实例

今天无意导入几个测试数据,发现测试数据中,有很多数据记录是相同的,我现在就想删除掉这些数据,
如何查询数据相同记录呢?这个好说,以下语句就可以查看相同记录的了:

 代码如下 复制代码
SELECT COUNT(*) AS c, key_word FROM search_keywrod GROUP BY key_word HAVING c > 1 

其中 HAVING c >1 代表相同记录数就有相同的了。

查询相同的是比较容易实现的了,但是想要删除这些重复的,估计就比较麻烦的了,因为你是要删除自身表里的记录,有些朋友可能就会使用到临时表,把相同需要删除的记录,导到时这个临时表,然后再通过临时表来删除主表。或者写一个临时程序,删除掉其中的一条重复记录。

以上二个方法删除重复记录,最大的麻烦就是操作繁琐。而我们今天介绍的就是利用MYSQL自身的语句,不创建临时表,不写程序来删除掉自身的重复记录。请看以下SQL语句:

 代码如下 复制代码
SELECT t1.id, t1.key_word  
FROM search_keywrod t1, ( 
SELECT key_word, MIN(id) AS minid  
FROM search_keywrod  
GROUP BY key_word HAVING COUNT(key_word) > 1 
)t2  
WHERE t1.key_word = t2.key_word AND t1.id = t2.minid 

这条语句就跟我们第一条语句是一样的功能,但是这条语句好处就是MIN(id),可以控制是删除大的id重复记录(MAX),还是删除小的id重复记录(MIN)。

OK,经过改良的语句实现了查询,现在就可以利用DELETE FROM语句来删除了。

 代码如下 复制代码

DELETE FROM search_keywrod WHERE id IN (SELECT id FROM ( 
 
SELECT t1.id 
FROM search_keywrod t1, ( 
SELECT key_word, MIN(id) AS minid FROM search_keywrod  
GROUP BY key_word HAVING COUNT(key_word) > 1 
)t2  
WHERE t1.key_word = t2.key_word AND t1.id = t2.minid 
 
)t3) 

执行时,请多执行几次,因为每次删除时,只删除掉重复的一次记录,如果你一条记录重复五次,那你就要执行五次的了。所以多执行几次,直到没有可删除的记录了,这样你直接一条语句删除掉重复的mysql记录功能就实现了

Statement
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Explain the InnoDB Buffer Pool and its importance for performance.Explain the InnoDB Buffer Pool and its importance for performance.Apr 19, 2025 am 12:24 AM

InnoDBBufferPool reduces disk I/O by caching data and indexing pages, improving database performance. Its working principle includes: 1. Data reading: Read data from BufferPool; 2. Data writing: After modifying the data, write to BufferPool and refresh it to disk regularly; 3. Cache management: Use the LRU algorithm to manage cache pages; 4. Reading mechanism: Load adjacent data pages in advance. By sizing the BufferPool and using multiple instances, database performance can be optimized.

MySQL vs. Other Programming Languages: A ComparisonMySQL vs. Other Programming Languages: A ComparisonApr 19, 2025 am 12:22 AM

Compared with other programming languages, MySQL is mainly used to store and manage data, while other languages ​​such as Python, Java, and C are used for logical processing and application development. MySQL is known for its high performance, scalability and cross-platform support, suitable for data management needs, while other languages ​​have advantages in their respective fields such as data analytics, enterprise applications, and system programming.

Learning MySQL: A Step-by-Step Guide for New UsersLearning MySQL: A Step-by-Step Guide for New UsersApr 19, 2025 am 12:19 AM

MySQL is worth learning because it is a powerful open source database management system suitable for data storage, management and analysis. 1) MySQL is a relational database that uses SQL to operate data and is suitable for structured data management. 2) The SQL language is the key to interacting with MySQL and supports CRUD operations. 3) The working principle of MySQL includes client/server architecture, storage engine and query optimizer. 4) Basic usage includes creating databases and tables, and advanced usage involves joining tables using JOIN. 5) Common errors include syntax errors and permission issues, and debugging skills include checking syntax and using EXPLAIN commands. 6) Performance optimization involves the use of indexes, optimization of SQL statements and regular maintenance of databases.

MySQL: Essential Skills for Beginners to MasterMySQL: Essential Skills for Beginners to MasterApr 18, 2025 am 12:24 AM

MySQL is suitable for beginners to learn database skills. 1. Install MySQL server and client tools. 2. Understand basic SQL queries, such as SELECT. 3. Master data operations: create tables, insert, update, and delete data. 4. Learn advanced skills: subquery and window functions. 5. Debugging and optimization: Check syntax, use indexes, avoid SELECT*, and use LIMIT.

MySQL: Structured Data and Relational DatabasesMySQL: Structured Data and Relational DatabasesApr 18, 2025 am 12:22 AM

MySQL efficiently manages structured data through table structure and SQL query, and implements inter-table relationships through foreign keys. 1. Define the data format and type when creating a table. 2. Use foreign keys to establish relationships between tables. 3. Improve performance through indexing and query optimization. 4. Regularly backup and monitor databases to ensure data security and performance optimization.

MySQL: Key Features and Capabilities ExplainedMySQL: Key Features and Capabilities ExplainedApr 18, 2025 am 12:17 AM

MySQL is an open source relational database management system that is widely used in Web development. Its key features include: 1. Supports multiple storage engines, such as InnoDB and MyISAM, suitable for different scenarios; 2. Provides master-slave replication functions to facilitate load balancing and data backup; 3. Improve query efficiency through query optimization and index use.

The Purpose of SQL: Interacting with MySQL DatabasesThe Purpose of SQL: Interacting with MySQL DatabasesApr 18, 2025 am 12:12 AM

SQL is used to interact with MySQL database to realize data addition, deletion, modification, inspection and database design. 1) SQL performs data operations through SELECT, INSERT, UPDATE, DELETE statements; 2) Use CREATE, ALTER, DROP statements for database design and management; 3) Complex queries and data analysis are implemented through SQL to improve business decision-making efficiency.

MySQL for Beginners: Getting Started with Database ManagementMySQL for Beginners: Getting Started with Database ManagementApr 18, 2025 am 12:10 AM

The basic operations of MySQL include creating databases, tables, and using SQL to perform CRUD operations on data. 1. Create a database: CREATEDATABASEmy_first_db; 2. Create a table: CREATETABLEbooks(idINTAUTO_INCREMENTPRIMARYKEY, titleVARCHAR(100)NOTNULL, authorVARCHAR(100)NOTNULL, published_yearINT); 3. Insert data: INSERTINTObooks(title, author, published_year)VA

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

DVWA

DVWA

Damn Vulnerable Web App (DVWA) is a PHP/MySQL web application that is very vulnerable. Its main goals are to be an aid for security professionals to test their skills and tools in a legal environment, to help web developers better understand the process of securing web applications, and to help teachers/students teach/learn in a classroom environment Web application security. The goal of DVWA is to practice some of the most common web vulnerabilities through a simple and straightforward interface, with varying degrees of difficulty. Please note that this software

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

Safe Exam Browser

Safe Exam Browser

Safe Exam Browser is a secure browser environment for taking online exams securely. This software turns any computer into a secure workstation. It controls access to any utility and prevents students from using unauthorized resources.

SublimeText3 Linux new version

SublimeText3 Linux new version

SublimeText3 Linux latest version

mPDF

mPDF

mPDF is a PHP library that can generate PDF files from UTF-8 encoded HTML. The original author, Ian Back, wrote mPDF to output PDF files "on the fly" from his website and handle different languages. It is slower than original scripts like HTML2FPDF and produces larger files when using Unicode fonts, but supports CSS styles etc. and has a lot of enhancements. Supports almost all languages, including RTL (Arabic and Hebrew) and CJK (Chinese, Japanese and Korean). Supports nested block-level elements (such as P, DIV),