我经常看到客户测试SQL Server 的插入(INSERT)或者批量导入大量数据的性能表现。其中有测试大量INSERT的TSQL脚本如下: use myDB go create table t1 ( id int primary key not null identity ( 1 , 1 ), name varchar ( 200 ), memo char ( 500 )); go trun
我经常看到客户测试SQL Server 的插入(INSERT)或者批量导入大量数据的性能表现。其中有测试大量INSERT的TSQL脚本如下:
use myDB
go
createtable t1(id intprimarykeynotnullidentity (1,1),name varchar(200),memo char(500));
go
truncatetable t1
go
declare @i int
set @i=0
declare @beginTime datetime,@endTime datetime
set @beginTime=GETDATE()
while(@i200000)
begin
insertinto t1(name,memo)values('test'+str(RAND()*100000000),'memo'+str(RAND()*100000000))
set @i=@i+1
end
set @endTime=GETDATE()
select @endTime-@beginTime
上面这个脚本测试插入200000行数据需要的时间。这个脚本有问题么?
语法没有问题,但是写法却不够优化。如果你使用performance monitor
来观察,就会发现在数据插入期间log flushes/sec的次数非常高。在我的机器上达到5000。Log flushes发生一次,意味着SQL server 需要写入事务日志(transaction log)一次。每秒5000次的日志写动作导致磁盘大量的写操作。正是这个磁盘写操作影响了上面的batch的性能。我上面脚本在我的机器上使用了大概40秒左右的时间。
如何改进这个脚本的性能呢?如何减少log flushes/sec从而减少磁盘的写操作? 答案是使用Batch Size如下面脚本所示。
truncatetable t1
go
declare @i int
set @i=0
declare @beginTime datetime,@endTime datetime
set @beginTime=GETDATE()
declare @batchSize int
set @batchSize=1000
while(@i00000)
begin
if (@i%@batchSize=0)
begin
if (@@TRANCOUNT>0)COMMITTRAN
BEGINTRAN
end
insertinto t1(name,memo)values('test'+str(RAND()*100000000),'memo'+str(RAND()*100000000))
set @i=@i+1
end
if (@@TRANCOUNT>0)COMMITTRAN
set @endTime=GETDATE()
select @endTime-@beginTime
黄色高亮是我改进的地方。我在同样的机器上跑了一下,奇迹发生了,它只使用了7秒多一点。快了5倍多!如果在看下performance

MySQLhandlesconcurrencyusingamixofrow-levelandtable-levellocking,primarilythroughInnoDB'srow-levellocking.ComparedtootherRDBMS,MySQL'sapproachisefficientformanyusecasesbutmayfacechallengeswithdeadlocksandlacksadvancedfeatureslikePostgreSQL'sSerializa

MySQLhandlestransactionseffectivelyusingtheInnoDBengine,supportingACIDpropertiessimilartoPostgreSQLandOracle.1)MySQLusesREPEATABLEREADasthedefaultisolationlevel,whichcanbeadjustedtoREADCOMMITTEDforhigh-trafficscenarios.2)Itoptimizesperformancewithabu

MySQLisbetterforspeedandsimplicity,suitableforwebapplications;PostgreSQLexcelsincomplexdatascenarioswithrobustfeatures.MySQLisidealforquickprojectsandread-heavytasks,whilePostgreSQLispreferredforapplicationsrequiringstrictdataintegrityandadvancedSQLf

MySQL processes data replication through three modes: asynchronous, semi-synchronous and group replication. 1) Asynchronous replication performance is high but data may be lost. 2) Semi-synchronous replication improves data security but increases latency. 3) Group replication supports multi-master replication and failover, suitable for high availability requirements.

The EXPLAIN statement can be used to analyze and improve SQL query performance. 1. Execute the EXPLAIN statement to view the query plan. 2. Analyze the output results, pay attention to access type, index usage and JOIN order. 3. Create or adjust indexes based on the analysis results, optimize JOIN operations, and avoid full table scanning to improve query efficiency.

Using mysqldump for logical backup and MySQLEnterpriseBackup for hot backup are effective ways to back up MySQL databases. 1. Use mysqldump to back up the database: mysqldump-uroot-pmydatabase>mydatabase_backup.sql. 2. Use MySQLEnterpriseBackup for hot backup: mysqlbackup--user=root-password=password--backup-dir=/path/to/backupbackup. When recovering, use the corresponding life

The main reasons for slow MySQL query include missing or improper use of indexes, query complexity, excessive data volume and insufficient hardware resources. Optimization suggestions include: 1. Create appropriate indexes; 2. Optimize query statements; 3. Use table partitioning technology; 4. Appropriately upgrade hardware.

MySQL view is a virtual table based on SQL query results and does not store data. 1) Views simplify complex queries, 2) Enhance data security, and 3) Maintain data consistency. Views are stored queries in databases that can be used like tables, but data is generated dynamically.


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

MantisBT
Mantis is an easy-to-deploy web-based defect tracking tool designed to aid in product defect tracking. It requires PHP, MySQL and a web server. Check out our demo and hosting services.

EditPlus Chinese cracked version
Small size, syntax highlighting, does not support code prompt function

SublimeText3 Chinese version
Chinese version, very easy to use

ZendStudio 13.5.1 Mac
Powerful PHP integrated development environment

SecLists
SecLists is the ultimate security tester's companion. It is a collection of various types of lists that are frequently used during security assessments, all in one place. SecLists helps make security testing more efficient and productive by conveniently providing all the lists a security tester might need. List types include usernames, passwords, URLs, fuzzing payloads, sensitive data patterns, web shells, and more. The tester can simply pull this repository onto a new test machine and he will have access to every type of list he needs.
