search
HomeBackend DevelopmentPHP Tutorialmysql - php导出十多万条数据有没有办法更快?

现在是用phpExcel导出大概需要十分钟多,而且内存报错...
有没有什么办法可以让大量数据导出可以更快呢?
...忘了说因为是一些数据客户需要用到的,所以要用excel格式

回复内容:

现在是用phpExcel导出大概需要十分钟多,而且内存报错...
有没有什么办法可以让大量数据导出可以更快呢?
...忘了说因为是一些数据客户需要用到的,所以要用excel格式

首先应该明确问题场景:
1.这个导出10w数据是用户的功能还是后台或者系统内的功能,如果是用户使用的功能,那么可能PHP很难处理,因为要考虑用户体验问题,那么这个功能可能需要使用第三方服务或者用其他扩展来做。但是如果不是用户使用,而是系统人员或者内部人使用。则办法有很多种。
分步骤分析问题
1.从mysql数据库导出10w条数据,2,将10万天数据导出。
首先看下,mysql数据库查询10w条的时间:
我刚刚测试了一下,我的11万数据大概导了205s。因为机器的问题所以每个人的都不一样。但是可以明确一点,这个时间不快。如果PHP的话,一般默认连接时间是30s,就会有第一个问题,超时的问题。这个很好解决
2.用PHP将10w条数据导出。10w条数据就会有第二个问题,内存不够的问题。处理10w条数据比较慢
建议:
1.如@悲惨的大爷的方案,将10条分次导出,由于是SELECT,所以不会占用MYSQL太多资源,可以多线程查询SQL。
但这只能解决查询的问题,查询倒是快了,这个可以优化第一个步骤的时间,但是用PHP导出10条数据的问题还是在,因为是需要把10w条数据导出成一个excel。针对这点可以使用csv.这会比phpexcel快不少,
2.如果数据量太多或者应用场景比较频繁可以做一个服务。

分多个文件导出.
比如现在要导出10w 条数据,那么可以分10次导出,一次1w 条.
可以用多进程来处理.

我猜你是查数据库一次取了十万条,然后导出的。这样非常占用内存。
你可以试试分页取十次,每次取一万条导出。最后把导出的文件合并。

用mysql自带的mysqldump不好么

如果一定要excel 可以odbc连接到数据库后导出到excel,具体操作搜索下

我导出500000一点问题都没的,记着释放内存

直接对数据库操作,mysql导出数据,写个别的桌面程序来转换?

CSV 没有一点压力

用csv格式啊,不要用phpexcel

用csv。千万不要用phpexcel。这种东西读取速度特别的慢。我是吃过亏的。

Statement
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Optimize PHP Code: Reducing Memory Usage & Execution TimeOptimize PHP Code: Reducing Memory Usage & Execution TimeMay 10, 2025 am 12:04 AM

TooptimizePHPcodeforreducedmemoryusageandexecutiontime,followthesesteps:1)Usereferencesinsteadofcopyinglargedatastructurestoreducememoryconsumption.2)LeveragePHP'sbuilt-infunctionslikearray_mapforfasterexecution.3)Implementcachingmechanisms,suchasAPC

PHP Email: Step-by-Step Sending GuidePHP Email: Step-by-Step Sending GuideMay 09, 2025 am 12:14 AM

PHPisusedforsendingemailsduetoitsintegrationwithservermailservicesandexternalSMTPproviders,automatingnotificationsandmarketingcampaigns.1)SetupyourPHPenvironmentwithawebserverandPHP,ensuringthemailfunctionisenabled.2)UseabasicscriptwithPHP'smailfunct

How to Send Email via PHP: Examples & CodeHow to Send Email via PHP: Examples & CodeMay 09, 2025 am 12:13 AM

The best way to send emails is to use the PHPMailer library. 1) Using the mail() function is simple but unreliable, which may cause emails to enter spam or cannot be delivered. 2) PHPMailer provides better control and reliability, and supports HTML mail, attachments and SMTP authentication. 3) Make sure SMTP settings are configured correctly and encryption (such as STARTTLS or SSL/TLS) is used to enhance security. 4) For large amounts of emails, consider using a mail queue system to optimize performance.

Advanced PHP Email: Custom Headers & FeaturesAdvanced PHP Email: Custom Headers & FeaturesMay 09, 2025 am 12:13 AM

CustomheadersandadvancedfeaturesinPHPemailenhancefunctionalityandreliability.1)Customheadersaddmetadatafortrackingandcategorization.2)HTMLemailsallowformattingandinteractivity.3)AttachmentscanbesentusinglibrarieslikePHPMailer.4)SMTPauthenticationimpr

Guide to Sending Emails with PHP & SMTPGuide to Sending Emails with PHP & SMTPMay 09, 2025 am 12:06 AM

Sending mail using PHP and SMTP can be achieved through the PHPMailer library. 1) Install and configure PHPMailer, 2) Set SMTP server details, 3) Define the email content, 4) Send emails and handle errors. Use this method to ensure the reliability and security of emails.

What is the best way to send an email using PHP?What is the best way to send an email using PHP?May 08, 2025 am 12:21 AM

ThebestapproachforsendingemailsinPHPisusingthePHPMailerlibraryduetoitsreliability,featurerichness,andeaseofuse.PHPMailersupportsSMTP,providesdetailederrorhandling,allowssendingHTMLandplaintextemails,supportsattachments,andenhancessecurity.Foroptimalu

Best Practices for Dependency Injection in PHPBest Practices for Dependency Injection in PHPMay 08, 2025 am 12:21 AM

The reason for using Dependency Injection (DI) is that it promotes loose coupling, testability, and maintainability of the code. 1) Use constructor to inject dependencies, 2) Avoid using service locators, 3) Use dependency injection containers to manage dependencies, 4) Improve testability through injecting dependencies, 5) Avoid over-injection dependencies, 6) Consider the impact of DI on performance.

PHP performance tuning tips and tricksPHP performance tuning tips and tricksMay 08, 2025 am 12:20 AM

PHPperformancetuningiscrucialbecauseitenhancesspeedandefficiency,whicharevitalforwebapplications.1)CachingwithAPCureducesdatabaseloadandimprovesresponsetimes.2)Optimizingdatabasequeriesbyselectingnecessarycolumnsandusingindexingspeedsupdataretrieval.

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

MantisBT

MantisBT

Mantis is an easy-to-deploy web-based defect tracking tool designed to aid in product defect tracking. It requires PHP, MySQL and a web server. Check out our demo and hosting services.

MinGW - Minimalist GNU for Windows

MinGW - Minimalist GNU for Windows

This project is in the process of being migrated to osdn.net/projects/mingw, you can continue to follow us there. MinGW: A native Windows port of the GNU Compiler Collection (GCC), freely distributable import libraries and header files for building native Windows applications; includes extensions to the MSVC runtime to support C99 functionality. All MinGW software can run on 64-bit Windows platforms.

VSCode Windows 64-bit Download

VSCode Windows 64-bit Download

A free and powerful IDE editor launched by Microsoft