search
HomeBackend DevelopmentPHP TutorialphpExcel导出(大数据报错)

phpExcel导出(大数据报错)

Jun 23, 2016 pm 01:49 PM
phpexcelTarsusExportReport an error

数据量:
列:155
行:10000(后续数据量可能上涨150倍)

前提:
1、因为这1w行的标题存在较为复杂的逻辑问题,所以需要合并单元格的操作,只能选择excel导出
2、原系统导出时,发现会内存溢出,程序崩溃,实际中不可能无限制的修改内存。所以从网上查找了资料,有两种方法,一种是追加(写一个excel,后面的去读取这个excel,然后追加),另一种是多sheet

问题:
1、根据方法做了测试,只是查了这10000个基本数据(没有涉及到原先的多表关联),发现追加的数据量过了500临界点之后程序就崩溃了(内存为php默认的内存),多个sheet也无法解决,难道追加也和内存有关系么?难道数据量超过了限制?有什么方法可以解决这个问题?或者有什么方法可以解决这样的大数据导出?
(需要考虑用户体验。。。)


回复讨论(解决方案)

导成 csv 吧,这个比较容易一些,或者导成 xml 格式,但是文件名可以写成 xls 。

不过,一下子导出150W 行数据,有人看吗?

客户要这么多数据,应该是要别的工具做统计

csv有合并单元格,字体样式的操作么?

导成 csv 吧,这个比较容易一些,或者导成 xml 格式,但是文件名可以写成 xls 。

不过,一下子导出150W 行数据,有人看吗?




客户要这么多数据,应该是要用别的工具做统计(总是各种需求..)

csv有合并单元格,字体样式的操作么? 


导成 csv 吧,这个比较容易一些,或者导成 xml 格式,但是文件名可以写成 xls 。

不过,一下子导出150W 行数据,有人看吗?




客户要这么多数据,应该是要用别的工具做统计(总是各种需求..)

csv有合并单元格,字体样式的操作么? 

对了,最主要的是这个excel里面有超连接,连接到服务器上用资源管理器查看非结构化数据,如果csv估计不行了吧?

这个可能对你有用  http://www.baidu.com/s?wd=phpExcel%E7%BC%93%E5%AD%98&ie=utf-8



导成 csv 吧,这个比较容易一些,或者导成 xml 格式,但是文件名可以写成 xls 。

不过,一下子导出150W 行数据,有人看吗?




客户要这么多数据,应该是要用别的工具做统计(总是各种需求..)

csv有合并单元格,字体样式的操作么? 

对了,最主要的是这个excel里面有超连接,连接到服务器上用资源管理器查看非结构化数据,如果csv估计不行了吧?

存在 xml  格式的吧,可以解决 超连接和 合并单元格式的需求 。 

你自己建一个excel 文件,随便录几个数据,再存在 xml 格式,你大概看看,没什么难度。

数据不要一次读出来,可以分页多次遍历追加csv或者exel

全部到处150W,excel的临界点都超过了,肯定不现实的。

PHPExcel 处理Excel的确有很多问题(不知道是不是自己没搞懂), 按document操作经常出现内存不够,或者CPU高飙的情况。数据量一大基本没法操作。
之前参考了几种修改其缓存的方法,对我来说没什么用。你可以参考下。PHPExcel缓存方式 http://pqbd69.blog.163.com/blog/static/2312720062013865360908/

Statement
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
PHP Performance Tuning for High Traffic WebsitesPHP Performance Tuning for High Traffic WebsitesMay 14, 2025 am 12:13 AM

ThesecrettokeepingaPHP-poweredwebsiterunningsmoothlyunderheavyloadinvolvesseveralkeystrategies:1)ImplementopcodecachingwithOPcachetoreducescriptexecutiontime,2)UsedatabasequerycachingwithRedistolessendatabaseload,3)LeverageCDNslikeCloudflareforservin

Dependency Injection in PHP: Code Examples for BeginnersDependency Injection in PHP: Code Examples for BeginnersMay 14, 2025 am 12:08 AM

You should care about DependencyInjection(DI) because it makes your code clearer and easier to maintain. 1) DI makes it more modular by decoupling classes, 2) improves the convenience of testing and code flexibility, 3) Use DI containers to manage complex dependencies, but pay attention to performance impact and circular dependencies, 4) The best practice is to rely on abstract interfaces to achieve loose coupling.

PHP Performance: is it possible to optimize the application?PHP Performance: is it possible to optimize the application?May 14, 2025 am 12:04 AM

Yes,optimizingaPHPapplicationispossibleandessential.1)ImplementcachingusingAPCutoreducedatabaseload.2)Optimizedatabaseswithindexing,efficientqueries,andconnectionpooling.3)Enhancecodewithbuilt-infunctions,avoidingglobalvariables,andusingopcodecaching

PHP Performance Optimization: The Ultimate GuidePHP Performance Optimization: The Ultimate GuideMay 14, 2025 am 12:02 AM

ThekeystrategiestosignificantlyboostPHPapplicationperformanceare:1)UseopcodecachinglikeOPcachetoreduceexecutiontime,2)Optimizedatabaseinteractionswithpreparedstatementsandproperindexing,3)ConfigurewebserverslikeNginxwithPHP-FPMforbetterperformance,4)

PHP Dependency Injection Container: A Quick StartPHP Dependency Injection Container: A Quick StartMay 13, 2025 am 12:11 AM

APHPDependencyInjectionContainerisatoolthatmanagesclassdependencies,enhancingcodemodularity,testability,andmaintainability.Itactsasacentralhubforcreatingandinjectingdependencies,thusreducingtightcouplingandeasingunittesting.

Dependency Injection vs. Service Locator in PHPDependency Injection vs. Service Locator in PHPMay 13, 2025 am 12:10 AM

Select DependencyInjection (DI) for large applications, ServiceLocator is suitable for small projects or prototypes. 1) DI improves the testability and modularity of the code through constructor injection. 2) ServiceLocator obtains services through center registration, which is convenient but may lead to an increase in code coupling.

PHP performance optimization strategies.PHP performance optimization strategies.May 13, 2025 am 12:06 AM

PHPapplicationscanbeoptimizedforspeedandefficiencyby:1)enablingopcacheinphp.ini,2)usingpreparedstatementswithPDOfordatabasequeries,3)replacingloopswitharray_filterandarray_mapfordataprocessing,4)configuringNginxasareverseproxy,5)implementingcachingwi

PHP Email Validation: Ensuring Emails Are Sent CorrectlyPHP Email Validation: Ensuring Emails Are Sent CorrectlyMay 13, 2025 am 12:06 AM

PHPemailvalidationinvolvesthreesteps:1)Formatvalidationusingregularexpressionstochecktheemailformat;2)DNSvalidationtoensurethedomainhasavalidMXrecord;3)SMTPvalidation,themostthoroughmethod,whichchecksifthemailboxexistsbyconnectingtotheSMTPserver.Impl

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

EditPlus Chinese cracked version

EditPlus Chinese cracked version

Small size, syntax highlighting, does not support code prompt function

SublimeText3 English version

SublimeText3 English version

Recommended: Win version, supports code prompts!

MantisBT

MantisBT

Mantis is an easy-to-deploy web-based defect tracking tool designed to aid in product defect tracking. It requires PHP, MySQL and a web server. Check out our demo and hosting services.

SublimeText3 Linux new version

SublimeText3 Linux new version

SublimeText3 Linux latest version

SecLists

SecLists

SecLists is the ultimate security tester's companion. It is a collection of various types of lists that are frequently used during security assessments, all in one place. SecLists helps make security testing more efficient and productive by conveniently providing all the lists a security tester might need. List types include usernames, passwords, URLs, fuzzing payloads, sensitive data patterns, web shells, and more. The tester can simply pull this repository onto a new test machine and he will have access to every type of list he needs.