Message 94545 - Python tracker

➜

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author	volker_siepmann
Recipients	volker_siepmann
Date	2009-10-27.07:13:47
SpamBayes Score	2.379036e-06
Marked as misclassified	No
Message-id	<[email protected]>
In-reply-to

Content
The readline() function in zipfile (in ZipExtFile) reads chunks of max 100 bytes (zipfile.py, line 525) into the linebuffer. A file of 500 MBytes therefore yields 5 million chunks. Changing the value 100 to 10000 bytes boosts performance by magnitudes, while it only requires 10k of memory. My fix in zipfile.py, line 525: buf = self.read(min(size, 10000)) # was 100 before Best regards / Volker Siepmann

The readline() function in zipfile (in ZipExtFile) reads chunks of max
100 bytes (zipfile.py, line 525) into the linebuffer. A file of 500
MBytes therefore yields 5 million chunks.
Changing the value 100 to 10000 bytes boosts performance by magnitudes,
while it only requires 10k of memory.

My fix in zipfile.py, line 525:

buf = self.read(min(size, 10000)) # was 100 before

Best regards / Volker Siepmann

History
Date	User	Action	Args
2009-10-27 07:13:52	volker_siepmann	set	recipients: + volker_siepmann
2009-10-27 07:13:51	volker_siepmann	set	messageid: <[email protected]>
2009-10-27 07:13:48	volker_siepmann	link	issue7216 messages
2009-10-27 07:13:47	volker_siepmann	create