Calibre Week in Review

This has been a busy week for me on the Calibre front. All of my changes were to pluginize and the first three I talk about also made it into trunk and will be appearing in the next release.

I re-worked the mobi metadata reader so that it does not read the entire file into memory. It only reads the parts of the file that hold the metadata. The advantage is reading the metadata is now about five times faster. These results are from unscientific testing by a the bug reporter. Basically he said that listing the books on his Kindle went from 5 minutes to about 1 minute.

The metadata writer for pdf files has been re-worked and is now enabled. Kovid did some work to my initial work so that it won’t lock up the GUI when working with large pdf files.

I (a bit of help from Kovid on this too) was able to fix bug 2112 (last few pdf files held open). Calibre relies on Python’s garbage collector and object scope for closing files. It does not explicitly close them. The bug as caused by pyPdf which is a Python library Calibre uses to read and write pdfs. For some reason pyPdf’s file reader wan’t allowing the files to be closed. They were no longer in use and the object went out of scope but the garbage collector didn’t close the file immediately. It would close it eventually. A wrapper object was created and is used so that pyPdf doesn’t have a direct reference to the open file and it now gets closed properly.

The GUI in the releases only supports displaying one storage card from a device. Not all device support two storage cards but the Sony PRS devices do. Support for the GUI to display two storage cards has been added.

To go along with the GUI supporting displaying two storage cards, Almost all device drivers have been made to support up to two storage cards. The USBMS base class supports two cards and as most device drivers use this base they all get support for it without much work. However, this doesn’t mean that a device that doesn’t physically a storage card or two storage card slots won’t magically support two cards. All except the PRS drivers don’t have any user visible changes. For anyone looking to write a device driver using USBMS if the device supports two cards USBMS has you covered.

The PRS505 and PRS700 drivers both received the two card treatment. They also received a bit of work. They have been moved to use the USBMS base class. This removed a lot of redundant code and puts them on the same code path as the other (except PRS500) drivers. Overall this change is to reduce work in finding and fixing bugs and maintenance.

Internal work on the PRS505 and PRS700 drives wasn’t all I did to them. They no longer dump all books into a single directory. Books are stored in author/title/book hierarchy. News items are stored in a news/title hierarchy. They also support the USBMS / tag as a custom layout path.

Earlier I said almost all device driver got two storage card support. The PRS500 driver did not. It still only supports one storage card. Due to the way the driver works I will not be touching it.

I’ve been working with ldolse from mobileread and with his help the processing rules for pdftohtml (used for pdf input) have been improved.