TXT input now auto detects the character encoding of the file. It isn’t 100% accurate but should work for the majority of cases. It’s using chardet for the detection. Unfortunately, cp1252 is the most common encoding that gives people issues and unless you’re using things like smart quotes and curly apostrophes it doesn’t always detect properly.

I started getting TXT input to detect the document structure. Mainly, are the paragraphs arranged in block, single line, or print fashion. Lee saw the detection code and modifying some of his preprocessing code he was able to greatly increased the detection accuracy over my initial work. He’s also added an unformatted type that assumes the text is one big blob and tries to determine paragraphs in much the same way PDF input tries to determine them. By unwrapping based upon punctuation and other factors.

In addition to detecting the paragraph style used in the document, TXT input now tries to detect the formatting of the text content. Markdown formatted text is detected. I’ve also added a heuristic processor which runs by default if either Markdown is not detected or if the user has not specified the formatting as none (which disables any type of formatting processing).

The heuristic processor uses some ideas from GutenMark. Specifically italicizing common words and certain contentions used in Project Gutenberg texts that denote italics. I started working on a set of heuristics to detect chapter headings but Lee quickly pointed out he had already created something similar using regular expressions in his preprocessing code. I quickly began using it in my heuristic processor and it’s working well. Chapter headings and subheadings are now formatted with the appropriate h tags. He has some plans to enhance the detection further using a word list.

TCR, PDB PalmDoc and PDB zTXT inputs all pass the extracted text to the TXT input plugin for processing. This allows them to take advantage of all the work that’s gone into TXT input. Also, with auto detection now being part of TXT input it should allow for one time conversion instead of convert, check, tweak some options, convert again. Especially since these formats don’t make it easy to see how the text is structured within the file without first converting.

TXT input wasn’t the only part of TXT support that was touched. I spent some time cleaning up the TXT output. Consistant spacing is now created around headings. Also, when using the –remove-paragraph-spacing option, headings are not indented with a tab. The output now looks much cleaner and I consider it acceptable for reading.

Not to be left out FB2 output got a small bug fix. With all the work rewriting it I broke having it read covers. If you were converting an EPUB for instance that specified the cover (or title page) in the guide rather than the spine it would not be included. Also, the –cover option was being ignored. Now that’s fixed and external covers are inserted properly.

I’ve started a Qt based library for reading ebooks in a generic manner. It is called lebookread! It is it’s early stages. So far I have it supporting epub, palmdoc pdb, ztxt pdb, tcr, and rb files. I plan to support ereader pdb, mobi, and plucker files in the near future.

The main goal of this project is to make reading ebooks easy for Qt based projects. I’ve chose to write the library in C++. This is also my first attempt at writing a library and it shows. I hope that it will be used by Sigil.

The real motivation of writing lebook read is I really want a good light weight ebook reader. The current offering have issues. I want something that is a bit more advanced in it’s rendering than FBReader. I also didn’t want anything with as large a dependency list as calibre. So, I plan on using lebookread to write my own ebook viewer.

I’ve added a metadata reader for the eReader format. However, eReader supports 3 ways to set the metadata in the file. 1) In the pdb header (only supports setting a short title). 2) In the metadata section of the file (supports the most information: title, author, publisher, copyright, isbn). 3) Embedded in the text as a comment. 2 and 3 are only accessible if the book does not contain DRM (or has been unlocked, but Calibre does not support this). 3 is not supported at all with this metadata reader. The reader first tires 2 then falls back to 1 if the book is DRMed or if the metadata section is non-existent.

Two new input and output formats have been added. ztxt and palmdoc. They are both pdb formats like eReader. For input the pdb input plugin will automatically determine the internal format and call the appropriate code path. For output the default is palmdoc but there is an option –format that can be used to change it to any other supported pdb output format (ztxt is the only other currently). The format option is also available in the conversion dialog in the GUI.

Speaking of conversion in the GUI. It now works. There are all new dialogs for single and bulk conversion. Pretty much anything that can be done using the command line ebook-convert can be done in the GUI. Bulk, single and auto conversion are all complete and working. Auto conversion will also honor a users preferences for formats set for the device interface plugin.