John's Blog

This week I finally sat down and spend some time with Markdown input and output. Both saw major changes. Markdown input was bumped to upstream version 2.0. Output was completely rewritten from scratch. Markdown output is now completely custom code (not using a third party output module like before). I based the new markdown code off of the Textile output classes I helped Perkin to create.

As with all new code and major changes there are probably bugs. I tested Markdown output with a variety of test material and kept working at it until everything converted acceptably. I also used a variety of the Markdown tests provided by John Gruber to ensure my output was correct. When converting the HTML output tests back to Markdown the output is similar enough to the original that I feel it is acceptable.

The last big change I made this week was adding a new OEB transformation to unsmarten punctuation. As the name implies it changes curly quotes, apostrophes and a few other characters to their plain text, straight equivalents. It basically does the opposite of smarten punctuation. I find this especially useful when converting to formatted (Textile or Markdown) plain text files (TXT).

Tags: calibre, markdown, txt.

About Formatting Tips.

Up to this point Formatting Tips have been focused on the EPUB format and working directly with the underlying XHTML and CSS. Not everyone wants or needs this level of control over the layout of their book. Often times a book only needs basic formatting such as headings, bold, and italic. There are other easier ways to format an ebook. However, in this case simpler does mean basic.

A very easy way to format an ebook is to start with a plain text file (TXT). Then use either Markdown or Textile to add the formatting. Both Markdown and Textile allow for simple text formatting and they are designed to be converted to HTML.

By using TXT with a formatting syntax you can use pretty much any text editor you want. Markdown and Textile are very simple formats that are much easier to learn than XHTML and CSS. Adding things like *bold* is as easy as putting a * before and after a segment of text.

I recommend looking at both Markdown and Textile. There are differences in what formatting they support but they both support the basics like bold, italic, and headings. I’ve found Markdown to be easier to use but Textile offers more options.

After adding your formatting to the text it’s very easy to turn the TXT file into your desired final format (EPUB or MOBI most likely). calibre supports TXT formatted with either Markdown or Textile. However, the Textile support is more robust. Simply convert to the output format of your choosing.

Tags: book, calibre, ebook, epub, formatting, FT, markdown, print, textile, tips, txt.

I’ve become active in contributing to calibre again. So far I’ve been focusing on fixing issues related to the output formats I maintain. I’ve been focusing on FB2 and TXT output at the moment.

With FB2 output my goals were to fix as many bugs with it as possible and to produce 100% valid output. The first goal corresponds nicely to the second because most of the open bugs dealt with invalid markup.

FB2 output underwent some very large changes with a large amount of code being re-written. Also, I’ve removed a number of options. The idea is to simplify the code while working toward valid output and to remove options that were really just work arounds for invalid output in certain cases.

Overall I’m pleased with the FB2 output changes. It’s 100% valid (at least with the test book I ran though it) and the code is simpler. As always if any issues are found with the output a ticket would be appreciated.

TXT output had one small bug fix and one major change. TXT output can now produce Markdown formatted text. However, I’m not fully satisfied with the markdown generation. I didn’t spend much time with it and as of now it doesn’t appear to be taking css styling into account. I only pushed the xhtml from the OEB intermediate stage into html2markdown. I need to spend some more time with it. My fear is I may have to abandon the use of html2markdown if it’s unable to cope with css.

One other change with me getting back into calibre development is my working branch. I’ve changed it to lp:~user-none/calibre/dev because of some issues relating to my previous branch and some failures with upgrading the branch format.

Tags: calibre, fb2, markdown, txt.

For the past three weeks I’ve been working on an editor for working with plain text files and making it easy to add markdown syntax to them. My main goal is to make it easier to format the large number of ebooks I have. Almost all of them are plain text files.

It’s a python project using PyQt4 and I’m hosting it on Launchpad. here is the project page and you can find some screen shots here.

The features of this application and what makes it more useful that a generic text editor are the tool box and the tools. The toolbox allows for a number of markdown syntax changes to be made with one click. The tools menu supports a number of options that make formatting text a bit easier.

The current tools are:

• Heading list which shows a listing of all headings in the document
• Link list which shows a listing of all links in the document
• Image list which shows a listing of all images in the document
• ASCIIize which will turn all unicode characters into an ASCII equivalent
• Remove leading spaces
• Remove trailing spaces
• Replace tabs with spaces
• Separate paragraphs
• Double line breaks
• Remove excessive line beaks

There are a number of other options such as line numbering, highlighting of the line and syntax, and inline spell check.

There is still a lot I would like to do with the project. For one thing I needs and icon. As well as build targets for Windows and OS X. Include image previews in the Image listing. Take a look at the TODO file to get a feel of what I have in mind in the near future.

For those of you who what to test it out you can find a tarball here. The dependencies are:

• Python 2.6
• Qt >= 4.5
• PyQt >=4.5
• python-markdown
• python-enchant (optional for spell check)

Tags: GUI, markdown, python, qt.