John's Blog

One pet peeve of I have with my Cybook Gen 3 is its inability to properly display unicode characters in plain text files. I don’t need anything fancy like Japanese characters just simple things like “ and ” (as opposed to ” and “). To solve this problem I’ve been thinking about adding an –asciize option to calibre. I say thinking because I didn’t really know where to start. Thankfully a user recently requested this very functionality in bug #2846. He even included a link to work to accomplish this very task.

I will be integrating transliteration of unicode to ascii into calibre soon. However, in the mean time here is a script and classes, uni2ascii.zip, to accomplish this task outside of calibre. This is my python port of the ruby unidecode gem. Which is a port of the original perl Text::Unidecode.

The major differences between my implementation and the others is it’s written in python and it uses a single dictionary instead of loading the code group files as needed.

You can find out more on how this all works at http://interglacial.com/~sburke/tpj/as_html/tpj22.html

Tags: ebook, python.

Following is a bit of python code that illustrates how to create a QComboBox that attaches to a model for listing history items. The main features of this code are items entered in the text area of the combo are added to the history. Selected items and items entered that already appear in the combo are moved to the top. When MAX_ITEMS is exceeded older items (items at the bottom of the drop down) are removed.

#!/usr/bin/env python

from PyQt4.Qt import *
from PyQt4.QtGui import *

class ComboModel(QAbstractListModel):

    MAX_ITEMS = 5
    items = [u'123', u'456', u'789']

    # Required to get a working model.
    def rowCount(self, parent=QModelIndex()):
        # This is a List model meaning all elements are root elements.
        if parent and parent.isValid():
            return 0
        return len(self.items)

    # Required to get a working model.
    def insertRows(self, row, count, parent=QModelIndex()):
        self.beginInsertRows(parent, 0, 1)
        self.endInsertRows()
        return True

    def data(self, index, role):
        if role in (Qt.DisplayRole, Qt.EditRole):
            return QVariant(self.items[index.row()])
        return QVariant()

    def setData(self, index, value, role):
        value = unicode(value.toString())
        if value in self.items:
            # Move the item to the top of the list.
            del self.items[self.items.index(value)]
            self.items.insert(0, value)
        else:
            # Add the new item to the top of the list.
            self.items.insert(0, value)
            self.remove_items()
        self.emit(SIGNAL('dataChanged(QModelIndex, QModelIndex)'),
            self.createIndex(0, 0), self.createIndex(len(self.items) - 1, 0))
        return True

    def remove_items(self):
        '''
        Checks the number of items in the list and if it has been exceeded
        removes extra items.
        '''
        if len(self.items) > self.MAX_ITEMS:
            count = len(self.items) - self.MAX_ITEMS
            del self.items[-count:]

    def order_items(self, index):
        '''
        Move the selected item tot he top of the list.
        '''
        # We only need to move the item to the top if it is not the first item.
        if index > 0:
            self.emit(SIGNAL('layoutAboutToBeChanged()'))
            value = self.items[index]
            del self.items[index]
            self.items.insert(0, value)
            self.emit(SIGNAL('layoutChanged()'))

class MComboBox(QComboBox):

    def __init__(self, parent=None):
        QComboBox.__init__(self, parent)
        self.setEditable(True)
        # The default policy is InsertAtBottom. InsertAtTop must be set
        # otherwise the model will move the item to the top and the QComboBox
        # will select the last index. In the case of reaching max items the
        # index will be invalid because the removal is after the QComboBox has
        # stored the value of the last index.
        self.setInsertPolicy(QComboBox.InsertAtTop)
        # Without this the QComboBox will not call set data for us to check
        # for items already in the model if the item is a duplicate.
        self.setDuplicatesEnabled(True)
        self._model = ComboModel()
        self.setModel(self._model)

        # We need to tell the model when an item is selected so it can be moved
        # to the top of the history list.
        self.connect(self, SIGNAL('activated(int)'), self._model.order_items)

def main():
    app = QApplication([])

    bx = MComboBox()
    bx.show()

    app.exec_()

if __name__ == '__main__':
    main()

One thing to note is that in this example the model stores the items in a list called items. This can be replaced with some other way to retrieve the history items. For example with a connection to an SQLite DB.

Tags: python, qt.

There was no week in review last week because I went on vacation this past week. So this week in review combines everything since the last week in review.

I’ve made a few bug fixes to some output formats, PDB metadata and FB2 output mainly. The major things I’ve been working on is a bit of restructuring for the GUI and fixing some small bugs.

The GUI has had the button in the status bar (jobs, tags, cover flow) moved to a side bar on the right hand side. The version information and device connected information has moved to the status bar. The donate button was moved to the side bar. The status bar is now collapsible. When collapsed it shows less information (the list of formats for the selected book). When expanded it shows the book info the same as it does currently. The location view that lists the library and the connected device is hidden when no device is connected. I’ve added an About button to the new sidebar that will show some information about calibre. Overall these changes have two major benefits. It makes the interface a lot more netbook friendly and makes the book table larger so more information can be seen.

These changes to the GUI will be part of the 0.6 series but I can’t say for certain if it will be included in the initial 0.6.0 release.

Tags: calibre, fb2, GUI, pdb.

This week has been a productive one. I’ve made a lot of small GUI enhancements and did some work on PDF input as well. All of these changes have not made it into trunk yet. This is mainly because Kovid has been away this week.

I’ve added auto complete to a number of the input control on the GUI. Authors, Publisher, and Tags all auto complete pretty much everywhere now. The Tags will even auto complete in the table view in the main window. However, Authors, Series and Publisher do not auto complete in the main windows as of yet.

I’ve also been working with the GUI’s search. ISBN, Rating, Cover fields are all included in the default search. They are also search field identifiers. Meaning you can do isbn:123 to search just isbn numbers. Searching for empty and filled fields has been implemented as well. Use field:false and field:true respectively.

PDF input, either is or last week, got the ability to specify an unwrap factor for unwrapping lines. Previously this was a fixed value. Now it can be changed by the user. I have some ideas to enhance this further but I’m not going to to into detail because they may not materialize. Use the option –unwrap-factor with a decimal value 0 – 1. It is used by the regular expression that determines the minimum line length required for unwrapping.

PDF input had another highly requested change. The ability to remove headers and footers. However, it’s not as user friendly as I would like. There are four new options in total. –remove-header, –remove-footer, –header-regex, and –footer-regex. If the the –remove-* options are used then a regular expression that can be customized by using –*-regex is used to match headers and footers. The header and footer matching happens before all other processing rules. Use the ebook-convert’s –debug-input option to see the HTML that the regex will be matched against.

$ ebook-convert input.pdf .epub --debug-input output_dir/

Tags: calibre, GUI, pdf.

Here is a python script demonstrating how to use QCompleter to complete multiple tags in a QLineEdit. A few features of this script are: removing tags from the drop down that already appear in the QLineEdit, caching the tags, and inserting a , after completion to ease adding more tags. There are a few parts of this script that I’m going to go into detail about.

#!/usr/bin/env python

'''
Copyright (c) 2009 John Schember 

Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
'''

import sys

from PyQt4.Qt import Qt, QObject, QApplication, QLineEdit, QCompleter, \
    QStringListModel, SIGNAL

TAGS = ['Nature', 'buildings', 'home', 'City', 'country', 'Berlin']

class CompleterLineEdit(QLineEdit):

    def __init__(self, *args):
        QLineEdit.__init__(self, *args)

        QObject.connect(self, SIGNAL('textChanged(QString)'), self.text_changed)

    def text_changed(self, text):
        all_text = unicode(text)
        text = all_text[:self.cursorPosition()]
        prefix = text.split(',')[-1].strip()

        text_tags = []
        for t in all_text.split(','):
            t1 = unicode(t).strip()
            if t1 != '':
                text_tags.append(t)
        text_tags = list(set(text_tags))

        self.emit(SIGNAL('text_changed(PyQt_PyObject, PyQt_PyObject)'),
            text_tags, prefix)

    def complete_text(self, text):
        cursor_pos = self.cursorPosition()
        before_text = unicode(self.text())[:cursor_pos]
        after_text = unicode(self.text())[cursor_pos:]
        prefix_len = len(before_text.split(',')[-1].strip())
        self.setText('%s%s, %s' % (before_text[:cursor_pos - prefix_len], text,
            after_text))
        self.setCursorPosition(cursor_pos - prefix_len + len(text) + 2)

class TagsCompleter(QCompleter):

    def __init__(self, parent, all_tags):
        QCompleter.__init__(self, all_tags, parent)
        self.all_tags = set(all_tags)

    def update(self, text_tags, completion_prefix):
        tags = list(self.all_tags.difference(text_tags))
        model = QStringListModel(tags, self)
        self.setModel(model)

        self.setCompletionPrefix(completion_prefix)
        if completion_prefix.strip() != '':
            self.complete()

def main():
    app = QApplication(sys.argv)

    editor = CompleterLineEdit()

    completer = TagsCompleter(editor, TAGS)
    completer.setCaseSensitivity(Qt.CaseInsensitive)

    QObject.connect(editor,
        SIGNAL('text_changed(PyQt_PyObject, PyQt_PyObject)'),
        completer.update)
    QObject.connect(completer, SIGNAL('activated(QString)'),
        editor.complete_text)

    completer.setWidget(editor)

    editor.show()

    return app.exec_()

if __name__ == '__main__':
    main()

Looking at the main() function the editor widget’s text_changed signal is connected to the completer’s update slot. This serves two purposes. It provides the completer with all tags that are in the editor. Also, it provides the completer with the prefix of the current text that is being entered. The prefix is used for listing matching tags that are stored in the completer’s cache.

editor’s complete_text function takes the text in the editor before the cursor and subtracts the length of the prefix from that position because the prefix will be included in the completed text. The text before, the completed text, a comma, a space, and the text after the cursor are combined. This becomes the text in the QLineEdit. The cursor is advanced the position it was at minus the length of the prefix and plus 2 characters (, ) so that typing can immediately continue.

TAGS can be replaced with a function that gets all relevant tags if caching is not wanted.

Also, note that completer.setWidget(editor) was used not QLineEdit’s setCompleter() function. If setCompleter is used completion will only take place at the beginning of the QLineEdit. Meaning it will match everything before as one string and ignore the , delimiter separating the tags.

Tags: python, qt.

This month my book club has chosen to read Ex Machina, Vol. 1: The First Hundred Days. The story is about the world’s first (looks like only) superhero who decides to go into politics. His renown gives him a win and he is elected the mayor of New York City. I’ve been looking forward to reading this one so I hope it turns out better than last months pick.

.