John's Blog

For the past three weeks I’ve been working on an editor for working with plain text files and making it easy to add markdown syntax to them. My main goal is to make it easier to format the large number of ebooks I have. Almost all of them are plain text files.

It’s a python project using PyQt4 and I’m hosting it on Launchpad. here is the project page and you can find some screen shots here.

The features of this application and what makes it more useful that a generic text editor are the tool box and the tools. The toolbox allows for a number of markdown syntax changes to be made with one click. The tools menu supports a number of options that make formatting text a bit easier.

The current tools are:

• Heading list which shows a listing of all headings in the document
• Link list which shows a listing of all links in the document
• Image list which shows a listing of all images in the document
• ASCIIize which will turn all unicode characters into an ASCII equivalent
• Remove leading spaces
• Remove trailing spaces
• Replace tabs with spaces
• Separate paragraphs
• Double line breaks
• Remove excessive line beaks

There are a number of other options such as line numbering, highlighting of the line and syntax, and inline spell check.

There is still a lot I would like to do with the project. For one thing I needs and icon. As well as build targets for Windows and OS X. Include image previews in the Image listing. Take a look at the TODO file to get a feel of what I have in mind in the near future.

For those of you who what to test it out you can find a tarball here. The dependencies are:

• Python 2.6
• Qt >= 4.5
• PyQt >=4.5
• python-markdown
• python-enchant (optional for spell check)

Tags: GUI, markdown, python, qt.

Tags: screenshot.

***Update: Simplified Highlighter.highlightBlock function

One thing Qt lacks is an integrated spell check in the text entry components. For a project I’m working on this is necessary. Using python-enchant and the QSyntaxHighlighter I was able to implement this functionality. Here is how to add an in line spell check support to a QPlainTextEdit.

#!/usr/bin/env python
# -*- coding: utf-8 -*-

__license__ = 'MIT'
__copyright__ = '2009, John Schember '
__docformat__ = 'restructuredtext en'

import re
import sys

import enchant

from PyQt4.Qt import QAction
from PyQt4.Qt import QApplication
from PyQt4.Qt import QEvent
from PyQt4.Qt import QMenu
from PyQt4.Qt import QMouseEvent
from PyQt4.Qt import QPlainTextEdit
from PyQt4.Qt import QSyntaxHighlighter
from PyQt4.Qt import QTextCharFormat
from PyQt4.Qt import QTextCursor
from PyQt4.Qt import Qt
from PyQt4.QtCore import pyqtSignal

class SpellTextEdit(QPlainTextEdit):

    def __init__(self, *args):
        QPlainTextEdit.__init__(self, *args)

        # Default dictionary based on the current locale.
        self.dict = enchant.Dict()
        self.highlighter = Highlighter(self.document())
        self.highlighter.setDict(self.dict)

    def mousePressEvent(self, event):
        if event.button() == Qt.RightButton:
            # Rewrite the mouse event to a left button event so the cursor is
            # moved to the location of the pointer.
            event = QMouseEvent(QEvent.MouseButtonPress, event.pos(),
                Qt.LeftButton, Qt.LeftButton, Qt.NoModifier)
        QPlainTextEdit.mousePressEvent(self, event)

    def contextMenuEvent(self, event):
        popup_menu = self.createStandardContextMenu()

        # Select the word under the cursor.
        cursor = self.textCursor()
        cursor.select(QTextCursor.WordUnderCursor)
        self.setTextCursor(cursor)

        # Check if the selected word is misspelled and offer spelling
        # suggestions if it is.
        if self.textCursor().hasSelection():
            text = unicode(self.textCursor().selectedText())
            if not self.dict.check(text):
                spell_menu = QMenu('Spelling Suggestions')
                for word in self.dict.suggest(text):
                    action = SpellAction(word, spell_menu)
                    action.correct.connect(self.correctWord)
                    spell_menu.addAction(action)
                # Only add the spelling suggests to the menu if there are
                # suggestions.
                if len(spell_menu.actions()) != 0:
                    popup_menu.insertSeparator(popup_menu.actions()[0])
                    popup_menu.insertMenu(popup_menu.actions()[0], spell_menu)

        popup_menu.exec_(event.globalPos())

    def correctWord(self, word):
        '''
        Replaces the selected text with word.
        '''
        cursor = self.textCursor()
        cursor.beginEditBlock()

        cursor.removeSelectedText()
        cursor.insertText(word)

        cursor.endEditBlock()

class Highlighter(QSyntaxHighlighter):

    WORDS = u'(?iu)[\w\']+'

    def __init__(self, *args):
        QSyntaxHighlighter.__init__(self, *args)

        self.dict = None

    def setDict(self, dict):
        self.dict = dict

    def highlightBlock(self, text):
        if not self.dict:
            return

        text = unicode(text)

        format = QTextCharFormat()
        format.setUnderlineColor(Qt.red)
        format.setUnderlineStyle(QTextCharFormat.SpellCheckUnderline)

        for word_object in re.finditer(self.WORDS, text):
            if not self.dict.check(word_object.group()):
                self.setFormat(word_object.start(),
                    word_object.end() - word_object.start(), format)

class SpellAction(QAction):

    '''
    A special QAction that returns the text in a signal.
    '''

    correct = pyqtSignal(unicode)

    def __init__(self, *args):
        QAction.__init__(self, *args)

        self.triggered.connect(lambda x: self.correct.emit(
            unicode(self.text())))

def main(args=sys.argv):
    app = QApplication(args)

    spellEdit = SpellTextEdit()
    spellEdit.show()

    return app.exec_()

if __name__ == '__main__':
    sys.exit(main())

The SpellTextEdit’s purpose is straightforward. It will mark misspelled words. Right clicking on a word in the SpellTextEdit will cause the word to become selected and display a context menu. If the word is misspelled and there are spelling suggestions the context menu will include a sub menu of those suggestions. Selecting a suggestion will replace the misspelled text with the selection.

The Highlighter class takes text, breaks it into words, checks if they are spelled correctly and if not underlines the misspelled ones with a red squiggle. I’m using a regular expression to split the words instead of using str.split because str.split will only split on whitespace and include punctuation (e.g. “.!*) as part of the words.

SpellAction is a simple class that allows for the action’s text to be sent with the signal. This is necessary for dynamically creating the list of possible correction words in the right click menu. The SpellAction is connected to a function that replaces the selected text with the signal text.

Tags: GUI, pyqt, python, qt, spell check, widget.

My last post was an implementation of a Qt widget which displays text with line numbers. I found that it has a few limitations. The biggest was a performance penalty when dealing with large documents. I’ve since re-factored and rewritten the class to make the performance acceptable. I’ve also cleaned up the code a bit and added a highlight to the current line.

'''
Text widget with support for line numbers
'''

from PyQt4.Qt import QFrame
from PyQt4.Qt import QHBoxLayout
from PyQt4.Qt import QPainter
from PyQt4.Qt import QPlainTextEdit
from PyQt4.Qt import QRect
from PyQt4.Qt import QTextEdit
from PyQt4.Qt import QTextFormat
from PyQt4.Qt import QVariant
from PyQt4.Qt import QWidget
from PyQt4.Qt import Qt

class LNTextEdit(QFrame):

    class NumberBar(QWidget):

        def __init__(self, edit):
            QWidget.__init__(self, edit)

            self.edit = edit
            self.adjustWidth(1)

        def paintEvent(self, event):
            self.edit.numberbarPaint(self, event)
            QWidget.paintEvent(self, event)

        def adjustWidth(self, count):
            width = self.fontMetrics().width(unicode(count))
            if self.width() != width:
                self.setFixedWidth(width)

        def updateContents(self, rect, scroll):
            if scroll:
                self.scroll(0, scroll)
            else:
                # It would be nice to do
                # self.update(0, rect.y(), self.width(), rect.height())
                # But we can't because it will not remove the bold on the
                # current line if word wrap is enabled and a new block is
                # selected.
                self.update()

    class PlainTextEdit(QPlainTextEdit):

        def __init__(self, *args):
            QPlainTextEdit.__init__(self, *args)

            #self.setFrameStyle(QFrame.NoFrame)

            self.setFrameStyle(QFrame.NoFrame)
            self.highlight()
            #self.setLineWrapMode(QPlainTextEdit.NoWrap)

            self.cursorPositionChanged.connect(self.highlight)

        def highlight(self):
            hi_selection = QTextEdit.ExtraSelection()

            hi_selection.format.setBackground(self.palette().alternateBase())
            hi_selection.format.setProperty(QTextFormat.FullWidthSelection, QVariant(True))
            hi_selection.cursor = self.textCursor()
            hi_selection.cursor.clearSelection()

            self.setExtraSelections([hi_selection])

        def numberbarPaint(self, number_bar, event):
            font_metrics = self.fontMetrics()
            current_line = self.document().findBlock(self.textCursor().position()).blockNumber() + 1

            block = self.firstVisibleBlock()
            line_count = block.blockNumber()
            painter = QPainter(number_bar)
            painter.fillRect(event.rect(), self.palette().base())

            # Iterate over all visible text blocks in the document.
            while block.isValid():
                line_count += 1
                block_top = self.blockBoundingGeometry(block).translated(self.contentOffset()).top()

                # Check if the position of the block is out side of the visible
                # area.
                if not block.isVisible() or block_top >= event.rect().bottom():
                    break

                # We want the line number for the selected line to be bold.
                if line_count == current_line:
                    font = painter.font()
                    font.setBold(True)
                    painter.setFont(font)
                else:
                    font = painter.font()
                    font.setBold(False)
                    painter.setFont(font)

                # Draw the line number right justified at the position of the line.
                paint_rect = QRect(0, block_top, number_bar.width(), font_metrics.height())
                painter.drawText(paint_rect, Qt.AlignRight, unicode(line_count))

                block = block.next()

            painter.end()

    def __init__(self, *args):
        QFrame.__init__(self, *args)

        self.setFrameStyle(QFrame.StyledPanel | QFrame.Sunken)

        self.edit = self.PlainTextEdit()
        self.number_bar = self.NumberBar(self.edit)

        hbox = QHBoxLayout(self)
        hbox.setSpacing(0)
        hbox.setMargin(0)
        hbox.addWidget(self.number_bar)
        hbox.addWidget(self.edit)

        self.edit.blockCountChanged.connect(self.number_bar.adjustWidth)
        self.edit.updateRequest.connect(self.number_bar.updateContents)

    def getText(self):
        return unicode(self.edit.toPlainText())

    def setText(self, text):
        self.edit.setPlainText(text)

    def isModified(self):
        return self.edit.document().isModified()

    def setModified(self, modified):
        self.edit.document().setModified(modified)

    def setLineWrapMode(self, mode):
        self.edit.setLineWrapMode(mode)

Tags: GUI, python, qt.

Here is a Qt4 widget written in Python that allows for line numbers next to a QTextEdit. Similar to what is seen in a number of text editors such as gedit and kate.

from PyQt4.Qt import QFrame, QWidget, QTextEdit, QHBoxLayout, QPainter

class LineTextWidget(QFrame):

    class NumberBar(QWidget):

        def __init__(self, *args):
            QWidget.__init__(self, *args)
            self.edit = None
            # This is used to update the width of the control.
            # It is the highest line that is currently visibile.
            self.highest_line = 0

        def setTextEdit(self, edit):
            self.edit = edit

        def update(self, *args):
            '''
            Updates the number bar to display the current set of numbers.
            Also, adjusts the width of the number bar if necessary.
            '''
            # The + 4 is used to compensate for the current line being bold.
            width = self.fontMetrics().width(str(self.highest_line)) + 4
            if self.width() != width:
                self.setFixedWidth(width)
            QWidget.update(self, *args)

        def paintEvent(self, event):
            contents_y = self.edit.verticalScrollBar().value()
            page_bottom = contents_y + self.edit.viewport().height()
            font_metrics = self.fontMetrics()
            current_block = self.edit.document().findBlock(self.edit.textCursor().position())

            painter = QPainter(self)

            line_count = 0
            # Iterate over all text blocks in the document.
            block = self.edit.document().begin()
            while block.isValid():
                line_count += 1

                # The top left position of the block in the document
                position = self.edit.document().documentLayout().blockBoundingRect(block).topLeft()

                # Check if the position of the block is out side of the visible
                # area.
                if position.y() > page_bottom:
                    break

                # We want the line number for the selected line to be bold.
                bold = False
                if block == current_block:
                    bold = True
                    font = painter.font()
                    font.setBold(True)
                    painter.setFont(font)

                # Draw the line number right justified at the y position of the
                # line. 3 is a magic padding number. drawText(x, y, text).
                painter.drawText(self.width() - font_metrics.width(str(line_count)) - 3, round(position.y()) - contents_y + font_metrics.ascent(), str(line_count))

                # Remove the bold style if it was set previously.
                if bold:
                    font = painter.font()
                    font.setBold(False)
                    painter.setFont(font)

                block = block.next()

            self.highest_line = line_count
            painter.end()

            QWidget.paintEvent(self, event)

    def __init__(self, *args):
        QFrame.__init__(self, *args)

        self.setFrameStyle(QFrame.StyledPanel | QFrame.Sunken)

        self.edit = QTextEdit()
        self.edit.setFrameStyle(QFrame.NoFrame)
        self.edit.setAcceptRichText(False)

        self.number_bar = self.NumberBar()
        self.number_bar.setTextEdit(self.edit)

        hbox = QHBoxLayout(self)
        hbox.setSpacing(0)
        hbox.setMargin(0)
        hbox.addWidget(self.number_bar)
        hbox.addWidget(self.edit)

        self.edit.installEventFilter(self)
        self.edit.viewport().installEventFilter(self)

    def eventFilter(self, object, event):
        # Update the line numbers for all events on the text edit and the viewport.
        # This is easier than connecting all necessary singals.
        if object in (self.edit, self.edit.viewport()):
            self.number_bar.update()
            return False
        return QFrame.eventFilter(object, event)

    def getTextEdit(self):
        return self.edit

Tags: GUI, python, qt.

This time I missed last weeks week in review because I simply forgot. I’m hoping to keep this to a minimum in the future.

The big news is calibre 0.6 has been released. Kovid is now back to his regular (weekly at the least) bug fix releases too. As of now the latest version is 0.6.4 and I get the feeling that 0.6.5 is right around the corner. For a listing of what’s gone into the 0.6 release take a look here.

Some features I’ve been working on that are included in the current release are: asciiize text, and iRex Iliad support.

The asciiize feature is one I’ve been wanting for a while and I’ve finally implemented it. It’s based on the ASCIIize text post I made last week. There is now an –asciiize option for the ebook-convert command and an option for the conversion dialog in the GUI. The basic premise of this feature is unicode characters often are not displayed correctly by ebook readers. My Cybook Gen 3 exhibits this behavior. ASCIIize transliterates the unicode text to an ASCII representation. Meaning it takes “Михаил Горбачёв” and converts it to “Mikhail Gorbachiov”.

The iRex Iliad is now as supported as I can get it. calibre detects it as a device, displays the list of books on the device and you can send books to it using send to device. The part I’m running into issues with is the manifest.xml file. It looks like it’s similar to the Sony PRS’s media.xml file, meaning it is a quick store for metadata. However, I don’t really know what goes into this file or should I say files (there are more than one). It also doesn’t look like the device updates it in any way because I had a user send me the files off of their Iliad and they were empty even though the user has put 20 or so books on the device with the Mobipocket desktop software. On the bright side it works well enough that I don’t think anyone will notice.

On the GUI tweaks front (these won’t be in a release until some point in the future) I’ve added a history drop down to the search field. There is a swap button for authors and title in the metadata bulk dialog, and you can hide the toolbars in the ebook viewer.

The remainder of what I accomplished over the past two weeks was bug fixes and code refactoring. Just th boring stuff that I would rather put off versus actually doing it.

Tags: calibre, ebook, GUI.