John's Blog

My last post was about the GUI plugin for calibre to generate APNX files. While it seems the Kindle apps (Kindle for Mac, Kindle4PC and Kindle for iPhone / iPad) don’t like APNX files with non Amazon ebooks.

A number of people complained that the APNX files generated with the GUI plugin were not working on their Kindle apps. I looked into it and what I discovered is not very encouraging. First the problem. If you take a MOBI file (generated with calibre for instance) and create an APNX file; then put those two files in the “My Kindle Content” directory the book will open but the page numbers will not display. The strange thing about this issue is the GUI plugin outputs the same APNX files as transferring to a Kindle via “Send to Device.” These generated APNX files are known to work on Kindles.

I installed the Kindle for Mac application and came to some startling conclusions about how Amazon is handling page numbers in their Kindle apps. Fortunately, the Kindle for Mac install comes with a few DRM free books that have APNX files. I looked at these Amazon provided APNX files and couldn’t find any real difference between them and the APNX files I generate with calibre.

When I first opened the application and selected a book I noticed there were no page numbers. Since I was only using this for testing and I was planning on removing the app after I finished I did not tie it to my Amazon account and I also had my firewall deny outbound connections for the app. After a time closing and opening the Kindle app page numbers were still not appearing.

I did some searching online and found an Amazon help page that had a few suggestions for page numbers not displaying. The first item is “Wait a few minutes: It may take several minutes for page numbers to become visible.” I can see this being the case if it needs to download the APNX file but all three of the books it comes with have APNX files…

I compared the Kindle for Mac’s configuration and cache directories before and after allowing it to connect to the internet. It appears that the Kindle for Mac app must connect to https://kindle-app-services.amazon.com (there is no web server at this url) before the app will display page numbers.

Once the app will exchange an encrypted question and receive an encrypted answer for Amazon. New encrypted values are then written to ~/Library/Application Support/Kindle/storage/.kindle-info.

The .kindle-info file holds a variety of different things. For instance, on my system removing the following will cause the app to ask me to agree to the Terms of Use.

[0TB-ZJBABgbzb3ZgZPBKZP0vbs0gB4Zb:bK0B0hZOBHbYZNBtZDBablZYZKbUbk0U]

mrsquash on MobileRead did some further testing of APNX files in the Kindle app. He was able to have the Kindle app display page numbers using calibre generate APNX files for Amazon provided ebooks. I did some testing myself and I can swap APNX files for Amazon ebooks.

This leads me to believe that the Kindle app(s) will only display page numbers once it has communicated with Amazon and verified the ebook file is from Amazon. The reasons I’ve come to this conclusion are: 1) Generated APNX files work on the Kindle and on Amazon ebooks. 2) APNX files can be swapped between Amazon ebooks. 3) Only non-Amazon ebooks do not display page numbers with a generated APNX or with an APNX from an Amazon ebook. My suspicion is Amazon is doing this purposely and restricting the displaying of page numbers to force people to buy from Amazon and not a third party.

Tags: amazon, apnx, calibre, kindle.

The Amazon APNX file generation added to the Kindle device interface has been wildly popular. So popular that people want to use the APNX files without a Kindle. It turns out a large number of calibre uses don’t actually read using a Kindle but using one of the many reading apps Amazon produces (PC, Mac, iPad…). So I’ve created a GUI Plugin that allows users to create and save APNX files from MobiPocket (MOBI, AZW, and PRC) files. It can be found here.

Due to this feature being highly niche (only users of Amazon reading apps will have a use for it) I decided not to make it a part of calibre proper. Instead is being hosted as a 3rd party plugin on. The good news is the new Plugin Updater plugin will support my APNX plugin.

Tags: amazon, apnx, calibre, kindle.

Coming with the Kindle 3.1 firmware is the ability to have real page numbers. Getting ready for this Amazon has put out a preview release of the 3.1 firmware and has started adding the necessary information to Kindle books to show the page numbers.

The page numbers themselves map to the pages of the corresponding print book. Over all it gives a very pleasant experience. Amazon has implemented the page mapping though a new auxiliary file that has the .apnx extension. Doing this they can easily add this feature to all existing books and not have to worry about incompatibilities with older Kindles.

There is an easy way to tell if a book is going to include the APNX file. Look for “Page Numbers Source ISBN:”in the Product Details. All books that map pages to a print book will specify which edition they map to.

Now on to the more technical part of this post. I’ve spent some time looking at various books that Amazon is distributing with the APNX file and I’ve been able to reverse engineer the format. It’s a very simple format and after the header information is simply a list of 4 byte big-endian integers that correspond to locations in the uncompressed text. The position of the integer in the list corresponds to its page number.

Following is the documentation of the APNX specification I’ve written:

APNX
----
 
apnx files are used by the Amazon Kindle (firmware revision 3.1+) to
map pages from a print book to the Kindle version. Integers within
the file are big-endian.
 
 
Layout
------
 
bytes   content             comments 
 
4       00010001            Format identifier. Value of 65537 little-endian.
4       start of next       The offset after ending location of the first header.
                            Starts a new sequence of header info
4       length              Length of first header
N       first header        String containing content header
Starts next sequence
2       unknown             Always 1
2       length              Length of second header
2       page count          Total number of bytes after second header that
                            represent pages. This total includes bytes that
                            are ignored by the pageMap.
2       unknown             Always 32
N       second header       String containing the page mapping header
4*N     padding             The first number given in the page mapping header indicates the number of 0 bytes.
4*N     page list           
 
 
Content Header
--------------
 
The content header is a string enclosed in {} containing key, value pairs.
 
content             comments
 
contentGuid         Guid.
asin                Amazon identifier for the Kindle version of the book.
cdeType             MOBI cdeType. Should always be EBOK for ebooks.
fileRevisionId      Revision of this file.
 
Example:
{"contentGuid":"d8c14b0","asin":"B000JML5VM","cdeType":"EBOK","fileRevisionId":"1296874359405"}
 
 
Page Mapping Header
-------------------
 
The page mapping header is a string enclosed in {} containing key, value pairs.
 
content             comments
 
asin                The ISBN 10 for the paper book the pages correspond to
pageMap             Three value tuple. Looks like: "(N,N,N)"
                    1) Number of bytes after header that starts the page numbering sequence
                    2) unknown
                    3) unknown
 
Example:
{"asin":"1906694184","pageMap":"(4,a,1)"}
 
 
Page List
---------
 
The page list is a sequence of offsets in the uncompressed HTML. Each
value is the beginning of a new page. Each entry is a 4 byte big endian
int. The list is ordered lowest to highest.

Tags: amazon, apnx, ebook, kindle, mobi, reverse engineer, specification.