E-books Galore

by Tina Holmboe 25th of January 2010 (archive)

I’m a bookworm. I love books. Always did. For definitions of «always» lasting some thirty odd years. The discerning reader will notice how I use «bookworm» and not «bibliophile» — in the majority of cases the format of the book is of little interest to me.

As you may now suspect, should the title of this piece have escaped you, e–books are acceptable to me. Although I do prefer a nice, physical, paper–based book and always will, e–books have advantages. The fact that I can carry an entire library of light reading material when on business– or pleasure trips is one; the fact that I hate risking my paper volumes on the subway is another.

But they also drive me mad. Or, perhaps, it’s just the current state of affairs — sad by any kind of measure — which do the mental damage.

Let’s backtrack.

Some Light History

Since the Palm III I have been an avid user of the PalmOS platform, and even more so when I discovered Plucker. This was around the same time my upgrade to a Tungsten T in 2002.

As with so many PalmOS programs, Plucker is simple and quite brilliant. It distills down material from the web and produce an datafile containing the site for offline viewing. The possibilities were endless — and among the first visits I made were to Project Gutenberg and the Baen Free Library. Suddenly I could carry literally hundreds of books in my pocket.

With a reasonable screen — 212dpi — and Plucker, the Palm TT suddenly became a worthwhile e–book reader. Pure genius. Granted, despite the exceptional dpi the 240x240 resolution was annoying. The T3 and T5 followed, each improving the experience, then the T|X which I still use today. 320x480 in 3.9 inches gave an acceptable 148dpi. This is very likely the single best PDA ever produced.

But time and creativity ran out at Palm/PalmOne, and it is very unlikely that anything new and improved will emerge from them. Yes. That includes the PalmPre.

Somewhere along the line came the idea that instead of producing reader software such as Plucker for PDAs, specialised hardware could do the job. 1998 saw the Rocket reader, Softbook, and French Cybook; in later years we’ve seen the Sony Reader and the now–famous Amazon Kindle — to mention but a few dedicated reader «boards».

The concept of dedicated hardware and software, capable of only one single function, did not impress. The idea of lugging around a 6 to 10 inch reading board did not make me any happier, and neither did the fact that even the latest Kindle has no more than 150dpi.

Yes, you are right. I am preoccupied with dots per inch. Fact remain that for reading more than just a short amount of time, the higher dpi the better. For e–book readers, designed to read, well, entire books, I insist on 200+ in 2010.

Luckily alternatives existed. I stress the latter term.

Hardware

My first commercial experience was with eReader.com and their dedicated reader software — which was available for free on the Palm. Mobipocket had one too, also for free. Books cost money, tho often much less than paper copies, and one had to live with DRM. So far so good. With Plucker, eReader and MobiPocket on my T|X I was content.

As the years passed I started getting annoyed with the idea of carrying two devices — as self–employed I need that cellphone, and the PDA is my portable office. Perhaps, perhaps, I could combine the two? It’s been easier thought than done. There are two problems.

First, the state of hardware. Most «smart» phones had exceptionally small screens; annoying for many reasons. Others have ridiculous dpi; some as low as the 90 or so on desktop screens. It’s wasn’t worth it.

This begun to change as touchscreen smartphones started to pop up. The concept isn’t new — IBM showed such a product in 1992; Nokia had their Communicator line from 1996. Despite this the devices were in their infancy. The Palm Treo was «nice», but lacked screen real–estate. The Blackberry line had the same problem; the iPhone lacked everything in way of extensibility and suffered from a non–functional touchscreen technology.

The Nokia 5800 was a breath of fresh air. Running Symbian 9.5 with the S60 5th Edition GUI on top, it had the benefits of a stable OS, a solid touchscreen, and a reasonable resolution — 640x360 and 229dpi.

Although the Nokia N810 — 800x480 and 225dpi — had a more comfortable screen ratio it, alas, was no phone. That’d lead to two devices. The N900, due Any Day Soon, will push to 800x480 and 266dpi, include a phone, and PDA functionality. If the rumours are true it may just be the best combination since the T|X. But it leads us to problem number two.

Software

Secondly, then, the state of the software … and the state of the e–book formats.

Plucker is discontinued. It exists, still, and the Palm reader works as beautifully as always. eReader still exists, but up until recently it has not been particularly useful on Symbian — and it still does not exist for Linux, which leaves out the N900’s Maemo. Mobipocket’s reader works fine on Symbian, but has not been ported to 5th Edition devices — this may, or may not, be due to Amazon buying Mobipocket. I suspect ‹may› . Linux is out as well.

Other software exist, of course, specifically the rather flexible FBReader which supports a variety of formats, including Plucker and non–DRM Mobipocket. Finally: it runs on the N900.

One would think life is again rosy. One would be wrong. The main problem, yet again, is the e–book format itself. There’s no standard — not unless you accept the International Digital Publishing Forum’s right to create standards as they have with the EPUB format. I’d be happy to — since FBReader supports it. eReader.com uses the aptly named eReader format. Mobipocket.com sell in…. Mobipocket.

And never should the twain meet. Or rather, never should the dozen or so meet — Open Electronic Package, TomeRaider, Flip Books, Amazon Kindle — Mobipocket with a twist — Plucker, PDF, Microsoft LIT … the difficulty should be obvious: an electronic library can suddenly contain a plethora of different formats, often requiring different readers.

Buy an Amazon Kindle and say goodbye to your eReader, Plucker and EPUB books. Buy a Barnes & Noble «Nook» and goodbye Mobipocket and Amazon Kindle books. Get the Sony Reader, and no more Mobipocket, eReader, or Plucker texts. FBReader can handle several — but only without DRM. Insanity, here I come.

Complicated Conversions

All is not lost, however. It just gets complicated. It’s possible to convert formats, even strip DRM if that gets in the way. Unless, of course, you live in the US — but I consider that a lifestyle choice ;)

First of all, since I live in Sweden I am legally allowed to remove DRM if it prevents me from reading legally purchased books. Make no noises about it. I am, naturally, not allowed to share books after removing said DRM, but then again … to share a book would mean letting it out of my sight ;)

The solution I am currently the most satisfied with is, perhaps surprisingly, plain text. It emerged out of an enjoyment of fanfiction and other freely shared original works online — works which are, more often than not, in bad shape. The equivalent of careful restoration is required before they are readable. Some use peculiar Windows–only character encodings; some use broken markup and strange stylesheets.

To sort this out I began writing a text–to–html parser. That allowed me to first dump a text–only version of the story with Lynx, and then re–create the markup and character encoding. I can now take the worst cases and convert with little to no manual intervention.

The logical next step was to store the book as text, and later convert to HTML at need. Taking it even further I created a program to download, convert to text, and insert — using a microformat – meta–data such as author, title, and various other details.

My process has an added advantage: the excellent Calibre e–book suite can convert, with good results, HTML to .mobi or .epub — and even more importantly it can do so using efficient command–line tools.

«But how» — I hear you ask — «does this help with REAL e–books?». Here’s the rub: I’m not alone. After some digging out on the ’net I came up with the ereader2html.py and mobidedrm.py programs, both by «The Dark Reverser». Name, and use of Python, aside they do as written on the tin — remove DRM and convert eReader books to HTML, and remove DRM from Mobipocket books. Calibre, in turn, can convert Mobipocket to text. Enter Lynx and htmlify — which, finally, leaves me with «real» e–books in plain text format ready to be converted back to, say, EPUB.

And they can be stored. Permanently. Without any reason to fear that the format will explode on me any time soon. Of course, FBReader has poor support for EPUB, which is why I use .mobi at the moment. That allows me to read the books on either the Nokia 5800 or the Nokia N900. Or, for that matter, anything else that can read Mobipocket.

Flexibility. Of course, now I am left to do the conversion on a couple of hundred e–books …

I’ll return with a more in–depth look at how the N900 actually work as a book reader when, and if, I decide to get one.

PS

Some will tell me, with indignation, that the iPhone is certainly entirely suitable for reading e–books. They’ll be wrong. It may suit them, but not me. Let me take an example.

Some months ago I did a job for a company in the Northern part of Stockholm, around Kista. At lunch I ate at the Kista galleria which have, as part of their food court, the most excellent Texas Burger Co. And they have a dish consisting of hot spicy chicken wings … and more hot spicy chicken wings. A forty–minute break with spicy food and a good book is just what the doctor ordered.

Now tell me how you change pages in your e–book reader on the iPhone? Yes? I tap the bottom–most part of the screen with the stylus. Perhaps you don’t eat and read, but I do, just as I take calls with gloves on, enjoy handwriting, and some times spend times in situations where bare, greasy fingers on a touchscreen is a no–no. ’nuff of that.