The end of ebook development 26 April 2012

The end of ebook development The Table of Contents and notes/annotations are to the sides of the main text. Annotation highlights are local links that open either in the sidebar or in a popup Start reading →

A bit of an utopian fantasy but, I think, an essential one. We need visions of where we're heading with this.

Everybody knows what it feels like. You’re thrown from crisis to crisis, locked into solving problems, resolving issues, keeping things from breaking down, grasping the duct tape with one hand, and your sanity with the other.

Every time you look up, you see more trouble coming your way and no time to avoid it.

It’s hard to think about long-term problems when you’re up to your neck in shit and you’re losing your toehold.

The publishing and ebook industry is completely in reaction mode. Publishers reacted to Amazon by colluding to set up the agency system. Kobo and B&N reacted to Amazon by mimicking its strategy. Even the IDPF’s EPUB3 and FXL standards are reactions to the runaway train that is HTML5 and Apple’s format extensions, respectively. Google’s publishing plans seem about as coordinated and planned as a piece of driftwood’s path through a hurricane.

The only two companies that seem to have a plan and try to act on it are Amazon and Apple, both tech companies. Unsurprisingly, they are the ones who are going to control the future of ebook publishing. Unless there are dramatic changes the rest of us can do very little except play along and accept that it’s their playground and their toys, leave, or have the patience to wait the decade it might take for somebody to disrupt them.

Or, we could stop reacting to what they are doing, take a moment, and figure out what the ebook industry should look like. Picture the ideal and then figure out how to get there, step by step.

I’m already on record as believing that ebook distribution and retail should be based on a modular ecosystem, open file formats and standardised services. I should be able to buy a book from any retailer, have it automatically download to any ereader (app or device), have that ereader use any bookmarking/note service I want, and have that service sync my notes on to Simplenote or Dropbox.

What we have in the ebook world is worse than the days of ‘best in IE’. It’s crazy that we don’t have one ebook file format that works in all readers.

Even the EPUB world is a mess. Try getting a complex book working in iBooks, Kobo, and Nook. More often than not, you’re going to run into problems.

What’s even more insane is having to open up an epub file to edit it by hand just to make it work or accomplish a design effect. It’s nuts.

So what’s the ideal?

How do we want ebooks to be made?

One button publishing. Themes. Design tools. Developers.

1. One button publishing

A writer should be able to open up a Scrivener or Word document – one that has been thrown back and forth between the writer and the editor until both are satisfied – click on something like “Export to EPUB” and have a ready-made EPUB file that works everywhere. Maybe have a bit of preferences and stuff to adjust but no fuss. No rendering errors. No worries about whether that blockquote is too long or whether the margins on that list will disappear for no reason. Maybe a quick conversion with kindlegen, but even that can be done automatically.

Write, rewrite, edit, click export: you have an ebook ready to publish.

Of course, I’m leaving out proofreading iterations and other vital, important things, but that doesn’t change the basic point. Generating a good, retail-ready, ebook from an authoring document should be a one-button affair.

This is unlikely to ever work as long as there are Kindles being used that only support KF7 because that particular ebook format suffers from extensive brain damage. It’s the reason why we have ebooks where all blockquotes are marked up as paragraphs. The only thing you can expect to work automatically and without problems is slightly dolled up plain text (headers, bold, italics, lists if you’re lucky). You can’t get reliable one-button publishing as long as these devices have any sort of marketshare.

Which is really unfortunate because a lot of titles would be better served if they were published without any styles whatsoever, just a solid structure.

And when I say solid, I mean a structure generated by an app. Any sort of human editing of production markup is a recipe for errors and trouble.

Focus on the text, let the ereader style the book appropriately and everybody is happy. But that particular vision of the future will have to wait until Amazon stops supporting KF7-only devices. As long as Amazon keeps selling and supporting these devices, the complications involved in preserving backwards-compatibility for these inane, mindless, critters mean that some sort of human hand is has to mess with the process.

Which will introduce errors in and of itself.

It’s as if Microsoft were shipping IE6 alongside IE9, letting their customers decide which to install while glossing over the limitations and toxicity of IE6 by pitching it as the light-weight alternative.

Amazon has good business reasons for creating and maintaining this situation, but we don’t have to shut up and like it.

2. Themes.

Even though most books would be served by basic styles, there are a number of books that require a visual aesthetic with specific characteristics for the book to work. These titles need to have the freedom to deviate or completely replace the ereader’s default styles. And none of that fake book chrome. You can’t do a modernist design, for example, when your book is framed by a retro, kitsch, facsimile of a book.

To accomplish this we only need to add a single one-time step to our one-button process: Write, click export, and pick a theme.

Maybe configure the theme a little. And the authoring system should remember your choice of theme and settings so that the next time you click ‘export’ we’d be back to it being a one-button, one-press, thing.

This would have a few implications for the publishing process:

A publisher would have a set of custom-made themes, one for each book series or type. Not unlike the book cover system that Penguin had back in the day. You’d pick a design for a book, then write it, instead of the other way around. That way you know exactly what it’ll look like to the reader throughout the writing, editing, and proofreading process. The ebook design and development industry would disappear and be replaced by a theme development industry, not too dissimilar from what has happened to bog-standard web design.

Of course, for this to work we would have to have ereaders that actually support designs and not each breaking the design in a different way by enforcing defaults that can’t be changed, either by the designer or the reader.

‘But, we do it for the reader. Honest.’ Don’t make me laugh. What readers need is the ability to enforce minimum font sizes, screenreader support, and always being able to change the font size. Page margin pedantry serves no one.

Then we have books that can’t be served by simply hanging a theme off a solid structure. These books need proper design. Or, if they don’t need it, they’re getting it because the author is a neurotic designer who is compensating for their inadequacies as a writer by making pap look gorgeous.

That’s where we get to the next level in the tools hierarchy: proper, fully featured, extensive design tools.

On one end of the spectrum we have iBooks Author, which is actually theme-based, but allows for adjustments and customisations to such a degree that it skips up into the design tool level.

On the other end we have Indesign, a great tool for print designers that is the single biggest cause of depression, schizophrenia, and other mental illnesses in interactive media designers since the demise of IE6.

(No sane person willingly uses Indesign to create interactive media. So, it’s handy for Adobe that it’s effective at driving developers insane which then maintains the user base. It’s a behemoth that almost makes you nostalgic for the bad old days of Director.)

The usefulness of these tools is directly proportional to their limitations.

A tool that is unlimited by media, supports print design, iPad magazine app design, ebooks, ebooks with layout, etc., is a tool that will drive you to drink and suicide through frustration. These apps do deliver on the power and flexibility they promise but their complexity and expense will make that power a lot less useful than you think.

A tool that only works for one ereader app on one device type is amazing to use and fills you with joy, all of which is drained away once you realise that only people with iPads can read your work.

Hypercard was of the second type: fun and amazing to use, but only worked on Macs and had several big limitations.

A capable, easy to use, design tool that delivers to multiple platforms is hard to create.

Of course, it’d be a lot easier to do if everybody in the ebook industry just settled on one format and, maybe, a couple of standard rendering engines. (Hah!)

If you’re going to create a design tool, my suggestion would be to follow the example of iBooks Author, not Indesign. Start off with theme-based development. Add tools to extend, modify, and adjust the theme. Target only one format or even just one platform and a limited range of devices.

4. Developers

In this world there wouldn’t be any ebook developers any more than you have .doc developers.

The fact that we have an industry of people whose job description is close to indistinguishable from ‘fixes office documents by hand in a hex editor’ is insane.

The only developers the ebook industry should have are tool developers, people who program and make the writing programs that export the ebooks, theming apps that add themes to ebooks, and the design tools for the edge cases.

Even theme design should be done in close collaboration with the app developers and, as such belongs to their sphere of influence.

This crowd is the one that would make everything happen. Their work is what enables the rest. As such they need extraordinary support from ereader vendors, retailers, and publishers.

Tool developers need:

Documentation. Lots and lots of it and in detail. If you’re going to make an app that generates a file that’s supposed to work on all platforms, you need those platforms to document exactly what they support. Exactly. Inspection tools. You need something akin to the web inspectors browser vendors offer to figure out what went wrong when it does go wrong. Otherwise you’re working blind. You need debuggers if you’re supporting javascript and inspectors if you’re supporting CSS and markup. Automation. Iteratively adjusting the ebook generation code, compiling, exporting, transferring to the ereader, paging, paging, paging to the right chapter to see if you fixed that goddamn bug is a reliable why to burn out a developer. You need a way to automate this process. You should be able to automate anything you do repeatedly. Transparency. At least a little bit. Developers need to know in advance about some of the things platform owners are planning.

This is a group of people who few and far between and you want them all to be on your side creating ebook tools (and not developing iPhone games, for example).

You don’t want to drive them away by having no documentation, a schizophrenic platform that undermines every attempt at design, no transparency, no automation, and no developer tools to speak of.

Which is what you’re doing today.