Category Archives: Writing

Charting the Amazon Sci-Fi jungle

Probably one of the more inspiring books I’ve read lately is Chris Fox’s Write to Market – it’s practical, grounded advice for building a writing career in today’s landscape. The principles contained within are solid, the first being: find an under-served market you can target your efforts on.

It makes complete sense from a supply/demand point of view – if you know ahead of time what readers are interested in buying, and they happen to align with what you enjoy writing, you can build a much clearer picture of what you’ll need to do to succeed. Modern content production has changed, after all.

The book got me thinking about how I might apply it to my own ambitions, and it became pretty clear that I’d have to take a very detailed look at the Sci-Fi book market on Kindle. Amazon accounts for a solid portion of global eBook sales, and should serve as a fantastic indicator for what’s trending.

So last night, I worked on exactly that – first, acquiring a snapshot view of the Top 100 books in each of the 21 sub-genres of Science Fiction, and how they relate to the global sales rank. I’ve got some information to share there, as well as some related insights on the composition of the market.

The Theory

The theory is relatively simple: Amazon lists over five million eBooks on Kindle (depending on what numbers you trust, I guess), and they’re all globally ranked on what Amazon calls their Best Seller rank (I call it ABS for short).

One book can exist in multiple categories – it can have a rank in the niche it serves (for instance, Science Fiction about Genetic Engineering), as well as a global ABS rank. The relation between these tell you how active a niche is.

For instance, if the top 5 books in a niche also exist on the top 10 ABS list, it means there’s a large amount of demand there. If books #80-100 in that same niche are in the high-thousands, that indicates under-served demand: People are buying books in that niche, but for whatever reason are not spending money on some of the lower-ranked books currently available.

This is the fertile ground – you know you have people heavily interested in a particular niche, and they are likely ready to buy anything new and interesting that might land in that category.

If the top 5 books in a niche are in the high-thousands, that means there’s very little demand for that niche. But if all Top 100 books in a niche appear in the top 500 ABS rank, that’s most likely an impenetrable market – and a wildly popular niche.

For the time being, anyway – the ground here shifts constantly as reader tastes evolve. Just like fashion, trends come and go. So despite all the charts I’m showing you in this post, they could be out of date as little as six months from now.

The Niches in Sci-Fi

I’m interested primarily in Sci-Fi, and so focused my analysis there. Things might look different in the other markets, but since I’m not likely to get into Suspense or Young Adult any time soon, I figured I’d give those a miss 😉

Amazon lists 21 niches (or sub-genres) under Sci-Fi:

Adventure Alien Invasion Alternative History
Anthologies & Short Stories Classics Colonization
Cyberpunk Dystopian First Contact
Galactic Empire Genetic Engineering Hard Science Fiction
LGBT Metaphysical & Visionary Military
Post-Apocalyptic Space Exploration Space Opera
Steampunk TV, Movie, Video Game Adaptations Time Travel

For each one, I set about gathering specific data:

  • The list of top 100 books in that niche, based on the niche’s own performance
  • For each book, what the global ABS rank is, and who the merchant is
  • Timestamped for once-a-day retrieval

 

Throw them all together in a chart, and you end up with something like this:

PBIDesktop_2017-07-03_20-59-51.png

Enlightening, right? Let’s rather go by genre, starting with the most hotly-contested one right now – Adventure:

PBIDesktop_2017-07-03_21-14-05.png

This is the dashboard of a very healthy sub-genre.

The top 20 books all have ABS ranks below 1000, with the top 5 being below 100 – these books are selling very well, and there is clear demand for this sub-genre right now. The market is also being very well served at the moment – none of the ABS ranks are above 10’000, so it’s unlikely that a first-time author, or someone without major existing traction, will be able to break in here right now.

Now let’s look at a less-contested genre – Hard Science Fiction.

PBIDesktop_2017-07-03_21-19-52.png

This is more like it! The Top 20 books are all under the 2000 ABS rank, and the book sitting at #40 is double that. The category bottoms out at over 12K, so if you’re looking for a place to start, this could be a good sub-genre to do it in.

Finally, the most uncontested sub-genre at the moment – LGBT.

PBIDesktop_2017-07-03_21-22-34.png

There are no official numbers for this, but the #1 book being at ABS rank 1973 would suggest that it’s selling around 100 copies a day. By comparison, the #1 book in Adventure should be doing around 6000 copies/day. This is according to TCK Publishing’s calculator.

100 copies/day on the top end is not much in terms of demand, so while you could almost definitely rank in this sub-genre, it probably won’t be worth the time investment right now.

All the charts above are looking at the total market though, regardless of whether or not titles were independently published. Let’s get into that next.

Independent publishing on Amazon

I published first versions of these on the Dragon Writers group, but now that I have updated information and time to properly process it, here’s a snapshot of how the Sci-Fi genres break down as of today.

PBIDesktop_2017-07-03_21-37-53.png

The vast majority of Kindle titles in the Sci-Fi genre are independently published – “Amazon Digital Services LLC” is the business name used there.

A word of warning on this: That same business name is used by Amazon itself on occasion – so far I’ve seen it used for special store listings of old, republished books. Unfortunately that’s just the nature of a project like this – the data is not going to be 100% accurate.

Other than the LLC, there are a few big names in this space, but they account for very few of the titles published.

But then there’s the quality-vs-quantity argument. Are independently-published novels doing better (or worse) than those published by traditional houses?

This one’s a tricky question to answer, so it’ll help to look at it in parts.

Let’s go with all titles under Sci-Fi with an ABS rank of 1000 or higher. At 1000, you’re selling around 185 books/day – it’s an arbitrary number, sure, but we need to start somewhere.

For each of the sub-genres that have books in that ABS range, what proportions were published independently vs traditionally?

PBIDesktop_2017-07-03_21-59-05.png

It’s no surprise that Traditional is dominating the Classics sub-genre – since that’s literally the genre in which Traditional companies re-package existing traditionally-published books.

But look at the rest – entire sub-genres are being dominated by independently-published titles! This is the encouraging part – on the largest eBook retail platform in the world, it’s possible for independently-published authors to dominate entire sub-genres.

What does the top end look like? Let’s take the top 100 books across all Sci-Fi sub-genres, sorted by ABS rank. The #1 Sci-Fi book is ranked 4th on the Best Seller list, and the #100th book comes in at rank 1429.

PBIDesktop_2017-07-03_23-30-26.png

That’s the most encouraging chart I’ve produced yet. Across the top 100 titles at the moment, 88 are independent titles – but more than that, there’s no clear bias attributable to the publishing method.

Or in other words: It doesn’t matter if you’re independently or traditionally published – both methods have a chance of reaching the top, and ultimately reaching customers.

Conclusions

None of the data above looks at sales or revenue – a lot is being inferred by the limited ranking information that Amazon makes available. For the most comprehensive report that actually looks at sales, AuthorEarnings is the best place to go.

The intention of this post wasn’t to dive into the industry as a whole, but rather to illustrate two things:

  1. There is opportunity here, possibly more so than via traditional publishing channels. The markets are wide-open to new entrants, and the opportunities might change over time, but they are always there.
  2. In the eBook space, it doesn’t matter whether you were published by a big name, or under your own name – both books will have equal treatment, and customers end up making the choices.

Publishing is definitely changing, and I’m excited to see where it goes next.

Project: write500.net

Check it out: https://write500.net

Whenever I’m not deliberately concentrating on a given task, my mind tends to wander in a very specific way: It tells me stories. All day, every day – characters, worlds, twists, inventing and reinventing themselves.

One of my perpetual New Year’s Resolutions (since roughly 2008) is to write more – almost entirely to pin some of those stories down on paper, and if I turn out to be any good at it, polish one or two up and publish them.

Good theory – much harder in practice.

A writer is someone for whom writing is more difficult than it is for other people.

~Thomas Mann

Writing, I’ve learned, is every bit a skill as software development. Sure: anyone can read someone else’s work, see the logic in it, and gain the (false) confidence to create something like it – but the moment the pen hits the paper, that evaporates completely, leaving you facing the sober reality that, actually, you have no experience.

This has been my problem for the last few years, and I suspect it’s not unique. Any writer who’s read more than half a writing craft book should know that writing every day is one of the critical components – that you need the ability to produce sheer quantity, before you can start obsessing about the quality.

So with the new year coming, I had the idea of setting up a system for 2017 to help me exercise that muscle. I know I can write 500 words pretty easily – this blog post, written on the spur of the moment, is 586 words – and so long as I have some sort of guidance as to what to write, I shouldn’t find it difficult at all.

500 words per day, for 365 days, is over 180’000 words. Sure, they’re not all congruent words, and I have no hope of getting a novel out of it – but if I can manage it, I’d have written a novel’s worth of words, and I’d have built up routine, momentum, and (hopefully) a bit of confidence in my ability.

On that theory, I grabbed a domain, and started building a system to deliver me a writing prompt every morning at 9am. I figured the workflow would be no different from managing my inbox – I get an email, I respond to it, and I carry on with my day. And if I can do that every day (and let’s be realistic, we spend way too much time on email anyway), then I could start developing a writing habit.

About a minute after that thought, it clicked that other people might also benefit from a system like this, so I’ve spent the last few days producing a polished version I could share. It still needs a ton of behind the scenes work, but I’ve got time over the next 10 days, and I intend on hitting the ground running on 1 January.

That system: write500.net

For now, it’s basically just a mailing list. I’m working on a batch of thematic writing prompts (not just the random nonsense you find via Google), and if I can finish this off as intended, I’ve got some other feature ideas to throw in. But right now, I shouldn’t get distracted 😉

(Interestingly enough, while Mailchimp (the list provider there) does have an automation system, setting up a chain of 365 emails would push it to its limit, so I’m working up a completely custom system, using Amazon SES and my own list management. I might do a write-up on this at some point, assuming I can get it all off the ground!)

Could have been a prophet

Back in 2013 I started learning about the “filter bubble” – a natural result of the behavior- and preference-driven algorithms that power major search engines. Between your search history, the links you click on, and the sites you visit that are tagged with Google Analytics (which is a lot of websites nowadays), search engines like Google can make a reasonable approximation of what results might interest you.

There’s nothing inherently nefarious about this. Google’s interest is in getting you to the right website as quickly as possible, and they’ve done a phenomenal job at it. The better results you get from Google, the more you use them – which means you’re more likely to click the ads that get served alongside search results.

The problem, though, is that it sacrifices diversity of ideas a bit. If you’re a habitual Fox News reader, you follow Donald Trump, and routinely watch his speeches on YouTube, the next time you search something like “abortion” or “gun rights”, the filter bubble will give you results from sites it thinks you want to see, and you’ll get a very right-wing view of the situation.

The search engine will dutifully give you the information it thinks you want to see, but may not give you the information you need, and that’s where the problem comes in. So if you’re in a really bad situation (say, unwanted pregnancy in a conservative and oppressive religious family), and you need level-headed information on whether abortions are safe, legal, and where to get them, Google won’t know to give you that.

That’s the inadvertent side-effect of the filter bubble, and it got me thinking – what would happen if it were made deliberate? A corporation with that much power could, theoretically, start deliberately adjusting their algorithms to subtly affect the worldview of the people using their service.

Facebook’s hilarious miscarriage aside, Google is now doing exactly this. Wired Magazine reports that a Google subsidiary is going to deliberately attempt to feed misinformation to potential ISIS recruits.

Much has been written about cyber warfare, and what it might look like – hackers, viruses, trojans, groups of dangerous people taking down power plants and military bases. Much less has been written about the more insidious form of information warfare that’s crept up on us over the last few years, and practically nothing about calling large search engines to account.

Today, Jigsaw (/Google) is trying to identify potential ISIS recruits, and change the results they get to feed them anti-propaganda, to dissuade them from signing up. Maybe it won’t work – I imagine that most recruiting is done peer-to-peer in any case – but maybe it will.

And if it does work, it sets a very worrying precedent. Up until this point, it’s been in Google’s best interests to vacuum up as much of the Internet as possible, and optimize it relentlessly to get you where you’re going. But what if Google decides that, for whatever reason, they’re a national security asset now, and they have a responsibility to tailor search results away from dangerous ideas?

That’s a slippery slope of note, because it opens the door for people to start redefining what those dangerous ideas are. To any reasonable person, a dangerous idea is one that could result in physical harm or a loss of property.

To a militaristic dictatorship, a dangerous idea is any one that can teach the common man to arm and defend themselves. In a police state, a dangerous idea is one that reminds people of their rights under their respective laws. In a communist dictatorship, a dangerous idea is that people are entitled to the fruits of their own labor, and that being constantly stripped of your wealth is not the best way to run a country.

Anything that upsets the balance of power could be considered dangerous, whether or not that power is being wielded fairly or equitably. And with the sheer amount of power we’re giving search engines over our lives, I think it’s worth asking whether or not we’re actually being shown a fair representation of ideas, not just the ones that are deemed “acceptable”.

In the past, the news media has always been the gatekeepers of that, and have been rightly criticized for withholding information that was of vital public interest. The internet has always acted as a bulwark against that, creating a forum where all speech is equal. And now it seems we’re slowly sliding back towards a world where there are gatekeepers, and fringe speech is marginalized at the behest of the powerful.

So anyway, the moral of the story here is that I regret not writing the short story I had in mind in 2013. It was a story that dealt more or less exactly with this – what would it look like if companies could start shaping information that we thought was being ranked on technical merit alone. Would people even notice that their ideas were being deliberately adjusted on a network they thought was free and open? Who would line up to pull the strings, to use information as the next theater for cyber warfare?

Had I written that then, it would have been topical now. Next time I’ll have to do better.

System Shock 2 theme for Q10

So I’m trying to get back into writing more fiction work, and I’ve found that the actual writing environment has a big impact on my productivity. When the look, feel, and sound of a tool matches the tone of the writing I’m working on, it gets so much easier to focus.

One of my favorite games of all time is System Shock 2, a first-person cyberpunk thriller RPG. It was released in the late 1990’s, was way ahead of it’s time, and occasionally I’ll play through it again just to immerse myself in the wonderful story-driven world it creates. So naturally, when I’m working on SF-heavy writing, a SF environment helps 🙂

To set that up, I used the following:

The sound files simply need to be extracted to the same folder where Q10.exe lives. The fonts can be set via the preferences pane inside Q10 (Ctrl+P).