It’s just about GDPR time!

Leave a reply

(Image credit: Flickr)

I still remember back when the Internet was an escape from the real world. Today, it seems that the real world is an escape from the Internet. Over the last 30 years, a substantial amount of daily life has moved online for billions of people: News, communication, sharing, commerce, banking, and entertainment.

Wherever human activity goes, regulation inevitably follows. This year it seems like the internet is truly going mainstream, with new regulations from 2 of the 3 largest economies in the world: the US’ FOSTA, and Europe’s GDPR.

I’m actually quite a fan of GDPR – or at least, what it’s trying to do. In GDPR, there’s an attempt to extend protection of EU citizens rights across the Internet, following their personal data wherever it may end up. And the regulations make a lot of sense from a consumer perspective: If companies want your personal data, they have to prove they can handle it responsibly, and you retain basically all the rights.

GDPR goes “live” on the 25th of May (just under 3 weeks!), and I’m hoping it heralds a new era of more responsible data practices. I think everything they’re trying to do is achievable, and should be baked into organizational and system design for new businesses (“privacy-first architecture”).

For existing businesses though, it’s going to be hell of a slog. I’ve worked on data audits and compliance for some of the world’s biggest brands, and it’s impossible to overstate how tricky full compliance is going to be. When your database software vendors have gone defunct, when your processes have spiraled out into spreadsheet nightmares, when your primary method of data exchange is email attachments, you’ve got a problem.

Over the last few days I’ve put some stuff live to help with that. For one, there’s already a wave of companies that are blocking the EU from accessing their services outright. It’s a sensible short-term move if you need time to assess the impact, and update your processes to comply. The rights of Erasure and Portability could require non-trivial work if you’ve been historically lax with how you manage data internally.

For that use-case, I pulled together a repo (gdpr-blackhole) of the IP ranges (IPv4 and IPv6) of all 28 EU member states, UK included. Brexit or not, early indications are that the UK will adopt GDPR in its entirety.

Then, since I work mainly in Laravel, I wrapped that up in a simple Middleware class that makes blocking EU IPs straightforward (laravel-gdpr-blocker).

Finally, I’ve just put a short blog post live on Amberstone: “What small businesses need to know about GDPR“. Large businesses already have armies of lawyers, auditors, and officers to assess their liability. Small businesses are not exempt, and if you want to participate in the second-largest economy on the planet, you’re going to need to know a bit about what GDPR asks of you.

Overall, it’s an exciting change. The ICO has already indicated that they’d sooner work with non-compliant organizations to improve their stewardship of personal data, making the eye-watering fines a last resort. And if all goes well, this is exactly the sort of bar-raising work we’ve needed on the world stage.

And with any luck, it means less Excel spreadsheets loaded with personal data. Time will tell 🙂

Oh boy, where to begin.

Leave a reply

On the 5th of May, 2008, I started my first day at work.

I doubt I’ll ever forget it: To beat the traffic, I carpooled with an intern from a nearby law firm, arriving in Cape Town at around 6am. I had to wander around the darkness of Kloof Street for a while, trying to keep busy (and warm) until someone showed up to let me in.

It was a surreal morning for me. The preceding family drama is complicated, but the summary of it was that I had absolutely no idea what I was going to do with my life at that point. I ended up on a bus from Pretoria to Cape Town the week before Christmas 2007, and lived with my father for a few (very strained) months before I got any semblance of an act together.

I was nervous as hell. At the time, I didn’t actually think I’d land the job. Luckily I had some experience building websites, and between a meager portfolio and an old friend’s insistence on listing at CareerJunction, a recruiter found me.

I failed my first interview. It still amuses me to know that the person who decided not to hire me back then still works at the company, and we’d ended up working together on a few things over the years. Sometimes I wonder if she regrets not having hired me, but I suspect I wouldn’t have lasted in her team anyway.

Luckily, I passed the second interview – between the recruiter and the HR director, they thought I might have some potential. In the end, I landed in the Paid Search team.

While my dress code (on the whole) has been very informal over the last 10 years, I dressed for the occasion on my first day. And it was while wandering around Kloof Street that the outsole on my right shoe became partially unstuck, making an embarrassing noise every time I walked.

Nervous, overdressed, surrounded by people I didn’t know in a city I’d never lived in, praying that my shoe didn’t completely come apart before I could get home. That was my introduction to the company that would carry me through the next ten years.

I get a weird look when I tell people my first job’s lasted this long. It doesn’t feel that way with all the roles I’ve held since 2008. I’ve done pretty much everything there is to do in digital marketing – search, email, display, analytics, consulting, architecture, compliance, project management, team leadership, training, and staying on top of the never-ending waves of technological and social progress.

Right now it feels more like I’m graduating from one of the most arduous post-secondary education experiences imaginable. I’ve had hundreds of hours of theory and thousands of hours of practice. If Gladwell’s Outliers is to be believed, I’ve sunk the requisite 10’000 hours required to achieve mastery in digital marketing. And then some.

And I’ve traveled. My god, have I traveled.

DbYkZptVQAIgrkH.jpg

Every airport I’ve flushed a toilet in.

I’ve been as far east as Phoenix, Arizona (connecting flight on a return trip from Salt Lake City), and as far west as Melbourne (my first on-site development/consulting gig). It’s been a privilege to see so many different parts of the world, and the exhaustion of business travel has thoroughly disabused my notions of the glamorous lifestyle I once thought it was.

The world has changed. 2008 was a different time: Facebook was only 4 years old, Twitter was a toy that gained some media traction during Barack Obama’s campaign. The iPhone 1 had been released just last year.

opera_2018-04-27_22-11-26.png

The top story that week – food riots in Somalia. Today, there’s food riots in South Africa.

And I’ve changed. In future, I might write more openly about the life I came from, the demons I’ve wrestled with over the years, and the moments of breakthrough that have set me free. For now, it’s enough to acknowledge that 2018 Wogan is a far cry from 2008 Wogan, and I’m grateful for every bit of progress in between.

Today’s my last day. It’s a mixed feeling – strange, to think that I’m moving on after so many years; a relief, knowing I’ve reached the end of this road; anticipation of what the future might bring, and the confidence that comes with real-world experience.

At last, it’s time to move on. I don’t know for sure what the next ten years are going to look like, but I’m eager to find out!

Subscription payments in South Africa

Leave a reply

So you’re a South African business that wants to take payments for a subscription-based service. You’re probably going to have a mix of local and international customers, and want to know what services are available.

Well, I’ve got some good news, and some bad news.

The good news is – there are ways of doing this. Even with South Africa’s relative backwardness in global economic participation there are still a few options.

The bad news is: the best options are currently unavailable to you, and are likely to stay unavailable in the long term. If you have the means to do so, incorporating and setting up banking in a major economy might be the better option.

Onwards!

Local Options – EFT

It goes without saying that local EFT is an option. If you don’t want to wait for interbank delays, there are a few vendors that offer Instant EFT solutions:

In my experience, PayFast has the easier onboarding path, but you might have a tougher time integrating it into your application. None of those solutions incorporate any sort of subscription management though – it’s on you to keep the accounts in check.

Depending on your capabilities, debit orders might be an option. If you can obtain debit order mandates from your customers (signed papers, recorded calls, etc), you can use Sage Pay’s NAEDO collection system: https://sagepay.co.za/services/debit-order-collection/

Local Options – Credit Cards

This is where it gets a bit more interesting!

PayFast allows you to accept credit cards online, and is relatively easy to set up if you’re not looking for advanced integration. They will make your life easier on one front – they understand subscription management: https://www.payfast.co.za/subscriptions/

I’ve worked with their API before though, and you’re going to need to exercise extreme patience with it in order to get anything done. I hope their systems and documentation improve over time!

PayGate also offers a subscription product, but they make it obvious that they’re geared towards larger businesses – you have to start the process with a sales inquiry: https://www.paygate.co.za/paygate-products/paysubs/

Global Options

There are too many global vendors to list, but when it comes to what you can feasibly use in South Africa, it narrows down to one pretty quickly: PayPal.

I implemented full-on subscription management using PayPal for Write500 – and it worked quite well. PayPal can accept credit cards, manage subscriptions (including pausing and resuming), and has a solid set of developer tools for integrating it into your application.

You’ll want to create a Billing Plan (your product), then a Billing Agreement (a paid subscription to it), so that Paypal can issue Invoices and settle them automatically. Start here: https://developer.paypal.com/docs/api

The one limitation: You need an FNB account to receive any of that money here in SA.

Going further abroad, you do have the option of incorporating a business remotely. It’s a ton of paperwork (depending on a ton of factors), but one vendor is offering a simple solution to it: Stripe Atlas.

For a one-time fee of $500, and a decent amount of effort, you can incorporate remotely in Delaware – including all the documentation and registration you need to run a corporation within the US. That includes a Stripe.com account, which is basically the global, golden standard for subscription credit card billing.

There’s a couple of major downsides on the administrative side, though – repatriating that money comes with its own set of tax challenges. From what I’ve seen, this is only a really viable option if you’re expecting to do a lot of subscription billing for global customers, and the Stripe fees are a more attractive option.

Other Options

I would be remiss if I didn’t mention PeachPayments – a Cape Town-based company that attempts to make all of the above easier for local companies. If you’re not keen on the idea of getting your hands dirty with integration logic and setting up subscription billing, those will be the folks to talk to.

Then of course, there’s bitcoin. If you really feel like jumping into the murky waters of cryptocurrency for your project, give Coinbase a try. They offer a recurring billing option denominated in Bitcoin. To cash those coins out locally, the simplest option will probably be to hold a Luno.com wallet, and sell those coins on the local exchange to recover your Rands.

I know that there are likely several other services out there, but if I were starting from scratch today, and wanted to be able to accept payments from a global customer base I’d probably still go with PayPal, myself.

It’s time to talk about charts

Leave a reply

Have you ever felt annoyed that someone tried using a world map chart to visualize country-level data?

Or is that just me?

(It’s probably just me.)

Over the last few weeks I’ve been picking up more books to read (as part of my drive to write more), one of them being Content Inc – recommended to me as a good introduction to content marketing, and how powerful it can be.

The book vacillates between content production at the individual level, and the corporate level. Some of the stories focus on single-person startups, and how they tested ideas and built businesses off the back of content production. The rest of it, haphazardly, deals with how to maintain that within a larger organization (team structures, responsibilities, and so on).

What struck me about the individual stories though was the relative simplicity of the focus. One person wrote about writing – another, about real estate. A third simply wrote about how to get more value out of your camera. All of those, over time, became profitable businesses – the key ingredients being effort, and no small amount of passion for the target subject.

It got me thinking about an idea I had years ago, when first starting to work with Domo. Without going into too much detail, one of the things that intuitively clicked for me during the first few weeks was the brightline relationship between business management, and data visualization.

Borderline-buzzword sentence, I know.

The practice seemed to hit at the intersection of a few of my interest areas – complex systems, data and numbers, and visual communication – and it wasn’t very long before I was already planning out an enormous series of content on how to get the best value out of different data visualization options.

That content never materialized. I had put it on my internal roadmap; to develop “added value” in the form of training content that our consultants could use to help plan best-practice dashboards.

In the end, Domo themselves reached a new level of maturity on their operating models, and that filtered through to the training we got. For that (and quite a few other reasons) that content was never built.

Reading Content Inc made me dust that idea off again. I know for a fact that I can produce useful, actionable content on this topic, having done it before. I’ve also learned, somewhat accidentally, that this is a passion of mine.

In retrospect it might be obvious, but the revelation really came to me on a recent customer project. We were planning out a series of dashboards, and someone wanted to include a world map chart where there didn’t need to be one. That led to a long (and I want to use the word “vibrant”) discussion on whether or not we should include it.

Afterwards, reflecting on that conversation, I realized how deeply I had internalized the principles I had been learning since 2013 – and how naturally they seemed to fit in with the rest of my thinking.

So between that, and my desire to write and publish content more frequently, I’ve decided to take a stab at maintaining a data visualization blog, with a specific focus on practicality: There are amazing interactive visualizations out there (Jer Thorp in particular will always be in my pantheon of data deities), but most of the visualizations we use in daily life are much more basic.

Software has, I think, tricked too many people into thinking charts are easy. I’ve seen so many presentations, Excel workbooks, and “professional”-level reporting that ends up being hard to get any sort of good understanding from.

Simple rule: If your chart is accompanied by a “how to read this chart” helper, you haven’t built a good chart.

And I think this might be the thing that I tackle next: A bit of theory, but mostly practical advice on how to construct good charts. And there are a lot of scenarios to consider – more than enough to build a solid resource for the “everyman” visualization work.

So I’ll be building out plans and content for this over the next few weeks, and hope to launch a new site before the year is out. If there’s one thing I’ve learned so far, is that good data visualization is timeless.

1786_Playfair_-_Exports_and_Imports_of_Scotland_to_and_from_different_parts_for_one_Year_from_Christmas_1780_to_Christmas_1781.jpg

In particular, we’ve had stacked bar charts since as far back as 1780.

Once it goes live, I’ll be posting about the site here. If you want to be alerted when that happens, consider subscribing to my blog – widget’s on the top right.

Rise of the Machines

Leave a reply

Right now, we’re living in one of the most momentous times in human history, and it could end up being one of the best (or, possibly, worst) things to unfold: our inevitable transition to what Maurice Conti calls the Augmented Age.

Computers have become part of mainstream life in every advanced economy, and basically all major cities around the world (into which people are packing in ever-greater numbers). The resulting efficiency gains have either been a huge boost to creativity and opportunity, or the death-knell of industries that employ tens of millions of people.

I’d like to share two different perspectives on this – both, conveniently, delivered as excellent TED talks. The first is by Maurice Conti, on how advances in computing have changed the way design could be done.

The most remarkable thing about the computer-derived inventions is how biological they look. It took nature millions of years to evolve a structure that their computers can do inside of a few days (referring to the drone chassis), and in future, could do on demand.

I think this is the best insight into how the leading edge of computing might change the way we design cities, vehicles, infrastructure, and the machines that help run our lives. It’s encouraging to note that human designers are still very much a part of the process, but will be able to do a lot more in a lot less time.

Which is a factor leading into the next TED talk – what happens when you centralize that amount of power (and consequently, the financial gains) in the hands of a relative few? People who are skilled at these technologies are able to create enormous value in a short space of time, relative to someone still doing the same task manually.

So what happens when you no longer have a need for the manual labor?

Another excellent talk that takes an unbiased view of Unconditional (I prefer Universal) Basic Income. It raises some good points, but misses at least one point I need to make a note of:

While it’s true that the top 5 tech companies are enormously valuable and employ relatively few people, the platforms they create have in turn generated opportunities for millions more. There are companies, products, services and entertainment channels that could not have existed were it not for the infrastructure and tools that Facebook and the like provide.

Google basically pulled the web development industry up out of the ground when it became clear to businesses that having a well-built site was a competitive advantage. I’m not sure anyone can count the amount of new jobs created in web development, creative design, copywriting, SEO optimization, consulting and education as a result of the platform Google built.

(Yes, I know Google didn’t build the internet. And yes, I know all these websites run on the internet that Google didn’t build, but everyone who’s ever been paid to build one has done so at the request of a customer who believed that being discoverable online would be beneficial to their business, and Google is still the king of discovery on the internet.)

Same goes for the use-cases enabled by Apple hardware, Facebook’s networking, Amazon’s fulfillment infrastructure, and the productivity tools released by Microsoft. Those companies themselves may employ relatively few, but they have empowered millions more.

Moving on.

I think UBI is feasible not so much because of productivity gains due to automation, but because of the ever-declining costs of providing an acceptable standard of living. An excellent, recent example of this is Apis Cor’s house printer.

On the one hand: This technology might end up putting a lot of construction workers out of jobs. While you’ll still need workers for big buildings and the like, simple 1-2 person houses can probably be built quickly, and very cheaply, as a result of this innovation.

But on the flip-side, the cost of houses will plummet. You may not need to work for 20 years to pay off a mortgage for a house that only costs $10k to build. While construction workers might be worried about this, the people who should be a lot more worried are ones with heavy investments in residential development companies 😉

I like to imagine a future unconstrained by urbanization. Cities are where the opportunities are – the best jobs are in cities, the best entertainment, the best healthcare, and overall, the best opportunities to live a good life. This is because it’s a lot easier, with the current limitations, to pile a lot of services into one place.

I don’t believe civilization needs to be so centralized, though. If you could get the same quality of food, healthcare, entertainment and job opportunity in an area 200km outside a major city, plus it was cheaper to live there – wouldn’t you?

And there may come a time when we have to. Most major cities (and by extension, most of the world’s population) are located relatively close to a coastline. Historically, cities were founded and grew near coastlines because those afforded the best opportunities for global trade.

Well, that’s under threat. Depending on who you believe, climate change is either a myth, or it’s a reality already underway – and one of the most dire consequences will be the rise of the ocean level. Which, if that happens, will start to make the large, coastal cities unlivable.

We will be forced to start again – massive inland migrations, the design of new cities, infrastructure and services to support the population, while simultaneously ensuring people have a shot at an acceptable standard of living. With the lessons we’re learning today, I imagine those cities (and societies) will look very different.

Between the work of engineers like Maurice and researchers like Federico, I’m optimistic that we’ll be well-equipped to meet those challenges in future.

The Rooms Are Made Of Death

Leave a reply

Many years ago, in a Game far far away, I had a bit of cash to spend (and absolutely no idea what to spend it on), so I wandered over to their discount bargain DVD bin and started poking through it. As I recall, I bought exactly 3 DVDs that day. Two of them are forever lost to me – because that was the day I discovered the Cube universe.

cubezero.jpg

Seriously, what teenager sees a cover like this and ISN’T curious?

At this point I think I’ve seen just about every trope there is, retold in hundreds of different ways. I have yet to see Fast and Furious: Middle Ages, with horse-drifting and choirs of singing inn-wenches though, but that’s probably something Hollywood would do if given the budget.

What I haven’t seen though, is something so beautifully executed as Cube. And I do mean “executed” literally – Cube Zero opens with a man getting sprayed with acid, then melting into a pile of bones right on screen while two remote operators watch it happen.

The idea behind it is fascinating to me. The world of Cube just exists. Three movies have been made, with no apparent connection between them, other than the existence of a giant cube that kills people.

Usually, when three movies are made about the same thing, the writers will strive for some semblance of continuity – world-building, rationalization, or something thematic to tie them together.

I know I’m guilty of this – whenever I think of a scene, character or technology that would be awesome to write about, I find myself stuck on trying to rationalize exactly how or why such a thing would exist.

But not Cube. The first movie was released in 1997, and apparently had no direction other than “hey what if six people woke up in a cube full of cube-shaped rooms, and some rooms have deadly traps in them”.

And that’s the entire movie – six characters, speculating about why they’re here, with no answers or resolution in sight, slowly going crazy and eventually trying to kill one another off.

Cube offered no answers, no rationalization, no explanation. The characters existed, were filled out somewhat, and were then killed off in various brutal ways, with only one (the weakest in the group) apparently managing to escape the cube – but even his fate is unclear, and the movie ends with no resolution.

A few years later it was followed up with Cube 2: Hypercube – a movie that succeeded in answering absolutely nothing about its predecessor. Same basic story: A bunch of strangers, stuck in a cube with cube-shaped rooms, and some of the rooms have traps in them.

Except: this was 2002, and the cube existed as a theoretical experiment (Military? Corporate? Who knows?) – a confusing mix of looping time, folding space, parallel realities, and the perfect end to any of the three movies: An ending that answers absolutely nothing.

It turns out that one of the people in the cube was actually working for some shadowy organization, and had to track down someone else inside the cube, even though this organization is supposedly the one that put everyone in the cube in the first place.

This person had some sort of storage device on them unlike anything else in the world, which was on them when they were kidnapped. At the end, it turns out that the experiment itself has an expiration date, and through a confusing sequence of events the last surviving character reappears in the real world, hands over the recording device, and is promptly shot in the head.

Perfection.

What was the experiment? Why were these people chosen? What was on the recording device? Why shoot your own agent? How do you even create something like this hypercube? No answers.

The latest movie, and the one I watched first, was the prequel – Cube Zero. It’s set before the events of the first Cube, and attempts to offer some sort of rationalization for why the cube exists – it’s either a highly-unethical experiment, or a government program, or some sort of secret military project.

Or it’s a public works project gone wrong, or it’s an elaborate execution facility, or something else entirely.

For no reason at all, random people are brought to this facility, given no instructions, and are left to figure out the rules before they all die (and they inevitably all die).

And this includes the people in charge of the place – the wardens are prisoners, and the people on the higher floors (to which you must never go) have some sort of threat hanging over them too. It’s never ultimately clear who’s in charge, or what’s actually being accomplished here.

Which makes it one of my favorite movies of all time. Nobody in the movie seems to know what’s going on, and the only person who does seem to know is either a plant, or crazy.

It’s a perfect, nightmarish machine – the love child of malice and incompetence – efficient, brutal and fundamentally irrational.

Despite all of that (or maybe because of it), on some level, it just works – it’s captivating to watch, it raises more questions than answers, and leaves the viewer to fill in the blanks. As compared to most Hollywood movies nowadays, which trip over themselves to explain what’s going on, Cube is a rare find.

If you enjoy movies that don’t have clear endings, raise questions about human morality, and leave the reasons for their existence wide open to speculation, you’ll enjoy these.

Charting the Amazon Sci-Fi jungle

Leave a reply

Probably one of the more inspiring books I’ve read lately is Chris Fox’s Write to Market – it’s practical, grounded advice for building a writing career in today’s landscape. The principles contained within are solid, the first being: find an under-served market you can target your efforts on.

It makes complete sense from a supply/demand point of view – if you know ahead of time what readers are interested in buying, and they happen to align with what you enjoy writing, you can build a much clearer picture of what you’ll need to do to succeed. Modern content production has changed, after all.

The book got me thinking about how I might apply it to my own ambitions, and it became pretty clear that I’d have to take a very detailed look at the Sci-Fi book market on Kindle. Amazon accounts for a solid portion of global eBook sales, and should serve as a fantastic indicator for what’s trending.

So last night, I worked on exactly that – first, acquiring a snapshot view of the Top 100 books in each of the 21 sub-genres of Science Fiction, and how they relate to the global sales rank. I’ve got some information to share there, as well as some related insights on the composition of the market.

The Theory

The theory is relatively simple: Amazon lists over five million eBooks on Kindle (depending on what numbers you trust, I guess), and they’re all globally ranked on what Amazon calls their Best Seller rank (I call it ABS for short).

One book can exist in multiple categories – it can have a rank in the niche it serves (for instance, Science Fiction about Genetic Engineering), as well as a global ABS rank. The relation between these tell you how active a niche is.

For instance, if the top 5 books in a niche also exist on the top 10 ABS list, it means there’s a large amount of demand there. If books #80-100 in that same niche are in the high-thousands, that indicates under-served demand: People are buying books in that niche, but for whatever reason are not spending money on some of the lower-ranked books currently available.

This is the fertile ground – you know you have people heavily interested in a particular niche, and they are likely ready to buy anything new and interesting that might land in that category.

If the top 5 books in a niche are in the high-thousands, that means there’s very little demand for that niche. But if all Top 100 books in a niche appear in the top 500 ABS rank, that’s most likely an impenetrable market – and a wildly popular niche.

For the time being, anyway – the ground here shifts constantly as reader tastes evolve. Just like fashion, trends come and go. So despite all the charts I’m showing you in this post, they could be out of date as little as six months from now.

The Niches in Sci-Fi

I’m interested primarily in Sci-Fi, and so focused my analysis there. Things might look different in the other markets, but since I’m not likely to get into Suspense or Young Adult any time soon, I figured I’d give those a miss 😉

Amazon lists 21 niches (or sub-genres) under Sci-Fi:

Adventure Alien Invasion Alternative History
Anthologies & Short Stories Classics Colonization
Cyberpunk Dystopian First Contact
Galactic Empire Genetic Engineering Hard Science Fiction
LGBT Metaphysical & Visionary Military
Post-Apocalyptic Space Exploration Space Opera
Steampunk TV, Movie, Video Game Adaptations Time Travel

For each one, I set about gathering specific data:

  • The list of top 100 books in that niche, based on the niche’s own performance
  • For each book, what the global ABS rank is, and who the merchant is
  • Timestamped for once-a-day retrieval

 

Throw them all together in a chart, and you end up with something like this:

PBIDesktop_2017-07-03_20-59-51.png

Enlightening, right? Let’s rather go by genre, starting with the most hotly-contested one right now – Adventure:

PBIDesktop_2017-07-03_21-14-05.png

This is the dashboard of a very healthy sub-genre.

The top 20 books all have ABS ranks below 1000, with the top 5 being below 100 – these books are selling very well, and there is clear demand for this sub-genre right now. The market is also being very well served at the moment – none of the ABS ranks are above 10’000, so it’s unlikely that a first-time author, or someone without major existing traction, will be able to break in here right now.

Now let’s look at a less-contested genre – Hard Science Fiction.

PBIDesktop_2017-07-03_21-19-52.png

This is more like it! The Top 20 books are all under the 2000 ABS rank, and the book sitting at #40 is double that. The category bottoms out at over 12K, so if you’re looking for a place to start, this could be a good sub-genre to do it in.

Finally, the most uncontested sub-genre at the moment – LGBT.

PBIDesktop_2017-07-03_21-22-34.png

There are no official numbers for this, but the #1 book being at ABS rank 1973 would suggest that it’s selling around 100 copies a day. By comparison, the #1 book in Adventure should be doing around 6000 copies/day. This is according to TCK Publishing’s calculator.

100 copies/day on the top end is not much in terms of demand, so while you could almost definitely rank in this sub-genre, it probably won’t be worth the time investment right now.

All the charts above are looking at the total market though, regardless of whether or not titles were independently published. Let’s get into that next.

Independent publishing on Amazon

I published first versions of these on the Dragon Writers group, but now that I have updated information and time to properly process it, here’s a snapshot of how the Sci-Fi genres break down as of today.

PBIDesktop_2017-07-03_21-37-53.png

The vast majority of Kindle titles in the Sci-Fi genre are independently published – “Amazon Digital Services LLC” is the business name used there.

A word of warning on this: That same business name is used by Amazon itself on occasion – so far I’ve seen it used for special store listings of old, republished books. Unfortunately that’s just the nature of a project like this – the data is not going to be 100% accurate.

Other than the LLC, there are a few big names in this space, but they account for very few of the titles published.

But then there’s the quality-vs-quantity argument. Are independently-published novels doing better (or worse) than those published by traditional houses?

This one’s a tricky question to answer, so it’ll help to look at it in parts.

Let’s go with all titles under Sci-Fi with an ABS rank of 1000 or higher. At 1000, you’re selling around 185 books/day – it’s an arbitrary number, sure, but we need to start somewhere.

For each of the sub-genres that have books in that ABS range, what proportions were published independently vs traditionally?

PBIDesktop_2017-07-03_21-59-05.png

It’s no surprise that Traditional is dominating the Classics sub-genre – since that’s literally the genre in which Traditional companies re-package existing traditionally-published books.

But look at the rest – entire sub-genres are being dominated by independently-published titles! This is the encouraging part – on the largest eBook retail platform in the world, it’s possible for independently-published authors to dominate entire sub-genres.

What does the top end look like? Let’s take the top 100 books across all Sci-Fi sub-genres, sorted by ABS rank. The #1 Sci-Fi book is ranked 4th on the Best Seller list, and the #100th book comes in at rank 1429.

PBIDesktop_2017-07-03_23-30-26.png

That’s the most encouraging chart I’ve produced yet. Across the top 100 titles at the moment, 88 are independent titles – but more than that, there’s no clear bias attributable to the publishing method.

Or in other words: It doesn’t matter if you’re independently or traditionally published – both methods have a chance of reaching the top, and ultimately reaching customers.

Conclusions

None of the data above looks at sales or revenue – a lot is being inferred by the limited ranking information that Amazon makes available. For the most comprehensive report that actually looks at sales, AuthorEarnings is the best place to go.

The intention of this post wasn’t to dive into the industry as a whole, but rather to illustrate two things:

  1. There is opportunity here, possibly more so than via traditional publishing channels. The markets are wide-open to new entrants, and the opportunities might change over time, but they are always there.
  2. In the eBook space, it doesn’t matter whether you were published by a big name, or under your own name – both books will have equal treatment, and customers end up making the choices.

Publishing is definitely changing, and I’m excited to see where it goes next.