Category Archives: Technology

Subscription payments in South Africa

So you’re a South African business that wants to take payments for a subscription-based service. You’re probably going to have a mix of local and international customers, and want to know what services are available.

Well, I’ve got some good news, and some bad news.

The good news is – there are ways of doing this. Even with South Africa’s relative backwardness in global economic participation there are still a few options.

The bad news is: the best options are currently unavailable to you, and are likely to stay unavailable in the long term. If you have the means to do so, incorporating and setting up banking in a major economy might be the better option.


Local Options – EFT

It goes without saying that local EFT is an option. If you don’t want to wait for interbank delays, there are a few vendors that offer Instant EFT solutions:

In my experience, PayFast has the easier onboarding path, but you might have a tougher time integrating it into your application. None of those solutions incorporate any sort of subscription management though – it’s on you to keep the accounts in check.

Depending on your capabilities, debit orders might be an option. If you can obtain debit order mandates from your customers (signed papers, recorded calls, etc), you can use Sage Pay’s NAEDO collection system:

Local Options – Credit Cards

This is where it gets a bit more interesting!

PayFast allows you to accept credit cards online, and is relatively easy to set up if you’re not looking for advanced integration. They will make your life easier on one front – they understand subscription management:

I’ve worked with their API before though, and you’re going to need to exercise extreme patience with it in order to get anything done. I hope their systems and documentation improve over time!

PayGate also offers a subscription product, but they make it obvious that they’re geared towards larger businesses – you have to start the process with a sales inquiry:

Global Options

There are too many global vendors to list, but when it comes to what you can feasibly use in South Africa, it narrows down to one pretty quickly: PayPal.

I implemented full-on subscription management using PayPal for Write500 – and it worked quite well. PayPal can accept credit cards, manage subscriptions (including pausing and resuming), and has a solid set of developer tools for integrating it into your application.

You’ll want to create a Billing Plan (your product), then a Billing Agreement (a paid subscription to it), so that Paypal can issue Invoices and settle them automatically. Start here:

The one limitation: You need an FNB account to receive any of that money here in SA.

Going further abroad, you do have the option of incorporating a business remotely. It’s a ton of paperwork (depending on a ton of factors), but one vendor is offering a simple solution to it: Stripe Atlas.

For a one-time fee of $500, and a decent amount of effort, you can incorporate remotely in Delaware – including all the documentation and registration you need to run a corporation within the US. That includes a account, which is basically the global, golden standard for subscription credit card billing.

There’s a couple of major downsides on the administrative side, though – repatriating that money comes with its own set of tax challenges. From what I’ve seen, this is only a really viable option if you’re expecting to do a lot of subscription billing for global customers, and the Stripe fees are a more attractive option.

Other Options

I would be remiss if I didn’t mention PeachPayments – a Cape Town-based company that attempts to make all of the above easier for local companies. If you’re not keen on the idea of getting your hands dirty with integration logic and setting up subscription billing, those will be the folks to talk to.

Then of course, there’s bitcoin. If you really feel like jumping into the murky waters of cryptocurrency for your project, give Coinbase a try. They offer a recurring billing option denominated in Bitcoin. To cash those coins out locally, the simplest option will probably be to hold a wallet, and sell those coins on the local exchange to recover your Rands.

I know that there are likely several other services out there, but if I were starting from scratch today, and wanted to be able to accept payments from a global customer base I’d probably still go with PayPal, myself.

Rise of the Machines

Right now, we’re living in one of the most momentous times in human history, and it could end up being one of the best (or, possibly, worst) things to unfold: our inevitable transition to what Maurice Conti calls the Augmented Age.

Computers have become part of mainstream life in every advanced economy, and basically all major cities around the world (into which people are packing in ever-greater numbers). The resulting efficiency gains have either been a huge boost to creativity and opportunity, or the death-knell of industries that employ tens of millions of people.

I’d like to share two different perspectives on this – both, conveniently, delivered as excellent TED talks. The first is by Maurice Conti, on how advances in computing have changed the way design could be done.

The most remarkable thing about the computer-derived inventions is how biological they look. It took nature millions of years to evolve a structure that their computers can do inside of a few days (referring to the drone chassis), and in future, could do on demand.

I think this is the best insight into how the leading edge of computing might change the way we design cities, vehicles, infrastructure, and the machines that help run our lives. It’s encouraging to note that human designers are still very much a part of the process, but will be able to do a lot more in a lot less time.

Which is a factor leading into the next TED talk – what happens when you centralize that amount of power (and consequently, the financial gains) in the hands of a relative few? People who are skilled at these technologies are able to create enormous value in a short space of time, relative to someone still doing the same task manually.

So what happens when you no longer have a need for the manual labor?

Another excellent talk that takes an unbiased view of Unconditional (I prefer Universal) Basic Income. It raises some good points, but misses at least one point I need to make a note of:

While it’s true that the top 5 tech companies are enormously valuable and employ relatively few people, the platforms they create have in turn generated opportunities for millions more. There are companies, products, services and entertainment channels that could not have existed were it not for the infrastructure and tools that Facebook and the like provide.

Google basically pulled the web development industry up out of the ground when it became clear to businesses that having a well-built site was a competitive advantage. I’m not sure anyone can count the amount of new jobs created in web development, creative design, copywriting, SEO optimization, consulting and education as a result of the platform Google built.

(Yes, I know Google didn’t build the internet. And yes, I know all these websites run on the internet that Google didn’t build, but everyone who’s ever been paid to build one has done so at the request of a customer who believed that being discoverable online would be beneficial to their business, and Google is still the king of discovery on the internet.)

Same goes for the use-cases enabled by Apple hardware, Facebook’s networking, Amazon’s fulfillment infrastructure, and the productivity tools released by Microsoft. Those companies themselves may employ relatively few, but they have empowered millions more.

Moving on.

I think UBI is feasible not so much because of productivity gains due to automation, but because of the ever-declining costs of providing an acceptable standard of living. An excellent, recent example of this is Apis Cor’s house printer.

On the one hand: This technology might end up putting a lot of construction workers out of jobs. While you’ll still need workers for big buildings and the like, simple 1-2 person houses can probably be built quickly, and very cheaply, as a result of this innovation.

But on the flip-side, the cost of houses will plummet. You may not need to work for 20 years to pay off a mortgage for a house that only costs $10k to build. While construction workers might be worried about this, the people who should be a lot more worried are ones with heavy investments in residential development companies 😉

I like to imagine a future unconstrained by urbanization. Cities are where the opportunities are – the best jobs are in cities, the best entertainment, the best healthcare, and overall, the best opportunities to live a good life. This is because it’s a lot easier, with the current limitations, to pile a lot of services into one place.

I don’t believe civilization needs to be so centralized, though. If you could get the same quality of food, healthcare, entertainment and job opportunity in an area 200km outside a major city, plus it was cheaper to live there – wouldn’t you?

And there may come a time when we have to. Most major cities (and by extension, most of the world’s population) are located relatively close to a coastline. Historically, cities were founded and grew near coastlines because those afforded the best opportunities for global trade.

Well, that’s under threat. Depending on who you believe, climate change is either a myth, or it’s a reality already underway – and one of the most dire consequences will be the rise of the ocean level. Which, if that happens, will start to make the large, coastal cities unlivable.

We will be forced to start again – massive inland migrations, the design of new cities, infrastructure and services to support the population, while simultaneously ensuring people have a shot at an acceptable standard of living. With the lessons we’re learning today, I imagine those cities (and societies) will look very different.

Between the work of engineers like Maurice and researchers like Federico, I’m optimistic that we’ll be well-equipped to meet those challenges in future.

You should probably invest in crypto

Obvious upfront disclaimer: Don’t take financial advice from the internet, least of all me. And don’t sue me if you make a bad bet and lose – like I’ll point out below, this is still an extremely volatile space.

I have “mixed” opinions on cryptocurrencies. On the one hand, I’m enormously frustrated with the quasi-religious evangelism that tells people how crypto is the future of literally all money, and their pooh-poohing of simple technical concerns like “how do you scale to hundreds of millions of concurrent real-time transactions”.

Crypto as we know it today won’t replace money as we know it today. Some future optimized version of crypto might, but it will be unrecognizable as compared to the algorithms of today.

On other aspects, I’m a lot more positive – the technology behind it (blockchain) is the real revolution, and a completely new way of thinking about storing, processing and ensuring the integrity of data. New applications built on blockchain technology have enormous potential, and it’s all open-source.

The reason I put “mixed” in quotes is that these positions seem inconsistent to fans of crypto (“how can you not believe it’s the future of money if you think it’s so cool?”), which is an argument entirely on its own, and more than I want to get into right now.

So what are cryptocurrencies? As simply as I can explain it: digital gold.

A bit more complex: A shared ledger, secured by a mathematical algorithm, calculated across tens of thousands of computers which will reach consensus on whether or not a single transaction on it is authentic.

In other words – it’s impossible to forge, and it’s impossible to steal (short of actually forcing someone at gunpoint to transfer coins to you). It is possible to lose, however (as the upcoming BIP 148 hiccup might end up demonstrating), and of course it’s always possible that governments might decide to regulate or outlaw it altogether.

It’s also impossible to devalue by manipulating volume – each currency has a set upfront limit of how many coins will ever be issued, and free markets are entirely responsible for determining the value. I’m sure this is one of the more attractive aspects to the folks who deeply distrust any government, bank or other institution.

Which is where the volatility comes into it. It’s very unlikely, but it is possible at any point that one of these currencies (and there are many) could simply stop operating.

Their existence depends entirely on the network of miners (networked computers) that agree to process transactions for that currency, and at some point, regular external market forces could make some of the smaller currencies untenable.

There’s also the regulation aspect hanging over this. Apart from the obvious criminal uses of a nearly-untraceable cryptocurrency, there’s the wildly unregulated speculation going on right now.

As of today, Bitcoin alone has a USD-denominated market cap of over $40 billion. What this means in practical terms: A good couple of investors (in the hundreds-of-thousands range by now, at least) have bought Bitcoins in exchange for real money, likely in the hope that it’ll appreciate in value over time.

That’s a really big roulette wheel, and so far (for the most part) it’s paid off quite handsomely. It has a lot of the hallmarks of historical gold rushes: Someone’s found something of value, and people are tripping over each other to get in on the action.

That value, though, is likely to only ever exist in the realm of financial speculation. Bitcoin is a desirable, limited-edition asset, which in and of itself is enough to drive speculators and gamblers to it. At $40bn, you have to assume there’s a lot of interest in keeping the entire system propped up, almost entirely for its own sake.

Which is not necessarily a bad thing, and it’s why I’m optimistic about crypto. $40bn is technically a tiny market cap, compared to assets like gold ($7 trillion) – but it has proven itself over the last few years to be stable and growing.

And it can only grow. Sure, getting into Bitcoin today is hard. If you’re planning on buying hardware and setting up as a miner, you better hope you can get free electricity – otherwise you’re losing money the moment you turn that machine on.

The market can always expand, though, and this is the main reason for my optimism. Bitcoin is the largest, oldest and most well-known, but it’s by far not the only cryptocurrency that can be traded for fiat. Here’s the top 10 by market cap, as per


Each symbol there represents an entirely different blockchain – a different asset. And while a single currency has a set limit on the amount of coins it can produce, there’s no limit to the amount of currencies that can be set up.

Unlike gold, wine, art or land, cryptocurrencies can be forked and spun up infinitely. As an asset class, it can expand to soak up every bit of market demand. Every person in the world, theoretically, could have holdings in at least one cryptocurrency.

So why the title of this post, and why do I think it’s worth investing in? Simple: It’s still early days, and it’s still got lots of room to grow.

With the amount of interest and investment going into the underlying systems, and the value of owning some cryptocurrency (both inherent and perceived), they’re all likely to continue to grow in value over the long term.

Today, a single Bitcoin costs around $2’500. There’s no reason to believe it won’t hit $10’000 someday. The same is true of most of the currencies in that screenshot above – whatever the cost is today, there’s a good chance it’ll be a great deal higher in a few years.

My quick anecdote on this: In January I moved R800 into a local crypto exchange, then found out they wanted to do KYC processes over unencrypted email, got annoyed, asked for a refund, and was denied. So in frustration I just cut my losses and forgot about it.

Until a few days ago, when I remembered the whole saga and logged in to see if anything had changed. And one thing had: In around 5 months, my R800 investment in Litecoin had turned into R4’720.

That’s a return you’re not going to find anywhere else – which is what makes it simultaneously alluring, and incredibly risky. There could just as easily have been a crash that wiped out my R800 forever.

Which is why I started small, and would advise anyone else interested in this to do the same. I’ve got accounts on three different exchanges, and holdings in two different cryptocurrencies. Every now and then when I get a bit of spare cash left over that I can afford to save, I move some of it into (what I now consider) my cryptocurrency portfolio.

Between all that spare cash, accumulating over time, it’s added up to very decent growth. As of today, that rate is sitting around 35%, and I’ve been doing this less than a year. That’s an absolutely insane growth rate as compared to other investment options.

And it could all disappear tomorrow, leaving me with absolutely no recourse. Which is why, as tempting as it is sometimes, I’m not gonna bet the farm on this just yet.

What it is, though, is the single riskiest position across all my investments, and the one that’s most likely to yield big returns in a short space of time. I’m expecting (and somewhat hoping) that this growth trend will continue for the next few years, too.

But I’m also expecting that at some point, all the valuations might just evaporate overnight, and I would have done no better than slowly setting that cash on fire. Tempered expectations are pretty much a requirement if you’re gonna ride this roller-coaster – at least, in my experience!

If you’re a South African and you want to get started, is your best bet. Otherwise, and are good options.

Nothing in this post should be construed as financial advice. Please consult a professional before making incredibly risky investment decisions. 

Getting started with Mastodon!

Mastodon Setup

Howdy, stranger! This document is the other half of this video, in which I set up a single-server instance of Mastodon. This was assembled on 9 April 2017, and there’s a good chance that some of the specifics here will change over time. I’ll keep an updated version up on

(What is Mastodon? I’ll do another post on that sometime!)

If you’d like, you can download a plain HTML, styled HTML or PDF version of this post instead – it might make copying some of the code easier.

UPDATE 17 April 2017: Mastodon has reached v1.2, and now requires Ruby 2.4.1. The post has been updated with new commands required as of today, and an upgrade guide is below.

0. Pre-Prerequisites

At a bare minimum, you’re going to need:

  • A domain name, with the ability to add an A record yourself
  • A free account, with the account verified and your sandbox enabled to send to you
  • A 1GB RAM machine with decent network access. This document uses a DigitalOcean VM.

This setup procedure skips a few things that you may want to do on a “productionized” or “community” instance of Mastodon, such as configuring S3 file storage, or using a non-sandbox email send account. You may also want a beefier machine than just 1GB RAM.

For reference, the OS version in use is Ubuntu 16.04.2 LTS and all the commands are being run from the root user unless explicitly specified.

1. Getting started!

The first couple steps:

  • Create the VM
  • Point your domain to it immediately, by setting the A record to the public IP
  • Log into the VM
  • Set your root password
  • Create a new Mastodon user: adduser mastodon
  • Update the apt cache: apt-get update

2. Install Prerequisites

Now we’ll grab all the prerequisite software packages in one go:

# apt-get install imagemagick ffmpeg libpq-dev libxml2-dev libxslt1-dev nodejs file git curl redis-server redis-tools postgresql postgresql-contrib autoconf bison build-essential libssl-dev libyaml-dev libreadline6-dev zlib1g-dev libncurses5-dev libffi-dev libgdbm3 libgdbm-dev git-core letsencrypt nginx

That’ll take a little while to run. When it’s done, you’ll need Node (version 4) and yarn:

# curl -sL | bash -
# apt-get install nodejs
# npm install -g yarn

You’ll also want to be sure that redis is running, so do:

# service redis-server start

3. Configure Database

With Postgres installed, you need to create a new user. Drop into the postgres user and create a mastodon account:

# su - postgres
$ psql
> \q
$ exit

Later on we’ll configure mastodon to use that.

4. Generate SSL certificate

Before configuring nginx, we can generate the files we’ll need to support SSL. First, kill nginx:

# service nginx stop

Now proceed through the LetsEncrypt process:

  • Run letsencrypt certonly
  • Enter your email address
  • Read and acknowledge the terms
  • Enter the domain name you chose

If the domain name has propagated (which is why it’s important to do this early), LetsEncrypt will find your server and issue the certificate in one go. If this step fails, you may need to wait a while longer for your domain to propagate so that LetsEncrypt can see it.

5. Configure nginx

With the SSL cert done, time to configure nginx!

# cd /etc/nginx/sites-available
# nano mastodon

Simply substitute your domain name where it says in this snippet (lines 9, 15, 23, 24), then paste the entire thing into the file and save it.

map $http_upgrade $connection_upgrade {
  default upgrade;
  ''      close;

server {
  listen 80;
  listen [::]:80;
  return 301 https://$host$request_uri;

server {
  listen 443 ssl;

  ssl_protocols TLSv1.2;
  ssl_ecdh_curve prime256v1;
  ssl_prefer_server_ciphers on;
  ssl_session_cache shared:SSL:10m;

  ssl_certificate     /etc/letsencrypt/live/;
  ssl_certificate_key /etc/letsencrypt/live/;

  keepalive_timeout    70;
  sendfile             on;
  client_max_body_size 0;
  gzip off;

  root /home/mastodon/live/public;

  add_header Strict-Transport-Security "max-age=31536000; includeSubDomains";

  location / {
    try_files $uri @proxy;

  location @proxy {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto https;

    proxy_pass_header Server;

    proxy_pass http://localhost:3000;
    proxy_buffering off;
    proxy_redirect off;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;

    tcp_nodelay on;

  location /api/v1/streaming {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto https;

    proxy_pass http://localhost:4000;
    proxy_buffering off;
    proxy_redirect off;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;

    tcp_nodelay on;

  error_page 500 501 502 503 504 /500.html;

Once you’ve saved and closed the file, enable it by creating a symlink:

# ln -s /etc/nginx/sites-available/mastodon /etc/nginx/sites-enabled/mastodon

Then test that the file is OK by running nginx -t. If it reports any errors, you’ll want to fix them before moving on. If the file comes back OK, fire it up!

# service nginx start

Open a browser tab and navigate to your domain. You should get a 502 Gateway Error, secured with your LetsEncrypt cert. If not, go back and make sure you’ve followed every preceding step correctly.

6. Configure Systemd

Mastodon consists of 3 services (web, sidekiq and streaming), and we need to create config files for each. You can use the code straight from this page, as-is.

# cd /etc/systemd/system/

The first file is called mastodon-web.service and consists of the following:


ExecStart=/home/mastodon/.rbenv/shims/bundle exec puma -C config/puma.rb


The next file is called mastodon-sidekiq.service and consists of the following:


ExecStart=/home/mastodon/.rbenv/shims/bundle exec sidekiq -c 5 -q default -q mailers -q pull -q push


The final file is called mastodon-streaming.service and consists of the following:


ExecStart=/usr/bin/npm run start


Once all those are saved, we’ve done all we can with the root user for now.

7. Switch to the Mastodon user

If you haven’t yet logged into the server as mastodon, do so now in a second SSH window. We’re going to set up ruby and pull down the actual Mastodon code here.

8. Install rbenv, rbenv-build and Ruby

As the mastodon user, clone the rbenv repo into your home folder:

$ git clone ~/.rbenv

When that’s done, link the bin folder to your PATH:

$ echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bash_profile

Then add the init script to your profile:

$ echo 'eval "$(rbenv init -)"' >> ~/.bash_profile

That line is valid for the OS we’re on (Ubuntu 16.04 LTS) but it may differ slightly for you. You can run ~/.rbenv/bin/rbenv init to check what line you need to use.

Once you’ve saved that, log out of the mastodon user, then log back in to complete the rest of this section.

Install the ruby-build plugin like so:

$ git clone ~/.rbenv/plugins/ruby-build

Then install Ruby v2.4.1 proper:

$ rbenv install 2.4.1

This could take up to 15 minutes to run!

When it’s done, change to your home folder and clone the Mastodon source:

$ cd ~
$ git clone live
$ cd live

Next up, dependencies! Always more dependencies – we’ll install bundler, then use that to install everything else:

$ gem install bundler
$ bundle install --deployment --without development test
$ yarn install

If all of those succeeded, we’re ready to configure!

9. Configure Mastodon

Before diving into the configuration file, generate 3 secret strings by running this command 3 times:

$ bundle exec rake secret

Copy those out to a text file – you’ll paste them back in later. Create the config file by copying the template, then editing it with nano:

$ cp .env.production.sample .env.production
$ nano .env.production

Inside this file we’re going to make several quick changes.


To enable federation, you need to set your domain name here:

Then, for these 3, paste in each key you generated earlier:


Finally, configure your SMTP details:

SMTP_LOGIN= (whatever your mailgun is)
SMTP_PASSWORD= (whatever your mailgun is)

Save and close the file.

10. Run installer

If you’ve done everything correctly, this command will install the database:

$ RAILS_ENV=production bundle exec rails db:setup

If that passes successfully (it’ll echo out every command it runs), you can then precompile the site assets, which may take a few minutes:

$ RAILS_ENV=production bundle exec rails assets:precompile

At this point, we’re almost ready to go!

11. Configure cronjob

This is technically optional, but highly recommended to keep your instance in good order. As the mastodon user, start by determining where your bundle command lives:

$ which bundle

That path will be substituted for $bundle. Now, edit your own crontab:

$ crontab -e

Select nano (2) if you’re prompted. As of version 1.2 (17 April 2017) you only need one daily task in your crontab:

5 0 * * * RAILS_ENV=production $bundle exec rake mastodon:daily

Save and close the crontab.

12. Log out and return to root

We’re done with the mastodon account. Log out and return to your root shell.

13. Start Mastodon

The moment of truth! Enable the Mastodon services (so that they start on boot):

# systemctl enable /etc/systemd/system/mastodon-*.service

Then fire up Mastodon itself:

# systemctl start mastodon-web.service mastodon-sidekiq.service mastodon-streaming.service

Open up a browser tab on your domain. Mastodon can take up to 30 seconds to warm up, so if you see an error page, don’t fret. Only fret if it’s there for longer than a minute – that requires troubleshooting, which is outside the scope of this document.

You should eventually get a signup page. Congratulations! Register an account for yourself, receive the confirmation email, and activate it. This should enable you (the first user) as an administrator.

14. Securing Mastodon

This is by no means a comprehensive guide to server security, but there are two quick things you can change while the root shell is open. Start by editing the passwd file:

# nano /etc/passwd

Find the mastodon entry (it’ll be near the bottom) and replace /bin/bash with /usr/sbin/nologin. Save and quit. This will prevent anyone from logging in as the mastodon user.

Next, configure ufw. First check if it’s disabled:

# ufw status

It should be off, since this is a brand new VM. Configure it to allow SSH (port 22) and HTTPS (port 443), then turn it on:

# ufw allow 22
# ufw allow 443
# ufw enable
? y

That will prevent any connection attempts on other ports.

15. Enjoy!

If you enjoyed this guide, I’d appreciate a follow! You can find me by searching in your Mastodon web UI. Give me a shout if you were able to get an instance set up with these instructions, or if you ran into any problems.

16. Upgrade to v1.2 (17 April 2017)

If you’ve installed Mastodon according to these instructions, you’ll need to do a few things to upgrade to the latest version.

Start by logging into your instance as the root user, then re-enabling your mastodon user shell (in step 14, change the mastodon user’s shell back to /bin/bash). We’ll use it in a bit to perform the upgrades themselves.

When that’s done, stop the Mastodon services like so:

# systemctl stop mastodon-*

That will shut down all the Mastodon services. In a new window, log into your mastodon user and install Ruby 2.4.1, the new preferred version:

$ cd live
$ rbenv install 2.4.1
$ gem install bundler --no-ri --no-rdoc

This will install the latest Ruby, and the version-appropriate bundler. Now pull down the latest source code:

$ git pull

There are a couple of one-time commands to run – in order, they are to install new dependencies, run database migrations, do a one-time avatar migration, and recompile the frontend.

$ bundle install
$ yarn install
$ RAILS_ENV=production bundle exec rails db:migrate
$ RAILS_ENV=production rake mastodon:maintenance:add_static_avatars
$ RAILS_ENV=production bundle exec rails assets:precompile

When this is all done, make sure your crontab has been updated to use the new mastodon:daily command. Refer to step 11 above for details.

Finally, the teardown – log out of the mastodon user, and switch back to your root connection. Set the mastodon user’s shell back to /usr/sbin/nologin (step 14) and restart the Mastodon services:

# systemctl start mastodon-web.service
# systemctl start mastodon-sidekiq.service
# systemctl start mastodon-streaming.service

Give it a few seconds to warm up, and check that they’re running with:

# systemctl status mastodon-web.service

If you get a green dot with “running”, you’re good to go!


A lot of this guide was sourced from the official Production guide on the Mastodon Github page. I reorded it into a logical sequence after running through it for a few tries.

This post was updated for v1.2 (and v1.1.2) upgrade notes on 17 April 2017.

Laravel 5.3 + Cloud9

I use Cloud9 as my primary IDE – it handles all my PHP projects, my Domo app projects, and when I start seriously developing Ionic-based mobile apps, I’ll find a way to use Cloud9 for that too.

I’ve found it particularly handy for Laravel development: A lot of the prerequisite software provided by Homestead comes pre-installed on the Ubuntu VM that Cloud9 provides, and with a few small config tweaks, it’s possible to get up and running very quickly.

1. VM Setup

One of my cardinal rules: All your code should exist in a repository. Especially with web development, there’s basically zero excuse to keeping substantial amounts of code locally.

My first step is to always create a blank repository, and use that as the clone URL for a new Cloud9 VM.

2017-01-20 05_48_50-Create a New Workspace.png

The Blank Ubuntu VM is good enough for our purposes – there’s no benefit to selecting the PHP VM, since we’ll be installing the latest PHP in any case.

2. Services Setup

Chances are, whatever you’re building in Laravel will need the MySQL service to be running, and in most cases, you’ll probably want Redis too. Cloud9 has made it a little bit more difficult to automate service startup (presumably to minimize unnecessary resource utilization), but there’s a way around that.

If you’re using a premium workspace, Cloud9 will keep it “hot” between sessions – everything you start will continue to run even after closing the IDE window itself. But that will expire after a while, and it can be a hassle to keep restarting your services.

So, the first change we’ll make to our new VM is editing the /.profile file, and adding two new lines to the bottom:

$ cd ~/
$ echo 'sudo service mysql start > /dev/null 2>&1' >> .profile
$ echo 'sudo service redis-server start > /dev/null 2>&1' >> .profile

That will start the services (and skip startup if they’re already running), won’t show any console output, and will run whenever a new terminal window is created. So now, every time you open up a new terminal window, you’re guaranteed to have those services running:

2017-01-20 06_11_31-cloud9-demo - Cloud9.png

It’s not as ideal as having the server start it automatically, but in my opinion, this is a small trade-off to make.

3. PHP 7 Setup

If you’re deploying code to a new server on DO, or AWS, or Rackspace, or most other places, chances are it’s running PHP 7. By default, Cloud9 does not include PHP 7 (for reasons which escape me), so we need to take a quick detour into apt-get land.

There are two ways to set up PHP here – apt, or phpbrew. For a while I was using phpbrew, which builds PHP from source. It’s a decent option, but using ondrej/php is faster.

$ sudo add-apt-repository ppa:ondrej/php

Then you’ll need to sudo apt-get update. When that’s done, you should have access to the php7 packages:

2017-01-20 06_18_23-Groove Music.png

I’m using php 7.1 here, because that’s what my DigitalOcean VM is running. Laravel will need that, plus a few other modules:

$ sudo apt-get install php7.1 php7.1-mysql php7.1-curl 
  php7.1-xml php7.1-mbstring

That shouldn’t take more than a minute. And now we’re good to go:

2017-01-20 06_21_26-cloud9-demo - Cloud9.png

4. Laravel Project Setup

There are quite a few ways to get Laravel installed. It has a CLI-type thing that lets you do laravel new, you can clone it directly from github, or you can use Composer.

I tend to favor composer since it’s a once-off command for this workspace. There’s no benefit to installing any more tools here, since we’re only going to create 1 Laravel project in this VM. And since we’ve already got our workspace folder linked to our project repository, git can get messy.

So my favorite – use composer to set up Laravel in a temporary directory, then move it all across:

$ cd ~/
$ composer create-project laravel/laravel tmp --prefer-dist
$ mv tmp/* workspace/
$ mv tmp/.* workspace/
$ rmdir tmp

Why two move commands? The first catches all the files, but Laravel does include a few dotfiles that aren’t caught by the first move command. Sure, you could configure dotglob to modify mv’s behavior, but we’re only doing this move once for the workspace.

Just one more setup step – the database. Start by adjusting the .env file to set the username to root, and a blank password:


Then we’ll create that database in one go:

$ mysql -uroot -e "CREATE DATABASE homestead;"

You can now run the initial migrations if you want.

5. Node Setup

If you’re planning on using Elixir to compile frontend assets, you’ll probably want to upgrade to the latest version of Node. Cloud9 makes this easy:

$ nvm install 7.2
$ nvm alias default 7.2

All node commands will now use the 7.2 binary. You can now install the necessary components in your workspace folder:

$ npm install

6. Running Laravel

Cloud9 features a reverse proxy – anything that binds to port 8080 in the VM will be accessible via a URL unique to your workspace. It also has a dedicated “runner” feature, which we’ll configure like so:

2017-01-20 06_33_15-cloud9-demo - Cloud9.png

The name can be anything, but the run command will be:

php artisan serve --host= --port=8080

Cloud9 does make mention of $HOST and $PORT variables, but they’re always the same values anyway. Fill that in and click the Run button on the left:

2017-01-20 06_33_53-cloud9-demo - Cloud9.png

Cloud9 should open up an in-IDE browser window that points to your site. You can click the little pop-out button on the right of the “address bar” to bring it up in a new browser tab.

2017-01-20 06_36_01-cloud9-demo - Cloud9.png

That domain ( will check for cloud9 authentication before proxying to your VM – in short, that URL will only work for you, and anyone else on cloud9 that you’ve shared the workspace with. Those settings are managed via the Share button on the top right:

2017-01-20 06_37_40-cloud9-demo - Cloud9.png

From that popup, you can invite new users to the workspace (cloud9 does issue free accounts), or you can just make the application URL public:

2017-01-20 06_38_34-cloud9-demo - Cloud9.png

You can now share that URL to anyone.

7. Get coding!

You’ve now got just about everything you need for proper Laravel development! It’s usually at this point I then take the extra step of configuring Amazon S3 as a cloud disk, push the blank Laravel project to bitbucket, and deploy my repo as a site on Forge.

Building a Domo dashboard for Write500

This post is part of a series of posts I’m doing about Domo (the Business Cloud) and how I’m using the free tier to analyze metrics around Click here for the full listing.

There’s a surprising amount of things to keep an eye on when it comes to a project like Write500.

Right now, only the free tier of Write500 exists – users get sent to a subscribe page, they opt-in, and receive daily writing prompts over email. It won’t always be so simple: I’ve got plans for a Plus (paid) version, and mobile apps to extend Write500’s reach.

So even this simple, free version, in terms of the architecture, already looks like this:


Every box in there has something that needs to be monitored in some way. Some of the monitoring is historical (looking back at performance over time), some of it is by exception (being alerted when something goes wrong).

That’s already a lot of potential metrics, from a lot of different sources. Individually, they aren’t particularly hard to get – it’s deciding which ones to get, how often, and what you’ll do with the information, that matters.

And that’s the product side of the equation. Not only am I building the product, I’m the one-person marketing team behind it.


Each of those boxes, too, can have a whole universe of metrics in them. Social in particular, which has a penchant for inventing crazy new metrics every few months.

Where to start?

If it seems overwhelming – it is. There’s a lot of information out there, not all of it useful. It’s way too easy to get hung up on the wrong metrics, and miss out on the ones that matter.

So the best place to start, in my experience, is not with the data. I’ve seen too many BI projects start out with what the team could get from their databases, and in the end, they delivered dashboards that looked identical to what they were already getting from their data in the first place.

What we’re after here is actionable, meaningful insight. So without further ado:

1. The Question

A much better place to start, is the question.

If I could answer one question about your business, what would it be?

For Write500 I have a lot of questions, but in this post we’ll deal with the one that’s most important to me:

Are my subscribers getting value from Write500?

I built Write500 because I wanted to be nudged to write 500 words per day. So far, as the user of my own product, I’ve managed to achieve that – but my experience doesn’t matter nearly as much as the experience of my subscribers. If they’re not getting the value they need, then I’m either aiming at the wrong audience, or there’s a problem with my product.

Now, that question could be answered anecdotally. I could point to a few places I’ve seen bloggers start their challenge, and take that as a sign that users are getting value. I’ve seen a lot of businesses do this – make gut judgements based on limited information.

I have learned, however, not to trust my gut. Let’s quantify this instead.

Getting metrics about my subscribers is easy – I can pull the subscribers table and chart the growth. Getting metrics about the value is a little bit harder – right now, I’m not equipped for that.

Ideally, the value I want subscribers to get out of my product is producing 500 words per day – and the only way to track that metric is to have them plug their 500 words into a system where I can measure their wordcount. That’s a feature planned for the next version of Write500, but right now, I don’t have that data.

The best I can do, with the information I have, is determine whether or not users are opening the emails I’m sending on a regular basis. That’s not the best metric for whether or not they’re getting value from Write500, but I’ll operate on the assumption that consuming the prompts is at least partially valuable.

Here, already, one of the biggest problems in BI becomes visible: The gap between what you need to know about your business, and the data you have on hand to answer it. So, so many BI vendors and consultants sell dashboards that chart what you have, but don’t give a second thought to what you need.

2. The Data

So based on my question, what data can I acquire to answer it?

Write500 has a bespoke tracking pixel. Every prompt that gets sent is uniquely identifiable, and includes embedded tracking that fires (email client willing) when the email is opened. This data is sent straight back to my server, instead of going through a third-party analytics tool.


The raw data, as you can see in the screenshot, includes the GUID for each send, and the server-side timestamp for when the email was opened. I also have the timestamp for when the email was sent. With this data, I can answer two sub-questions:

  1. Are subscribers opening the emails they’re receiving?
  2. Do subscribers open the email on the same day they receive it?

Together, the answers to those questions would at least indicate whether or not the emails themselves are being utilized (and whether my subscribers are getting value). So looking at the data I have available, this is what I can get to help answer those questions:

Per subscriber: (dimension)

  • The amount of prompts they have been sent (metric)
  • The amount of prompts they have opened (metric)
  • The open rate (calculated)

Per day: (dimension)

  • The total amount of prompts sent (metric)
  • The amount opened within 24 hours (metric)
  • The amount opened within 48 hours (metric)
  • The amount opened within 72 hours (metric)
  • The amount opened after 72 hours (metric)
  • The amount not opened (metric)

Dimensions and Metrics

I should explain what these are before continuing, and I’m doing this in the context of Domo’s capabilities. Within any tabular data set, you’ll have two types of fields:

  • Numbers
  • Everything that’s not numbers

Metrics are numbers you can measure, and Dimensions are the categories you group/slice/breakdown those metrics in. So:

  • New Subscribers per day
  • Recurring Visitors by mobile device
  • Purchases over $50 by persona

In the samples above, Day is a dimension (because it’s not a number). But Subscriber is also a dimension, even though it’s technically a numeric ID – except that it’s a number that makes no sense to measure. This is the exception to the rule.

It can’t be summed, averaged, min’d or max’d and present any meaningful result: What’s the point of knowing the average subscriber ID in your database? The count of subscribers is more useful, and in that case, the count becomes the metric.

3. The Acquisition

So now we know what question we’re answering, and the data that we’re going to use to answer it. This is where we start getting into the implementation.

The first step, obviously, is to acquire the data. In a previous post, I dealt with this issue for Write500, eventually settling on a SFTP upload for my subscribers dataset. Since then, I’ve added a better stats system to the app.

So for Write500 specifically, I now have this happening:

Write500 Data - New (1).png

I have a series of Artisan commands, each of which generates a single CSV file. Those commands are scheduled using the cron system on Forge, and when they run, the results are uploaded to a dedicated S3 bucket. Domo, in turn, pulls that information into its Vault:


Now that I have the data coming in, it’s time to build some charts!

4. Visualization

Visualization is a problem domain all its own, and has way too much science behind it to cram into a 3000-word blog post. However, we’re doing visualization for BI, which is very different to visualization for analysts, engineers and scientists.

BI visualization has to meet several fundamental criteria (at least, the type of BI I do):

  • Self-contained – the chart should explain what it is without requiring an appendix. Any chart that has a “How to read this chart” next to it, is a bad chart.
  • Key call-out – Whatever the purpose of the chart is, it should be obvious at a glance, preferably from across the room.
  • Honest – Too many “analysts” will fiddle with colors, min/max axes dimensions, sorting and scale to try to massage the truth behind the numbers. Don’t.
  • Actionable Next Step – In charts that are meant to drive decision-making, make the decision obvious.

In my experience, this tends to mean the following:

  • You’re going to re-use a lot of the same 3-4 chart types (line, bar, area, donut)
  • You’re going to create lots of smaller charts, instead of consolidated/complex ones
  • Colors are really important
  • Short titles are really important

I will eventually build the charts that answer my key question, but let’s start with some simple ones.


First, we start with a blank canvas.

Domo includes the ability to group charts into Collections, which make it simpler to manage complicated dashboards. In this case, all the charts that pertain to my subscribers will go in one collection – other collections will include marketing and product data.

Note: All the charts here are built on real data, as of today.

0. Warming up

Easiest thing to chart – one metric vs a date.


Here’s an example of what I think is a good chart:

  • The title and subtitle make it clear what I’m looking at,
  • The summary number makes the current status clear,
  • The bar chart shows the growth nicely, and
  • Using Domo’s linear regression Projection feature, I can see that I should end today with over 100 new subscribers

So overall – I’m getting new subscribers, and it’s visually obvious from the chart that the velocity is increasing.

But now let’s answer our first sub-question:

Are subscribers opening the emails they’re receiving?

Now we get to define a proprietary metric! I’m going to create a metric called “Engaged”:

Engaged: A user that has no more than 1 unopened prompt email

The logic behind this: Users receive one email per day, and there will be a gap between receiving an email and opening it. However, for as long as that gap exists, the email will show as “1 unopened” in my data, which is not fair – the user just hasn’t had a chance to open the most recent email yet.

“2 unopened” means they’ve neglected at least one email, so I’ll consider them less engaged. Yes, this is a high standard.

How do we calculate this metric? Here’s the Beast Mode formula I’m using, directly in the card.


This gets us the total users that have between 0 and 1 unopened emails, as a proportion of the total number of users in the data.

That can be represented in a single large number card, like so:


That’s a pretty impressive result! Almost 70% of my subscribers are opening their emails on a regular basis, and it could be even higher: remember that I cannot track opens from email clients that block the call to my tracking pixel.

I also want to get a sense of the proportions, though – so we’ll take that summary metric and use it as a summary number on a pie chart. I’ll group my subscribers like so, using another Beast Mode.


Honestly, these sorts of CASE statements represent 90% of what I use Beast Mode for, in Domo – to create custom categories.

We’ll apply that formula to the data, and spin up a pie chart. Here’s what we get:

2017-01-07 15_24_03-Write500 - Domo.png

So of all my subscribers, 60 of them are neglecting their prompts completely. This is also useful information – I know who those subscribers are, so I can segment them out and try different email subject lines, to see if I can maybe get them more engaged and bolster the overall rate.

There’s one more chart we can create – we can compare our engagement rate to an industry open rate. Using this source data, I’ll take Hobbies as my benchmark.

2017-01-07 15_28_21-Write500 - Domo.png

Not bad at all! But then, that’s to be expected – my emails are not marketing, they’re more transactional in nature, and transactional emails have much higher open rates.

So let’s go back to our question:

Are subscribers opening the emails they’re receiving?

Resoundingly, yes: Close to 70% of my subscribers are opening their emails.

On to the next question!

Do subscribers open the email on the same day they receive it?

The answer to our previous question can cover this one too – if Unopened is between zero and one, then yes – most open the same day they receive it. However, that’s an aggregate view, and I want to see some more specific on how long it takes subscribers to engage with their emails.

So for this, we’re looking at the time difference between the email being sent, and opened. I’ve already done the heavy lifting of categorizing the data inside my data-providing Artisan commands, so the data I’m visualizing is already close to useful:


Our last charts looked at Subscriber Engagement, which is a higher-level metric than the actual individual emails being opened or not.  For these charts, we’re taking a different angle, looking at the actual emails. We’ll get similar rates, but not identical – that’s to be expected.



Of all the emails that do get opened, practically all of them get opened on the same day (within 24 hours of being sent). And it looks like, overall, about 40% of the emails sent by the system have not been opened yet. Or, at least, they haven’t fired the tracking pixel yet.

The thing about tracking email open rates in particular, though – naturally, it’s not a trendable metric.

For example: You can send 100 emails on Monday, then calculate the open rate every day of the week, and get a different (higher) number every day.

Why? Because subscribers take time to open their emails (if at all). So looking at that chart above, right now, 40% of the emails I’ve sent are unopened. Those emails could have been sent today, or a week ago.

Right now, I care about whether or not subscribers open on the same day – and according to this chart, nearly all opened emails are opened on the same day. Emails opened at +1 Day and up are completely dwarfed. This is what happens when we remove Unopened emails from the chart:


So that’s my question answered!

5. Taking Action

I’ve got my questions, they’re being answered by data, and now I need to think about the next step in this: What actions can I take, based on this data, to drive my business goals?

Right now, my goals are two-fold:

  1. Drive more subscribers to Write500
  2. Improve the email open rate

For (1), what I really need to look at is my subscriber acquisition machine – the marketing I’m doing, which is outside the scope of this post. I do plan on writing another one dealing specifically with marketing metrics though.

For (2), I now have a tight group of subscribers that are either not opening their emails, or not opening them on a daily basis – meaning they’re not writing every day. I can segment these users out and try a few things to remedy that.

I could also simply reach out to the subscribers that haven’t opened any emails yet. There might be a good reason why those emails aren’t being opened:

  • maybe they’re getting caught in spam,
  • maybe the user hasn’t been at their mailbox in the last week, or
  • maybe they no longer want to receive them, but haven’t bothered opting out.

My action would be to find out what it is, and remedy it where possible.

6. (Finally) Measuring Results

The final piece of the puzzle, and the one question that every BI dashboard should answer:

Are our actions having the intended effect?

Since the insights and actions are data-driven, the results should be too. For every action that you take to impact your metrics, you need to think about how you’re going to measure the results in a quantifiable way.

Let’s take the Open Rate chart above. When I start taking action to improve it, I expect to see that, over time, the overall percentage of Unopened emails goes down.

Since it’s an email open rate though, I can only get snapshots at regular intervals. The data generator I wrote does just that – it gathers point-in-time statistics, and appends them to the dataset. Right now, it’s running once per day, and it’s grouping the sends by the date they went out.

So putting that into a 100% Stacked Bar Chart, I get this:

2017-01-07 17_02_06-Write500 - Domo.png

Every day, I will get another bar on this chart, and the chart itself will be completely recalculated (if emails sent on previous days are opened, which we know from the data above it will be).

However, over time, that red portion should get smaller. If I wanted to create very precise tracking on this, I’d create a recursive dataflow to record the Unopened rate in a dedicated dataset – but for right now, I think I have enough BI going on!




I hope this post has been informative!

I’m planning on doing a deeper dive post into Write500’s metrics at the end of the month – that post will deal more with my expectations vs the reality, the charts I ended up using on a daily basis, what decisions I took, and how I measured those results.

2017-01-07 17_17_53-Write500 _ IG_ write500 - Domo.png

The dashboard I’m sticking with for now.

If you want to be notified when I write more of this stuff, check the Subscribe widget in my sidebar, top right. Or you can follow me on Twitter.

Pulling arbitrary data into Domo

This post is part of a series of posts I’m doing about Domo (the Business Cloud) and how I’m using the free tier to analyze metrics around Click here for the full listing.

There are almost as many ways to integrate data, as there are types of data, and source systems that hold data. There’s basically no way to cover them all in the space of one blog post.

In terms of Domo, though, all data is tabular – Domo doesn’t deal with binaries, log streams, delimited text files or complex JSON documents. Just plain old CSVs, which makes sense if you’re doing BI for business – most of the things that matter to decision-makers can be represented in table form.

So long as you can get your data in CSV format, there are a good couple of options to get it into Domo. In this case, I want to pull anonymous user data from write500 – just enough to chart user growth.

I’m also working within the limitations of what the free tier of Domo can do. The Enterprise tier is capable of more advanced acquisition, including basic JSON and CSV API endpoints.

In this example, I do have the benefit of being both the developer of the product, and the business user consuming the dashboards – so I can write any solution I need. Based on what the free tier is capable of, I have the following options:


That’s a lot of options – and at some point, I’ve used every one.

From top to bottom:

1. Cloud Storage
I could write a script on the server to package the data in a format that can be uploaded to cloud storage – so a zipped file pushed to Box, OneDrive or Dropbox for Business. Domo then has cloud connectors that will let me pull that down as usable data.

I could expose the data through a simple JSON API, and use Google Sheets as an intermediary. It’s possible to use the built-in scripting language and some scheduling to populate a Google Sheet, which is then trivial to import into Domo. It’s not very scalable though.

3. Export to CSV
I could write a script to dump out my required data as a CSV file, then pull that over SFTP. This is easier to set up, but still requires some scripting work. I can then pull the resulting data with the CSV/SFTP connector.

4. Direct Database Access
I could use the MySQL connector to hit the write500 database directly. I’d have to open firewall ports, add users, and do a bunch of other setup first. I’m not in favor of doing that much work right now, though.

So we’ll go with 3 – I’ll write a script to produce the exact CSV file I need, then set up the SFTP connector to fetch it.

Deciding what to fetch

There’s something of an art to deciding what metrics to actually pull – you first need to decide what’s actually useful to you, in terms of answering your business questions. In this example, I know I simply need a dataset that looks like this:

User ID Date Registered Is Subscribed
1 2017-01-01 05:30:00 1

How you generate that CSV is up to you – in my case, I wrote a basic Bash script to do that:


Ok, so “basic” might be the wrong word to use – but all that’s really doing is generating a CSV file, with the right header line and correctly-formatted data.

That script is set up to run every 30 minutes, and it will keep a fresh copy of users.csv in a dedicated user’s home directory.

Now to import that! Domo has a set of connectors for working with files:


1. That (+) icon is everywhere.


2. We’re looking for this one.

On the next screen, typical SFTP setup stuff – host, username and password. If you run a tightly-secured system, you might also need to whitelist an IP range so that Domo can reach your server.


3. Just needs a valid SSH (/SFTP) user.


4. Voila! Our file.


5. I want frequent updates!


6. Name, Describe, and Save


So what we just did there – we set up the SFTP connector to fetch that CSV file once every hour. Every time it does that, it will overwrite the dataset we have with new data – that’s what the Replace method means, in the Scheduling tab.

Finally, named and described. It’s helpful to prefix all your datasets with a short project name – makes it easier to find later.

In no time at all, we’ve got our data!


Sweet, sweet data.

From here, the next step is to visualize it. That’s a whole topic all on its own (so I won’t go into detail here), but here’s a dead-simple line chart built from that data, to show the trending over time:


54 opted-in users from a total of 80 or so

So that’s a basic rundown of a CSV connector, end-to-end. This example does lean towards the “more complex” side of data acquisition when it comes to Domo. Luckily, most of the high-value stuff exists in systems that Domo already has great connectors for.

If you want to be notified when I write more of this stuff, check the Subscribe widget in my sidebar, top right. Or you can follow me on Twitter.