Category Archives: Software

Casting off for new shores

One thing I’ve learned this week – even if you know about the sunk cost fallacy, it can still somehow creep up on you unexpectedly.

Writing has been a lifelong dream for me. For about as long as I can remember, I’ve always been fascinated with stories, and would constantly dissect and re-imagine them in new ways.

But it never felt like the right time to actually write. The reasons are varied, complex, mostly depressing, some aggravating, and all in the past – the sum of it is that I’ve never really been able to settle on something that I actually wanted to do, and as a result of that paralysis, have done largely nothing.

Last year, May 2016, I redid this blog – archived all the old content, changed the domain name (supremely happy to have landed wogan.blog, let me tell you), and wrote this introductory post.

Fast forward a year, and while I’ve done some measure of that privately, I haven’t done nearly enough of it publicly. What I have done, instead, is retreat back to the same comfort zones I’ve always had.

 

Towards the end of 2016 I started getting agitated that I wasn’t making progress with my writing, and decided to tackle it as a pure productivity problem. People are writing and publishing books every day – there has to be a system that will work, right?

That’s where the idea for Write500 came from – my own desire to set specific goals that I could hit, every day, and make progress as a result.

But then my comfort zones kicked in.

When I talk about the “sunk cost fallacy”, the first assumption you might make is that I’m referring to the time I spent on this specific project – which is a factor. Over the last few weeks I’ve been working on Write500 more or less because it exists, and because I know there are people that are interested in how it evolves. Not so much because I think it can actually solve the problems I need it to solve.

So that’s one level of fallacy right there – working on something because I’ve been working on it before, and the cost of killing it is somehow (inexplicably and irrationally) unacceptably high.

But there’s another level to this, and it’s only in this last week that it’s really been driven home for me: My career so far is, itself, a sunk cost.

My approach to solving problems is almost always rooted in software, which shouldn’t be a surprise – I learned to program at a very young age, and because of my knack for it, I was able to get a job, which I was then able to turn into something resembling a career.

As a result, when I’m deciding the best way to add value to the world (which I believe is something we should all try and do, in our own way), my main inclination is almost always software. I keep coming back to it, thanks to how far its gotten me in life so far.

Over the last year though, that inclination’s been challenged somewhat. Last year around this time, I was in the process of handing over my biggest freelance client to another agency, thanks almost entirely to burnout.

The idea of working late nights developing software for paying customers had lost its appeal entirely, despite how absolutely brilliant it was to be able to monetize my nights and weekends, and build up some savings as a result.

I’ve had a similar inclination at work, too – whenever a new problem presented itself, and if it could be solved with some custom development, that would always be the first suggestion I’d make.

Which had not been a problem, really, up until the end of 2015 – I had latitude to develop things that I thought needed to be developed. The change of my job role had also forced me to change the way I solve problems – including occasionally not solving them at all.

I imagine it’s at this point that a lot of developers would quit out of frustration – feeling like they’re adding no value, or taking umbrage at not being able to use or grow their skills.

I didn’t quit, though, and I ended up learning something new: That it was possible for me to be productive (and add value) without writing a single line of code.

I’m sure that seems obvious to a lot of people, but it’s only recently become clear to me how big of a mental shift this actually is. And it brings me back to the thing about writing.

I’ve wanted to be a writer for as long as I can remember – the idea of spending time in my own world, creating characters and stories within it to share with other people – is incredibly appealing.

But with my very narrow view of problem-solving, I’d always look at my lack of writing as something that could be solved with software. And so instead of actually writing, I’d set out to shave as many yaks as possible.

It’s the old “if all you have is a hammer” adage – the problem of me not writing started to look like a nail. A problem that could be solved if I just found the perfect combination of tools, frameworks, and the right approach.

Which as it turns out, is horribly wrong – at least for me.

Most of my work on Write500 was underpinned by that. The first, most basic thing it was meant to do was deliver daily writing prompts (a tool I always wanted to build anyway). But beyond that, I wanted Write500 to solve two other problems: Be a daily go-to tool to produce new content, and be a revenue-generating SaaS product.

Except that neither of those things (and it’s obvious now) actually move me any closer to me being a writer. It’s actually the exact opposite: I’m creating new tasks for myself that specifically prevent me from writing, but justify it by telling myself that once I build this, I’ll be equipped to write.

Which is bullshit, and I think I always knew it was bullshit, but I let myself believe that anyway.

Another big dimension to all of this is that I’m doing all of this work in my spare time. What little of it I have, anyway. Time to work on these sorts of things is a scarce resource for me, and I haven’t been making very good use of it by focusing almost entirely on things that move me in the exact opposite direction of my goals.

And so last night, while processing all of this (and failing to fall asleep) I came to the eventual realization that I have to kill Write500. Specifically, the extensions to it – the daily prompts thing is still quite useful, and low-maintenance on my part.

Once I actually go through the process of producing and publishing something, I’m sure I’ll uncover lots of problems from that experience – and I might find a gap that could be filled with software.

For now, though, I’m rolling everything back and parking this project. A part of me still hates to do it, but the reality is that I have limited time available to me, and I’m not actually making the progress I want to make.

Instead, I now know I should be focusing on the things that are outside my comfort zone: Namely, writing things I think are shitty, and sharing them with people that might have nothing good to say about it. Which will be a start 🙂

Write500 and the Abyss of Reluctance

The story of Write500 so far can be best summed up in this commit chart from Bitbucket:

Screen Shot 2017-05-30 at 6.11.39 PM

The project started in December 2016, with the intention to create a tool to help writers write every day. At the time, I was pretty optimistic about my ability to retain focus:

…if I can finish this off as intended, I’ve got some other feature ideas to throw in. But right now, I shouldn’t get distracted 😉

I wrapped up and launched the first version before December drew to a close, and I was able to open registrations on 1 January 2017 – ringing in the New Year with a new project.

I then engaged the marketing engines, trying out a few different formats, and it wasn’t long before I reached the 500 subscriber mark. That, too, was hugely motivating – and it was less than a week later that I created the write500-app repository, starting work on the “real” version of Write500.

This was going to be the web app that I had originally envisioned, plus a ton of new ideas that I had picked up over the first few weeks. It was going to be an all-in-one of sorts: Writing and statistics, a built-in social feature, built-in community, and enough features to support two pricing tiers.

At the time I wrote my 28 January update, I had not actually done any creative writing since the 13th (about the time I started planning out the full app). I figured it was okay to let that lapse, since I was focused on building something that would eventually help me get back on track.

That momentum carried all the way through into the first two weeks of February, which is when the inertia set in. It’s pretty clear from the chart above: my code commits to the project simply fell flat. Looking back on it now, there were two main reasons:

Feature Fatigue

Too much, too early. I had actually managed to build up (at least, in my head) a beast of a system, but for every new feature I added, it felt like more features were missing. Pretty soon, the lists of things I planned on implementing had eclipsed the intention of the project itself (seriously, I was reinventing activity feeds towards the end).

I could actually start to feel the drift: More and more hours were being spent productively, but it felt like every new commit was pulling me further away from launching a usable product. Worse, it was getting harder for me to justify why the code I was working on, would actually help writers write more.

In the space of a few weeks, Write500 turns from an exciting project, to something resembling dread – towards the end, I just couldn’t bring myself to open the IDE and carry on working. I was firmly in the Abyss.

No Dogfooding

I had stopped writing every day – the irony of which does not escape me. At some point it became more important to me to work on Write500, than it did to actually write every day – the very problem Write500 purports to solve.

As the necessity to write every day started to wane, so did my motivation for solving the problem. It was a spiral I managed to trap myself in pretty effectively – as the scope of the product expanded, and my capacity for daily writing diminished, I thought I could solve the problem by designing extra features.

I had lost sight of trying to solve the problem for my hypothetical users in the simplest way, and was instead trying to solve a problem that existed entirely in my mind.

With the marketing campaigns long-expired (and subscribers only trickling into the free list), and my own capacity for writing eaten up by software development, that vicious cycle finally ground me down to complete ineffectiveness around mid-February.

It’s usually around this point in my projects where I just give up – I decommission, archive, and shelve, chalk it up to my inability to stay on-target, and move on. And I came close to doing that several times.

How is Write500 different? In the end, I think it had everything to do with this chart:

Screen Shot 2017-05-30 at 6.36.35 PM.png

After almost 5 months on the daily list, less than 25% of users had unsubscribed. More than 600 people were still getting value out of those daily prompts.

That chart gave me a different perspective on the problem entirely. Where I had been trying to solve problems with introducing ever-more-complex features, most users to date had simply been carrying on with the free list.

Maybe I was over-engineering it? That thought only occurred to me around mid-April. Maybe it would be possible (even, desirable) to throw away everything except the core experience (getting a new prompt every day), and basically start over.

The Great Purge

And so on 11 May 2017, I started doing just that.

Screen Shot 2017-05-30 at 6.42.57 PM.png

I gutted the entire project – all the controllers, views, models, migrations, resources, assets, most of the configuration, most of composer.json. And then I started over.

By the end of the first day, I had re-implemented the basics – authentication, prompts, the basic writing interface, the streak display. All perfectly-functional components of the behemoth project, brought over more-or-less intact.

The remarkable thing here? In the old project, those exact same components felt like smaller by-products of a larger vision. In the rebuild, with a fresh perspective, they actually felt like core components again. I found myself able to chart a much clearer path between the code I was writing, and the value I expected my users to be able to get out of this.

A week later, I had the subscription mechanism and Paypal integration restored, and documented better than before. I added a new Statistics mechanism, which now tracks and records wordcount and speed pretty much in realtime. I added the export options which were initially high on my list of priorities, but had fallen by the wayside.

This purge-and-refactor process brought back all the motivation I had lost before. Write500 transformed again – from something seemingly without end, to a project I could conceivably finish.

All the commits from the 21st onwards were mostly cleanup and polish – fixing typos, rearranging screen elements, testing in Browserstack (unbelievably useful) for the major mobile devices, adding a streamlined migration onboarding path from the free list service, and so on.

This past Sunday (the 28th) I rounded it off by adding the Terms and Privacy pages (TermsFeed.com was enormously helpful for the former), and finally pushed the v0.3.0 tag. I did this while sitting in a hotel room in London, having just landed a few hours before.

On Monday (thankfully, a bank holiday in the UK), I was able to refine and run the new dispatch system, and gave it a full day to test. And today, I completed the migration of all users from the free list service to the new version of Write500.

Less really is more

There’s an excellent quote – the origins of which I have long since forgotten – which I routinely forget to apply to my own work (and I’m paraphrasing a bit):

Every project eventually exceeds the developer’s capability to maintain it.

Write500 outgrew my ability to maintain it even before it had made it out of my dev environment – which is not smart, and is the reason I failed to launch it in the first place.

With limited time and resources, the smartest approach is almost certainly the leanest one. The version of Write500 I have deployed right now (0.3.5) is a far cry from the vision I have for it, but it has one compelling thing in its favor: It exists.

 

Carrying on

It exists, but it’s also only getting started. The real test is whether or not there is actually a market for this. I’m happy with the way the free list has performed – there’s clearly some demand out there for tools that make consistent writing easier to achieve.

Is there enough demand, though, to turn this into a paid product? I guess only time (and marketing!) will tell.

At the very least, I’m glad to have been able to make this amount of progress. Write500 is the first project that actually came back from the Abyss of Reluctance, and made it into production.

Which, right now, is enough for me!

Integrating Laravel and Flarum

One of the features I wanted to add to Write500 was a community forum. I’ve worked with Flarum before, and while it’s still ostensibly beta software, it has some really nifty features and works quite well across desktop and mobile. I decided early on that I’d plan to integrate a standalone instance of Flarum into Write500.

Flarum has a JSON API, which as it turns out is really easy to use (despite being insufficiently documented at the moment).

I wanted to set up an integration such that Write500 handled the authentication (you could only log in from the site, not the forum itself), and that changes to a user’s Write500 profile would be synced across.

Laravel Flarum - Page 1 (4).png

This integration broke down into several parts. I’ll go into detail on each one here – hopefully it’s useful!

Contents

  1. Laravel Setup
  2. Flarum Setup
  3. Integration: Creating a new user
  4. Integration: Authenticate the user
  5. Integration: Avatars
  6. Conclusion and Additional Notes

Laravel Setup

At the outset, I implemented a “nickname” field on my User model, with the following constraints (in the validator() method in RegisterController).

'nickname' => 'required|max:32|unique:users'

Those are the same constraints Flarum uses. So now, when I integrate the two (using the nickname as the Flarum username), I can be at least 99% sure it’ll work every time – barring the inevitable weird-unicode-character issue, or some other edge case.

I also ended up adding the following to my User table:

$table->string('nickname');
$table->integer('forum_id')->nullable()->default(null);
$table->string('forum_password')->nullable()->default(null);

Those fields will be populated when the integration runs, and are critical for everything else.

Finally, I needed a few configurable fields. I added these to my config/app.php file:

'forum_url' => env('FORUM_URL', 'https://forum.write500.net'),
'forum_admin_username' => env('FORUM_USER', 'default_admin'),
'forum_admin_password' => env('FORUM_PASS', 'default_pass'),

Note that in the samples below, I don’t reference any of these fields directly.

Flarum Setup

There are a few tweaks made to Flarum to enable all of this. In summary:

  • Installed the latest version of Flarum
  • Set up HTTPS using LetsEncrypt
  • Added the Flarum Links extension
  • Set up the default admin user as a generic account (not named after myself)
  • Disabled new-user registration

About the Links extension

When a user navigates to the community, I want them to be able to navigate back to any other part of the site, without feeling like they’ve really left Write500. To do this, I used the Links extension to create a navbar with the same links as on the main site.

Screen Shot 2017-02-13 at 2.31.49 PM.png

Header on main site

Screen Shot 2017-02-13 at 2.31.58 PM.png

Header on Flarum

I could have used Flarum’s Custom HTML Header feature for that, and created my own menu bar to replace the forum’s one, but that would have lost me the search bar and notification indicator.

About the default admin user

I use the username and password for the admin user to drive the integration. I could technically create an API key on Flarum’s side and just use that instead (it would save me a CURL call), but that also means going directly into the Flarum database to do stuff, which I wasn’t keen on.

One day, when you can generate API keys in the Flarum interface itself, that would obviously be the way to go.

Note about the code samples

The code samples below (all the gist embeds) are not taken directly from my Write500 source – they’ve been adapted for standalone illustration.

Integration: Creating a new User

My first integration requirement:

Whenever a new user is created in Laravel, that same user must be created and activated in Flarum.

The following has to happen:

  1. Authenticate to the Flarum API with my admin account
  2. Create a new user
  3. Activate that user
  4. Link that user to our Laravel user

I ended up putting all of that in a Job, and dispatching it when a new User object was created (in my RegisterController):

dispatch(new \App\Jobs\SyncUserToForum($user));

And now for the actual code!

Step 1: Authenticate

Nothing complicated here – just POSTing to the API to get a token back for this session. This would be the step I could skip if I had a preconfigured API token.

This same method is used whenever I need to log into the Flarum API, so for brevity I’m excluding this code from the rest of the samples.

Step 2: Create a new user

I should, strictly, be checking the API first to see if that nickname or email exists – to prevent any attempts to create duplicate users. However, both Laravel and Flarum itself should fail a new registration if the nickname or email address fields are not unique – so I should be okay there.

Step 3: Activate the user

This could possibly be done in the registration call itself (sending isActivated=true as an attribute upfront), but I’m breaking this out for another reason.

I know that, at some point, I’m going to set up a registration workflow on Write500 that requires users to activate their account before they can log in, and I’m only going to trigger the Flarum activation at that point.

What this means, practically, is that a user’s nickname will be reserved across the system, but will only be @-mentionable once their Write500 account is activated.

Step 4: Link to our Laravel user

When I created this new user, I generated a random password like so, which was used to create the user on Flarum:

$generated_password = substr(md5(microtime()), 0, 12);

The user never has to know this password, since the actual authentication is handled by the site. So we just need to update a few things on our user model:

// $new_user is the result of the POST /api/users call
$user->forum_id = $new_user->data->id;
$user->forum_password = $generated_password;
$user->save();

The forum user ID is not strictly required, but it makes it easier to check whether the user has a forum account yet or not – we don’t want to present the sign-in link to the user before they have a forum account to sign in to:

if($user->forum_id != null) { /* Integrated successfully */ }

Summary

So we now have a job that registers a new user account on Flarum once we have a valid Laravel user – none of it requiring any modifications to Flarum’s internal database.

Integration: Authenticate the user

Next, we need users to be able to authenticate to the forum.

Create a one-click link on Laravel’s side that authenticates and redirects the user to the Flarum instance.

There are a couple of ways to handle this, and I’ll admit upfront that the solution I’ve come up with (as of right now) may not be the best, security-wise.

The following has to happen:

  1. Create a new session token
  2. Set that as a cookie, then redirect to the forum
  3. Update the forum to only allow authenticated users access

Flarum makes this quite easy – the Token you get from the /api/token endpoint can be used as a flarum_remember cookie.

Step 1: Create session token

Once a user has been provisioned in Flarum (they have a numeric forum_id value on their model), we can use their nickname and password to generate a new session token, in the same way we authenticated the admin user.

Step 2: Set token value as cookie

I butted heads with this step for a while, before coming to a solution. On the Flarum side, I had to create a new auth.php file in the forum web root (where index.php and api.php are), with the following:

All that file does is take whatever token value you give it, set it as a cookie, and then bounce you to the forum homepage.

In terms of security, this setup is making me a bit nervous. However, it’s worth noting:

  • The token is sent over HTTPS, and HTTPS URLs and parameters are encrypted
  • The only way to hijack someone else’s account using this, would be knowing their session ID – at which point you can manually add that as a browser cookie anyway
  • The only other exploit that’s possible here is someone trying a script injection attack, but again, that’s nothing you cannot set in your browser manually
  • If a user tries hitting this route manually with a bad token value, they’ll be bounced to the forum, invalidated (see below) and then sent back to the login page.

With that script set up, you can now redirect the user to it from Laravel (as in Step 1), and they’ll be bounced to the forum, fully authenticated.

return redirect()->to('https://my.flarum.url/auth.php?token=' . $session->token);

Step 3: Only permit authenticated users to view the forum

This was another ugly script hack. Flarum itself lets you set permissions so that guests cannot read content – they’ll just see a “no discussions” message when opening the forum.

However, since I’m also planning on removing the Login button on the forum, the only way an unauthenticated user will be able to log into the forum, is by going via my login page.

So I need to add a redirect before Flarum itself fires, to check whether the user is authenticated. I ended up with this:

When a user authenticates with Flarum (which all runs via the api route eventually), a new access token is generated and stored in that table. If the value matches, they’re logged in – if not, they’re bounced to my login page. After logging into Write500, hitting that Community login route will authenticate and take them through.

That could be improved by reading the database connection details from config.php instead, but those values are not likely to change for this setup. I’d also have preferred it if Flarum used some sort of file storage for sessions, in the same way basic Laravel does – but we can’t always get what we want 😉

Integration: Flarum Modification

So we now have the basics down – new users are registered, users can log in via my site, and users cannot reach the forum without being authenticated.

Modify Flarum to integrate the navigation with the main site, and prevent users from logging out via Flarum.

On the Flarum side, I want to change what some of the buttons in the navigation do – for instance, they shouldn’t be able to log out via Flarum (since that still leaves them logged into Write500), and I don’t want them to be able to manage their forum settings directly in Flarum.

After some trial and error, and almost giving up and hacking on the core scripts, I eventually landed on this neat hack.

In the Flarum administration section, under Appearance, you can add your own custom HTML. I used that to deploy a script block that looks like this:

That script is loaded in the page header, and executes immediately. At the end of the page, Flarum loads its app script and creates an app object. Before that happens, there’s nothing to modify, and I cannot force my javascript to run at the end of the page. So instead, my script keeps checking to see if the app variable is available, and when it is, it makes some changes.

With that script in place, the forum is now about as integrated as it can get – there’s just one more thing to cover – Avatars.

Integration: Avatars

Users can upload and crop avatars to Write500.

Automatically synchronize the user’s avatar from Laravel, to Flarum.

I wanted those avatars to sync across to the forum, whenever they were created or updated on Write500’s side. I also have a “Reset to Default” which deletes the current avatar, and that deletion should be synced across as well.

Delete Avatars

This one was pretty easy. On the route that resets my avatar, I dispatch a job that basically does this:

It authenticates to the Write500 API, and issues a delete request against the avatar resource. Simple.

Upload avatars

This one’s a bit more complicated. The API endpoint is expecting a form-data upload, which takes a bit of fiddling to get right on PHP’s end.

After a user uploads, crops and saves an avatar, it’s stored in the public/ directory with a GUID, which is saved against the User model. All I really need to do is read that file from disk and upload it to the Flarum API. Which I achieve like so:

That bit of code will blindly upload a new avatar. Flarum doesn’t require you to delete an existing avatar before uploading a new one, so I can trigger this all I want. Right now, it’s just dispatching as a job after a new avatar is saved in Write500.

I’m not sure whether or not the User-Agent is strictly required for this, but I haven’t had time to go back and test it. I do know that the Flarum API does behave differently depending on whether it thinks it’s being hit from a browser.

Conclusion

So that’s an exhaustive look at how I’ve enabled the following integrations:

Laravel Flarum - Page 1 (5).png

I’m sure that some elements of this could be improved, and if I make any substantial updates to this, I’ll likely write a follow-up post here.

Additional Considerations

This is all still a work in progress, and I’m sure I’ll find more bugs as I go along.

Right now, for instance, I think it might make more sense to keep the user token cached in the User model, and only issue a new one when the current one expires. Right now, if you upload an avatar (for instance), it generates a new token that might invalidate the old one. So you’d have to log into the community from scratch to get access again.

There’s also notification integration. There’s an API endpoint that returns information about new notifications (@-mentions, etc), which I would want to display in the Write500 navbar.

I’d also want to take over the profile view eventually. I’m aiming to build a user profile page in Write500 to show some high-level stats, and I’d rather integrate forum activity in there, as opposed to having Flarum display any user profile page at all. Finally, I need to consider what will happen when users cancel their Write500 subscriptions.

Those are all questions for another day though!

Write500 – One Month In

I promised myself I’d write a post at the end of January, reflecting on the progress and lessons learned from Write500 so far. It’s been a busier month than I was expecting!

My goals

At the start of the month I set a few goals for myself:

  • I’d write to the same prompts that users get every day
  • I’d broadcast 1 image per day on social media to promote Write500
  • I’d work towards a web app that could go into beta by the end of the month

As it turns out, doing all of that with consistency, on top of having a regular 9-5 is a bit more of a challenge than I initially thought!

Some numbers

Before I get into the full run-down, here’s the part most people are likely to care about: the stats!

I started Write500 as a free mailing list, launched on 1 January of this year, and promoted in a few places (notably, Instagram). All the campaigns have now expired, and as we approach the end of the month, here are the highlights for that list’s performance:

Screen Shot 2017-01-28 at 3.57.28 PM.png

Of the 709 users currently subscribed to the free daily prompt list:

  • 683 (96.2%) are still active, and receive daily emails
  • 26 (3.7%) have opted out
  • 144 (20.3%) open their prompt emails every day

Since migrating the user database to the new list system on 17 Jan (more details on that below), I’ve had an additional 23 users join the list – without spending any money, and barely any effort, on marketing.

Screen Shot 2017-01-28 at 4.19.54 PM.png

My traffic curve looks exactly like you’d expect – lots of interest during the time the Instagram campaign was running, and very little when it ended. Interestingly, the campaign only officially concluded in the early hours of 13 January, after interest to the site started slowing down.

Filtering for all traffic starting 14 January, this is what my Acquisition Overview looks like:

Screen Shot 2017-01-28 at 4.22.08 PM.png

It’s encouraging to see Referral and Organic Search showing up in Channels. I’ve still got a ton of work to do, in terms of content marketing – a full site+blog overhaul, to start.

Finally, and probably my favorite part:

Screen Shot 2017-01-28 at 4.24.42 PM.png

Write500 is now ranking #1 for its own keyword on Google, Bing and Yahoo (hence DuckDuckGo putting it at #1 too). Which, to be fair, wasn’t much of a challenge – there is only 1 other domain (write500.com) that could be said to be competing for that keyword.

I do feel that this bodes well for offline and word-of-mouth marketing, though!

Actual achievements

Goal #1: Writing

In terms of the writing, I started out pretty strong – I produced 12249 words between 1 Jan and 12 Jan. There were a few blog posts in there, but most of the words came from the daily prompts.

I started losing traction on this around 13 January, which is no coincidence – that’s when my mental bandwidth for this project started to narrow, with dayjob-related work ramping up. I had to start making compromises, and figured that it would be better to prioritize the app itself.

I’m still pretty happy with what I achieved here, though – 12k words in January represents more than I wrote across most of 2016. And I now know I’m capable of this, I just need to find the mental space again.

Goal #2: Image Publishing

When I started working on the prompt database for Write500, my first step was to collect 366 inspirational, motivational and literary quotes. They were originally intended to be used to decorate the daily emails, providing something else to read other than the prompt and the instructions itself.

When I spun up the Write500 Instagram account, I realized I could find a secondary use for that content – publishing each quote as an image. So I created a batch of 25 images – here’s a sample:

I used one of those as the creative on my Instagram ad, and it performed really well, so I decided to keep publishing those images every day. Even after the campaign ended, those images kept getting likes (as much as 80 per image).

However, this too suffered from a lack of time. I published every image on schedule, from 1 Jan to 18 Jan, until I missed my first day. I then managed to publish daily until the 21st, at which point I hit a bit of a wall.

I needed to make time to create more images in order to continue publishing daily, but at this point there wasn’t an immediate return on it. Even though the images are well-received on Instagram, I got very few signups that way. The only benefit to continuing might have been to keep building up my profile, which felt a bit like putting the cart before the horse: There’s still no app I can send people to.

I will eventually pick this up again, and produce the remaining 340-or-so images. That’ll be a marketing exercise though, and right now, the app is more important.

Goal #3: A Write500 Beta

I might actually make this one! Right now, it’s Saturday afternoon. I’m taking a break from a few hours of dev to write this post, and I’m intending to continue when it’s published. There’s only one core feature left to add, a few style cleanups, and then I’ll be able to start inviting beta users.

I had to jump through quite a few hoops for this one. When I started Write500, my plan was to build one app that allowed both free and paid signups, and I’d port over the users from the free mailing list into it.

When I actually tried building that out though, I started running into issues. Small ones, sure, but I could already see the technical debt forming – I’d have to make lots of exclusions and compromises to have both personas co-exist in one database.

So instead, I took a few days to break out the free mailing list feature into its own project. It now lives at https://list.write500.net as a standalone automated mailer, and exists independently from the main project.

I might even polish that up and release it as a standalone product someday. It just needs a nicer content editor and subscriber management features, but is otherwise a fairly capable bulk mailer (including a bespoke interaction tracker), using the very-reasonably-priced Amazon SES as a backend. Who can argue with $0.10 per thousand mails, and not having to send your subscriber or interaction data to a third party?

With the List system out of the way, I started building the app itself. I didn’t have much of an architecture or feature layout in mind when I started – I just knew the broad strokes of what I wanted to accomplish. Which, to date, has been the following:

  • Subscription integration with Paypal (the only global payment provider that I can use right now), using their PHP SDK to create and execute Billing Agreements.
  • An MVP for the main interaction loop (receive prompt, write words, get statistics), in a basic but functional Bootstrap-based UI
  • A Programs feature, that delivers the prompts on a schedule – this replaces the mailing list in many ways.

And I do have a few screenshots to share! Click to enlarge.

Up Next?

So right now, Write500 addresses the write-in-isolation use case: the writers who prefer to work alone, and only need to see their statistics and streaks building up over time for motivation.

Of course, there are other types of writers out there. I’m in a slightly different cohort myself: I prefer writing as part of a small group, sharing notes and progress as I go. I find that being surrounded by other writers helps the motivation somewhat.

Logically, Write500 needs a Groups feature of some kind, but I’ve given that far more thought, and I’m going to try something kinda new: Tribes.

About Tribes

One of the problems I have with normal groups (the sort you have on Facebook, for instance), is that as the size of the group goes up, the quality of the interactions goes down.

Not all interactions, of course – users still post useful things, and engage in useful ways. But when a group gets too big, it loses its initial sense of closeness and community. More members eventually means more rules, more moderation, and inevitably, users going quiet, leaving the group, or splintering off to form their own.

For Write500, I need a Groups feature that encourages everyone in the group to engage on a regular (daily, preferably) basis, while still feeling that they’re getting good personal engagement with other users. Doing this in a classic open-ended group would be difficult. That sort of interaction would be deafening, for one: signal-to-noise will be going way down.

And there’s also the fact that different personas want different things out of their groups. Some prefer lively debate, some prefer terse updates, some prefer checking in multiple times per day, some prefer checking in once a week.

So with Tribes, I’m going to create a groups feature that has the following characteristics:

  • Max of 10 users per Tribe (to start), and Tribes need to split in order to grow.
  • Users join on a time-limited tryout (or need to be invited), and every other member of the Tribe has to explicitly (and anonymously) vote to include them permanently
  • Notifications and events from Tribes will have a dedicated section in Write500 (Tribe Newsfeed), and be the most visible form of notification available
  • Users can only belong to one Tribe at a time

Completely antithetical to standard community growth tactics? You bet!

With Tribes, I specifically want groups of writers to form strong relationships with other writers in their genre/pace/orbit, and feel at ease about sharing more about their work than they’d regularly share on an open group.

I also want to make sure that being in a Tribe is a rewarding experience, and that other members of the Tribe pay attention to what you have to say – not always a given in a group that can grow uncontrollably.

When this comes to the actual writing, there’d be integration there too – the ability to broadcast a completed prompt to your Tribe, letting them review and comment on your work. Which feels like a better solution to me, than just broadcasting your work in a public square and hoping to catch people’s attention.

Of course, this entire experiment could fail and I just end up going back to standard groups! I’m optimistic, though, that a format like this would create an environment that some users would find useful.

Shoutout: StartupStudyGroup

If you read this far down, well done! You’re clearly someone who’s interested in the details, and how new products and services come about -so you really should check out startupstudygroup.com!

I joined the SSG Slack group earlier this year, and the community there has provided me with valuable insights and encouragement so far. If you join the SSG Slack, come say hello in #write500!

And now, back to the grindstone.

Laravel 5.3 + Cloud9

I use Cloud9 as my primary IDE – it handles all my PHP projects, my Domo app projects, and when I start seriously developing Ionic-based mobile apps, I’ll find a way to use Cloud9 for that too.

I’ve found it particularly handy for Laravel development: A lot of the prerequisite software provided by Homestead comes pre-installed on the Ubuntu VM that Cloud9 provides, and with a few small config tweaks, it’s possible to get up and running very quickly.

1. VM Setup

One of my cardinal rules: All your code should exist in a repository. Especially with web development, there’s basically zero excuse to keeping substantial amounts of code locally.

My first step is to always create a blank repository, and use that as the clone URL for a new Cloud9 VM.

2017-01-20 05_48_50-Create a New Workspace.png

The Blank Ubuntu VM is good enough for our purposes – there’s no benefit to selecting the PHP VM, since we’ll be installing the latest PHP in any case.

2. Services Setup

Chances are, whatever you’re building in Laravel will need the MySQL service to be running, and in most cases, you’ll probably want Redis too. Cloud9 has made it a little bit more difficult to automate service startup (presumably to minimize unnecessary resource utilization), but there’s a way around that.

If you’re using a premium workspace, Cloud9 will keep it “hot” between sessions – everything you start will continue to run even after closing the IDE window itself. But that will expire after a while, and it can be a hassle to keep restarting your services.

So, the first change we’ll make to our new VM is editing the /.profile file, and adding two new lines to the bottom:

$ cd ~/
$ echo 'sudo service mysql start > /dev/null 2>&1' >> .profile
$ echo 'sudo service redis-server start > /dev/null 2>&1' >> .profile

That will start the services (and skip startup if they’re already running), won’t show any console output, and will run whenever a new terminal window is created. So now, every time you open up a new terminal window, you’re guaranteed to have those services running:

2017-01-20 06_11_31-cloud9-demo - Cloud9.png

It’s not as ideal as having the server start it automatically, but in my opinion, this is a small trade-off to make.

3. PHP 7 Setup

If you’re deploying code to a new server on DO, or AWS, or Rackspace, or most other places, chances are it’s running PHP 7. By default, Cloud9 does not include PHP 7 (for reasons which escape me), so we need to take a quick detour into apt-get land.

There are two ways to set up PHP here – apt, or phpbrew. For a while I was using phpbrew, which builds PHP from source. It’s a decent option, but using ondrej/php is faster.

$ sudo add-apt-repository ppa:ondrej/php

Then you’ll need to sudo apt-get update. When that’s done, you should have access to the php7 packages:

2017-01-20 06_18_23-Groove Music.png

I’m using php 7.1 here, because that’s what my DigitalOcean VM is running. Laravel will need that, plus a few other modules:

$ sudo apt-get install php7.1 php7.1-mysql php7.1-curl 
  php7.1-xml php7.1-mbstring

That shouldn’t take more than a minute. And now we’re good to go:

2017-01-20 06_21_26-cloud9-demo - Cloud9.png

4. Laravel Project Setup

There are quite a few ways to get Laravel installed. It has a CLI-type thing that lets you do laravel new, you can clone it directly from github, or you can use Composer.

I tend to favor composer since it’s a once-off command for this workspace. There’s no benefit to installing any more tools here, since we’re only going to create 1 Laravel project in this VM. And since we’ve already got our workspace folder linked to our project repository, git can get messy.

So my favorite – use composer to set up Laravel in a temporary directory, then move it all across:

$ cd ~/
$ composer create-project laravel/laravel tmp --prefer-dist
$ mv tmp/* workspace/
$ mv tmp/.* workspace/
$ rmdir tmp

Why two move commands? The first catches all the files, but Laravel does include a few dotfiles that aren’t caught by the first move command. Sure, you could configure dotglob to modify mv’s behavior, but we’re only doing this move once for the workspace.

Just one more setup step – the database. Start by adjusting the .env file to set the username to root, and a blank password:

DB_CONNECTION=mysql
DB_HOST=127.0.0.1
DB_PORT=3306
DB_DATABASE=homestead
DB_USERNAME=root
DB_PASSWORD=

Then we’ll create that database in one go:

$ mysql -uroot -e "CREATE DATABASE homestead;"

You can now run the initial migrations if you want.

5. Node Setup

If you’re planning on using Elixir to compile frontend assets, you’ll probably want to upgrade to the latest version of Node. Cloud9 makes this easy:

$ nvm install 7.2
$ nvm alias default 7.2

All node commands will now use the 7.2 binary. You can now install the necessary components in your workspace folder:

$ npm install

6. Running Laravel

Cloud9 features a reverse proxy – anything that binds to port 8080 in the VM will be accessible via a URL unique to your workspace. It also has a dedicated “runner” feature, which we’ll configure like so:

2017-01-20 06_33_15-cloud9-demo - Cloud9.png

The name can be anything, but the run command will be:

php artisan serve --host=0.0.0.0 --port=8080

Cloud9 does make mention of $HOST and $PORT variables, but they’re always the same values anyway. Fill that in and click the Run button on the left:

2017-01-20 06_33_53-cloud9-demo - Cloud9.png

Cloud9 should open up an in-IDE browser window that points to your site. You can click the little pop-out button on the right of the “address bar” to bring it up in a new browser tab.

2017-01-20 06_36_01-cloud9-demo - Cloud9.png

That domain (c9users.io) will check for cloud9 authentication before proxying to your VM – in short, that URL will only work for you, and anyone else on cloud9 that you’ve shared the workspace with. Those settings are managed via the Share button on the top right:

2017-01-20 06_37_40-cloud9-demo - Cloud9.png

From that popup, you can invite new users to the workspace (cloud9 does issue free accounts), or you can just make the application URL public:

2017-01-20 06_38_34-cloud9-demo - Cloud9.png

You can now share that URL to anyone.

7. Get coding!

You’ve now got just about everything you need for proper Laravel development! It’s usually at this point I then take the extra step of configuring Amazon S3 as a cloud disk, push the blank Laravel project to bitbucket, and deploy my repo as a site on Forge.

Use Amazon S3 with Laravel 5

Laravel’s Filesystem component makes it very easy to work with cloud storage drivers, and the documentation does an excellent job of covering how the Storage facade works – so I won’t repeat that here.

Instead, here’s the specifics on getting Laravel configured to use S3 as a cloud disk. These instructions are valid as of 4 January 2017.

The AWS Setup

On the AWS side, you need a few things:

  • An S3 bucket
  • An IAM user
  • An IAM policy attached to that user to let it use the bucket
  • The AWS Key and Secret belonging to the IAM user

Step 1: The S3 Bucket

Assuming you don’t already have one, of course.

This is the easiest part – log into AWS, navigate to S3, and create a bucket with any given name. For this example, I’m using write500-backups (mainly because I just migrated the automated backups for write500.net to S3):

2017-01-04 00_33_39-S3 Management Console.png

1. Easy button to find

Then:

2017-01-04 01_41_14-S3 Management Console.png

2. Select your region – with care

US Standard is otherwise known as North Virginia, and us-east-1. You can choose any region, but then you’ll need to use the corresponding region ID in the config file. Amazon keeps a list of region names here.

If you’re using this as a cloud disk for your app, it would make sense to locate the bucket as close as physically possible to your main servers – there are transfer time and latency benefits. In this case, I’m selecting Ireland because I like potatoes.

Step 2: The IAM User

Navigate to IAM and create a new user. AWS has made some updates to this process recently, so it has a lot more of a wizard look and feel.

2017-01-04 00_37_17-IAM Management Console.png

1. Add a new user from the Users tab

2017-01-04 00_37_33-IAM Management Console.png

2. Make sure the Programmatic Access is ticked, so the system generates a Key and Secret

Step 3: The IAM Policy

The wizard will now show the Permissions page. AWS offers a few template policies we’ll completely ignore, since they grant far too much access. We need our user to only be able to access the specific bucket we created.

Instead, we’ll opt to attach existing policies:

2017-01-04 01_45_53-IAM Management Console.png

3. This one

And then create a new policy:

2017-01-04 01_46_01-IAM Management Console.png

4. And then this one

This will pop out a new tab. On that screen, Select “Create Your Own Policy”.

  • Policy name: Something unique
  • Policy description: Something descriptive
  • Policy document: Click here for the sample

Paste that gist into the Policy Document section, taking care that there are no blank spaces preceding the {. Replace “bucket-name” with your actual bucket name, then save:

2017-01-04 01_50_40-IAM Management Console.png

If only insurance were this easy

Go back to the IAM wizard screen and click Refresh – you should see your brand new policy appear at the top of the list.

2017-01-04 00_40_36-IAM Management Console.png

Perfect!

Tick the box, then click Next: Review, and then Create user. It’ll give you the Access key ID and Secret like so:

2017-01-04 00_40_50-IAM Management Console.png

3. When you complete the wizard, you’ll get these.

The Access Key ID (key) and Secret access key (secret) will be plugged into the config file.

Step 4: Configure Laravel

You’ll want to edit the filesystem details at:config/filesystem.php

Near the bottom you should see the s3 block. It gets filled in like so:

fixed.png

Remember to set the correct region for your bucket

And done! The filesystem is now configured. If you’re making your app portable, it would be smart to use env() calls with defaults instead, but I’ll leave you figure that one out 🙂

Step 5: Test

The simplest way to test this is to drop into a tinker session and try working with the s3 disk.

2017-01-04 01_57_16-forge@write500_ ~_write500.net.png

And you should see the corresponding file in the S3 interface itself:

2017-01-04 01_57_43-S3 Management Console.png

Step 6: parrot.gif

Now that you have cloud storage configured, you should use it!.

First (and this will take minimal effort), you should set up the excellent spatie/laravel-backup package. It can do file and db backups, health monitoring, alerts, and can be easily scheduled. Total win.

You can also have Laravel do all its storage there. Just change the default disk:

2017-01-04 02_01_34-write500 - Cloud9.png

config/filesystems.php

This has the benefit of ensuring that even if your server crashes/dies horribly, nothing gets lost. You can also have multiple server instances all talking to the same s3 disk.

In my case, I’m using S3 as the storage for regular backups from write500. I’ll also use the same connection and attempt to publish my internal statistics dumps as CSV directly to S3 – meaning I can pull the data in via Domo’s Amazon S3 connector. I can then undo the SFTP setup I created previously, further securing my server.

Project: Newskitten

The world is a big, messy, complex, and sometimes-scary place – and it’s often made to seem more scary than it really is, thanks to mainstream news media.

Now don’t get me wrong, I’m not some sort of tinfoil-hatted alt-truther who believes there are secret forces pulling the strings behind what we see and hear in the news. I know it for a fact.

News outlets need to generate revenue to survive (they’re not charities), and over time, the proven revenue generators have been bad news. The day Oscar Pistorius shot Reeva Steenkamp was a fantastic day for online news, and for online ad revenue – that’s the sort of market force that ends up shaping editorial policy.

strip-les-journalistes-aujourdhui-650-finalenglish2

News organizations are optimizing for what works, and unfortunately, what works is not good. They’re forced to give audiences less of what they need, and more of what they want.

What people want, it seems, is sensationalism. They want news that angers and upsets them, that gives them something to talk and argue about, that conforms to their biases and reminds them their opinion of the world is still valid. They want to be entertained, not informed – because real information is boring.

At some point, the content that news providers put out there stopped being informative. It stopped being information that applied to you, or information that you needed in order to make an informed decision. There’s almost nothing that you can take from a news website today and use to improve the world you live in.

For example, here’s the top-line news on News24.com right now:

2016-12-22-12_43_40-news24-south-africas-premier-news-source-provides-breaking-news-on-national

This is what the largest and most popular news website in South Africa thinks I need to know today. None of it is relevant or useful to me.

  • Police pursuing N1 City Mall robber, high alert at shopping centres – I’m not in the police. I don’t control their budgets or deployments, I have no involvement in shopping center security, and the only way this applies to me is if I go shopping at a mall – at which point the people who’s jobs it is to keep the mall safe, will do their job to the best of their ability.
  • Helen Zille is a chief racist – ANC Western Cape – An empty statement from a political party. I don’t know Zille, I have nothing to do with either party (no say in their internal structures, no membership), and when it comes time to vote in a few years, I’ll cast my vote based on what the parties have actually done for me. News like this does nothing to sway my opinion.
  • We will show you how the ‘soccer’ game is played – KZN’s ANC – more political noise. I’m not in the ANC, I’m not in KZN, I’m not attending that conference, or have any interest in its outcome. Even if I hopped a plane right now and showed up at the conference venue door, there’d be nothing I could do. Hell, even if I was in the KZN ANC itself, chances are I’d have no voice, owing to the fact that my career is not politics, and I’d have no decision-making ability.
  • CONFIRMED: Mashaba sacked as Bafana coach – Zero relevance to me, since I don’t follow sports at all. But even to a sports fan, still zero relevance – hiring decisions are made by team management, not fans. Team managers are doing their jobs, and if there are better ways to do it, then a case can be made for change.
  • The end of Everest as we know it? China plans to build a mega-resort on the mountain – Everest is literally a world away from me. Everything I know about Everest is thanks to movies, books and music – this news is irrelevant to me. If China does build a resort there (which will be interesting seeing as Everest is in Tibet, and the nations have a cold relationship), I’ll have zero say in how it’s designed, resourced, built or maintained. I definitely, as a South African, have less than zero input into any decision made by a Chinese firm.

I could go on – headlines about SA’s nuclear plan, an SPCA investigation into a rodent supplier, Hlaudi Motsoeneng mouthing off: none of this information is useful to me. I doubt it’s useful to most people, since most people are not in a position to do anything with this information.

Except get upset. The mall-robbers story is a great springboard for complaining about how South Africa is chronically unsafe and the police aren’t doing their jobs. The Zille story is the perfect fuel to stoke the continually-burning race war in our national discourse. The Everest story is Christmas to people who think nature should be left alone, and people (or the Chinese specifically) are evil for building things.

None of these stories leave you with anything constructive – you just come away feeling depressed, angry and hopeless. And then, most of the time, you do your friends the disservice of sharing that on social media, spreading the infection further. Worse if you believe that your Facebook likes are actually saving lives.

how-idiots-think-facebook-works

And if you’re like most people, you rationalize it by calling it “being informed”, and that it’s better to be informed, than ignorant. You think that being informed is very important, almost crucial to daily life, and that the news you’re reading is an accurate depiction of the state of things.

Except that it’s not. On top of being driven by what sells, news organizations occasionally employ very biased editors. Editors that would happily tarnish the reputation of their organization for the sake of running, say, a politically-motivated smear campaign.

That was not news. That was a deliberate, blatant fabrication with no fact-checking, presented as news. Even if it had been true, what would you do with information that serves only to confirm your existing biases?

Imagine for a second that it was true, that Maimane was getting lessons from FW. If it was a problem, the DA has its own means of dealing with it. If it’s not a problem, the DA will have spun it to Maimane’s credit. In either case, on the outside, there’s nothing meaningful you could have contributed to the process.

If that’s what an editor is willing to put on the front page, in the middle of an election season, imagine what the editors are willing to do with the every day stories you read online. How many of those have been edited to provide the worst version of events? How many are actually just political hit-pieces disguised as news, aimed at discrediting a person, a party, or a part of society? How can you even tell the difference anymore?

And how many editors are just flat-out ignorant? Like the story of Andrew Kenny, a respected engineer and columnist who visited the Afrikaner town of Orania. His story shows the community in a positive light, and his editor decided not to run the story, since it might offend readers.

http://www.biznews.com/undictated/2015/10/30/heres-andrew-kennys-orania-column-the-citizen-doesnt-want-you-to-read/

News that might offend readers? News is meant to be facts, right? Well-researched, well-documented, delivered responsibly, and if it upsets people, then that’s unavoidable – hell, it’s necessary. Being offended is a natural and healthy part of living in a civilized society – but that’s a whole other topic.

So that’s news, as of 2016. It’s either designed to generate revenue, slanted to an editor’s personal bias, or simply ignored because it doesn’t fit the outlet’s story. The information you really need (civics, opportunities, policies) are all obtained elsewhere, and any time you spend on a typical news website is probably (balance of probability) time spent consuming content that won’t have a net-positive effect on your health or wellbeing.

Which is what I built a Chrome extension to fix.

ktn2

It’s called Newskitten, and you can download it here: Newskitten – Chrome Web Store

It’s still in its early stages. The concept is simple – if it sees a news website, it replaces it with that message above instead.

I’ve been using it for over a week now, and it’s already made a noticeable difference. I’ve clicked through from depressing-sounding headlines on my Facebook feed, only to be met with a relaxing cat. I’ve habitually opened up news websites in moments of boredom, hoping to find something new, and getting that gif instead.

It already feels like the news is leeching a lot less of my time and energy away from me. So much so, that I found the time (and mental clarity) to write this really long blog post, without feeling like the world is about to collapse on me at any moment.