Monthly Archives: February 2017

Integrating Laravel and Flarum

One of the features I wanted to add to Write500 was a community forum. I’ve worked with Flarum before, and while it’s still ostensibly beta software, it has some really nifty features and works quite well across desktop and mobile. I decided early on that I’d plan to integrate a standalone instance of Flarum into Write500.

Flarum has a JSON API, which as it turns out is really easy to use (despite being insufficiently documented at the moment).

I wanted to set up an integration such that Write500 handled the authentication (you could only log in from the site, not the forum itself), and that changes to a user’s Write500 profile would be synced across.

Laravel Flarum - Page 1 (4).png

This integration broke down into several parts. I’ll go into detail on each one here – hopefully it’s useful!

Contents

  1. Laravel Setup
  2. Flarum Setup
  3. Integration: Creating a new user
  4. Integration: Authenticate the user
  5. Integration: Avatars
  6. Conclusion and Additional Notes

Laravel Setup

At the outset, I implemented a “nickname” field on my User model, with the following constraints (in the validator() method in RegisterController).

'nickname' => 'required|max:32|unique:users'

Those are the same constraints Flarum uses. So now, when I integrate the two (using the nickname as the Flarum username), I can be at least 99% sure it’ll work every time – barring the inevitable weird-unicode-character issue, or some other edge case.

I also ended up adding the following to my User table:

$table->string('nickname');
$table->integer('forum_id')->nullable()->default(null);
$table->string('forum_password')->nullable()->default(null);

Those fields will be populated when the integration runs, and are critical for everything else.

Finally, I needed a few configurable fields. I added these to my config/app.php file:

'forum_url' => env('FORUM_URL', 'https://forum.write500.net'),
'forum_admin_username' => env('FORUM_USER', 'default_admin'),
'forum_admin_password' => env('FORUM_PASS', 'default_pass'),

Note that in the samples below, I don’t reference any of these fields directly.

Flarum Setup

There are a few tweaks made to Flarum to enable all of this. In summary:

  • Installed the latest version of Flarum
  • Set up HTTPS using LetsEncrypt
  • Added the Flarum Links extension
  • Set up the default admin user as a generic account (not named after myself)
  • Disabled new-user registration

About the Links extension

When a user navigates to the community, I want them to be able to navigate back to any other part of the site, without feeling like they’ve really left Write500. To do this, I used the Links extension to create a navbar with the same links as on the main site.

Screen Shot 2017-02-13 at 2.31.49 PM.png

Header on main site

Screen Shot 2017-02-13 at 2.31.58 PM.png

Header on Flarum

I could have used Flarum’s Custom HTML Header feature for that, and created my own menu bar to replace the forum’s one, but that would have lost me the search bar and notification indicator.

About the default admin user

I use the username and password for the admin user to drive the integration. I could technically create an API key on Flarum’s side and just use that instead (it would save me a CURL call), but that also means going directly into the Flarum database to do stuff, which I wasn’t keen on.

One day, when you can generate API keys in the Flarum interface itself, that would obviously be the way to go.

Note about the code samples

The code samples below (all the gist embeds) are not taken directly from my Write500 source – they’ve been adapted for standalone illustration.

Integration: Creating a new User

My first integration requirement:

Whenever a new user is created in Laravel, that same user must be created and activated in Flarum.

The following has to happen:

  1. Authenticate to the Flarum API with my admin account
  2. Create a new user
  3. Activate that user
  4. Link that user to our Laravel user

I ended up putting all of that in a Job, and dispatching it when a new User object was created (in my RegisterController):

dispatch(new \App\Jobs\SyncUserToForum($user));

And now for the actual code!

Step 1: Authenticate

Nothing complicated here – just POSTing to the API to get a token back for this session. This would be the step I could skip if I had a preconfigured API token.

This same method is used whenever I need to log into the Flarum API, so for brevity I’m excluding this code from the rest of the samples.

Step 2: Create a new user

I should, strictly, be checking the API first to see if that nickname or email exists – to prevent any attempts to create duplicate users. However, both Laravel and Flarum itself should fail a new registration if the nickname or email address fields are not unique – so I should be okay there.

Step 3: Activate the user

This could possibly be done in the registration call itself (sending isActivated=true as an attribute upfront), but I’m breaking this out for another reason.

I know that, at some point, I’m going to set up a registration workflow on Write500 that requires users to activate their account before they can log in, and I’m only going to trigger the Flarum activation at that point.

What this means, practically, is that a user’s nickname will be reserved across the system, but will only be @-mentionable once their Write500 account is activated.

Step 4: Link to our Laravel user

When I created this new user, I generated a random password like so, which was used to create the user on Flarum:

$generated_password = substr(md5(microtime()), 0, 12);

The user never has to know this password, since the actual authentication is handled by the site. So we just need to update a few things on our user model:

// $new_user is the result of the POST /api/users call
$user->forum_id = $new_user->data->id;
$user->forum_password = $generated_password;
$user->save();

The forum user ID is not strictly required, but it makes it easier to check whether the user has a forum account yet or not – we don’t want to present the sign-in link to the user before they have a forum account to sign in to:

if($user->forum_id != null) { /* Integrated successfully */ }

Summary

So we now have a job that registers a new user account on Flarum once we have a valid Laravel user – none of it requiring any modifications to Flarum’s internal database.

Integration: Authenticate the user

Next, we need users to be able to authenticate to the forum.

Create a one-click link on Laravel’s side that authenticates and redirects the user to the Flarum instance.

There are a couple of ways to handle this, and I’ll admit upfront that the solution I’ve come up with (as of right now) may not be the best, security-wise.

The following has to happen:

  1. Create a new session token
  2. Set that as a cookie, then redirect to the forum
  3. Update the forum to only allow authenticated users access

Flarum makes this quite easy – the Token you get from the /api/token endpoint can be used as a flarum_remember cookie.

Step 1: Create session token

Once a user has been provisioned in Flarum (they have a numeric forum_id value on their model), we can use their nickname and password to generate a new session token, in the same way we authenticated the admin user.

Step 2: Set token value as cookie

I butted heads with this step for a while, before coming to a solution. On the Flarum side, I had to create a new auth.php file in the forum web root (where index.php and api.php are), with the following:

All that file does is take whatever token value you give it, set it as a cookie, and then bounce you to the forum homepage.

In terms of security, this setup is making me a bit nervous. However, it’s worth noting:

  • The token is sent over HTTPS, and HTTPS URLs and parameters are encrypted
  • The only way to hijack someone else’s account using this, would be knowing their session ID – at which point you can manually add that as a browser cookie anyway
  • The only other exploit that’s possible here is someone trying a script injection attack, but again, that’s nothing you cannot set in your browser manually
  • If a user tries hitting this route manually with a bad token value, they’ll be bounced to the forum, invalidated (see below) and then sent back to the login page.

With that script set up, you can now redirect the user to it from Laravel (as in Step 1), and they’ll be bounced to the forum, fully authenticated.

return redirect()->to('https://my.flarum.url/auth.php?token=' . $session->token);

Step 3: Only permit authenticated users to view the forum

This was another ugly script hack. Flarum itself lets you set permissions so that guests cannot read content – they’ll just see a “no discussions” message when opening the forum.

However, since I’m also planning on removing the Login button on the forum, the only way an unauthenticated user will be able to log into the forum, is by going via my login page.

So I need to add a redirect before Flarum itself fires, to check whether the user is authenticated. I ended up with this:

When a user authenticates with Flarum (which all runs via the api route eventually), a new access token is generated and stored in that table. If the value matches, they’re logged in – if not, they’re bounced to my login page. After logging into Write500, hitting that Community login route will authenticate and take them through.

That could be improved by reading the database connection details from config.php instead, but those values are not likely to change for this setup. I’d also have preferred it if Flarum used some sort of file storage for sessions, in the same way basic Laravel does – but we can’t always get what we want ­čśë

Integration: Flarum Modification

So we now have the basics down – new users are registered, users can log in via my site, and users cannot reach the forum without being authenticated.

Modify Flarum to integrate the navigation with the main site, and prevent users from logging out via Flarum.

On the Flarum side, I want to change what some of the buttons in the navigation do – for instance, they shouldn’t be able to log out via Flarum (since that still leaves them logged into Write500), and I don’t want them to be able to manage their forum settings directly in Flarum.

After some trial and error, and almost giving up and hacking on the core scripts, I eventually landed on this neat hack.

In the Flarum administration section, under Appearance, you can add your own custom HTML. I used that to deploy a script block that looks like this:

That script is loaded in the page header, and executes immediately. At the end of the page, Flarum loads its app script and creates an app object. Before that happens, there’s nothing to modify, and I cannot force my javascript to run at the end of the page. So instead, my script keeps checking to see if the app variable is available, and when it is, it makes some changes.

With that script in place, the forum is now about as integrated as it can get – there’s just one more┬áthing to cover – Avatars.

Integration: Avatars

Users can upload and crop avatars to Write500.

Automatically synchronize the user’s avatar from Laravel, to Flarum.

I wanted those avatars to sync across to the forum, whenever they were created or updated on Write500’s side. I also have a “Reset to Default” which deletes the current avatar, and that deletion should be synced across as well.

Delete Avatars

This one was pretty easy. On the route that resets my avatar, I dispatch a job that basically does this:

It authenticates to the Write500 API, and issues a delete request against the avatar resource. Simple.

Upload avatars

This one’s a bit more complicated. The API endpoint is expecting a form-data upload, which takes a bit of fiddling to get right on PHP’s end.

After a user uploads, crops and saves an avatar, it’s stored in the public/ directory with a GUID, which is saved against the User model. All I really need to do is read that file from disk and upload it to the Flarum API. Which I achieve like so:

That bit of code will blindly upload a new avatar. Flarum doesn’t require you to delete an existing avatar before uploading a new one, so I can trigger this all I want. Right now, it’s just dispatching as a job after a new avatar is saved in Write500.

I’m not sure whether or not the User-Agent is strictly required for this, but I haven’t had time to go back and test it. I do know that the Flarum API does behave differently depending on whether it thinks it’s being hit from a browser.

Conclusion

So that’s an exhaustive look at how I’ve enabled the following integrations:

Laravel Flarum - Page 1 (5).png

I’m sure that some elements of this could be improved, and if I make any substantial updates to this, I’ll likely write a follow-up post here.

Additional Considerations

This is all still a work in progress, and I’m sure I’ll find more bugs as I go along.

Right now, for instance, I think it might make more sense to keep the user token cached in the User model, and only issue a new one when the current one expires. Right now, if you upload an avatar (for instance), it generates a new token that might invalidate the old one. So you’d have to log into the community from scratch to get access again.

There’s also notification integration. There’s an API endpoint that returns information about new notifications (@-mentions, etc), which I would want to display in the Write500 navbar.

I’d also want to take over the profile view eventually. I’m aiming to build a user profile page in Write500 to show some high-level stats, and I’d rather integrate forum activity in there, as opposed to having Flarum display any user profile page at all. Finally, I need to consider what will happen when users cancel their Write500 subscriptions.

Those are all questions for another day though!

DigitalOcean vs AWS Lightsail

I’ve been a big fan of DigitalOcean pretty much since they launched. Their pricing was cheap and simple, and their service was a joy to use. What made it different (as compared to other VPS hosts of the time) was the sheer simplicity of setup. The first time I used it, I was up and running with an SSD-backed VPS in under a minute – and blown away, of course.

In 2016, Amazon launched Lightsail – presumably in an attempt to tap into the market for developers who need quick and cheap VPSes. It got me thinking whether or not it’d be worth it to actually run some of my VMs there. At the lower tiers, at least, it looked like Lightsail had a cheaper offering.

A word on features

Each VPS host offers the same thing, fundamentally: CPU, RAM, SSD storage and bandwidth. They do diverge on the added-value features – for instance, DigitalOcean includes free DNS and monitoring, whereas Amazon expects you to pay for Route53 and CloudWatch respectively.

In this case though, I’m looking purely at the cost of the servers themselves.

The Basics

For this pricing comparison, I’m first looking at the per-hour cost – since that’s what you get billed on.

2017-02-11 02_33_07-VPS.xlsx - Excel.png

  • 0.5GB RAM: Out of the gate, Lightsail and DigitalOcean have the same pricing and features for their smallest instance.
  • 1GB RAM: One tier up, Lightsail actually works out fractionally cheaper for the features offered by DigitalOcean. But that lead doesn’t last.
  • 2GB RAM: In this category, Lightsail is slightly cheaper, but DigitalOcean offers double the CPU capacity at a┬ásimilar price point.
  • 4GB RAM: There’s parity again here with a comparative saving of just under 10%, if you were to select Lightsail over DigitalOcean.
  • 8GB RAM: And at this point, Lightsail is slightly cheaper (<10%), but again, DigitalOcean offers double the CPU.

This is how it works out on a Monthly basis:

2017-02-11 02_33_22-VPS.xlsx - Excel.png

Interestingly, despite having higher per-hour costs than some Lightsail options, DigitalOcean’s advertised monthly cost is lower. They must be using a strange definition of “Monthly” in calculating that, so to make it fair, I’m basing this on a 744-hour (31-day solid) month. That’s the upper bound for what you’d need to budget for.

My conclusion: at these instance sizes, DigitalOcean is better value for money across the board. With the possible exception of the 4GB RAM instance offered by LightSail – there you’d save roughly 10% over DigitalOcean.

Let’s talk about Transfer

In the next section, I’m going to compare these prices to raw AWS EC2 prices. One of the stated benefits of Lightsail is that once you outgrow your initial servers, you can migrate and extend by leveraging the AWS cloud. Which sounds nice in theory.

DigitalOcean (and Lightsail, by the looks of it) bundle a bandwidth allocation in at each price point.On DigitalOcean, that Transfer number counts for incoming and outgoing traffic on the public network interface (meaning that transfer on a private network is free).

AWS EC2 has a different approach. Most transfer into EC2 is free, and transfer out (uploading from your VPS to somewhere else) is charged differently depending on the destination. If it’s to another internal AWS service you usually get a much cheaper rate, as compared to transfer to the Internet.

While DigitalOcean and Lightsail both make huge bandwidth allocations available, the assumption (on their end) is that most users won’t actually use all of that bandwidth. If users did actually manage to max it out every month, the pricing would be very different.

Comparing to EC2

So let’s look at what it would cost to get the same features and bandwidth allocation directly from Amazon EC2. In this comparison, I’m basing everything off the N. Virginia region (their largest, oldest and cheapest), and I’m assuming On-Demand pricing for Linux VMs. I’ll compare it against Lightsail, which is only marginally more expensive than DigitalOcean to begin with.

2017-02-11 02_30_52-VPS.xlsx - Excel.png

Say what? Must be a calculation error, right?

EC2 charges for each component separately, and in excruciating detail. You’ll rent a Compute instance for RAM and CPU, then attach an Elastic Block Store volume to serve as the storage, and you’ll pay separately for the bandwidth. Complicated? You bet!

So in that table, the cost of each component breaks down like so:

2017-02-11 02_31_43-VPS.xlsx - Excel.png

Here’s where that bundled transfer stuff comes into play. If you look at just the Instance and Storage costs, it’s about on-par with Lightsail. The moment you want to serve traffic to the Internet, though, you’re paying $0.09/GB – and budgeting to be able to do terabytes worth of transfer every month is really expensive.

(Incidentally, pushing everything to AWS CloudFront won’t save you, since they start at $0.085/GB for transit to the Internet).

In truth, the bundled transfer included by DigitalOcean and Lightsail is what makes the difference.

Conclusions

If you were already on DigitalOcean, you’re probably congratulating yourself right now for making the smarter choice. And you’d be right.

If you’re on Lightsail, there’s no real reason to move. But if you’re running a couple of smallish EC2 VMs, and are sweating the bandwidth costs every month, it might be worth switching and taking advantage of the free bundled bandwidth.