Slings and Arrows

09 Feb 2019

In our last exciting episode of The New Ben, our hero had embarked on a new adventure in programming, a trip into ElixirLand. Well, this journey is not all wine and roses. It would be silly to expect learning any new programming language, ecosystem, and web framework to be trivial. In this episode I’ll catch you up on what has happened and hopefully record some problems and solutions that will help others down the road. At the very least it will give me notes to refer back to and help me assess where I’m going.

Starting over

At the time of my last post, almost three weeks ago now, I had started on the backend for my startup’s application using Elixir and Phoenix. In the interest of saving time, I had chosen an open-source authentication library for Phoenix called Coherence. I was a little leery about this because it hid the implementation of an important part of my app in a separate Elixir application. Also, the project appeared to have gone silent.

I soon ran into weirdness when I was trying to write controller tests for authenticated users. I decided I was going down the wrong path with this library. Better to write it all from scratch and know it inside out than to build my new empire on an opaque and unmaintained foundation. I managed to get my user registration and authentication code in place fairly quickly. I don’t have all the niceties of Coherence – email verification, password recovery, brute force blocking, etc – but I can add those when the time comes. It’s better to focus on the core minimum viable product for now.

Strange Frustrations and New Lessons

So I regenerated my app, again doing so as an Elixir Umbrella app. I’ve slowly gotten used to working in an umbrella. I even added my first app other than the generated domain (repo) and web apps: a stand-alone API gateway app. For a while, I struggled with strange dependency issues. Often when I tried to run mix in one of the app directories, I would find I needed to reload my dependencies and recompile. I had this persistent issue with getting this error in my repo directory:

** (Mix) Could not start application phoenix_html: could not find application file: phoenix_html.app

I finally stumbled onto this Elixir github issue, which didn’t even show the same error, but sounded similar and was fixed in Elixir 1.8.1. I was still using 1.8. So I gave upgrading a try and it fixed this error!

So now, things are moving along fairly smoothly, albeit more slowly than I had hoped. Really though, I shouldn’t have expected anything less. While Elixir presents itself with a somewhat Ruby-like syntax, it has many dissimilarities. Divorcing the data structures from their logic seems straightforward on the face of it. But when you’ve spent your whole career writing object-oriented programming, it’s hard not to fall into old patterns.

Similarly, the Phoenix Framework adopts an architecture that seems intended to make Rails devs feel at home. But once you get past the initial similarity, you find yourself spending plenty of time scratching your head. Where does this go? What’s the preferred pattern here? Et cetera.

Phoenix definitely does not try to be as magical as Rails. You’re going to write more code. Also, the HTML generator (the equivalent of the Rails scaffold generator) is nearly useless. You probably won’t use all the functions it creates and the CRUD views and actions it creates only work for a single model. If your model has an association, you’re hacking that in yourself. Then again, the same is true of Rails scaffold, and to be honest, I haven’t used Rails scaffold in years. I mostly used the Phoenix HTML generator to have code to look at.

New Tools for a New World

The advice I got in the excellent Elixir Slack channel is to stop comparing Elixir and Phoenix to Ruby and Rails. They have surface similarities and similar goals, but there are plenty of differences.

One example of this stems from Elixir being built on top of the Erlang/OTP ecosystem. When I needed to write some code to make an API call, I knew that I didn’t want the web request/response cycle to have to wait on this call. My natural inclination, coming from the Rails world, was to install a background job worker similar to Sidekiq or DelayedJob. I found the straightforward-looking Ecto-Job and was about to start using it, but after doing a little more research, I realized this was overkill. Others agreed

Really, all I need is for the API call to happen in a separate thread or process. The controller action can just let the client know the job is underway so that the client can show a spinner and poll the backend for the result, which will take the form of a record showing up in the database. This means my “job” can be a simple “fire-and-forget” task. And in fact, Elixir has the Task module for doing just this. The client can keep track of how long it has been waiting for the task to finish and give up and show an error after a certain amount of time. The user can just try again if that happens. There is no need for job preservation or state management.

Sometimes the differences can be fun too. This API wrapper that I had to write was a good example. Doing this sort of thing is where Elixir pattern matching really shines:

  def decode_vin_values(vin) do
    case get("/api/vehicles/decodevinvalues/" <> vin <> "?format=json") do
      {:ok, resp} ->
        handle_response(resp.body)

      {:error, resp} ->
        {:error, resp}
    end
  end

  defp handle_response(%{
         "Message" => "Results returned successfully",
         "Results" => [vehicle_data_map | _]
       }) do
    handle_success_response(vehicle_data_map)
  end

  defp handle_response(%{
         "Message" => _,
         "Results" => [%{"Message" => error_message}]
       }) do
    {:error, error_message}
  end

  defp handle_success_response(data_map) do
    if data_map["ErrorCode"] =~ ~r{^0 - VIN decoded clean} do
      {:ok, data_map}
    else
      {:error, data_map["ErrorCode"]}
    end
  end

The first function calls the API and has conditions for a success or failure. In the success case it calls a multi-headed function to decide what to do with the response (this API can return a successful response with an error message inside). The success version of this calls another function which pattern-matches on the ErrorCode in the response being 0, which means there was no error. Writing this was very fun and rewarding. In general, I have felt very good writing Elixir. Working with Phoenix has been a bit more challenging, probably mostly because there is a lot more to it, a lot more to keep in mind. I am slowly finding my way around.

Carry On, My Wayward Son

I would be lying if I said there haven’t been times when I’ve questioned my sanity for choosing Elixir and Phoenix for my startup’s backend. I have wondered several times how far along I would be if I had just done this in Ruby on Rails. But I explained my reasons in my previous post and they still hold true.

The key thing I keep coming back to is to keep my focus as narrow as possible. Just work on one small problem at a time and then move on to another. Of course, I have to have some idea what I’m building overall and where I’m going. But it is easy to try to flesh out too big a piece of this at once.

So, back to it. My goal right now is to complete the functions for creating a vehicle record and its associations. Onward and upward!