Phoenix 1.7 is still a Release Candidate while being used here so things may change. Will keep this updated.

TLDR

mix local.hex
mix archive.install hex phx_new 1.7.0-rc.2
mix phx.new app_name --binary-id
mix phx.gen.auth Accounts User users

Plus do the UTC Datetime part - there's no TLDR variant of it.

Update Elixir / Erlang

I use asdf to handle versions.

asdf list all elixir                          # find latest version
asdf install elixir 1.14.3-otp-25             # install latest
asdf global elixir 1.14.3-otp-25              # use latest

asdf list all erlang
asdf install erlang 25.2
asdf global erlang 25.2

mix local.hex                                 # update hex
mix archive.install hex phx_new 1.7.0-rc.2    # phx.new for phoenix 1.7

New LiveView Project

mix phx.new app_name --binary-id

--binary_id makes Ecto use UUID instead of number id's by default.

Since Phoenix 1.7 TailwindCSS is already included, so no extra steps for that.

Then follow guide in console and git commit -am "Phoenix boilerplate".

Authentication

Phoenix has a handy command to generate a simple register / login flow. This generates the code and doesn't import new deps. That's great since we can then change the behaviour to how we want. Like to use magic link for example.

mix phx.gen.auth Accounts User users

Use UTC Datetime everywhere

You should use :utc_datetime unless you have a reason not to. The reason why Ecto’s timestamps() are naive by default is backwards compatibility. Since they are always UTC anyway, you should use :utc_datetime for them (and I believe the default will be changed to that in the future).

That's from Elixir Forum.

Ecto uses NaiveDatetime by default so lets change it.

# config/config.exs
config :app, App.Repo, migration_timestamps: [type: :utc_datetime]

And every Schema file you create must use:

@timestamps_opts [type: :utc_datetime]

Since we just generated schemas, let's change their datetimes:

# lib/app/accounts/user_token.ex

defmodule App.Accounts.UserToken do
  use Ecto.Schema
  [...]

  @primary_key {:id, :binary_id, autogenerate: true}
  @foreign_key_type :binary_id
  @timestamps_opts [type: :utc_datetime]           # add this line
  schema "users_tokens" do
    field :token, :binary
  
  [...]

# lib/boocs/accounts/user.ex

defmodule App.Accounts.User do
  use Ecto.Schema
  [...]
  @timestamps_opts [type: :utc_datetime]                # add this
  schema "users" do
    [...]
    field :confirmed_at, :utc_datetime                  # change this
    timestamps()
  end
  [...]

We could use a custom Schema macro to do the Schema change for us, but then we'd still have to remember to use it when generating files, so doesn't seem like we can get over manual work for now.

Staying up-to-date

Since we're using generators we'd like to know what changes new versions of the generators come with. For Phoenix itself you can use the official upgrade guide or https://phoenixdiff.org to see what's new.

You may also generate a new project, copy files on top of your current one and then just check the diff. I would not suggest making changes this was as some detail may be missed. Instead look at the diff and make changes to your project manually.

Bonus: Libraries

These are the libraries that I use most often. Not all of them evey project, but it's a good list to check.

• httpoison for making requests
• decimal for accurate calculations
• credo helper for clean code
• gradient gradual typechecker
• sobelow security-focused static analysis tool
• shorter_maps destruct maps
• typed_struct write types with ease
• typed_ecto_schema type writer for ecto schemas
• timex easier datetime formatting
• kaffy quick admin dashboard, like PHPMyAdmin
• ecto_shorts simpler Ecto API
• ex_machina generate DB items for testing
• ex_money don't roll your own currency
• oban jobs queue

Bonus 2: I made a search engine for hexdocs!

Direct all routes to a single function

Optional. I like my static page traffic handled by a single function.

lib/platform_web/router.ex

for path <- PlatformWeb.HomeController.available_paths() do
    get "/#{path}", HomeController, :static_page
end

Divide the traffic

I'll show you the whole file and comment in between.
lib/platform_web/controllers/home_controller.ex

defmodule PlatformWeb.HomeController do
    use PlatformWeb, :controller

I don't want a catch-all routing disaster so I've defined available routes manually. We could map the actual templates here, but then the AB testing templates are available too. Manual is safe.

def available_paths do
    ["/", "/contact", "projects", "pricing", "about"]
end

The "handle any route" function just takes the requested page, asks the ab_test() function to adjust the actual file rendered and then renders it.

def static_page(conn, _params) do
    path = Enum.at(conn.path_info, 0, "index")
    {conn, page} = ab_test(conn, path)
    render(conn, page, %{})
end

The ab_test() function mainly deals with session memory. If this user has already seen a page, then show the same one again. Lets not get our visitors confused. If this page has not yet been seen, choose a random_variation().

  defp ab_test(conn, requested_path) do
    memory = get_session(conn, :ab_test_memory) || %{}
    page = Map.get(memory, requested_path) || random_variation(requested_path)
    new_memory = Map.put(memory, requested_path, page)
    conn = put_session(conn, :ab_test_memory, new_memory)
    {conn, page}
  end

Choosing a random template was easy after figuring out the View actually holds all of the filenames.

  defp random_variation(page) do
    PlatformWeb.HomeView.__templates__()
    |> elem(2)
    |> Enum.filter(&String.starts_with?(&1, page))
    |> Enum.random()
  end
end

And voila. Pretty basic, once you have the template list at hand. If you've also implemented AB Testing, but in some way cooler manner - let me know.

Curiously sometimes the error shows up. But after a hard refresh on the browser - again nothing. Just

[info] GET /learn
[info] Sent 500 in 2ms

Where's the error?

Solution

I was advised to use Sentry, so I dug in and tried to figure out what they do differently. Their elixir integration is open-source so I found the plug that's responsible for capturing errors and implemented that into my own app.

Amazingly it actually logs the hidden error. So this time the problem was actually Phoenix Endpoint not handling its error properly.

Here's my module.

defmodule PlatformWeb.EndpointErrorCatcher do
  @moduledoc """
  Since Phoenix doesn't capture all errors we've got to try-catch them ourselves. I have no idea how Phoenix is meant to be working in this regard, but I just had an invisible error in prod.
  Module is ripped from https://github.com/getsentry/sentry-elixir/blob/02a2cc31c7444af88d079f7c32a789b9b1b4e6f1/lib/sentry/plug_capture.ex
  #### Usage
  In a Phoenix application, it is important to use this module before
  the Phoenix endpoint itself. It should be added to your endpoint.ex:
      defmodule MyApp.Endpoint
        use EndpointErrorCatcher
        use Phoenix.Endpoint, otp_app: :my_app
        # ...
      end
  In a Plug application, it can be added below your router:
      defmodule MyApp.PlugRouter do
        use Plug.Router
        use EndpointErrorCatcher
        # ...
      end
  """
  defmacro __using__(_opts) do
    quote do
      @before_compile PlatformWeb.EndpointErrorCatcher
    end
  end

  defmacro __before_compile__(_) do
    quote do
      defoverridable call: 2

      def call(conn, opts) do
        try do
          super(conn, opts)
        rescue
          e in Plug.Conn.WrapperError ->
            stack =
              e.stack
              |> List.first()
              |> elem(3)
              |> then(fn [file: file, line: line] -> "#{file}:#{line}" end)

            Logger.error("EndpointWrapperError at #{stack}: #{e.reason.message}")
            Plug.Conn.WrapperError.reraise(e)

          e ->
            stack =
              __STACKTRACE__
              |> List.first()
              |> elem(3)
              |> then(fn [file: file, line: line] -> "#{file}:#{line}" end)

            Logger.error("EndpointUnknownError at #{stack}: #{e.reason.message}")

            :erlang.raise(:error, e, __STACKTRACE__)
        catch
          kind, reason ->
            message = "Uncaught #{kind} - #{inspect(reason)}"
            stack = __STACKTRACE__
            Logger.error(message, stacktrace: stack)
            :erlang.raise(kind, reason, stack)
        end
      end
    end
  end
end

It logs the file:line and reason for error.

Use it as per instructed in @moduledoc.

Started Koodikool Platform.
[info] Running PlatformWeb.Endpoint with cowboy 2.8.0 at :::4010 (http)
[info] Access PlatformWeb.Endpoint at http://app.koodikool.ee
[info] GET /
[debug] Processing with Phoenix.LiveView.Plug.index/2
  Parameters: %{}
  Pipelines: [:browser]
[info] Sent 500 in 1ms
[debug] Converted error :function_clause to 500 response

Great.. So what it's telling me is that there's an error with a function so it turned the error into a 500 response. Thank you very much, but what's the error??

Elixir is supposed to log all errors by default, but my experience has proven otherwise. It's not too helpful either that people in the Slack respond with "You should see something logged as an error" (actual response, no fault on their end, I'm the one not understanding my system).

Not knowing what to do I enable debug_errors: true in the Endpoint config. So now the browser points me in the right direction:

Looking at code it's apparent that the problem is something to do with the server file structure (data directory). So I literally can't test in local. And making a staging server also beats the point since it's not identical with production.

So what are we supposed to do in this situation? Why is there no error? I'll update if I learn what I should really do.

The main product I made was Bashboard. It currently has 1 user and he doesn't reply by email. I've thought about possible pivots, but really I don't think there's a lot to do with it, besides keep it as a portfolio project.

Overall we've been running our software agency. It's been hard, then fun, then hard again. The best part about it are the people. The worst part that it the work itself doesn't really speak to me.

January 1st my daughter was born. What a special day that was. What a special few months it's been. I'm happy to be able to just sit there and spend time with her.

Through some stumbling around I've managed to start a new project, or rather reboot an old love of mine - Koodikool (Code School). It used to be a local group of kids and adults who came together to program something. Now many pieces have come together to perhaps be a startup instead. It feels like backwind.

Yet today (2021 march) I'm not sure where to go from here. If Koodikool fails then I don't feel like working on any more ideas. I also don't feel like going back and finding a place in the agency (from which I've detached and drifted quite far). We have an awesome client, but I'm not sure my goals align with him or the agency.

What are my goals? I'd like to see pro's making products. I'd like to make a change in education (hoping for Koodikool). I'd like challenges in service design. Code wise I'd love to write Elixir. And of course on top of that I don't want to give away what I have already: freedom and autonomy.

Realistically what the agency needs now is systematic company building skills. No matter how much I try - that is not me. I'm more of a product building tornado (tuulispask). The only difference really seems to be that company building is for a few select people and product building is for a larger audience.

In 5 years this company will be many times larger and more successful. I wouldn't be surprised if then it will also have their own product. But it wont be because I'm a better product developer. No, I'd be the same. The product will come from the outside. Which would be fine if it wasn't a waiting game.

If I do leave it will likely go on the wall of companies that got successful after I left. I'll take it easy for now. These are just my current feelings. Perhaps Koodikool will turn out to be something.

(edit: I was already offered a chance to train myself into a product lead, which is an amazing idea and opportunity. Things are looking up all of a sudden)

You want each test to run in a clean database so they don't interfere with each other. So how do you clear the database?

Best is to keep your tests in a sandbox. A sandbox is basically a temporary database that gets deleted automatically. Here's how to set it up.

1. Config

First configure the testing environment to use sandbox in config/test.exs:

config :bashboard, Bashboard.Repo,
  # your connection settings
  pool: Ecto.Adapters.SQL.Sandbox

2. Overwrite the test command

The command mix test runs tests. Since we also want to migrate the database before tests are run, make sure you overwrite the test command in mix.exs:

  defp aliases do
    [
      test: ["ecto.create", "ecto.migrate", "test"]
    ]
  end

That should be there by default, if you used Phoenix to generate your project structure.

3. Start sandbox

The test_helper.ex file is what actually starts the tests. In here we also enable sandbox mode:

Ecto.Adapters.SQL.Sandbox.mode(MyApp.Repo, :manual)

:manual here means that the database is only available in the test process. So if inside the test you start another process (with Task for example) then that will fail. This is the preferred way. Only enable :shared more if needed (example later).

4. Define pre-test action

You need to checkout a new database connection before each test. This ensures that the database is reduced to the starting state.

The simplest way is to include a setup function in the beginning of the test module.

setup do
  Ecto.Adapters.SQL.Sandbox.checkout(MyRepo)
end

Usually you have many test files though so rewriting this makes no sense. Instead you can define a function somewhere. Let's write a test/test_conn.exs file:

defmodule MyApp.TestConn do

  def checkout(_context \\ nil) do
    Ecto.Adapters.SQL.Sandbox.checkout(MyApp.Repo)
  end
  
end

Now you can just import it in the beginning of any test module and that setup function will run in all the tests in that file:

defmodule Bashboard.UserTest do
  use ExUnit.Case
  use Plug.Test
  import MyApp.TestConn

  setup :checkout

  test "description" do
    # your test
  end
end

Troubleshooting

1. process ownership error

** (RuntimeError) cannot find ownership process for #PID<0.35.0>

This error tells you that you tried to use the DB connection outside of the intended test do process. For example by invoking a Task. This happens because of that :manual setting. The idea is to limit the scope of the sandbox so it's explicit.

To fix this tell the connection to be :shared instead:

defmodule MyApp.TestConn do

  def checkout(_context \\ nil) do
    Ecto.Adapters.SQL.Sandbox.checkout(MyApp.Repo)
    Ecto.Adapters.SQL.Sandbox.mode(MyApp.Repo, {:shared, self()})
  end
  
end

Now any process that needs the database can access the same one the test process is using. Make sure you always checkout a connection before sharing it though! Sharing a connection means you can't run in concurrent mode though (running multiple tests at the same time).

To keep tests running concurrently, you can use the more cumbersome, but specific allowances.

2. Database still not empty

You may have some data in the test database from previous runs. So either just empty it once with mix ecto.drop or add the same command to the alias list:

  defp aliases do
    [
      test: ["ecto.drop", "ecto.create", "ecto.migrate", "test"]
    ]
  end

Generally this should not be needed.

The first pair I got the right earbud was dead. Wouldn't even turn on. Support has been good though and 4 months later I got a new pair.

This time the touch controls only register single taps. Much worse however is that they keep disconnecting every 10 minutes. This is incredibly annoying and I will be sending these back also. Again, support is a tad slow but still good and I'm sure I'll get a working pair eventually.

I do believe they are working at their max capacity so I that's all I'll say about these issues for now.

My over-sensitive ears

I have over-sensitive ears. 4 people speaking will make my ears crackle and deaf after a while. Whether I have hyperacusis or misophonia the doctors have not diagnosed, but the problem is real.

I got the new IQ Buds Max 2 to deal with this. So I can turn down loud sounds and keep conversations.

Audio tests show that my hearing is perfect. Even the app's Ear ID test says so. The rings are complete, thus I hear all sounds great.

What nobody seems to test is how much sound can I handle and what happens then.

IQ Buds usefulness to my overly sensitive ears

Not much. The ANC is much less than the physical sealing. The different modes of letting me hear conversations end up greatly over-powering a fan in the corner and leaving me deaf for the person speaking in front of me. When a dog barks.. well let's say I'd rather keep incoming sound off in most occasions.

Overall quality

Sound is about the same as 7€ JBL wired in-ear buds. ANC is around 20% as strong as the physical sealing, so pretty much pointless. Otherwise they stay in the ear very well and I enjoy using them. I just wonder if the same result would have been achieved with 100€ buds instead of 300€.

Summary

My current experience is that they are overpriced and useless. If you have hearing loss I think these can actually benefit you a lot. But for my hearing problems, a solution does not exist currently.

I write irregularly. Some months there's many posts every week. Some months not a single one. That's not going to change. No promises. But I'm going to focus on making posts that share my learning's in leading a company.

Me and two other founders (and some more awesome people) run a software development company. I'm otherwise a startup kind of guy, but this is the only thing that's stayed in business for 2 years now. I learn a lot every day and I want to share it.

So topics are going to be: business, programming, schools (I like teaching). Probably some random personal stuff every now and again. My most popular post is a technical configuration tutorial, yet my most emailed about post is orbits stress and depression. People have found a variety of topics helpful.

I'm going to email these posts out for a while just to see what happens. What feedback I get. Feel free to unsubscribe. Just like I do most newsletters. I don't have an aim to grow this audience as much as let people find me, who actually benefit from this. I'd like to hone my writing skills.

So either enjoy or just see you later.

A less apparent surprise was that even when the project is super small, having extra two people to work with boosts work morale through the roof! It's a whole other experience. Motivation loss is barely a thing. Now it really is a matter of figuring out a work routine.

I'm going on child leave for 3 months soon. I hope the two can stay motivated together. We've experienced the single developer burnout multiple times already.

In order to keep these two happy I'm involving self-development time regularly. If the project is small and the direction is dependent on the clients feelings, motivation will naturally disappear at one point. So I'm doing my best to take care of the employees.

In any case. Having a team is one of those obvious things that took years to find out. I'm happy I'm here now.

Check this out. On the 15th of May I was supposed to have 38 users.

I had put a custom fetch request into that page and sent the data to Bashboard.

    const previousVisit = localStorage.getItem('log-0-blog')
    if (!previousVisit) {
        localStorage.setItem('log-0-blog', true)
        fetch('https://bashboard.io/cpconf/users/funnel', {
            method: 'POST',
            body: JSON.stringify('0-blog')
        })
    }

15th of May actually had 20 visitors!

So either my localStorage trick is more faitful than Google Analytics megascript, or the problem is a lot of bots don't run fetch() commands. They do however run JS otherwise Google wouldn't see the visitor either.

If you know what's up with this - let me know in the comments (or IH comments).

Which is not a suprise. I was however pleasently surprised how many people voted for me when posting to my social groups. I mean 15 isn't much, but it's a lot more than what I was expecting :)

So anyway what are the stats?

So on PH launch day there were 329 visitors. Next day a bit less. Those days actually include the HN launch I did in the same evening since the DB was beefed up anyway and no tmuch traffic was happening. HN resulted in 0.

The most popular page besides landing was /price and then /examples. The least popular were /roadmap, /usecases and /about. I probably spent too much time on those.

The stats show 18 registered users, but 7 of those are either spam or myself. So 11. Not the worst, but there are no active users since launch - and that's the important bit.

What's next?

Marketing. I'll go into feature avoiding mode and only implement something if:

1. An active user asks for it.
2. I need it to see how the marketing is doing.

Le Grande Marketing Plan

I'm a noob. But I have some ideas.

1. Hang out in communities and build a reputation and suggest Bashboard when actually appropriate.
2. Keep myself up to date with competitors blogs to learn about my own customer.
3. Find individual users who may be interested by utilizing a few specific growth hacks.
4. Blog about trendy topics like how to get live charts into Notion :)
5. Make public charts that are useful/interesting and spread those.

That's it! Yeah it's not much I know. But it's a marathon, not a sprint. New ideas will arise, old ones will fall. A few actions will prove more beneficial than others. Just keep moving.

But first!

During the three month bashing out of Bashboard - I learned some things. And had a few ideas. I want to try them out. It'll only take a few weeks, I promise! :D

But actually yes, this blog has some attention from the internet, I want to try and use it. Also Sales Safari has been a mind blower. Definitely want to go in and find some watering holes of the internet that I could perhaps serve.