TDD Phoenix

Adding History to Rooms

We almost have our app fully functional: our users can create accounts, sign in, create chat rooms, and chat to their hearts' content. From here, we can now see the finish line. It's time for the last stretch.

In this chapter, we want to add history to chat rooms, allowing conversations to persist through time. Currently, when a user joins a channel, they cannot see any previous messages. Just refreshing the browser will cause all messages to be lost. That's not a great user experience for a chat application, so let's store messages in our database.

Writing our feature test

We'll start by writing a new test inside user_can_chat_test.exs. In the test, a user will post a message in a chat room, then a different user will log in, and we should expect the second user to see the first user's message. As we've done before, we'll write out the test as we want it to exist and let the errors drive us:

# test/chatter_web/features/user_can_chat_test.exs

test "new user can see previous messages in chat room", %{metadata: metadata} do
  room = insert(:chat_room)
  user1 = insert(:user)
  user2 = insert(:user)

  metadata
  |> new_session()
  |> visit(rooms_index())
  |> sign_in(as: user1)
  |> join_room(room.name)
  |> add_message("Welcome future users")

  metadata
  |> new_session()
  |> visit(rooms_index())
  |> sign_in(as: user2)
  |> join_room(room.name)
  |> assert_has(message(user1, "Welcome future users"))
end

The test probably looks familiar, but let's walk through what we're doing:

  • Since we're starting two sessions in the test, we take the %{metadata: metadata} as params instead of the %{session: session}.
  • We create our chat room and two users.
  • We create our first session, sign in user1, and immediately post a message. Note that user2 is not in the chat room yet.
  • We then create another session, sign in user2, and expect to see the message user1 posted before user2 signed in.

Without further ado, let's run our test:

mix test test/chatter_web/features/user_can_chat_test.exs:39
Excluding tags: [:test]
Including tags: [line: "39"]



  1) test new user can see previous messages in chat room (ChatterWeb.UserCanChatTest)
     test/chatter_web/features/user_can_chat_test.exs:34
     ** (Wallaby.ExpectationNotMetError) Expected to find 1, visible element with the attribute 'data-role' with value 'message' but 0, visible elements with the attribute were found.

     code: |> assert_has(message(user1, "Welcome future users"))
     stacktrace:
       test/chatter_web/features/user_can_chat_test.exs:51: (test)



Finished in 5.6 seconds
2 tests, 1 failure, 1 excluded

Good. Everything is working (or not working 😁) as expected: the second user cannot see the message sent before they joined. Let's fix that.

Saving messages

To show historical messages, we first have to save messages when users send them. Open up our ChatRoomChannel, and update the handle_in/3 callback to call Chat.new_message/2:

# lib/chatter_web/channels/chat_room_channel.ex

 defmodule ChatterWeb.ChatRoomChannel do
   use ChatterWeb, :channel

+  alias Chatter.Chat
+
   def join("chat_room:" <> _room_name, _msg, socket) do
     {:ok, socket}
   end

   def handle_in("new_message", payload, socket) do
-    author = socket.assigns.email
+    %{room: room, email: author} = socket.assigns
     outgoing_payload = Map.put(payload, "author", author)
+
+    Chat.new_message(room, outgoing_payload)
     broadcast(socket, "new_message", outgoing_payload)
+
     {:noreply, socket}
   end

Note that we expect the socket.assigns to have both a room and an email (author). Since that's not true yet, we expect that to be a point of failure. Let's rerun our test:

mix test test/chatter_web/features/user_can_chat_test.exs:39
Excluding tags: [:test]
Including tags: [line: "39"]
Compiling 1 file (.ex)
warning: function Chatter.Chat.new_message/2 is undefined or private
  lib/chatter_web/channels/chat_room_channel.ex:16


03:35:42.003 [error] GenServer #PID<0.573.0> terminating
** (MatchError) no match of right hand side value: %{email: "super0@example.com"}
    (chatter) lib/chatter_web/channels/chat_room_channel.ex:11: ChatterWeb.ChatRoomChannel.handle_in/3
    (phoenix) lib/phoenix/channel/server.ex:284: anonymous fn/4 in Phoenix.Channel.Server.handle_info/2
    (chatter) lib/chatter_web/endpoint.ex:1: ChatterWeb.Endpoint.instrument/4
    (stdlib) gen_server.erl:637: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:711: :gen_server.handle_msg/6
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Last message: %Phoenix.Socket.Message{event: "new_message", join_ref: "1", payload: %{"body" => "Welcome future users"}, ref: "2", topic: "chat_room:chat room 0"}


  1) test new user can see previous messages in chat room (ChatterWeb.UserCanChatTest)
     test/chatter_web/features/user_can_chat_test.exs:34
     ** (Wallaby.ExpectationNotMetError) Expected to find 1, visible element with the attribute 'data-role' with value 'message' but 0, visible elements with the attribute were found.

     code: |> assert_has(message(user1, "Welcome future users"))
     stacktrace:
       test/chatter_web/features/user_can_chat_test.exs:51: (test)



Finished in 5.7 seconds
2 tests, 1 failure, 1 excluded

Just as we expected, socket.assigns do not have a room in it, so we get the following error: (MatchError) no match of right hand side value: %{email: "super0@example.com"}.

Before going further, remember that channels are a good place to "step in" from outside to inside testing. So, instead of running our feature test, let's run our chat room channel test, and see what error we get:

$ mix test test/chatter_web/channels/chat_room_channel_test.exs:5
Excluding tags: [:test]
Including tags: [line: "5"]

03:39:01.051 [error] GenServer #PID<0.547.0> terminating
** (MatchError) no match of right hand side value: %{email: "random@example.com"}
    (chatter) lib/chatter_web/channels/chat_room_channel.ex:11: ChatterWeb.ChatRoomChannel.handle_in/3
    (phoenix) lib/phoenix/channel/server.ex:284: anonymous fn/4 in Phoenix.Channel.Server.handle_info/2
    (chatter) lib/chatter_web/endpoint.ex:1: ChatterWeb.Endpoint.instrument/4
    (stdlib) gen_server.erl:637: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:711: :gen_server.handle_msg/6
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Last message: %Phoenix.Socket.Message{event: "new_message", join_ref: nil, payload: %{"body" => "hello world!"}, ref: #Reference<0.3053985511.2766667778.258553>, topic: "chat_room:general"}


  1) test new_message event broadcasts message to all users (ChatterWeb.ChatRoomChannelTest)
     test/chatter_web/channels/chat_room_channel_test.exs:5
     ** (EXIT from #PID<0.545.0>) an exception was raised:
         ** (MatchError) no match of right hand side value: %{email: "random@example.com"}
             (chatter) lib/chatter_web/channels/chat_room_channel.ex:11: ChatterWeb.ChatRoomChannel.handle_in/3
             (phoenix) lib/phoenix/channel/server.ex:284: anonymous fn/4 in Phoenix.Channel.Server.handle_info/2
             (chatter) lib/chatter_web/endpoint.ex:1: ChatterWeb.Endpoint.instrument/4
             (stdlib) gen_server.erl:637: :gen_server.try_dispatch/4
             (stdlib) gen_server.erl:711: :gen_server.handle_msg/6
             (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3



Finished in 0.06 seconds
1 test, 1 failure

Good. Because we're getting the same match error — ** (MatchError) no match of right hand side value: %{email: "random@example.com"} — we can safely "step in" and use the channel test as our guiding test.

To fix the error, we need to add room to our socket.assigns. But where should we do that?

Well, we have that chat room information when a user first joins a chat room. So let's do that. So far, when joining a channel, we've been ignoring the subtopic in join("chat_room:" <> _room_name). Let's grab that room name, fetch the room from the database, and set it in the socket.assigns. Doing so will also force us to update our channel to use a persisted chat room in the setup.

Let's dive in. Update the join/3 function as follows:

# lib/chatter_web/channels/chat_room_channel.ex

-  def join("chat_room:" <> _room_name, _msg, socket) do
-    {:ok, socket}
+  def join("chat_room:" <> room_name, _msg, socket) do
+    room = Chat.find_room_by_name(room_name)
+    {:ok, assign(socket, :room, room)}
   end

Since Chat.find_room_by_name/1 doesn't exist yet, we can expect that to fail. Let's run our test:

$ mix test test/chatter_web/channels/chat_room_channel_test.exs:5
Compiling 1 file (.ex)
warning: function Chatter.Chat.find_room_by_name/1 is undefined or private. Did you mean one of:

      * find_room/1

  lib/chatter_web/channels/chat_room_channel.ex:7

warning: function Chatter.Chat.new_message/2 is undefined or private
  lib/chatter_web/channels/chat_room_channel.ex:17

Excluding tags: [:test]
Including tags: [line: "5"]

03:47:08.576 [error] an exception was raised:
    ** (UndefinedFunctionError) function Chatter.Chat.find_room_by_name/1 is undefined or private
        (chatter) Chatter.Chat.find_room_by_name("general")
        (chatter) lib/chatter_web/channels/chat_room_channel.ex:7: ChatterWeb.ChatRoomChannel.join/3
        (chatter) lib/chatter_web/endpoint.ex:1: ChatterWeb.Endpoint.instrument/4
        (phoenix) lib/phoenix/channel/server.ex:219: Phoenix.Channel.Server.init/1
        (stdlib) gen_server.erl:374: :gen_server.init_it/2
        (stdlib) gen_server.erl:342: :gen_server.init_it/6
        (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3


  1) test new_message event broadcasts message to all users (ChatterWeb.ChatRoomChannelTest)
     test/chatter_web/channels/chat_room_channel_test.exs:5
     ** (MatchError) no match of right hand side value: {:error, %{reason: "join crashed"}}
     code: {:ok, _, socket} = join_channel("chat_room:general", as: email)
     stacktrace:
       test/chatter_web/channels/chat_room_channel_test.exs:7: (test)



Finished in 0.05 seconds
1 test, 1 failure

That is a big error message, and it has several warnings. Though it seems daunting, we expect all of those errors and warnings.

First, Elixir is warning us about two undefined functions (both of which we know are undefined): Chat.find_room_by_name/1 and Chat.new_message/2. Then, we get the error that crashed the process: ** (UndefinedFunctionError) function Chatter.Chat.find_room_by_name/1 is undefined or private. That error matches the first warning we saw. Interestingly, our channel bubbles up a nice error for our test when we try to join the channel. Instead of returning the {:ok, _, socket} tuple we expected, the error tells us that the join crashed: {:error, %{reason: "join crashed"}}.

So the warnings and errors are telling us that joining the channel failed because Chat.find_room_by_name/1 is undefined. Let's fix that next. Since Chat.find_room_by_name/1 is part of our core business logic, let's "step in" one level deeper.

Creating Chatter.Chat.find_room_by_name/1

Since we're "stepping in" one more level, let's get a failing error that matches the undefined function error we saw in our channel test. Open up test/chatter/chat_test.exs, and add a new test:

# test/chatter/chat_test.exs

  describe "find_room_by_name/1" do
    test "retrieves a room by name" do
      room = insert(:chat_room)

      found_room = Chat.find_room_by_name(room.name)

      assert room == found_room
    end
  end

Now, run the test:

$ mix test test/chatter/chat_test.exs:58
Excluding tags: [:test]
Including tags: [line: "58"]



  1) test find_room_by_name/1 retrieves a room by name (Chatter.ChatTest)
     test/chatter/chat_test.exs:55
     ** (UndefinedFunctionError) function Chatter.Chat.find_room_by_name/1 is undefined or private
     code: found_room = Chat.find_room_by_name(room.name)
     stacktrace:
       (chatter) Chatter.Chat.find_room_by_name("chat room 0")
       test/chatter/chat_test.exs:58: (test)



Finished in 0.1 seconds
6 tests, 1 failure, 5 excluded

Good. We get the same undefined function error. Let's add an implementation:

# lib/chatter/chat.ex

  def find_room_by_name(name) do
    Chat.Room |> Repo.get_by!(name: name)
  end

We use the ! version of Repo.get_by/3 because we don't want to return nil and assign it as the chat room name in our socket.assigns. Sometimes raising an exception is too aggressive, but in our case, we want the process to fail if a user is trying to join a non-existent chat room.

Let's rerun our test:

$ mix test test/chatter/chat_test.exs:58
Compiling 1 file (.ex)
warning: function Chatter.Chat.new_message/2 is undefined or private
  lib/chatter_web/channels/chat_room_channel.ex:17

Excluding tags: [:test]
Including tags: [line: "58"]

.

Finished in 0.1 seconds
6 tests, 0 failures, 5 excluded

Good, our test passes! We still see the warning for Chat.new_message/2 being undefined, but we'll "step back out" to the channel test and later "step in" to handle that warning.

Updating ChatRoomChannelTest

Let's run our channel test to see what to do next:

$ mix test test/chatter_web/channels/chat_room_channel_test.exs:5
Compiling 1 file (.ex)
warning: function Chatter.Chat.new_message/2 is undefined or private
  lib/chatter_web/channels/chat_room_channel.ex:17

Excluding tags: [:test]
Including tags: [line: "5"]

04:11:29.388 [error] an exception was raised:
    ** (DBConnection.OwnershipError) cannot find ownership process for #PID<0.556.0>.

When using ownership, you must manage connections in one
of the four ways:

* By explicitly checking out a connection
* By explicitly allowing a spawned process
* By running the pool in shared mode
* By using :caller option with allowed process

The first two options require every new process to explicitly
check a connection out or be allowed by calling checkout or
allow respectively.

The third option requires a {:shared, pid} mode to be set.
If using shared mode in tests, make sure your tests are not
async.

The fourth option requires [caller: pid] to be used when
checking out a connection from the pool. The caller process
should already be allowed on a connection.

If you are reading this error, it means you have not done one
of the steps above or that the owner process has crashed.

See Ecto.Adapters.SQL.Sandbox docs for more information.
        (ecto_sql) lib/ecto/adapters/sql.ex:602: Ecto.Adapters.SQL.raise_sql_call_error/1
        (ecto_sql) lib/ecto/adapters/sql.ex:538: Ecto.Adapters.SQL.execute/5
        (ecto) lib/ecto/repo/queryable.ex:147: Ecto.Repo.Queryable.execute/4
        (ecto) lib/ecto/repo/queryable.ex:18: Ecto.Repo.Queryable.all/3
        (ecto) lib/ecto/repo/queryable.ex:74: Ecto.Repo.Queryable.one!/3
        (chatter) lib/chatter_web/channels/chat_room_channel.ex:7: ChatterWeb.ChatRoomChannel.join/3
        (chatter) lib/chatter_web/endpoint.ex:1: ChatterWeb.Endpoint.instrument/4
        (phoenix) lib/phoenix/channel/server.ex:219: Phoenix.Channel.Server.init/1
        (stdlib) gen_server.erl:374: :gen_server.init_it/2
        (stdlib) gen_server.erl:342: :gen_server.init_it/6
        (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3


  1) test new_message event broadcasts message to all users (ChatterWeb.ChatRoomChannelTest)
     test/chatter_web/channels/chat_room_channel_test.exs:5
     ** (MatchError) no match of right hand side value: {:error, %{reason: "join crashed"}}
     code: {:ok, _, socket} = join_channel("chat_room:general", as: email)
     stacktrace:
       test/chatter_web/channels/chat_room_channel_test.exs:7: (test)



Finished in 0.06 seconds
1 test, 1 failure

Whoa! That is an unexpected (and rather large) Ecto error. Thankfully, it comes with a helpful message: we have multiple processes trying to use a database connection, but only the process that checked out the connection (our test process) is allowed to use it.

We can fix that by using the third option in the error's list: running the pool in shared mode. Note what the message says:

The third option requires a {:shared, pid} mode to be set. If using shared mode in tests, make sure your tests are not async.

So we have to set Ecto's SQL Sandbox mode to {:shared, pid}, but that means we can no longer run our channel test asynchronously. This trade-off is so common that if you open ChatterWeb.ChannelCase, you'll see that the setup sets the mode to {:shared, pid} when we run tests as async: false. So, all we have to do is change async to false in our test, and we're good to go!

# test/chatter_web/channel/chat_room_channel_test.exs

 defmodule ChatterWeb.ChatRoomChannelTest do
-  use ChatterWeb.ChannelCase, async: true
+  use ChatterWeb.ChannelCase, async: false

Now run our test again:

$ mix test test/chatter_web/channels/chat_room_channel_test.exs:5
Excluding tags: [:test]
Including tags: [line: "5"]

04:13:46.217 [error] an exception was raised:
    ** (Ecto.NoResultsError) expected at least one result but got none in query:

from r0 in Chatter.Chat.Room,
  where: r0.name == ^"general"

        (ecto) lib/ecto/repo/queryable.ex:76: Ecto.Repo.Queryable.one!/3
        (chatter) lib/chatter_web/channels/chat_room_channel.ex:7: ChatterWeb.ChatRoomChannel.join/3
        (chatter) lib/chatter_web/endpoint.ex:1: ChatterWeb.Endpoint.instrument/4
        (phoenix) lib/phoenix/channel/server.ex:219: Phoenix.Channel.Server.init/1
        (stdlib) gen_server.erl:374: :gen_server.init_it/2
        (stdlib) gen_server.erl:342: :gen_server.init_it/6
        (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3


  1) test new_message event broadcasts message to all users (ChatterWeb.ChatRoomChannelTest)
     test/chatter_web/channels/chat_room_channel_test.exs:5
     ** (MatchError) no match of right hand side value: {:error, %{reason: "join crashed"}}
     code: {:ok, _, socket} = join_channel("chat_room:general", as: email)
     stacktrace:
       test/chatter_web/channels/chat_room_channel_test.exs:7: (test)



Finished in 0.1 seconds
1 test, 1 failure

That's more like it! Since our test doesn't try to join the channel with an existing chat room's name (it simply passes "general"), our application cannot find the room in the database and fails to join the channel. Let's update our test to join a real chat room:

# test/chatter_web/channels/chat_room_channel_test.exs

   describe "new_message event" do
     test "broadcasts message to all users" do
       email = "random@example.com"
-      {:ok, _, socket} = join_channel("chat_room:general", as: email)
+      room = insert(:chat_room)
+      {:ok, _, socket} = join_channel("chat_room:#{room.name}", as: email)
       payload = %{"body" => "hello world!"}

Rerun our test:

$ mix test test/chatter_web/channels/chat_room_channel_test.exs:5
Excluding tags: [:test]
Including tags: [line: "5"]


== Compilation error in file test/chatter_web/channels/chat_room_channel_test.exs ==
** (CompileError) test/chatter_web/channels/chat_room_channel_test.exs:7: undefined function insert/1
    (elixir) src/elixir_locals.erl:107: :elixir_locals."-ensure_no_undefined_local/3-lc$^0/1-0-"/2
    (elixir) src/elixir_locals.erl:108: anonymous fn/3 in :elixir_locals.ensure_no_undefined_local/3
    (stdlib) erl_eval.erl:680: :erl_eval.do_apply/6
    (elixir) lib/code.ex:767: Code.require_file/2
    (elixir) lib/kernel/parallel_compiler.ex:211: anonymous fn/4 in Kernel.ParallelCompiler.spawn_workers/6

Aha! We do not have an insert/1 function in our channel tests because we did not import Chatter.Factory in ChannelCase. Let's do that:

test/support/channel_case.ex

       # Import conveniences for testing with channels
       use Phoenix.ChannelTest

+      import Chatter.Factory
+
       # The default endpoint for testing
       @endpoint ChatterWeb.Endpoint

And run the test:

$ mix test test/chatter_web/channels/chat_room_channel_test.exs:5
Excluding tags: [:test]
Including tags: [line: "5"]

05:17:42.450 [error] GenServer #PID<0.547.0> terminating
** (UndefinedFunctionError) function Chatter.Chat.new_message/2 is undefined or private
    (chatter) Chatter.Chat.new_message(%Chatter.Chat.Room{__meta__: #Ecto.Schema.Metadata<:loaded, "chat_rooms">, id: 452, inserted_at: ~N[2020-03-06 10:17:42], name: "chat room 0", updated_at: ~N[2020-03-06 10:17:42]}, %{"author" => "random@example.com", "body" => "hello world!"})
    (chatter) lib/chatter_web/channels/chat_room_channel.ex:15: ChatterWeb.ChatRoomChannel.handle_in/3
    (phoenix) lib/phoenix/channel/server.ex:284: anonymous fn/4 in Phoenix.Channel.Server.handle_info/2
    (chatter) lib/chatter_web/endpoint.ex:1: ChatterWeb.Endpoint.instrument/4
    (stdlib) gen_server.erl:637: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:711: :gen_server.handle_msg/6
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Last message: %Phoenix.Socket.Message{event: "new_message", join_ref: nil, payload: %{"body" => "hello world!"}, ref: #Reference<0.3684083830.809238531.87989>, topic: "chat_room:chat room 0"}


  1) test new_message event broadcasts message to all users (ChatterWeb.ChatRoomChannelTest)
     test/chatter_web/channels/chat_room_channel_test.exs:5
     ** (EXIT from #PID<0.545.0>) an exception was raised:
         ** (UndefinedFunctionError) function Chatter.Chat.new_message/2 is undefined or private
             (chatter) Chatter.Chat.new_message(%Chatter.Chat.Room{__meta__: #Ecto.Schema.Metadata<:loaded, "chat_rooms">, id: 452, inserted_at: ~N[2020-03-06 10:17:42], name: "chat room 0", updated_at: ~N[2020-03-06 10:17:42]}, %{"author" => "random@example.com", "body" => "hello world!"})
             (chatter) lib/chatter_web/channels/chat_room_channel.ex:15: ChatterWeb.ChatRoomChannel.handle_in/3
             (phoenix) lib/phoenix/channel/server.ex:284: anonymous fn/4 in Phoenix.Channel.Server.handle_info/2
             (chatter) lib/chatter_web/endpoint.ex:1: ChatterWeb.Endpoint.instrument/4
             (stdlib) gen_server.erl:637: :gen_server.try_dispatch/4
             (stdlib) gen_server.erl:711: :gen_server.handle_msg/6
             (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3



Finished in 0.08 seconds
1 test, 1 failure

Great. Now, we're getting the error that matches the second warning we've seen: ** (UndefinedFunctionError) function Chatter.Chat.new_message/2 is undefined or private. Let's "step in" by getting the same failure in chat_test.exs.

Creating Chatter.Chat.new_message/2

Our new_message/2 function takes a chat room and a map that includes the message's body and author. We want our function to save a message with the body, the author, and the associated chat room. Write the following test:

# test/chatter/chat_test.exs

  describe "new_message/2" do
    test "inserts message associated to room" do
      room = insert(:chat_room)
      params = %{"body" => "Hello world", "author" => "random@example.com"}

      {:ok, message} = Chat.new_message(room, params)

      assert message.chat_room_id == room.id
      assert message.body == params["body"]
      assert message.author == params["author"]
      assert message.id
    end
  end

Now, let's run the test:

$ mix test test/chatter/chat_test.exs:70
Excluding tags: [:test]
Including tags: [line: "70"]



  1) test new_message/2 inserts message associated to room (Chatter.ChatTest)
     test/chatter/chat_test.exs:65
     ** (UndefinedFunctionError) function Chatter.Chat.new_message/2 is undefined or private
     code: {:ok, message} = Chat.new_message(room, params)
     stacktrace:
       (chatter) Chatter.Chat.new_message(%Chatter.Chat.Room{__meta__: #Ecto.Schema.Metadata<:loaded, "chat_rooms">, id: 487, inserted_at: ~N[2020-03-06 10:25:17], name: "chat room 0", updated_at: ~N[2020-03-06 10:25:17]}, %{"author" => "random@example.com", "body" => "Hello world"})
       test/chatter/chat_test.exs:69: (test)



Finished in 0.08 seconds
7 tests, 1 failure, 6 excluded

Good. We have successfully "stepped in" by getting the same error as our channel test. Now, let's add a basic but incomplete implementation to move the test forward:

# lib/chatter/chat.ex

  def new_message(_room, _params) do
    {:ok, %{}}
  end

Run the test:

mix test test/chatter/chat_test.exs:70
Compiling 1 file (.ex)
Excluding tags: [:test]
Including tags: [line: "70"]



  1) test new_message/2 inserts message associated to room (Chatter.ChatTest)
     test/chatter/chat_test.exs:65
     ** (KeyError) key :chat_room_id not found in: %{}
     code: assert message.chat_room_id == room.id
     stacktrace:
       test/chatter/chat_test.exs:71: (test)



Finished in 0.09 seconds
7 tests, 1 failure, 6 excluded

The function exists, so we make it one step further. But since we return an empty map, there's no chat_room_id key. Let's add a more realistic implementation. We will create a new message that is associated with the room. Replace our previous implementation with this:

# lib/chatter/chat.ex

  def new_message(room, params) do
    room
    |> Ecto.build_assoc(:messages)
    |> Chat.Room.Message.changeset(params)
    |> Repo.insert()
  end

We use Ecto's build_assoc/2 to associate the message with the chat room. We then use a yet-to-be-created Message.changeset/2 function and then insert the new message into the database. Let's run our test:

$ mix test test/chatter/chat_test.exs:70
Compiling 1 file (.ex)
warning: function Chatter.Chat.Room.Message.changeset/2 is undefined (module Chatter.Chat.Room.Message is not available)
  lib/chatter/chat.ex:30

Excluding tags: [:test]
Including tags: [line: "70"]



  1) test new_message/2 inserts message associated to room (Chatter.ChatTest)
     test/chatter/chat_test.exs:65
     ** (ArgumentError) schema Chatter.Chat.Room does not have association :messages
     code: {:ok, message} = Chat.new_message(room, params)
     stacktrace:
       (ecto) lib/ecto/association.ex:138: Ecto.Association.association_from_schema!/2
       (ecto) lib/ecto.ex:462: Ecto.build_assoc/3
       (chatter) lib/chatter/chat.ex:29: Chatter.Chat.new_message/2
       test/chatter/chat_test.exs:69: (test)



Finished in 0.09 seconds
7 tests, 1 failure, 6 excluded

It seems our Chat.Room doesn't have a messages association. Let's define it:

# lib/chatter/chat/room.ex

   schema "chat_rooms" do
     field :name, :string
+    has_many :messages, Chatter.Chat.Room.Message

     timestamps()
   end

Rerun our test:

$ mix test test/chatter/chat_test.exs:70
Compiling 2 files (.ex)
warning: invalid association `messages` in schema Chatter.Chat.Room: associated schema Chatter.Chat.Room.Message does not exist
  lib/chatter/chat/room.ex:1: Chatter.Chat.Room (module)

warning: function Chatter.Chat.Room.Message.changeset/2 is undefined (module Chatter.Chat.Room.Message is not available)
  lib/chatter/chat.ex:30

Excluding tags: [:test]
Including tags: [line: "70"]



  1) test new_message/2 inserts message associated to room (Chatter.ChatTest)
     test/chatter/chat_test.exs:65
     ** (UndefinedFunctionError) function Chat.Room.Message.__schema__/1 is undefined (module Chat.Room.Message is not available)
     code: room = insert(:chat_room)
     stacktrace:
       Chat.Room.Message.__schema__(:primary_key)
       (ecto) lib/ecto/changeset/relation.ex:140: Ecto.Changeset.Relation.change/3
       (ecto) lib/ecto/repo/schema.ex:710: anonymous fn/4 in Ecto.Repo.Schema.surface_changes/3
       (elixir) lib/enum.ex:1940: Enum."-reduce/3-lists^foldl/2-0-"/3
       (ecto) lib/ecto/repo/schema.ex:697: Ecto.Repo.Schema.surface_changes/3
       (ecto) lib/ecto/repo/schema.ex:235: Ecto.Repo.Schema.do_insert/4
       (ecto) lib/ecto/repo/schema.ex:164: Ecto.Repo.Schema.insert!/4
       test/chatter/chat_test.exs:66: (test)



Finished in 0.07 seconds
7 tests, 1 failure, 6 excluded

Good! Our application is finally complaining that Chat.Room.Message does not exist! It's often surprising how far we can go without introducing a database table. Now let's add that module, schema, and table.

Creating Chat.Room.Message

We'll use Phoenix's schema generator for this. If you frequently forget the order of arguments for Phoenix's schema generator — I know I do — you can always find help with mix help phx.gen.schema. For now, we'll use the generator to create the migration and schema files, and we'll modify them by hand:

$ mix phx.gen.schema Chat.Room.Message chat_room_messages
* creating lib/chatter/chat/room/message.ex
* creating priv/repo/migrations/20200309101741_create_chat_room_messages.exs

Open up the migration to add columns for the chat_room_id, the body and the author:

# priv/repo/migrations/20200309191741_create_chat_room_messages.exs

defmodule Chatter.Repo.Migrations.CreateChatRoomMessages do
  use Ecto.Migration

  def change do
    create table(:chat_room_messages) do
      add :chat_room_id, references(:chat_rooms), null: false
      add :body, :text, null: false
      add :author, :string, null: false

      timestamps()
    end
  end
end

You may have noticed that we did not reference the users table for the author. Instead, we only keep the email to render the messages in history. Other applications might need that reference. In our application, however, we're only interested in the historical context, not in associating all of a user's messages back to them. So we keep it simple.

Now let's add the corresponding fields to the schema:

# lib/chatter/chat/room/message.ex

defmodule Chatter.Chat.Room.Message do
  use Ecto.Schema
  import Ecto.Changeset

  schema "chat_room_messages" do
    field :author, :string
    field :body, :string
    belongs_to :chat_room, Chatter.Chat.Room

    timestamps()
  end

  @doc false
  def changeset(message, attrs) do
    message
    |> cast(attrs, [])
    |> validate_required([])
  end
end

The schema generator also created a changeset/2 function for us. Though we're not using it now, we'll use it very soon, so we'll leave it defined.

Finally, run mix ecto.migrate to migrate the database, and rerun our chat test:

$ mix test test/chatter/chat_test.exs:70
Excluding tags: [:test]
Including tags: [line: "70"]



  1) test new_message/2 inserts message associated to room (Chatter.ChatTest)
     test/chatter/chat_test.exs:65
     ** (KeyError) key :room_id not found
     code: {:ok, message} = Chat.new_message(room, params)
     stacktrace:
       (ecto) lib/ecto/association.ex:606: Ecto.Association.Has.build/3
       (chatter) lib/chatter/chat.ex:29: Chatter.Chat.new_message/2
       test/chatter/chat_test.exs:69: (test)



Finished in 0.09 seconds
7 tests, 1 failure, 6 excluded

Our test fails when we try to use Ecto.build_assoc/2. It seems the function assumes that Chat.Room.Messages will have a room_id instead of a chat_room_id. Let's update the foreign_key option in our Chat.Room module:

# lib/chatter/chat/room.ex

   schema "chat_rooms" do
     field :name, :string
-    has_many :messages, Chatter.Chat.Room.Message
+    has_many :messages, Chatter.Chat.Room.Message, foreign_key: :chat_room_id

     timestamps()
   end

Now run the test again:

$ mix test test/chatter/chat_test.exs:70
Compiling 2 files (.ex)
Excluding tags: [:test]
Including tags: [line: "70"]



  1) test new_message/2 inserts message associated to room (Chatter.ChatTest)
     test/chatter/chat_test.exs:65
     ** (Postgrex.Error) ERROR 23502 (not_null_violation) null value in column "body" violates not-null constraint

         table: chat_room_messages
         column: body

     Failing row contains (1, 3, null, null, 2020-03-09 10:29:06, 2020-03-09 10:29:06).
     code: {:ok, message} = Chat.new_message(room, params)
     stacktrace:
       (ecto_sql) lib/ecto/adapters/sql.ex:605: Ecto.Adapters.SQL.raise_sql_call_error/1
       (ecto) lib/ecto/repo/schema.ex:649: Ecto.Repo.Schema.apply/4
       (ecto) lib/ecto/repo/schema.ex:262: anonymous fn/15 in Ecto.Repo.Schema.do_insert/4
       test/chatter/chat_test.exs:69: (test)



Finished in 0.08 seconds
7 tests, 1 failure, 6 excluded

Good, the association is now set correctly. Our test fails because we tried inserting a message with an empty body, which is not allowed. But why is the body empty? Well, our Message.changeset/2 function is not casting or validating any fields, so let's update that next.

Implementing Chat.Room.Message.changeset/2

Create a new file to test the Chat.Room.Message module and add the following test:

# test/chatter/chat/room/message_test.exs

defmodule Chatter.Chat.Room.MessageTest do
  use Chatter.DataCase, async: true

  alias Chatter.Chat.Room.Message

  describe "changeset/2" do
    test "validates that an author and body are provided" do
      changes = %{}

      changeset = Message.changeset(%Message{}, changes)

      refute changeset.valid?
      assert "can't be blank" in errors_on(changeset).body
      assert "can't be blank" in errors_on(changeset).author
    end
  end
end

We first want to test that our Message.changeset/2 validates the presence of the body and author. To do that, we pass an empty map, and then expect the returning changeset to be invalid, having errors on both the body and author fields.

Let's run the message_test:

$ mix test test/chatter/chat/room/message_test.exs:10
Excluding tags: [:test]
Including tags: [line: "10"]



  1) test changeset/2 validates that an author and body are provided (Chatter.Chat.Room.MessageTest)
     test/chatter/chat/room/message_test.exs:7
     Expected false or nil, got true
     code: refute changeset.valid?()
     stacktrace:
       test/chatter/chat/room/message_test.exs:12: (test)



Finished in 0.04 seconds
1 test, 1 failure

The changeset came back valid because we didn't cast or validate the author and body fields. Let's update that:

# lib/chatter/chat/room/message.ex

   def changeset(message, attrs) do
     message
-    |> cast(attrs, [])
-    |> validate_required([])
+    |> cast(attrs, [:author, :body])
+    |> validate_required([:author, :body])
   end

Rerun the test:

$ mix test test/chatter/chat/room/message_test.exs:10
Compiling 2 files (.ex)
Excluding tags: [:test]
Including tags: [line: "10"]

.

Finished in 0.03 seconds
1 test, 0 failures

Good! Let's add one more test to ensure the chat_room_id is required:

# test/chatter/chat/room/message_test.exs

  test "validates that record is associated to a chat room" do
    changes = %{"body" => "hello world", "author" => "person@example.com"}

    changeset = Message.changeset(%Message{}, changes)

    refute changeset.valid?
    assert "can't be blank" in errors_on(changeset).chat_room_id
  end

Run the new test:

$ mix test test/chatter/chat/room/message_test.exs:17
Excluding tags: [:test]
Including tags: [line: "17"]



  1) test changeset/2 validates that record is associated to a chat room (Chatter.Chat.Room.MessageTest)
     test/chatter/chat/room/message_test.exs:17
     Expected false or nil, got true
     code: refute changeset.valid?()
     stacktrace:
       test/chatter/chat/room/message_test.exs:22: (test)



Finished in 0.05 seconds
2 tests, 1 failure, 1 excluded

The returning changeset is valid, even though it should not be. Let's add a validation for chat_room_id:

  def changeset(message, attrs) do
    message
-    |> cast(attrs, [:author, :body])
-    |> validate_required([:author, :body])
+    |> cast(attrs, [:author, :body, :chat_room_id])
+    |> validate_required([:author, :body, :chat_room_id])
  end

And rerun our test:

$ mix test test/chatter/chat/room/message_test.exs:17
Compiling 2 files (.ex)
Excluding tags: [:test]
Including tags: [line: "17"]

.

Finished in 0.04 seconds
2 tests, 0 failures, 1 excluded

Good! Now run both of our message tests to make sure they pass.

$ mix test test/chatter/chat/room/message_test.exs
..

Finished in 0.05 seconds
2 tests, 0 failures

Nice! Before moving forward, let's complete the red-green-refactor cycle for the two tests we just added by refactoring Room.Message.changeset/2. The list of fields we pass to cast/2 and validate_required/2 is the same, so we can extract that into a module attribute to avoid the repetition:

# lib/chatter/chat/room/message.ex

+  @valid_fields [:author, :body, :chat_room_id]

   @doc false
   def changeset(message, attrs) do
     message
-    |> cast(attrs, [:author, :body, :chat_room_id])
-    |> validate_required([:author, :body, :chat_room_id])
+    |> cast(attrs, @valid_fields)
+    |> validate_required(@valid_fields)
   end

In the future, we may only want a subset of the fields we cast to be required. If that's the case, we can separate them then. For now, our refactoring is slightly cleaner. Let's rerun our tests one more time to make sure they still pass.

$ mix test test/chatter/chat/room/message_test.exs                  ..

Finished in 0.05 seconds
2 tests, 0 failures

Good! It's time to step back out to chat_test.

Stepping out to Chatter.ChatTest

Having implemented the Message.changeset/2 function, let's now step back out and run our chat_test:

$ mix test test/chatter/chat_test.exs:70
Excluding tags: [:test]
Including tags: [line: "70"]

.

Finished in 0.1 seconds
7 tests, 0 failures, 6 excluded

Excellent! It seems the changeset was all we needed. Before we step back out another level, however, I'd like to test the behavior of new_message/2 when we fail to create a message. On failure, we expect an {:error, changeset} not an {:ok, message}. Let's write that test:

# test/chatter/chat_test.exs

  test "returns a changeset if insert fails" do
    room = insert(:chat_room)
    params = %{}

    {:error, changeset} = Chat.new_message(room, params)

    assert errors_on(changeset).body
  end

To make the test fail, we pass an empty map for params into Chat.new_message/2. Note that we want to test the failure behavior of Chat.new_message/2. That includes Chat.new_message/2 returning an error tuple with a changeset that has some type of error. But we don't care about the exact error messages returned (since that's part of the Message's responsibility). So we only assert that errors are present without concerning ourselves with the actual error message.

Let's run the test:

$ mix test test/chatter/chat_test.exs:76
Excluding tags: [:test]
Including tags: [line: "76"]

.

Finished in 0.1 seconds
8 tests, 0 failures, 7 excluded

Good! Now, let's step back out one more level to our chat room channel test.

Refactoring ChatRoomChannel

Now that we've successfully implemented the Chat.new_message/2 function, we should expect our ChatRoomChannelTest to pass. Let's see if that's the case by running it:

$ mix test test/chatter_web/channels/chat_room_channel_test.exs:5
Excluding tags: [:test]
Including tags: [line: "5"]

.

Finished in 0.1 seconds
1 test, 0 failures

Excellent! Now that our chat room channel is passing, we should consider refactoring our implementation of ChatRoomChannel. I am concerned that we create a new message in our database before broadcasting it. That means our chat message will not be broadcast until we've made a round trip to the database.

Depending on the business requirements, the current behavior could be something we want — to ensure the integrity of message history. But I consider broadcasting messages to be more important than storing them. Broadcasting them is essential to our application. A failure to store a message should not prevent that message from being broadcast.

Since we care more about the broadcast than about saving the messages, let's make it so that saving the chat message doesn't block the broadcast. To do so, we could use Task.async to send the message asynchronously, or we can send our channel process a message that will then save the chat room in the database. Let's do the latter.

First, let's send our process a new message via send/2:

# lib/chatter_web/channels/chat_room_channel.ex

  def handle_in("new_message", payload, socket) do
    %{room: room, email: author} = socket.assigns
    outgoing_payload = Map.put(payload, "author", author)

+   send(self(), {:store_new_message, outgoing_payload})
    Chat.new_message(room, outgoing_payload)
    broadcast(socket, "new_message", outgoing_payload)

    {:noreply, socket}
  end

Now run our test:

$ mix test test/chatter_web/channels/chat_room_channel_test.exs:5
Compiling 1 file (.ex)
Excluding tags: [:test]
Including tags: [line: "5"]

.

Finished in 0.08 seconds
1 test, 0 failures

Now, let's move the pattern matching of room and the call to Chat.new_message/2 to a new handle_info/2 function that handles the {:store_new_message, payload} message:

# lib/chatter_web/channels/chat_room_channel.ex

   def handle_in("new_message", payload, socket) do
-    %{room: room, email: author} = socket.assigns
+    %{email: author} = socket.assigns
     outgoing_payload = Map.put(payload, "author", author)

     send(self(), {:store_new_message, outgoing_payload})
-    Chat.new_message(room, payload)
     broadcast(socket, "new_message", outgoing_payload)

     {:noreply, socket}
   end

+  def handle_info({:store_new_message, payload}, socket) do
+    %{room: room} = socket.assigns
+    Chat.new_message(room, payload)
+
+    {:noreply, socket}
+  end

And rerun our test:

$ mix test test/chatter_web/channels/chat_room_channel_test.exs:5
Compiling 1 file (.ex)
Excluding tags: [:test]
Including tags: [line: "5"]

.

Finished in 0.07 seconds
1 test, 0 failures

Good! The test still passes, but saving the message in the database no longer blocks the broadcast. We're now ready to step back out to our feature test.

Stepping out to our feature test

Let's run our test in test/chatter_web/features/user_can_chat_test.exs to see what we need to do next:

$ mix test test/chatter_web/features/user_can_chat_test.exs:34
Compiling 1 file (.ex)
Excluding tags: [:test]
Including tags: [line: "34"]



  1) test new user can see previous messages in chat room (ChatterWeb.UserCanChatTest)
     test/chatter_web/features/user_can_chat_test.exs:34
     ** (Wallaby.ExpectationNotMetError) Expected to find 1, visible element with the attribute 'data-role' with value 'message' but 0, visible elements with the attribute were found.

     code: |> assert_has(message(user1, "Welcome future users"))
     stacktrace:
       test/chatter_web/features/user_can_chat_test.exs:51: (test)



Finished in 5.5 seconds
2 tests, 1 failure, 1 excluded

The first part of our feature is complete: we save messages when they are sent. But Wallaby still can't find the messages because we don't retrieve them when a user first joins a chat room. That's what we'll work on next.

Fetching a chat room's message history

We completed the first part of our feature: saving messages. But now, when users join a chat room, we need to fetch the room's history for them participate in an ongoing conversation.

Let's start by updating our JavaScript code to expect messages in the response when joining a channel. If you aren't doing so already, run mix assets.watch in a terminal.

Now, open up chat_room.js, and update how we respond to a successful channel.join() like this:

// assets/js/chat_room.js

  channel.join();
    .receive("ok", resp => {      let messages = resp.messages      messages.map(({ author, body }) => {        let messageItem = document.createElement("li");        messageItem.dataset.role = "message";        messageItem.innerText = `${author}: ${body}`;        messagesContainer.appendChild(messageItem);      });    })

We expect a set of messages (with author and body) and iterate over them, creating a list item for each, and appending them to the messages container.

Let's run our test:

$ mix test test/chatter_web/features/user_can_chat_test.exs:39
Excluding tags: [:test]
Including tags: [line: "39"]



  1) test new user can see previous messages in chat room (ChatterWeb.UserCanChatTest)
     test/chatter_web/features/user_can_chat_test.exs:34
     ** (Wallaby.ExpectationNotMetError) Expected to find 1, visible element with the attribute 'data-role' with value 'message' but 0, visible elements with the attribute were found.

     code: |> assert_has(message(user1, "Welcome future users"))
     stacktrace:
       test/chatter_web/features/user_can_chat_test.exs:51: (test)



Finished in 5.3 seconds
2 tests, 1 failure, 1 excluded

As in the last chapter, the error doesn't give guidance on how to proceed next. But we know a few tricks to unblock ourselves: let's throw a console.log to see the value of resp.messages (hint: it should be undefined since we haven't updated our Elixir code to send them).

// assets/js/chat_room.js

    channel.join()
      .receive("ok", resp => {
        let messages = resp.messages
+       console.log(messages)
        messages.map(({ author, body }) => {
          // code omitted
        });
      })
  }

Now rerun our tests:

$ mix test test/chatter_web/features/user_can_chat_test.exs:39
Excluding tags: [:test]
Including tags: [line: "39"]

undefined
undefined


  1) test new user can see previous messages in chat room (ChatterWeb.UserCanChatTest)
     test/chatter_web/features/user_can_chat_test.exs:34
     ** (Wallaby.ExpectationNotMetError) Expected to find 1, visible element with the attribute 'data-role' with value 'message' but 0, visible elements with the attribute were found.

     code: |> assert_has(message(user1, "Welcome future users"))
     stacktrace:
       test/chatter_web/features/user_can_chat_test.exs:51: (test)



Finished in 5.3 seconds
2 tests, 1 failure, 1 excluded

Just as we expected: no messages are sent when a user joins a channel. Remove the console.log, and let's move to the ChatRoomChannel code.

Returning a chat room's history on join

Let's open up our chat room channel and update the join/3 function to return a list of messages on join.

# lib/chatter_web/channels/chat_room_channel.ex

   def join("chat_room:" <> room_name, _msg, socket) do
     room = Chat.find_room_by_name(room_name)
+    messages = Chat.room_messages(room)

-    {:ok, assign(socket, :room, room)}
+    {:ok, %{messages: messages}, assign(socket, :room, room)}
   end

We expect to get the messages from Chat.room_messages/1, a function that does not exist yet. Then, we return the messages to the client in a map via the reply in {:ok, reply, socket}.

Let's run our feature test:

$ mix test test/chatter_web/features/user_can_chat_test.exs:34
Compiling 1 file (.ex)
warning: function Chatter.Chat.room_messages/1 is undefined or private
  lib/chatter_web/channels/chat_room_channel.ex:8

Excluding tags: [:test]
Including tags: [line: "34"]

04:56:26.945 [error] an exception was raised:
    ** (UndefinedFunctionError) function Chatter.Chat.room_messages/1 is undefined or private
        (chatter) Chatter.Chat.room_messages(%Chatter.Chat.Room{__meta__: #Ecto.Schema.Metadata<:loaded, "chat_rooms">, id: 19, inserted_at: ~N[2020-03-19 08:56:25], messages: #Ecto.Association.NotLoaded<association :messages is not loaded>, name: "chat room 0", updated_at: ~N[2020-03-19 08:56:25]})
        (chatter) lib/chatter_web/channels/chat_room_channel.ex:8: ChatterWeb.ChatRoomChannel.join/3
        (chatter) lib/chatter_web/endpoint.ex:1: ChatterWeb.Endpoint.instrument/4
        (phoenix) lib/phoenix/channel/server.ex:219: Phoenix.Channel.Server.init/1
        (stdlib) gen_server.erl:374: :gen_server.init_it/2
        (stdlib) gen_server.erl:342: :gen_server.init_it/6
        (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
04:56:27.753 [error] an exception was raised:
    ** (UndefinedFunctionError) function Chatter.Chat.room_messages/1 is undefined or private
        (chatter) Chatter.Chat.room_messages(%Chatter.Chat.Room{__meta__: #Ecto.Schema.Metadata<:loaded, "chat_rooms">, id: 19, inserted_at: ~N[2020-03-19 08:56:25], messages: #Ecto.Association.NotLoaded<association :messages is not loaded>, name: "chat room 0", updated_at: ~N[2020-03-19 08:56:25]})
        (chatter) lib/chatter_web/channels/chat_room_channel.ex:8: ChatterWeb.ChatRoomChannel.join/3
        (chatter) lib/chatter_web/endpoint.ex:1: ChatterWeb.Endpoint.instrument/4
        (phoenix) lib/phoenix/channel/server.ex:219: Phoenix.Channel.Server.init/1
        (stdlib) gen_server.erl:374: :gen_server.init_it/2
        (stdlib) gen_server.erl:342: :gen_server.init_it/6
        (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
04:56:27.989 [error] an exception was raised:
    ** (UndefinedFunctionError) function Chatter.Chat.room_messages/1 is undefined or private
        (chatter) Chatter.Chat.room_messages(%Chatter.Chat.Room{__meta__: #Ecto.Schema.Metadata<:loaded, "chat_rooms">, id: 19, inserted_at: ~N[2020-03-19 08:56:25], messages: #Ecto.Association.NotLoaded<association :messages is not loaded>, name: "chat room 0", updated_at: ~N[2020-03-19 08:56:25]})
        (chatter) lib/chatter_web/channels/chat_room_channel.ex:8: ChatterWeb.ChatRoomChannel.join/3
        (chatter) lib/chatter_web/endpoint.ex:1: ChatterWeb.Endpoint.instrument/4
        (phoenix) lib/phoenix/channel/server.ex:219: Phoenix.Channel.Server.init/1
        (stdlib) gen_server.erl:374: :gen_server.init_it/2
        (stdlib) gen_server.erl:342: :gen_server.init_it/6
        (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
04:56:28.770 [error] an exception was raised:
    ** (UndefinedFunctionError) function Chatter.Chat.room_messages/1 is undefined or private
        (chatter) Chatter.Chat.room_messages(%Chatter.Chat.Room{__meta__: #Ecto.Schema.Metadata<:loaded, "chat_rooms">, id: 19, inserted_at: ~N[2020-03-19 08:56:25], messages: #Ecto.Association.NotLoaded<association :messages is not loaded>, name: "chat room 0", updated_at: ~N[2020-03-19 08:56:25]})
        (chatter) lib/chatter_web/channels/chat_room_channel.ex:8: ChatterWeb.ChatRoomChannel.join/3
        (chatter) lib/chatter_web/endpoint.ex:1: ChatterWeb.Endpoint.instrument/4
        (phoenix) lib/phoenix/channel/server.ex:219: Phoenix.Channel.Server.init/1
        (stdlib) gen_server.erl:374: :gen_server.init_it/2
        (stdlib) gen_server.erl:342: :gen_server.init_it/6
        (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
04:56:29.991 [error] an exception was raised:
    ** (UndefinedFunctionError) function Chatter.Chat.room_messages/1 is undefined or private
        (chatter) Chatter.Chat.room_messages(%Chatter.Chat.Room{__meta__: #Ecto.Schema.Metadata<:loaded, "chat_rooms">, id: 19, inserted_at: ~N[2020-03-19 08:56:25], messages: #Ecto.Association.NotLoaded<association :messages is not loaded>, name: "chat room 0", updated_at: ~N[2020-03-19 08:56:25]})
        (chatter) lib/chatter_web/channels/chat_room_channel.ex:8: ChatterWeb.ChatRoomChannel.join/3
        (chatter) lib/chatter_web/endpoint.ex:1: ChatterWeb.Endpoint.instrument/4
        (phoenix) lib/phoenix/channel/server.ex:219: Phoenix.Channel.Server.init/1
        (stdlib) gen_server.erl:374: :gen_server.init_it/2
        (stdlib) gen_server.erl:342: :gen_server.init_it/6
        (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
04:56:30.778 [error] an exception was raised:
    ** (UndefinedFunctionError) function Chatter.Chat.room_messages/1 is undefined or private
        (chatter) Chatter.Chat.room_messages(%Chatter.Chat.Room{__meta__: #Ecto.Schema.Metadata<:loaded, "chat_rooms">, id: 19, inserted_at: ~N[2020-03-19 08:56:25], messages: #Ecto.Association.NotLoaded<association :messages is not loaded>, name: "chat room 0", updated_at: ~N[2020-03-19 08:56:25]})
        (chatter) lib/chatter_web/channels/chat_room_channel.ex:8: ChatterWeb.ChatRoomChannel.join/3
        (chatter) lib/chatter_web/endpoint.ex:1: ChatterWeb.Endpoint.instrument/4
        (phoenix) lib/phoenix/channel/server.ex:219: Phoenix.Channel.Server.init/1
        (stdlib) gen_server.erl:374: :gen_server.init_it/2
        (stdlib) gen_server.erl:342: :gen_server.init_it/6
        (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3


  1) test new user can see previous messages in chat room (ChatterWeb.UserCanChatTest)
     test/chatter_web/features/user_can_chat_test.exs:34
     ** (Wallaby.ExpectationNotMetError) Expected to find 1, visible element with the attribute 'data-role' with value 'message' but 0, visible elements with the attribute were found.

     code: |> assert_has(message(user1, "Welcome future users"))
     stacktrace:
       test/chatter_web/features/user_can_chat_test.exs:51: (test)



Finished in 5.6 seconds
2 tests, 1 failure, 1 excluded

Wow, that's a large error message, though there is only one major error: our Chat.room_messages/1 function is undefined, ** (UndefinedFunctionError) function Chatter.Chat.room_messages/1 is undefined or private.

Since we're going into our core business logic territory, it's a good time to "step in" by writing another test in ChatTest for room_messages/1.

Stepping into Chat

Open up chat_test.exs, and add the following test:

# test/chatter/chat_test.exs

  describe "room_messages/1" do
    test "returns all messages associated to given room" do
      room = insert(:chat_room)
      messages = insert_pair(:chat_room_messages, chat_room: room)
      _different_room_message = insert(:chat_room_message)

      found_messages = Chat.room_messages(room)

      assert found_messages == messages
    end
  end

Note that we add a _different_room_message to test implicitly that we aren't returning chat room messages for a different room. Now, let's run it:

$ mix test test/chatter/chat_test.exs:87
Compiling 2 files (.ex)
warning: function Chatter.Chat.room_messages/1 is undefined or private
  lib/chatter_web/channels/chat_room_channel.ex:8

Excluding tags: [:test]
Including tags: [line: "87"]



  1) test room_messages/1 returns all messages associated to given room (Chatter.ChatTest)
     test/chatter/chat_test.exs:87
     ** (ExMachina.UndefinedFactoryError) No factory defined for :chat_room_message.

     Please check for typos or define your factory:

         def chat_room_message_factory do
           ...
         end

     code: messages = insert_pair(:chat_room_message, chat_room: room)
     stacktrace:
       (ex_machina) lib/ex_machina.ex:205: ExMachina.build/3
       (chatter) test/support/factory.ex:2: Chatter.Factory.insert/2
       (elixir) lib/stream.ex:1341: Stream.do_repeatedly/3
       (elixir) lib/enum.ex:2486: Enum.take/2
       test/chatter/chat_test.exs:89: (test)



Finished in 0.1 seconds

Aha! We are missing a chat_room_message factory. Let's define it:

# test/support/factory.ex

  def chat_room_message_factory do
    %Chatter.Chat.Room.Message{
      body: sequence(:body, &"hello there #{&1}"),
      author: sequence(:email, &"user#{&1}@example.com"),
      chat_room: build(:chat_room)
    }
  end

Rerun the test:

$ mix test test/chatter/chat_test.exs:87
Compiling 2 files (.ex)
warning: function Chatter.Chat.room_messages/1 is undefined or private
  lib/chatter_web/channels/chat_room_channel.ex:8

Excluding tags: [:test]
Including tags: [line: "87"]



  1) test room_messages/1 returns all messages associated to given room (Chatter.ChatTest)
     test/chatter/chat_test.exs:87
     ** (UndefinedFunctionError) function Chatter.Chat.room_messages/1 is undefined or private
     code: found_messages = Chat.room_messages(room)
     stacktrace:
       (chatter) Chatter.Chat.room_messages(%Chatter.Chat.Room{__meta__: #Ecto.Schema.Metadata<:loaded, "chat_rooms">, id: 25, inserted_at: ~N[2020-03-20 14:43:03], messages: #Ecto.Association.NotLoaded<association :messages is not loaded>, name: "chat room 0", updated_at: ~N[2020-03-20 14:43:03]})
       test/chatter/chat_test.exs:92: (test)



Finished in 0.2 seconds
9 tests, 1 failure, 8 excluded

Good. We have successfully "stepped in" because we have the same test failure: ** (UndefinedFunctionError) function Chatter.Chat.room_messages/1 is undefined or private. We can now use this test as our guidance. Let's add an empty function:

# lib/chatter/chat.ex

  def room_messages(room) do
  end

And run the test:

$ mix test test/chatter/chat_test.exs:87
Compiling 1 file (.ex)
warning: variable "room" is unused (if the variable is not meant to be used, prefix it with an underscore)
  lib/chatter/chat.ex:34

Excluding tags: [:test]
Including tags: [line: "87"]



  1) test room_messages/1 returns all messages associated to given room (Chatter.ChatTest)
     test/chatter/chat_test.exs:87
     Assertion with == failed
     code:  assert found_messages == messages
     left:  nil
     right: [
              %Chatter.Chat.Room.Message{
                __meta__: #Ecto.Schema.Metadata<:loaded, "chat_room_messages">,
                author: "user0@example.com",
                body: "hello there 0",
                chat_room: %Chatter.Chat.Room{
                  __meta__: #Ecto.Schema.Metadata<:loaded, "chat_rooms">,
                  id: 27,
                  inserted_at: ~N[2020-03-20 14:45:36],
                  messages: #Ecto.Association.NotLoaded<association :messages is not loaded>,
                  name: "chat room 0",
                  updated_at: ~N[2020-03-20 14:45:36]
                },
                chat_room_id: 27,
                id: 18,
                inserted_at: ~N[2020-03-20 14:45:36],
                updated_at: ~N[2020-03-20 14:45:36]
              },
              %Chatter.Chat.Room.Message{
                __meta__: #Ecto.Schema.Metadata<:loaded, "chat_room_messages">,
                author: "user1@example.com",
                body: "hello there 1",
                chat_room: %Chatter.Chat.Room{
                  __meta__: #Ecto.Schema.Metadata<:loaded, "chat_rooms">,
                  id: 27,
                  inserted_at: ~N[2020-03-20 14:45:36],
                  messages: #Ecto.Association.NotLoaded<association :messages is not loaded>,
                  name: "chat room 0",
                  updated_at: ~N[2020-03-20 14:45:36]
                },
                chat_room_id: 27,
                id: 19,
                inserted_at: ~N[2020-03-20 14:45:36],
                updated_at: ~N[2020-03-20 14:45:36]
              }
            ]
     stacktrace:
       test/chatter/chat_test.exs:94: (test)



Finished in 0.1 seconds
9 tests, 1 failure, 8 excluded

Good. We received nil, but we expected a set of messages. Let's add an implementation:

# lib/chatter/chat.ex

  import Ecto.Query
# code omitted

  def room_messages(room) do    Chat.Room.Message    |> where([m], m.chat_room_id == ^room.id)    |> Repo.all()  end

Run the test again:

$ mix test test/chatter/chat_test.exs:87
Compiling 1 file (.ex)
Excluding tags: [:test]
Including tags: [line: "87"]



  1) test room_messages/1 returns all messages associated to given room (Chatter.ChatTest)
     test/chatter/chat_test.exs:87
     Assertion with == failed
     code:  assert found_messages == messages
     left:  [%Chatter.Chat.Room.Message{__meta__: #Ecto.Schema.Metadata<:loaded, "chat_room_messages">, author: "user0@example.com", body: "hello there 0", chat_room_id: 29, id: 21, inserted_at: ~N[2020-03-20 14:51:09], updated_at: ~N[2020-03-20 14:51:09], chat_room: #Ecto.Association.NotLoaded<association :chat_room is not loaded>}, %Chatter.Chat.Room.Message{__meta__: #Ecto.Schema.Metadata<:loaded, "chat_room_messages">, author: "user1@example.com", body: "hello there 1", chat_room_id: 29, id: 22, inserted_at: ~N[2020-03-20 14:51:09], updated_at: ~N[2020-03-20 14:51:09], chat_room: #Ecto.Association.NotLoaded<association :chat_room is not loaded>}]
     right: [%Chatter.Chat.Room.Message{__meta__: #Ecto.Schema.Metadata<:loaded, "chat_room_messages">, author: "user0@example.com", body: "hello there 0", chat_room_id: 29, id: 21, inserted_at: ~N[2020-03-20 14:51:09], updated_at: ~N[2020-03-20 14:51:09], chat_room: %Chatter.Chat.Room{__meta__: #Ecto.Schema.Metadata<:loaded, "chat_rooms">, id: 29, inserted_at: ~N[2020-03-20 14:51:09], messages: #Ecto.Association.NotLoaded<association :messages is not loaded>, name: "chat room 0", updated_at: ~N[2020-03-20 14:51:09]}}, %Chatter.Chat.Room.Message{__meta__: #Ecto.Schema.Metadata<:loaded, "chat_room_messages">, author: "user1@example.com", body: "hello there 1", chat_room_id: 29, id: 22, inserted_at: ~N[2020-03-20 14:51:09], updated_at: ~N[2020-03-20 14:51:09], chat_room: %Chatter.Chat.Room{__meta__: #Ecto.Schema.Metadata<:loaded, "chat_rooms">, id: 29, inserted_at: ~N[2020-03-20 14:51:09], messages: #Ecto.Association.NotLoaded<association :messages is not loaded>, name: "chat room 0", updated_at: ~N[2020-03-20 14:51:09]}}]
     stacktrace:
       test/chatter/chat_test.exs:94: (test)



Finished in 0.1 seconds
9 tests, 1 failure, 8 excluded

Hmmm. That might have come as a surprise. At first glance, it looks as though we got the same records. But something is different about them. Thankfully, ExUnit shows us the difference in colors (though you cannot see it in my code snippet above): the %Chat.Room.Message{} structs returned from the database do not have the chat_room association loaded, so we see #Ecto.Assocation.NotLoaded<association :chat_room is not loaded>. But the messages built with our testing factory have the chat_room associations loaded. So what to do?

We should ask ourselves, what is the desired behavior? In this case, we don't care whether the associations are loaded or not. So, let's modify our test assertions to check that the messages are the same in essence without caring about all the details.

Let's assert the following about the messages we get from Chat.room_messages/1:

  1. that they have the correct ids,
  2. that they have the correct text bodys, and
  3. that they have the correct text authors.
# test/chatter/chat_test.exs

    test "returns all messages associated to given room" do
      room = insert(:chat_room)
      messages = insert_pair(:chat_room_message, chat_room: room)
      _different_room_message = insert(:chat_room_message)

      found_messages = Chat.room_messages(room)

-     assert found_messages == messages
+     assert map_ids(found_messages) == map_ids(messages)
+     assert map_bodies(found_messages) == map_bodies(messages)
+     assert map_authors(found_messages) == map_authors(messages)
    end

+   defp map_ids(structs), do: Enum.map(structs, fn %{id: id} -> id end)
+   defp map_bodies(structs), do: Enum.map(structs, fn %{body: body} -> body end)
+   defp map_authors(structs), do: Enum.map(structs, fn %{author: author} -> author end)

We map through the structs, grabbing each of the properties we care about, and we ensure those are the same. It's not a perfect test, nor is it exactly clean, but works for our needs.

Now run the test once more.

$ mix test test/chatter/chat_test.exs:87
Compiling 1 file (.ex)
Excluding tags: [:test]
Including tags: [line: "87"]

.

Finished in 0.1 seconds
9 tests, 0 failures, 8 excluded

Perfect! Let's step back out to our feature test and see what fails next.

Back to the future feature test

Let's run our feature test to see what comes next. But be warned, a huge error is about to show up. Here's the pertinent part of the error message:

11:12:12.207 [error] GenServer #PID<0.1223.0> terminating
** (RuntimeError) cannot encode metadata from the :__meta__ field for Chatter.Chat.Room.Message to JSON. This metadata is used internally by ecto and should never be exposed externally.

You can either map the schemas to remove the :__meta__ field before encoding to JSON, or explicit list the JSON fields in your schema:

    defmodule Chatter.Chat.Room.Message do
      # ...

      @derive {Jason.Encoder, only: [:name, :title, ...]}
      schema ... do

    (ecto) lib/ecto/json.ex:26: Jason.Encoder.Ecto.Schema.Metadata.encode/2
    (jason) lib/encode.ex:163: Jason.Encode.map_naive/3
    (jason) lib/encode.ex:140: Jason.Encode.list/3
    (jason) lib/encode.ex:163: Jason.Encode.map_naive/3
    (jason) lib/encode.ex:149: Jason.Encode.list_loop/3
    (jason) lib/encode.ex:150: Jason.Encode.list_loop/3
    (jason) lib/encode.ex:141: Jason.Encode.list/3
    (jason) lib/encode.ex:35: Jason.Encode.encode/2
    (jason) lib/jason.ex:197: Jason.encode_to_iodata!/2
    (phoenix) lib/phoenix/socket/serializers/v2_json_serializer.ex:21: Phoenix.Socket.V2.JSONSerializer.encode!/1
    (phoenix) lib/phoenix/socket.ex:772: Phoenix.Socket.encode_reply/2
    (phoenix) lib/phoenix/socket.ex:686: Phoenix.Socket.handle_in/4
    (phoenix) lib/phoenix/endpoint/cowboy2_handler.ex:76: Phoenix.Endpoint.Cowboy2Handler.websocket_handle/2
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Last message: {:DOWN, #Reference<0.2391617946.3979083777.196268>, :process, #PID<0.1221.0>, {%RuntimeError{message: "cannot encode metadata from the :__meta__ field for Chatter.Chat.Room.Message to JSON. This metadata is used internally by ecto and should never be exposed externally.\n\nYou can either map the schemas to remove the :__meta__ field before encoding to JSON, or explicit list the JSON fields in your schema:\n\n    defmodule Chatter.Chat.Room.Message do\n      # ...\n\n      @derive {Jason.Encoder, only: [:name, :title, ...]}\n      schema ... do\n"}, [{Jason.Encoder.Ecto.Schema.Metadata, :encode, 2, [file: 'lib/ecto/json.ex', line: 26]}, {Jason.Encode, :map_naive, 3, [file: 'lib/encode.ex', line: 163]}, {Jason.Encode, :list, 3, [file: 'lib/encode.ex', line: 140]}, {Jason.Encode, :map_naive, 3, [file: 'lib/encode.ex', line: 163]}, {Jason.Encode, :list_loop, 3, [file: 'lib/encode.ex', line: 149]}, {Jason.Encode, :list_loop, 3, [file: 'lib/encode.ex', line: 150]}, {Jason.Encode, :list, 3, [file: 'lib/encode.ex', line: 141]}, {Jason.Encode, :encode, 2, [file: 'lib/encode.ex', line: 35]}, {Jason, :encode_to_iodata!, 2, [file: 'lib/jason.ex', line: 197]}, {Phoenix.Socket.V2.JSONSerializer, :encode!, 1, [file: 'lib/phoenix/socket/serializers/v2_json_serializer.ex', line: 21]}, {Phoenix.Socket, :encode_reply, 2, [file: 'lib/phoenix/socket.ex', line: 772]}, {Phoenix.Socket, :handle_in, 4, [file: 'lib/phoenix/socket.ex', line: 686]}, {Phoenix.Endpoint.Cowboy2Handler, :websocket_handle, 2, [file: 'lib/phoenix/endpoint/cowboy2_handler.ex', line: 76]}, {:cowboy_websocket, :handler_call, 6, [file: '/Users/germanvelasco/tdd-phoenix/chatter/deps/cowboy/src/cowboy_websocket.erl', line: 471]}, {:cowboy_http, :loop, 2, [file: '/Users/germanvelasco/tdd-phoenix/chatter/deps/cowboy/src/cowboy_http.erl', line: 233]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 249]}]}}
11:12:12.229 [error] exited in: DBConnection.Holder.checkout(#PID<0.555.0>, [log: #Function<11.63566995/1 in Ecto.Adapters.SQL.with_log/3>, source: "chat_rooms", timeout: 15000, pool_size: 10, pool: DBConnection.Ownership])
    ** (EXIT) shutdown: "owner #PID<0.554.0> exited"


  1) test new user can see previous messages in chat room (ChatterWeb.UserCanChatTest)
     test/chatter_web/features/user_can_chat_test.exs:34
     ** (Wallaby.ExpectationNotMetError) Expected to find 1, visible element with the attribute 'data-role' with value 'message' but 0, visible elements with the attribute were found.

     code: |> assert_has(message(user1, "Welcome future users"))
     stacktrace:
       test/chatter_web/features/user_can_chat_test.exs:51: (test)



Finished in 5.4 seconds
2 tests, 1 failure, 1 excluded

The crucial part is this: ** (RuntimeError) cannot encode metadata from the :__meta__ field for Chatter.Chat.Room.Message to JSON. This metadata is used internally by ecto and should never be exposed externally. What is that?

By attempting to pass our Chat.Room.Message structs in the payload, Phoenix is trying to encode them into JSON, but it fails to do so. Thankfully, we have a helpful Ecto error message:

You can either map the schemas to remove the :__meta__ field before encoding to
JSON, or explicit list the JSON fields in your schema:

    defmodule Chatter.Chat.Room.Message do
      # ...

      @derive {Jason.Encoder, only: [:name, :title, ...]}
      schema ... do

Let's follow Ecto's advice and list the JSON fields by usint a @derive declaration in our Chat.Room.Message module:

# lib/chatter/chat/room/message.ex

defmodule Chatter.Chat.Room.Message do
  use Ecto.Schema
  import Ecto.Changeset

  @derive {Jason.Encoder, only: [:author, :body, :chat_room_id]}  schema "chat_room_messages" do

Let's run our test again:

$ mix test test/chatter_web/features/user_can_chat_test.exs:35
Compiling 3 files (.ex)
Generated chatter app
Excluding tags: [:test]
Including tags: [line: "35"]

.

Finished in 2.6 seconds
2 tests, 0 failures, 1 excluded

Congratulations! Take a deep breath and give yourself a round of applause. We've done something fantastic. But before we celebrate too much, let's remember to refactor.

Refactoring

I am happy with most of the code we introduced. But I'd like to clean up some duplication in the chat_room.js file. It's a small change, but I think it worth doing.

Currently, we have the same logic for creating messages and appending them to the messages container: once when we receive a "new_message" and once when we iterate through a channel's. Let's extract that logic to have a single way of creating messages in the DOM.

Refactoring chat_room.js

Let's run both tests in our user_can_chat_test.exs file:

$ mix test test/chatter_web/features/user_can_chat_test.exs
..

Finished in 4.9 seconds
2 tests, 0 failures

Good. With our baseline set up, let's extract the logic from "new_message" and put it in a new function:

# assets/js/chat_room.js

+  addMessage(author, body) {
+    let messageItem = document.createElement("li");
+    messageItem.dataset.role = "message";
+    messageItem.innerText = `${author}: ${body}`;
+    messagesContainer.appendChild(messageItem);
+  }


   channel.on("new_message", payload => {
-    let messageItem = document.createElement("li");
-    messageItem.dataset.role = "message";
-    messageItem.innerText = `${payload.author}: ${payload.body}`;
-    messagesContainer.appendChild(messageItem);
+    addMessage(payload.author, payload.body);
   });

Now rerun the test:

$ mix test test/chatter_web/features/user_can_chat_test.exs
..

Finished in 6.1 seconds
2 tests, 0 failures

Let's now use that function when we join the channel and get the history of messages:

# assets/js/chat_room.js

   channel
     .join()
     .receive("ok", resp => {
       let messages = resp.messages;
       messages.map(({ author, body }) => {
-        let messageItem = document.createElement("li");
-        messageItem.dataset.role = "message";
-        messageItem.innerText = `${author}: ${body}`;
-        messagesContainer.appendChild(messageItem);
+        addMessage(author, body);
       });
     })

Run the test one more time:

$ mix test test/chatter_web/features/user_can_chat_test.exs
..

Finished in 6.3 seconds
2 tests, 0 failures

Excellent! I like this version much better.

Wrap up

This concludes the last significant feature of our application. Our users can sign up, sign in, create chat rooms, and chat for days on end without worrying about closing the browser or refreshing the page. And when someone new joins, they can see the thread of the conversation. Yes, I think we have made the world a little bit better.

But this is about test-driven development. What shall we do next about that? How can you sharpen this new tool you have gained? Let's talk about that next.