Scrolls
7 min read
On this page

Implementing Protocols

Knowing what protocols are is half the picture. The other half is implementing them for your own types — both the built-in protocols when you want your struct to play nice with Enum, for, to_string, and inspect, and your own protocols when you want extensibility points in a library.

This is also where you run into the production-only concerns: protocol consolidation, fallback dispatch, and the gotchas of changing implementations in a long-running release. None of these matter on day one, but all of them matter the first time you ship.

Implementing Enumerable for a Custom Structure

Enumerable is the protocol that opens the door to every function in Enum and Stream. If you build a tree, a graph, a ring buffer, or a priority queue, implementing Enumerable makes it work with the rest of the language for free.

The protocol has four callbacks: reduce/3, count/1, member?/2, and slice/1. reduce/3 is the only one that is strictly required; the others let Enum pick faster paths when they exist.

Here is Enumerable for a simple binary tree, traversed in-order.

defmodule Tree do
  defstruct [:value, :left, :right]

  def new(value, left \\ nil, right \\ nil) do
    %Tree{value: value, left: left, right: right}
  end
end

defimpl Enumerable, for: Tree do
  def count(_tree), do: {:error, __MODULE__}
  def member?(_tree, _value), do: {:error, __MODULE__}
  def slice(_tree), do: {:error, __MODULE__}

  def reduce(tree, acc, fun), do: do_reduce(flatten(tree), acc, fun)

  defp flatten(nil), do: []
  defp flatten(%Tree{value: v, left: l, right: r}) do
    flatten(l) ++ [v] ++ flatten(r)
  end

  defp do_reduce(_list, {:halt, acc}, _fun), do: {:halted, acc}
  defp do_reduce(list, {:suspend, acc}, fun) do
    {:suspended, acc, &do_reduce(list, &1, fun)}
  end
  defp do_reduce([], {:cont, acc}, _fun), do: {:done, acc}
  defp do_reduce([h | t], {:cont, acc}, fun) do
    do_reduce(t, fun.(h, acc), fun)
  end
end

Now the whole Enum API works on trees:

tree = Tree.new(2, Tree.new(1), Tree.new(3))

Enum.to_list(tree)       # [1, 2, 3]
Enum.map(tree, &(&1 * 10))  # [10, 20, 30]
Enum.member?(tree, 2)    # true
for v <- tree, do: v * 2 # [2, 4, 6]

The verbose do_reduce/3 logic with :cont, :halt, and :suspend exists because Enumerable.reduce/3 is a resumable fold — that is how Stream builds its lazy pipelines and how Enum.zip/2 walks two enumerables in lockstep. You do not have to understand every state to implement the protocol correctly; you copy the four-clause pattern above and replace the traversal in flatten/1.

Returning {:error, __MODULE__} from count/1, member?/2, and slice/1 tells Enum to fall back to walking the whole structure. If you have an O(1) count or O(1) membership check, return {:ok, value} and Enum.count/1 will use it directly. Maps, MapSets, and ranges all take advantage of this.

Implementing Collectable

Collectable is what makes Enum.into/2 and for ... into: %YourThing{} work. The protocol has one function — into/1 — and it returns a {initial, collector} tuple where the collector is a reducer that accumulates values and finalizes the structure.

For a custom container that should accept items via Enum.into:

defmodule Bag do
  defstruct counts: %{}

  def new, do: %Bag{}
  def to_list(%Bag{counts: counts}), do: counts
end

defimpl Collectable, for: Bag do
  def into(%Bag{counts: counts} = bag) do
    collector = fn
      acc, {:cont, item} ->
        Map.update(acc, item, 1, &(&1 + 1))

      acc, :done ->
        %{bag | counts: acc}

      _acc, :halt ->
        :ok
    end

    {counts, collector}
  end
end

Now you can do:

Enum.into(["apple", "apple", "banana"], Bag.new())
# %Bag{counts: %{"apple" => 2, "banana" => 1}}

for word <- ~w(red red blue green green green), into: Bag.new() do
  word
end
# %Bag{counts: %{"red" => 2, "blue" => 1, "green" => 3}}

This is the same protocol MapSet, Map, and File.Stream implement. It is also how Ecto's batch insert helpers and Phoenix's response writers work under the hood.

Fallback Implementations with @fallback_to_any

Sometimes you want a protocol that should "just work" for any type, with type-specific overrides where they matter. The @fallback_to_any flag plus an Any implementation gets you there.

defprotocol Loggable do
  @fallback_to_any true
  def to_log_entry(data)
end

defimpl Loggable, for: Any do
  def to_log_entry(data), do: %{value: inspect(data), type: :unknown}
end

defimpl Loggable, for: Map do
  def to_log_entry(map), do: %{value: map, type: :map, keys: Map.keys(map)}
end

Without @fallback_to_any, calling Loggable.to_log_entry/1 on a tuple would crash with Protocol.UndefinedError. With it, anything you have not explicitly implemented falls through to the Any clause.

The cost is that you lose the safety net of "if I forgot an implementation, I will find out at runtime." Use this when the fallback is genuinely sensible — logging, serialization with a default, audit trail formatting. Avoid it when the protocol is supposed to be type-specific and a missing implementation is a real bug.

For structs specifically, there is also @derive. If a struct module says @derive Loggable, the compiler generates defimpl Loggable, for: ThatStruct automatically using the Any implementation as a template. This is what @derive Jason.Encoder does in Phoenix codebases — opts a struct into JSON encoding without writing the implementation by hand.

defmodule Event do
  @derive Loggable
  defstruct [:id, :type, :payload]
end

Protocol Consolidation

By default, every protocol call does a runtime lookup. The protocol module checks the value's type, looks up the implementation module, and dispatches. This is fast — a single map access — but it is not free, and it precludes some compiler optimizations.

In production releases, you want the lookup to be a direct function call instead. That is what protocol consolidation does. When you build a release with mix release, the compiler walks every protocol in the project and generates a consolidated dispatcher with all implementations baked in. The runtime lookup becomes a hardcoded clause for each known type.

mix compile.protocols

You can also run this manually during development to test the consolidated path. Phoenix and most production Elixir apps have it enabled by default in mix.exs:

def project do
  [
    consolidate_protocols: Mix.env() != :test,
    # ...
  ]
end

Why disable it in :test? Because once a protocol is consolidated, you cannot add new implementations to it without recompiling. In tests, you sometimes want to define a quick struct and implement a protocol for it inside a single test file. Consolidation breaks that. In production, where the set of types is fixed, consolidation gives you measurable throughput improvements on protocol-heavy code paths — Discord and Pinterest have both written about the wins from making sure consolidation is on for hot paths.

You can check whether a protocol is consolidated in IEx:

Protocol.consolidated?(Enumerable)  # true in a release

The Gotcha of Changing Implementations at Runtime

You can technically define a new defimpl in a running system. The module gets loaded, the protocol dispatcher picks it up, and subsequent calls route to it. This works in development. It does not work the same way in a consolidated release.

A consolidated protocol has its dispatch table compiled in. Loading a new implementation module does not update the dispatcher. Until you rebuild and redeploy the release, calls to that protocol for that new type will still hit the old fallback or raise.

This shows up in two real situations: hot code reloading in a long-running release and dynamic struct generation. If you load a new Ecto schema at runtime and try to call Enumerable.reduce/3 on an instance, the consolidated Enumerable dispatcher does not know about it. Most apps never hit this — you ship code as a release and restart — but if you build a system that depends on plug-in style hot reloading, factor this in.

The honest answer: do not change protocol implementations at runtime in production. Build, ship, restart. If you genuinely need runtime extensibility, model it explicitly with a registry or a behaviour callback rather than reaching for protocols.

Implementing Multiple Protocols on One Struct

Most production structs implement several protocols. A typical Ecto schema has Inspect, Jason.Encoder, possibly Phoenix.Param, sometimes String.Chars. You list them next to the struct:

defmodule Article do
  @derive {Inspect, except: [:body]}
  @derive {Jason.Encoder, only: [:id, :title, :slug]}
  @derive {Phoenix.Param, key: :slug}
  defstruct [:id, :title, :slug, :body, :inserted_at]
end

defimpl String.Chars, for: Article do
  def to_string(%Article{title: title}), do: title
end

The @derive form covers protocols that have a sensible default for any struct. The hand-written defimpl covers the cases where you need real logic. Both compose freely on the same type.

Common Pitfalls

Implementing Enumerable.reduce/3 as a non-resumable fold. The four-state pattern with :cont, :halt, and :suspend is not optional. Skipping :suspend breaks any caller that uses your enumerable through Stream, and skipping :halt breaks Enum.take/2 and Enum.find/2. Copy the pattern from a working example.

Forgetting to enable protocol consolidation in releases. It is on by default for new Mix projects, but old codebases sometimes have it disabled and nobody remembers why. Check mix.exs. The performance difference on protocol-heavy code paths is real.

Using @fallback_to_any as a crutch for missing implementations. The fallback is a feature, not a way to silence errors. If the only reason you added it was to stop seeing Protocol.UndefinedError, you usually want an explicit implementation for that type instead.

Implementing Inspect and then breaking IEx debugging. A custom Inspect that omits fields is great for safety but a problem when those fields are exactly what you need to see in a debugging session. The except option on @derive {Inspect, except: [...]} is usually the right balance — fields are hidden by default but a developer can still pattern match on the underlying struct to inspect the real values.

Defining a protocol with one implementation. A protocol is overkill if only one type will ever implement it. Just write a function in a module. Protocols pay off when the contract is open-ended and other code is expected to plug in.

Mutating data inside a Collectable reducer. The collector function gets called for every {:cont, item} and must be pure with respect to its accumulator argument. Side effects in the reducer — writing to a file, sending a message — work but make the protocol harder to reason about. Save side effects for the :done terminal.

Key Takeaways

  • Implementing Enumerable for a custom structure makes the entire Enum and Stream API work on it. reduce/3 is mandatory; count/1, member?/2, and slice/1 are optional fast paths.
  • Collectable is the inverse — implement it when your structure should accept values via Enum.into/2 or for ... into:.
  • @fallback_to_any true plus defimpl Protocol, for: Any gives you a default for unknown types. @derive Protocol on a struct opts that struct into the Any implementation at compile time.
  • Protocol consolidation turns runtime dispatch into a direct call. It is on by default in releases, off in :test so that tests can register new implementations. Confirm it is on for hot paths.
  • Do not rely on changing protocol implementations at runtime in production. Once consolidated, the dispatch table is fixed. Build, ship, restart.
  • Most production structs derive several protocols at once (Inspect, Jason.Encoder, Phoenix.Param) and hand-roll any that need real logic.