Scrolls
8 min read
On this page

Tasks and Async

Task is the everyday concurrency primitive in Elixir. When you want to run something in parallel — a slow API call, a batch of database queries, a CPU-bound calculation — Task is what you reach for. It hides the bookkeeping of spawn, send, and receive behind a simple async/await interface, and adds the supervisor and parallel-iteration variants you need for real systems.

Task.async and Task.await

The simplest pattern: kick off a piece of work, do something else, then wait for the result.

task = Task.async(fn ->
  expensive_computation()
end)

other_work()

result = Task.await(task)

Task.async/1 spawns a process that runs the function, links and monitors it, and returns a Task struct. Task.await/2 blocks until the task finishes, then returns its return value. If the task crashes, await raises with the same reason — your code finds out about the failure cleanly.

The default await timeout is 5 seconds. For longer-running tasks, pass an explicit timeout:

Task.await(task, 30_000)  # 30 seconds
Task.await(task, :infinity)

A timeout means the calling process raises an :timeout exit. Because tasks are linked, this also brings down the task. That is usually what you want — if you stopped caring about the result, the task should stop running.

Running Many Tasks in Parallel

The standard pattern for parallel work is to spawn multiple tasks and await them all:

urls
|> Enum.map(fn url -> Task.async(fn -> HTTPoison.get(url) end) end)
|> Enum.map(&Task.await(&1, 10_000))

This fires all the requests concurrently, then collects results in the order they were started. Total time is roughly the slowest single request, not the sum.

A few hundred concurrent HTTP calls is fine on the BEAM. A few thousand might saturate your network or your downstream. Tens of thousands is where you need to think about backpressure and bounded concurrency, and that is what Task.async_stream/3 is for (covered below).

Task.start vs Task.async

Task.async/1 is for tasks where you care about the result and will wait for it. Task.start/1 and Task.start_link/1 are for fire-and-forget work where you do not.

# Fire and forget — log a metric without blocking
Task.start(fn -> Metrics.send("user_signed_up", %{id: id}) end)

If a Task.start_link/1 task crashes, the link makes the caller crash too. If you want the caller to survive the task's failure, use Task.start/1 (no link) or run the task under a Task.Supervisor.

A common antipattern: using Task.async/1 for fire-and-forget, never awaiting. The Task's result message piles up in the caller's mailbox, and the caller's death takes the task with it. Use the right primitive for the use case.

Task.Supervisor

For long-lived applications, you do not want orphan tasks running outside any supervision tree. Task.Supervisor solves this — it is a supervisor designed to start and watch tasks dynamically.

In your application's supervision tree:

defmodule MyApp.Application do
  use Application

  def start(_type, _args) do
    children = [
      {Task.Supervisor, name: MyApp.TaskSupervisor},
      # ... other children
    ]

    Supervisor.start_link(children, strategy: :one_for_one)
  end
end

Then start tasks under it:

Task.Supervisor.async(MyApp.TaskSupervisor, fn ->
  send_welcome_email(user)
end)
|> Task.await()

Or fire-and-forget:

Task.Supervisor.start_child(MyApp.TaskSupervisor, fn ->
  send_welcome_email(user)
end)

The supervised task survives the death of the caller, gets restarted according to the supervisor's strategy, and shows up in :observer so you can see what is running. Production systems should generally route Task work through a supervisor rather than calling Task.async/1 from a controller and hoping for the best.

Task.async_stream: Parallel Iteration

This is the most useful function in the entire Task module, and you will use it constantly once you know it exists. Task.async_stream/3 runs a function over a collection in parallel, with bounded concurrency, returning a stream of results.

urls
|> Task.async_stream(fn url -> HTTPoison.get(url) end, max_concurrency: 50, timeout: 30_000)
|> Enum.to_list()

The shape of each result is {:ok, value} for successes and {:exit, reason} for failures. By default, a single failure does not bring the others down — they all complete and you see the failures alongside the successes.

max_concurrency defaults to the number of schedulers (usually CPU count). For I/O-bound work like HTTP calls, you almost always want it higher — 50, 100, sometimes 500 — because each call spends most of its time waiting on the network, not using CPU.

A common pipeline: fetch a list of records from a database, process each one in parallel, collect the results.

{successes, failures} =
  records
  |> Task.async_stream(&process_record/1, max_concurrency: 20, timeout: 60_000)
  |> Enum.split_with(&match?({:ok, _}, &1))

successes = Enum.map(successes, fn {:ok, v} -> v end)

Pinterest's batch jobs, Discord's notification fan-outs, and Bleacher Report's content ingestion all use this pattern at scale — bounded concurrency, supervised tasks, results streamed back.

For tasks that should be supervised, use Task.Supervisor.async_stream_nolink/4 or async_stream/4:

Task.Supervisor.async_stream_nolink(MyApp.TaskSupervisor, records, &process/1, max_concurrency: 20)
|> Enum.to_list()

The _nolink variant means a crashing task does not crash the caller — it just shows up as {:exit, reason} in the stream.

Comparison With Goroutines and async/await

If you come from Go, Task.async/1 is roughly go func() { ... }() plus a channel to receive the result. The differences:

  • Tasks are real processes with their own heap and mailbox. Goroutines are stackful coroutines on shared memory.
  • A task that crashes does not corrupt anything else — its process dies in isolation. A panicking goroutine that escapes its recover takes the whole program down.
  • Tasks integrate with supervision. Restart strategy, telemetry, observation are all built in. Goroutines are bare.
  • Task.async_stream gives you bounded parallel iteration in one line. The Go equivalent (worker pool with a semaphore channel) is 30-50 lines of correctness-prone code.

If you come from Rust async, the comparison is closer in shape. Task.async/1 looks like tokio::spawn, Task.await/1 looks like .await. The differences are:

  • The BEAM has preemptive scheduling. A long-running computation does not block the executor. Rust async is cooperative — a task that does not .await blocks its thread until it does.
  • Task crashes are isolated. A Rust panic in a task is similarly isolated, but unwinding-vs-aborting and shared state makes the boundary fuzzier.
  • Task.async_stream covers what would be futures::stream::iter(...).buffer_unordered(n) plus a supervisor in Rust.

The shape of "spawn, do work, collect" is similar across all three. What is different in Elixir is that the boundary between concurrent units is enforced by the runtime, not by discipline.

When to Use Task vs Spawn vs GenServer

A rough guide:

Task when you want to run a function and either get its result back or fire and forget. One-shot work.

spawn directly rarely. Almost always you want either Task or GenServer instead. The exceptions are educational examples and very low-level code.

GenServer when you have a long-lived, stateful process that responds to many messages over time. A connection, a user session, a worker that processes a queue.

A useful test: if the process exists primarily to "do this one thing and return a result," it is a Task. If it exists to "manage some state and respond to requests over time," it is a GenServer.

Yield and Cleanup

Task.await/2 raises on timeout and brings down the task. Sometimes you want softer behavior — check whether the task finished, and if not, give up cleanly without crashing. That is what Task.yield/2 and Task.shutdown/2 are for.

task = Task.async(fn -> slow_work() end)

case Task.yield(task, 5_000) || Task.shutdown(task) do
  {:ok, result} -> result
  {:exit, reason} -> Logger.error("task failed: #{inspect(reason)}"); :error
  nil -> Logger.warning("task did not finish in time"); :timeout
end

Task.yield/2 returns {:ok, value} if the task finished, {:exit, reason} if it crashed, or nil if the timeout fired and it is still running. Task.shutdown/2 cleanly terminates a task that did not finish in time and reaps any reply that arrived in the meantime. This pair is the right approach when timeouts are an expected outcome rather than a failure to crash on.

Awaiting Many

For awaiting a list of tasks, Task.await_many/2 is more efficient than mapping Task.await/2:

tasks = Enum.map(urls, &Task.async(fn -> fetch(&1) end))
results = Task.await_many(tasks, 30_000)

await_many collects results in input order and uses a single timeout for the whole batch rather than per-task. For dozens of tasks this rarely matters, but for hundreds it does — the per-task version pays setup cost on each await.

Common Pitfalls

Calling Task.async from a process that might die. The task is linked to the caller. Caller dies, task dies. For background work that should outlive a request, use Task.Supervisor.async_nolink/2 or Task.Supervisor.start_child/2.

Using the default 5-second await timeout for slow work. A 5-second default catches many real timeouts, but if your task takes 10 seconds, you will see false failures. Set an explicit timeout that matches the work.

Spawning a million tasks at once. Tasks are cheap, but the work they do may not be. Task.async_stream/3 with a sensible max_concurrency is almost always what you want for bulk parallel work.

Forgetting to collect from Task.async_stream. Task.async_stream returns a stream. If you do not consume it (with Enum.to_list, Enum.reduce, etc.), no work happens. The stream is lazy.

Ignoring {:exit, reason} results. Task.async_stream returns {:ok, _} or {:exit, reason}. If you only pattern-match on {:ok, _}, you silently drop failures. Either handle both shapes or use on_timeout: :kill_task plus :zip_input_on_exit if you need to know which input failed.

Mixing Task.async with receive directly. Tasks send their reply as a message of a specific shape ({ref, value}). If you receive for that yourself instead of using Task.await, you can break monitoring and timeout handling. Use await unless you have a strong reason not to.

Key Takeaways

  • Task.async/1 runs a function in a linked, monitored process. Task.await/2 blocks for the result.
  • Default await timeout is 5 seconds. Pass an explicit timeout for slower work.
  • Task.start/1 and Task.start_link/1 are for fire-and-forget work where the result is not needed.
  • For production work, route tasks through a Task.Supervisor so they appear in your supervision tree.
  • Task.async_stream/3 is the workhorse for bounded parallel iteration. max_concurrency defaults to scheduler count; raise it for I/O-bound work.
  • Compared to goroutines and Rust async, tasks give you stronger isolation, supervision integration, and a one-liner for parallel iteration.
  • Use Task for one-shot work, GenServer for long-lived stateful processes, and spawn directly almost never.