LlmComposer.Agent: a tool-calling loop for your Elixir apps

Back in September we announced llm_composer, our Elixir library for talking to LLMs across OpenAI, OpenRouter, Ollama, Bedrock, and Google. Since then the library kept growing, and llm_composer 0.20.0 shipped a feature worth a post of its own: LlmComposer.Agent, an automatic tool-calling loop.

Instead of another feature list, this post is a showcase: we’ll build a small shopping assistant that searches a real product catalog (Doofinder’s own public demo store), fetches details, and does the price math — all through tool calls the model decides to make on its own.

The problem

llm_composer already supported function calling before Agent existed, through LlmComposer.FunctionExecutor and LlmComposer.FunctionCallHelpers. But it was manual: every app that wanted a tool-using assistant had to write its own loop.

{:ok, resp} = LlmComposer.simple_chat(settings, user_prompt)

case LlmResponse.function_calls(resp) do
  nil ->
    # model answered directly
    resp.main_response.content

  function_calls ->
    executed_calls =
      Enum.map(function_calls, fn call ->
        case FunctionExecutor.execute(call, functions) do
          {:ok, executed} -> executed
          {:error, _} -> call
        end
      end)

    tool_messages = FunctionCallHelpers.build_tool_result_messages(executed_calls)

    assistant_with_tools =
      FunctionCallHelpers.build_assistant_with_tools(
        LlmComposer.Providers.OpenAI,
        resp,
        user_message,
        provider_opts
      )

    messages = [user_message, assistant_with_tools] ++ tool_messages

    # ...and now call run_completion/2 again, and check function_calls again,
    # and again, until the model finally answers without requesting a tool.
    {:ok, final} = LlmComposer.run_completion(settings, messages)
    final.main_response.content
end

That’s fine for a single round of tool calls. But a genuinely useful assistant often needs several rounds — search, then fetch details, then compute a total — and that means every app ends up hand-rolling the same recursive loop, plus its own iteration limit, plus its own handling for a tool that raises. That’s exactly what LlmComposer.Agent replaces.

The solution

LlmComposer.Agent.run/3 automates the whole cycle:

ask → model requests tool calls → execute them → feed the results back → repeat
      → until the model returns a final, tool-free answer

To show it doing real work, we built a small shopping assistant with three tools:

• search_products — searches a live product catalog via HTTP.
• get_product_details — fetches the full record for one product.
• calculate_total — pure Elixir math, applying a discount.

The catalog is Doofinder’s own public demo search engine (a toy store), so the example calls a real search API and returns real products — no mocked data. And to keep it copy-paste-runnable without any paid API key, the model behind it is gemma4 running locally through Ollama, using llm_composer’s OpenAI-compatible provider support.

The tools

defmodule DemoSearch.Tools do
  @moduledoc """
  Tool implementations exposed to the agent: search the catalog, fetch one
  product's details, and do the final price math.
  """

  @hashid Application.compile_env!(:demo_search, :hashid)
  @api_token Application.compile_env!(:demo_search, :api_token)
  @product_fields ~w(id title description best_price availability brand link)

  @spec search_products(map()) :: String.t()
  def search_products(%{"query" => query} = args) do
    results = search(%{"query" => query})["results"]

    results
    |> Enum.map(&%{id: &1["id"], title: &1["title"], price: &1["best_price"]})
    |> filter_by_max_price(args["max_price"])
    |> Jason.encode!()
  end

  defp filter_by_max_price(products, nil), do: products
  defp filter_by_max_price(products, max_price), do: Enum.filter(products, &(&1.price <= max_price))

  @spec get_product_details(map()) :: String.t()
  def get_product_details(%{"id" => id}) do
    case search(%{"query" => "", "filter[id][]" => id})["results"] do
      [product | _] -> product |> Map.take(@product_fields) |> Jason.encode!()
      [] -> Jason.encode!(%{error: "product not found"})
    end
  end

  @spec calculate_total(map()) :: String.t()
  def calculate_total(%{"prices" => prices} = args) do
    discount_pct = args["discount_pct"] || 0
    subtotal = Enum.sum(prices)
    total = subtotal * (1 - discount_pct / 100)

    Jason.encode!(%{subtotal: subtotal * 1.0, discount_pct: discount_pct, total: total})
  end

  defp search(params) do
    url = "https://eu1-search.doofinder.com/6/#{@hashid}/_search"
    headers = [{"authorization", "Token #{@api_token}"}]

    Req.get!(url, params: params, headers: headers).body
  end
end

Each tool is a plain function taking the arguments the model supplied (already decoded from JSON into a map) and returning a string result — llm_composer feeds that string straight back to the model as the tool’s output.

Describing the tools

defmodule DemoSearch.Functions do
  alias LlmComposer.Function

  @spec all() :: [Function.t()]
  def all do
    [
      %Function{
        mf: {DemoSearch.Tools, :search_products},
        name: "search_products",
        description: "Search the product catalog by free-text query, optionally capped by max_price.",
        schema: %{
          "type" => "object",
          "properties" => %{
            "query" => %{"type" => "string"},
            "max_price" => %{"type" => "number"}
          },
          "required" => ["query"]
        }
      },
      %Function{
        mf: {DemoSearch.Tools, :get_product_details},
        name: "get_product_details",
        description: "Get full details for a single product by id.",
        schema: %{
          "type" => "object",
          "properties" => %{"id" => %{"type" => "string"}},
          "required" => ["id"]
        }
      },
      %Function{
        mf: {DemoSearch.Tools, :calculate_total},
        name: "calculate_total",
        description: "Calculate the total for a list of prices, with an optional discount percentage.",
        schema: %{
          "type" => "object",
          "properties" => %{
            "prices" => %{"type" => "array", "items" => %{"type" => "number"}},
            "discount_pct" => %{"type" => "number"}
          },
          "required" => ["prices"]
        }
      }
    ]
  end
end

Running the agent

Application.put_env(:llm_composer, :open_ai, url: "http://localhost:11434/v1", api_key: "ollama")

settings = %LlmComposer.Settings{
  providers: [
    {LlmComposer.Providers.OpenAI,
     [model: "gemma4:latest", functions: DemoSearch.Functions.all()]}
  ],
  system_prompt: """
  You are a shopping assistant for an online toy store. Use the search_products and
  get_product_details tools to find real products, and calculate_total to compute prices.
  Never invent products or prices that didn't come from a tool result.
  """
}

{:ok, result} =
  LlmComposer.Agent.run(settings, "Find me a mouse toy under $15 and tell me the total with a 10% discount applied.")

IO.puts(result.response.main_response.content)

That’s the entire application code — no loop, no manual re-prompting, no bookkeeping for how many turns have happened.

Seeing it run

Running the snippet above against the real Doofinder demo store produces:

=== Final answer ===
I found the **HEXBUG Mouse Robotic Cat Toy (GREY)** for $10.00.

With a 10% discount applied, the total price would be **$9.00**.

=== Iterations: 3 ===

=== Tool calls ===
search_products({"max_price":15,"query":"mouse toy"})
  => [{"id":"6544618127530","title":"HEXBUG Mouse Robotic Cat Toy (GREY)","price":10.0},
      {"id":"6544615473322","title":"Hot Wheels Star Wars Battle of Geonosis Play Set","price":9.99}]

calculate_total({"discount_pct":10,"prices":[10]})
  => {"total":9.0,"discount_pct":10,"subtotal":10.0}

Three model turns, two tool calls, zero lines of loop code: the model searched, picked the cheapest match, then reached for calculate_total on its own to apply the discount. Everything in result.function_calls is a LlmComposer.FunctionCall struct with its result attached — a ready-made audit trail of what the agent actually did.

What Agent buys you

• No hand-written loop. run/3 keeps calling the model and executing tools until it gets a final, tool-free answer, or hits :max_iterations (10 by default).
• Parallel tool execution. Pass tool_execution: :parallel and independent tool calls run concurrently via Task.async_stream/3, still returned in order.
• Tool errors don’t abort the run. An unknown tool, bad arguments, or a raised exception is turned into an "Error: ..." string and fed back to the model, which gets a chance to recover or explain — only model/network errors stop the loop.
• Streaming with progress built in. With stream_response: true, run/3 returns a stream of text_delta chunks for the final answer plus :tool_call chunks emitted right after each tool executes — enough to drive a live UI without wiring up telemetry.
• Telemetry per iteration. [:llm_composer, :agent, :iteration, :stop] carries that turn’s cost_info, so cost tracking (a recurring theme across recent llm_composer releases) can be recorded incrementally instead of only at the end of a run.

Try it yourself

The full guide covers streaming, telemetry events, and the options table in more depth:

• Guide: Agent Loop
• HexDocs: LlmComposer
• GitHub: doofinder/llm_composer
• Changelog: 0.20.0

As always, questions and feedback are welcome on the ElixirForum thread or on GitHub.