Agentic Operators

Overview

Agentic semantic operators run tool-using agents over a body of work (a The Corpus) in parallel. Unlike an LLM semantic operator — which applies one model call per record — an agentic operator gives each unit a full tool-calling agent that can take multiple steps, call tools (for example a sandboxed Python REPL), and reason before producing a result.

They are the right choice when a task is complex or ambiguous and benefits from more than a single LLM call — running code to compute an exact value, parsing a file, sweeping a codebase, or making a keep/drop decision that requires real investigation.

The single entry point is Corpus.agent:

result = corpus.agent(task="...", ops=["map", "reduce"], tools=[PythonREPLTool()])

You give it a corpus, a natural-language task, an ordered list of ops, and optional tools. LOTUS plans how to split the work, runs the ops over the corpus, and returns a Result.

The operators

An agentic pipeline is an ordered list of ops. Each op is either corpus → corpus (chainable) or corpus → answer (terminal):

Op

Shape

What each agent does

map

Corpus → Corpus

Transforms each unit in parallel; emits one output unit per input unit.

filter

Corpus → Corpus (subset)

map projected to a keep/drop verdict per unit; drops the ones that fail the criterion.

reduce

Corpus → single answer (terminal)

Aggregates all current units into one result. Has tools, so aggregation is computed, not estimated. Must be the last op.

How to think about them

  • Ops compose into a pipeline over the corpus. The current corpus is threaded through each op in order:

    corpus --filter--> corpus --map--> corpus --reduce--> answer

  • map and filter are chainable (Corpus → Corpus); reduce is terminal (it collapses the corpus to a single answer), so it must come last. LOTUS validates the ordering.

  • Use ops standalone or together. ops=["map"] maps and returns a corpus; ops=["filter"] returns the surviving subset; ops=["filter", "map", "reduce"] filters, then maps the survivors, then reduces to one answer. The default is ["map", "reduce"].

  • Tools are handled transparently. The tools you pass are available to every op; LOTUS describes them to the agents in a system-generated prompt, so your task never has to mention them.

  • You give one task. A planner derives the per-op instruction (the map, the filter criterion, the reduce) plus sharding and parallelism. You can override any op’s instruction (see Agentic Map-Reduce).

  • filter is an instantiation of map. Both run the same execution core over the corpus; map turns each unit’s agent result into a new unit, while filter reads that result as a keep/drop verdict and returns the surviving units. So anything true of map’s execution (parallelism, tools, strategies below) is true of filter.

Execution strategies

Each corpus op (map/filter) runs under a strategy that controls how much context each per-unit decision gets — and how many agents run. The planner chooses one per op from the task and a look at the corpus; you can override it per op via strategies= (and contexts= for the shared background):

Strategy

What it does / when to use

per_unit (default)

One unit per agent, decided independently. Best for self-contained per-item work, or when units are large.

batched

Several units per agent (they see each other as context); the agent still returns one result per unit. Best when the criterion is comparative/relative (“the strongest”, dedup), or when units are tiny and many (batching cuts cost). Uses shard_size units per agent.

shared_context

One unit per agent, plus a shared background injected into every agent (a reference definition, schema, or keep/drop exemplars). Best when every unit is judged against the same fixed background.

 
# Let the planner choose (default), or pin a strategy explicitly:
corpus.agent(task="Keep the strongest arguments.", ops=["filter"],
             strategies={"filter": "batched"})

corpus.agent(task="Keep files that use the deprecated API.", ops=["filter"],
             strategies={"filter": "shared_context"},
             contexts={"filter": "Deprecated: foo(x, y). A call counts if it invokes foo()."})

Choosing between agentic and LLM operators

Reach for agentic operators when the per-item work needs tool calls or multi-step reasoning, or when the task is open-ended. Reach for LLM semantic operators (sem_map, sem_filter, sem_agg, sem_join, …) when the task is well defined and one model call per record suffices — they invoke far fewer calls and are transparently optimized by the query engine.

In this section