Choose a layer
The guides cover the layers above the request:
policies and tools,
rollout outcomes and retries,
conversation records and checkpoints, and
Docent readings. Users simulated by a base model, and
steered by a pilot model, are in
Simulated users. behaviors supplies no
general judge: build one for your experiment from these interfaces, as the
judge step does.
Where things are imported from
behaviors.typesexports the data types: messages and content blocks, requests, responses, sampling and reasoning parameters, failures, retry settings, stream events, rollouts and their turns and statuses, and transcripts.behaviors.implementations.modelsexportschat_session_model_from_api,generate_with_retries,DEFAULT_FAILURE_CATEGORIES_TO_RETRY, andSamplingOverrides.behaviors.implementations.models.providersholdsSingleModelProviderandModelOverrides.behaviors.implementations.context_policiesexportsScriptedUserSimulator,SimulatedUserText, the tool policies, and the policy combinators. The model-driven user simulators are in modules of their own there:base_model_user_simulator, with the profile types inbase_model_prompt, andpilot_steered_user_simulator, withPilotSteering.behaviors.interfacesholds the protocols:model_apis(includingModelProvider),chat_session_models,context_policies, andprojections.- The provider adapters and their model providers are each in a module of their own (below),
imported explicitly, because they need the provider SDKs. So does
base_model_user_simulator.
Sessions and factories
ModelAPI keeps no state between calls: every request carries the whole context. A
ChatSessionModel is a factory whose sessions (ChatSession) keep a conversation’s history, and
a ContextPolicy is a factory whose sessions supply the model’s initialization and then each next
ContextTurn, or EndRollout to finish. Keep mutable state for one conversation in its sessions
(or a generator’s locals), never on a factory that every conversation shares.
run_chat_rollout starts a session of each factory and closes both when it returns, including on
cancellation. It does not own the provider client, which the caller closes with
await api.aclose(), or await provider.aclose() for a model provider. Use
a fresh session for each independent conversation. The sessions of
chat_session_model_from_api accept None or SetSystemMessage as their initialization; any
other kind needs a session model that understands it. Close sessions you start yourself with
aclose().
Provider adapters
Install the provider SDKs with themodels extra, behaviors[models]. The core types, context
policies, and rollouts with a fake model need no SDK.
Construct an adapter with the provider’s model ID, as in
OpenAIResponsesAPI("gpt-6-luna");
Choosing models shows how to find IDs. Adapters read
credentials from the environment (OPENAI_API_KEY, ANTHROPIC_API_KEY). Each also takes
api_key and base_url, for another account or a compatible service, or an SDK client of your
own with client=..., but not both. An adapter closes a client it created, and leaves an
injected client to its caller. Never store credentials in experiment inputs or entities.
The adapters turn off the SDKs’ own retries, so that behaviors decides when a call is tried again
(see Rollout outcomes and retries).
Adapter options
Each adapter also takes keyword options that apply to every request it sends.AnthropicMessagesAPI:
default_max_tokens(16,384) is themax_tokenssent when a request’s sampling leaves it unset, since Anthropic requires one.final_block_cache_control, aCacheControl, puts a prompt-cache breakpoint on the final content block of each request, when that block is non-empty text or a tool result the request didn’t already mark. With it, each turn of a conversation reads the cached prefix the previous turn wrote, without marking messages yourself. Adding it raisesModelErrorwhen the request would then have more than Anthropic’s four breakpoints, or a 1-hour breakpoint after a 5-minute one.thinking_display,"summarized"(the default) or"omitted", is sent with enabled thinking."summarized"streams summarized thinking text;"omitted"returns only signed, empty thinking blocks. The adapter always sends it, since newer models omit thinking text unless asked.fast_mode=Trueasks for fast-mode inference on every request (speed: "fast", with its beta header). A model that doesn’t offer fast mode rejects the request.
OpenAIResponsesAPI and OpenAICompatibleAPI take extra_body, a dictionary added to every
request’s body; a request’s SamplingParams.extra takes precedence over it. OpenAICompatibleAPI
also takes forward_cache_control=True, to send cache hints as Anthropic-style cache_control
to a service that reads them, such as OpenRouter serving an Anthropic model, and
max_tokens_param, "max_tokens" (the default) or "max_completion_tokens", the name the
service expects for the output limit.
Model providers
AModelProvider, from behaviors.interfaces.model_apis, looks up a ModelAPI by model ID:
provider.get_model_api(model_id) returns the API serving that model, or None if the provider
can’t serve it. Use one where code picks models by ID as it runs, such as a harness whose main
and helper calls name different models, or to send requests for many models through one client.
Each adapter module has a provider (the table above), which builds that module’s adapter for any
model ID. It takes the adapter’s credentials and options, less the model ID; it builds one
adapter per model ID, on the first lookup, and returns the same one after that; and every
adapter it builds sends its requests through the provider’s one client, its client property.
Close the provider, not the adapters it built: await provider.aclose() closes the client if the
provider created it, and leaves a client passed with client=... to its caller. Closing an
adapter that a provider built does nothing.
behaviors.implementations.models.providers work with any adapter, and compose
others:
SingleModelProvider(model_api)serves one API for every model ID, such as a fake model that stands in for every model in a test.ModelOverrides(overrides={model_id: api, ...}, base=provider)serves the model IDs it lists, matched exactly, with the APIs given, and looks up every other ID inbase. With nobase, it serves only the IDs it lists. Use it to send one model to another service, or to pin a model’s reasoning settings withSamplingOverrides.
aclose: close the APIs and providers you give them.
Requests and responses
AModelRequest has context (the messages), tools, tool_choice, parallel_tool_calls, and
sampling, all required: pass () or None where they don’t apply. generate returns a
ModelResponse with id, output (a tuple of messages), stop_reason, and usage, or a
ModelCallFailure; check which before reading the response’s fields. In usage, the counts of
cached and reasoning tokens are None when the provider doesn’t report them.
Sampling settings
SamplingParams has max_tokens, temperature, top_p, stop_sequences, reasoning, and a
provider-specific extra. A setting left None leaves the choice to the adapter or the
provider, and which settings take effect depends on the adapter and the model:
OpenAIResponsesAPI, for example, doesn’t apply stop_sequences. Leave a setting unset unless
your adapter and model support it, and don’t expect extra settings to carry over from one
adapter to another. Write numeric fields with their declared types, such as temperature=0.0
rather than 0, since stored records are checked strictly.
ReasoningParams has enabled, budget_tokens, and effort, one of "minimal", "low",
"medium", "high", "xhigh", and "max". Each adapter turns them into its provider’s
parameters:
- The OpenAI adapters send an
effortyou set as it is,"xhigh"and"max"included, asreasoning.effort(Responses) orreasoning_effort(Chat Completions). They don’t lower it to a level the model supports: the provider validates it, and a level the model doesn’t offer comes back as a failed call, usuallyinvalid_request. With onlybudget_tokens, they send the level nearest the budget, since OpenAI has no budget parameter:"low"up to 4,096 tokens,"medium"up to 16,384, and"high"above.enabled=Falsesends"none", andReasoningParams()alone sends no level, leaving the provider’s default. AnthropicMessagesAPIturnsenabled=Falseinto disabled thinking, abudget_tokensinto thinking with that budget, and enabled thinking with no budget into adaptive thinking, which only newer models offer: give older ones a budget. Aneffortis sent asoutput_config.effort, except"minimal", which Anthropic doesn’t offer, sent as"low".
SamplingOverrides(inner=api, reasoning=...) wraps an adapter and pins the reasoning settings of
every request; despite its name, it overrides reasoning only.
Provider prompt-cache hints (CacheableContentText, CacheableToolParam, CacheControl) ask the
provider to cache a prompt’s prefix. They are unrelated to fxtr’s
step cache, which keeps completed step results.