API reference¶
Generated from the docstrings. Everything listed here is importable from simsim
unless the heading names a module.
Models¶
simsim.llm ¶
Model backends.
Every backend takes OpenAI-style chat messages and returns an LLMResponse.
One interface covers API models (OpenAI, Anthropic, Together, ...), locally
served open models (vLLM, SGLang, Ollama all speak the OpenAI protocol), and
ScriptedLLM, a fake model for free, deterministic tests.
All backends retry transient failures (rate limits, timeouts, server errors)
with exponential backoff, and count tokens (and dollars, if you pass price).
LLM ¶
Base class. Subclasses implement _chat (and optionally _retryable).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
temperature
|
float
|
sampling temperature sent with every call. |
0.7
|
max_tokens
|
int
|
cap on generated tokens per call. |
1024
|
seed
|
int | None
|
makes runs reproducible; each request gets a seed derived from this one
and the prompt (see |
None
|
retries
|
int
|
attempts after the first on transient errors. |
6
|
price
|
tuple[float, float] | None
|
(dollars per 1M input tokens, dollars per 1M output tokens), to track cost. |
None
|
max_concurrency
|
int | None
|
at most this many calls in flight at once (across threads). |
None
|
rpm
|
float | None
|
at most this many calls per minute (for API rate limits). |
None
|
choice_probs ¶
choice_probs(messages: list[Message], options: list[str]) -> dict[str, float]
Probability of each option as the model's next token (renormalized over options).
Used for survey-style probes: label options "A", "B", ... or "1".."7". Needs a backend that returns token log-probs (OpenAI, vLLM); others raise NotImplementedError.
complete ¶
complete(prompt: str, system: str | None = None, **kwargs) -> str
Convenience for one-shot text calls (summaries, importance scores, judges).
request_seed ¶
request_seed(messages: list[Message]) -> int | None
A per-request seed derived from seed, the prompt, and how often this exact prompt
was sent before. Runs stay reproducible, but different calls get different random
draws (sending one fixed seed with every call makes samples correlated).
OpenAIChat ¶
Bases: LLM
Any OpenAI-compatible endpoint: OpenAI, vLLM, SGLang, Ollama, Together, Anthropic's compat API.
extra_body is sent with every request, e.g. {"chat_template_kwargs": {"enable_thinking": False}}
to turn off thinking on Qwen3-style models served by vLLM.
chat_raw ¶
chat_raw(params: dict[str, Any])
One raw API call with this backend's retry policy (for features beyond chat).
LiteLLMChat ¶
Bases: LLM
Routes through LiteLLM (pip install litellm) for providers without an OpenAI-compatible API.
ScriptedLLM ¶
Bases: LLM
A fake model for tests and dry runs. Costs nothing and never retries.
script is either a list of responses (str or LLMResponse) returned in order,
or a function (messages, tools) -> str | LLMResponse.
LLMResponse
dataclass
¶
assistant_message ¶
assistant_message() -> Message
The message to append to the history after this response.
ToolCall
dataclass
¶
make_llm ¶
make_llm(spec: str, **kwargs) -> LLM
Build a backend from a short spec string.
make_llm("openai:gpt-4o-mini") make_llm("anthropic:claude-haiku-4-5") make_llm("vllm:Qwen/Qwen2.5-7B-Instruct") # localhost:8000 make_llm("vllm:Qwen/Qwen2.5-7B-Instruct@http://gpu012:8000/v1") make_llm("litellm:gemini/gemini-2.5-flash")
simsim.cache.CachedLLM ¶
Bases: LLM
Wraps any backend with a persistent SQLite cache (safe across threads and processes).
Agents¶
simsim.agent ¶
Agents: a model + a persona + a memory + tools.
Agent ¶
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
unique name in the simulation. |
required |
llm
|
LLM
|
the model backend. |
required |
persona
|
Persona | str
|
a |
''
|
memory
|
Memory | None
|
any |
None
|
tools
|
Iterable[Tool]
|
the agent's own tools (environments can add more per turn). |
()
|
instructions
|
str
|
task or role instructions, placed after the persona. |
''
|
template
|
str
|
system prompt template with {name}, {persona}, {instructions}, {memory}. Override it to control exactly what the model sees. |
DEFAULT_TEMPLATE
|
action_format
|
str
|
"tools" uses the API's native function calling; "react" uses Thought/Action/Final Answer text, for models or servers without tool calling. |
'tools'
|
output_schema
|
Schema | None
|
if set, every reply must be JSON matching this schema (see
|
None
|
public_field
|
str
|
with |
'reply'
|
remember_private
|
bool
|
store private fields in the agent's own memory. |
True
|
remember_observations
|
bool | int
|
True stores the full observation in memory; an int keeps only its first N characters; False stores only what the agent did. |
True
|
memory_max_chars
|
int | None
|
cap on memory text in the prompt (keeps the most recent part). |
None
|
components
|
Iterable[Component]
|
plans, impressions, goals, inner states... (see |
()
|
max_tool_steps
|
int
|
cap on tool-call rounds per turn. |
8
|
act ¶
act(observation: str, t: int = 0, tools: Iterable[Tool] = (), recorder: Recorder | None = None) -> Turn
Respond to observation, calling tools as needed, then store the turn in memory.
ask ¶
ask(question: str, schema: Schema | None = None, remember: bool = False, t: int = -1, recorder: Recorder | None = None) -> str | dict[str, Any] | None
Interview the agent: it answers in character, from its memory, without tools.
Nothing is written to memory unless remember=True, so probing mid-run does
not change what happens next. With schema, returns a validated dict (or None).
save ¶
save(path: str | Path) -> None
Save memory and component state, e.g. to continue this agent in a later experiment.
state_dict ¶
state_dict() -> dict[str, Any]
Everything the agent has learned (memory and components), JSON-serializable.
survey ¶
survey(question: str, options: list[str] | dict[str, str], t: int = -1, recorder: Recorder | None = None) -> dict[str, float]
A closed-ended survey item, answered with the model's next-token probabilities.
options is a list of answer texts (labeled A, B, C...) or a dict {label: text},
e.g. {"1": "Strongly disagree", ..., "7": "Strongly agree"}. Returns {label: prob}.
Thinking models must have thinking turned off for this (see OpenAIChat extra_body).
Persona
dataclass
¶
Who the agent is. description is free text; attributes are structured fields
(age, party, occupation...) that also appear in the prompt and in the logs.
load_jsonl
classmethod
¶
load_jsonl(path: str | Path) -> list[Persona]
One JSON object per line: {"name": ..., "description": ..., any other keys -> attributes}.
Turn
dataclass
¶
Everything an agent did in one turn.
text is what the agent said publicly. With an output_schema, private
holds the other fields (e.g. reasoning the other agents never see).
reasoning holds a thinking model's reasoning and ReAct "Thought:" lines.
ScriptedAgent ¶
Bases: Agent
A rule-based agent in the same simulation as LLM agents (confederates, bots, baselines).
policy is a list of replies used in order (then repeated from the last), or a
function (observation, t) -> str.
HumanAgent ¶
simsim.tools ¶
Tools: plain Python functions the agent can call.
@tool
def get_order(order_id: str) -> str:
"Look up an order by its id."
return db[order_id]
The function's signature and docstring become the JSON schema the model sees.
Tool
dataclass
¶
tool ¶
tool(fn: Callable | None = None, *, name: str | None = None, description: str | None = None)
Decorator turning a function into a Tool. Works bare or with arguments.
emulated_tool ¶
emulated_tool(llm, name: str, description: str, parameters: dict[str, type], world: str = '') -> Tool
A tool whose output is generated by an LLM instead of real code (ToolEmu-style).
Lets you test agents on tools that would be dangerous, expensive, or not yet
built (bank transfers, email, file deletion). world describes the hidden
state the emulator should stay consistent with (e.g. "The user's balance is $40.").
Earlier calls are remembered so outputs stay consistent within a run.
simsim.structured ¶
Structured outputs: ask for JSON with a schema, validate it, and repair on failure.
A schema is a dict from field name to a spec:
str, int, float, bool a value of that type
["agree", "disagree"] one of these strings
(int, 1, 7) / (float, 0, 1) a number in a range
("str", "what this field is") any spec plus a description, as (spec, description)
schema = {
"analysis": (str, "your private reasoning; the other side never sees it"),
"reply": str,
"stance": ["support", "oppose", "neutral"],
"confidence": (int, 1, 7),
}
If the model's answer does not parse or validate, the error goes back to the
model and it tries again (max_repairs times), the same way Sotopia, AgentDojo
and tau2-bench repair bad outputs.
StructuredOutputError ¶
Bases: ValueError
structured_chat ¶
structured_chat(llm: LLM, messages: list[Message], schema: Schema, max_repairs: int = 2, add_instructions: bool = True) -> tuple[dict[str, Any] | None, list[Message], list[str]]
Call llm until its answer matches schema.
Returns (parsed or None, the full message list including repair turns, error messages).
extract_json ¶
extract_json(text: str) -> dict[str, Any]
The first JSON object in text, tolerating code fences, thinking tags and chatter around it.
Memory and components¶
simsim.memory ¶
Memory modules.
All memories share one small interface, so swapping memory is a one-line change and can be an experimental condition:
write(entry) -> store what happened (called after every agent turn)
read(query) -> text placed in the agent's prompt for its next turn
tools() -> optional tools that let the agent manage memory itself
dump() / load() -> save and restore the contents (agent state, checkpoints)
The environment decides what is new for an agent; memory decides what is remembered. Keep those separate and you can vary one without touching the other.
Memory ¶
MemoryEntry
dataclass
¶
FullHistory ¶
WindowMemory ¶
MarkdownMemory ¶
Bases: Memory
A notes file the agent edits itself through tools (like Claude Code's memory.md).
Nothing is written automatically; the agent decides what to keep. Pass path
to persist the file across runs, or initial to seed it (e.g. a planted memory).
ScoredMemory ¶
Bases: Memory
Generative Agents-style retrieval (Park et al., 2023).
read(query) returns the top-k entries by a weighted sum of
recency decay ** (steps since the entry was last retrieved)
importance importance(text) in [0, 1]; with llm, the LLM rates 1-10
relevance cosine similarity of embeddings (with embedder), else keyword overlap
Each component is min-max normalized over the candidates (as in the original),
so the weights are comparable. Retrieved entries count as accessed.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
k
|
int
|
entries returned. |
8
|
embedder
|
Embedder | None
|
an |
None
|
llm
|
LLM | None
|
rates importance if |
None
|
importance
|
Callable[[str], float] | None
|
function |
None
|
decay
|
float
|
recency decay per step. |
0.99
|
weights
|
tuple[float, float, float]
|
(recency, importance, relevance); Park et al. used (0.5, 3, 2). |
(1.0, 1.0, 1.0)
|
ReflectiveMemory ¶
Bases: ScoredMemory
ScoredMemory plus Generative Agents-style reflection.
When the summed importance of new entries passes threshold, the LLM asks
itself n_questions high-level questions about recent memories, retrieves
evidence for each, and writes n_insights insights back as memories
(kind="reflection"), which can then be retrieved like any other memory.
llm_importance ¶
llm_importance(llm: LLM) -> Callable[[str], float]
Generative Agents' importance rating: the LLM scores each memory 1-10 (returned as 0.1-1).
simsim.embed ¶
Text embeddings for semantic memory retrieval.
emb = OpenAIEmbedder("text-embedding-3-small") # OpenAI emb = OpenAIEmbedder("BAAI/bge-m3", base_url="http://gpu012:8002/v1") # vLLM --task embed emb = HashEmbedder() # no model; bag of words
memory = ScoredMemory(embedder=emb)
simsim.components ¶
Agent components: pluggable pieces of an agent's mind.
A component adds a section to the agent's system prompt each turn (context) and
updates itself after each turn (update). Plans, impressions of others, goals,
and inner states such as emotions or beliefs are all components, so you can
add, remove, or swap them as experimental conditions:
agent = Agent("Ana", llm, persona=..., components=[
Goal("Sell the bike for at least $150."),
Plan(every=4),
Impressions(),
InnerState({"mood": ["calm", "annoyed", "angry"], "trust": (int, 1, 7)}),
])
Every component's state after each turn is logged (turn.components), so you can
analyze trajectories such as trust over time. Components use the agent's own
model unless you pass llm=.
Component ¶
Base class. heading titles the prompt section; empty context adds nothing.
context may update the component (e.g. write a new plan); view must not,
and is used for probes so interviewing an agent never changes it.
Goal ¶
Impressions ¶
Bases: Component
What the agent thinks of each other agent, updated after turns where they appear (Generative Agents summarizes the relationship before each conversation line).
InnerState ¶
Bases: Component
Structured inner state (emotions, beliefs, trust, stance...) updated after every turn.
schema uses the structured-output format (see simsim.structured), e.g.
{"mood": ["calm", "anxious", "angry"], "trust_in_seller": (int, 1, 7)}.
The current values are shown to the agent unless visible=False (then the
state is only measured, not fed back).
Environments¶
simsim.envs.base.Environment ¶
The world the agents live in.
The simulation loop calls, for each scheduled agent at step t: observe(agent, t) -> what the agent sees now (None to skip its turn) tools_for(agent, t) -> extra tools the agent may use this turn apply(turn, t) -> update the world with what the agent did and stops when done(t) is True. metrics() is the result saved per run.
To end the run from inside the environment, call self.stop("reason");
the reason is saved as the run's termination reason.
add_agent ¶
add_agent(agent: Agent) -> None
An agent joins mid-run. Subclasses with per-agent state should extend this.
simsim.envs.conversation ¶
Conversation ¶
Bases: Environment
Agents talk in a shared room: negotiation, debate, persuasion, interviews, co-writing, or a user simulator talking to an assistant.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
scenario
|
str
|
the setup every agent sees on its first turn. |
required |
private
|
dict[str, str] | None
|
per-agent private info, e.g. {"buyer": "Your budget is $500."}. |
None
|
show
|
str
|
"new" shows each agent only messages since its last turn (memory keeps the rest); "all" shows the full transcript every turn. |
'new'
|
end_token
|
str | None
|
an agent ends the conversation by putting this on the last line of its message (mentions elsewhere, e.g. in quotes, do not count). |
'[END]'
|
enders
|
list[str] | None
|
agents allowed to end it (default: anyone). For a user simulator, only the user should end. |
None
|
min_turns
|
int
|
end tokens before this many messages are removed and ignored (models otherwise tend to "end" in their first message). |
2
|
tools
|
dict[str, list[Tool] | Callable[[dict], list[Tool]]] | None
|
per-agent tools, e.g. {"assistant": [lookup_order, refund]}, or per-agent
factories |
None
|
max_turns
|
int | None
|
stop after this many messages. |
None
|
judge
|
Callable[[str], bool] | None
|
optional |
None
|
score
|
Callable[[list[dict], dict], dict[str, Any]] | None
|
optional |
None
|
world
|
dict | None
|
shared mutable state for tools and |
None
|
user_simulator ¶
user_simulator(name: str, llm, goal: str, persona: str = '', end_token: str = '###STOP###', **kwargs) -> Agent
An LLM that plays a user with a hidden goal, as in tau-bench and SimulatorArena.
Pair it with an assistant agent in a Conversation(..., end_token="###STOP###",
enders=[name], tools={assistant: [...]}).
simsim.envs.tasks ¶
Task
dataclass
¶
One episode.
score(turn, world) returns a dict of metrics (e.g. {"success": 1.0}); world
is the agent's mutable state that tools read and write.
TaskSequence ¶
Bases: Environment
Each agent works through the tasks in order; its memory carries across tasks.
This is the usual setup for memory experiments: earlier tasks plant information, later tasks test whether (and how) the agent uses it. With several agents, each gets its own copy of the world and its own progress.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
tasks
|
list[Task]
|
the episodes, in order. |
required |
tools
|
Callable[[dict], list[Tool]] | list[Tool]
|
a function |
()
|
world
|
dict | None
|
initial state (a dict), deep-copied per agent. |
None
|
simsim.envs.feed ¶
Feed ¶
Bases: Environment
A minimal social-media platform: post, reply, repost, like, follow.
Each turn an agent sees a timeline (recent posts from accounts it follows, or from everyone if it follows nobody) plus an optional news headline, and acts through tools.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
follows
|
dict[str, list[str]] | None
|
initial follow graph {agent: [agents it follows]}. |
None
|
news
|
list[str] | None
|
headlines; one is shown per turn (cycled) if given. |
None
|
timeline_size
|
int
|
posts shown per turn. |
10
|
ranking
|
str
|
"recent" or "popular" (likes, then recency). Swap in your own
by overriding |
'recent'
|
inject_post ¶
inject_post(author: str, text: str, t: int = -1) -> Post
Put a post on the platform from outside the agents (news, misinformation, a confederate).
Post
dataclass
¶
Running¶
simsim.sim ¶
The simulation loop, plus interventions (things that happen to the simulation at chosen steps).
Simulation ¶
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
env
|
Environment
|
the environment. |
required |
agents
|
list[Agent]
|
the agents (names must be unique). |
required |
scheduler
|
Scheduler
|
who acts each step; see |
round_robin
|
max_steps
|
int
|
hard cap on steps. |
20
|
seed
|
int
|
seeds the scheduler/environment RNG. |
0
|
simultaneous
|
bool
|
if True, all agents scheduled in a step observe the same world state and act in parallel; their actions are applied afterwards in schedule order. If False, each sees the previous agent's action. |
False
|
max_concurrency
|
int
|
with |
16
|
clock
|
Clock | None
|
a |
None
|
interventions
|
list[Intervention]
|
things that happen at chosen steps ( |
()
|
evaluators
|
dict[str, Callable[[Result], dict[str, Any]]] | None
|
{name: fn(result) -> dict}, run at the end and added to
metrics under |
None
|
checkpoint
|
str | Path | None
|
path to save the whole simulation every |
None
|
Result
dataclass
¶
Intervention ¶
Runs fn(sim, t) at the start of every step where when(t) is true (or at the end).
at ¶
at(steps: int | Iterable[int], fn: Callable[[Simulation, int], None]) -> Intervention
Run fn at the given step(s). Use END for after the final step.
every ¶
every(k: int, fn: Callable[[Simulation, int], None], start: int = 0, at_end: bool = False) -> Intervention
Run fn every k steps from start (and after the final step if at_end).
whisper ¶
whisper(agent: str, text: str) -> Callable[[Simulation, int], None]
Plant a memory in an agent (Generative Agents' 'whisper'): it believes this happened.
add_agent ¶
add_agent(agent: Agent) -> Callable[[Simulation, int], None]
A new agent joins the simulation.
probe ¶
probe(question: str, schema: Schema | None = None, agents: list[str] | None = None, name: str = 'probe') -> Callable[[Simulation, int], None]
Interview agents (without changing their memory); answers go to Result.probes.
survey ¶
survey(question: str, options: list[str] | dict[str, str], agents: list[str] | None = None, name: str = 'survey') -> Callable[[Simulation, int], None]
A closed-ended survey item answered with token probabilities; goes to Result.probes.
simsim.schedule ¶
Schedulers decide who acts at each step: (agents, t, rng) -> list of agents.
A scheduler may also accept a sim keyword argument to see the environment
(e.g. the conversation so far) or the clock; see NextSpeaker and by_hour.
NextSpeaker ¶
An LLM moderator picks who speaks next from the conversation so far
(Concordia's NextActing, MARBLE's planner). Works with Conversation.
Falls back to round-robin order if the answer names no agent.
round_robin ¶
round_robin(agents: list[Agent], t: int, rng: Random) -> list[Agent]
One agent per step, in a fixed order. The usual choice for conversations.
everyone ¶
everyone(agents: list[Agent], t: int, rng: Random) -> list[Agent]
All agents act every step, in a fixed order.
everyone_shuffled ¶
everyone_shuffled(agents: list[Agent], t: int, rng: Random) -> list[Agent]
All agents act every step, in a fresh random order (removes first-mover effects).
random_one ¶
random_one(agents: list[Agent], t: int, rng: Random) -> list[Agent]
One uniformly random agent per step.
bernoulli ¶
bernoulli(p: float | Callable[[Agent], float]) -> Scheduler
Each agent independently acts with probability p (or p(agent)) each step.
zipf ¶
zipf(s: float = 1.0, k: int = 1) -> Scheduler
k agents per step, drawn with Zipfian activity: a few 'power users' act most.
Ranks are assigned by agent order, so shuffle the agent list (with your seed) if rank should not follow construction order.
by_hour ¶
by_hour(activity: dict[int, float] | list[float], base: Scheduler | None = None) -> Scheduler
Each agent acts with a probability that depends on the simulated hour (needs a Clock).
activity maps hour (0-23) to probability, e.g. low at night, high in the evening,
as in OASIS's hourly activity profiles.
simsim.clock.Clock ¶
simsim.experiment ¶
Run a simulation across conditions x seeds, with logs and a results table.
def build(cond, seed):
llm = make_llm(cond["model"], seed=seed)
agent = Agent("assistant", llm, memory=cond["memory"]())
return Simulation(TaskSequence(tasks), [agent], seed=seed)
exp = Experiment(build, grid(model=["openai:gpt-4o-mini", "vllm:Qwen/Qwen2.5-7B-Instruct"],
memory={"none": NoMemory, "notes": MarkdownMemory}),
seeds=range(5), out="runs/memory_v1")
rows = exp.run(workers=8)
Each run writes out/<condition>/seed<k>.jsonl (every LLM call and tool call)
and one line in out/results.jsonl. Re-running skips runs already in
results.jsonl, so an interrupted experiment resumes where it stopped.
Experiment ¶
results ¶
results() -> list[dict[str, Any]]
All finished rows, including runs from earlier invocations.
grid ¶
grid(**factors: list | dict) -> dict[str, dict[str, Any]]
Cross factors into named conditions.
A list's values name themselves; a dict maps short names to values: grid(model=["a", "b"], memory={"none": NoMemory, "md": MarkdownMemory}) -> {"model=a,memory=none": {"model": "a", "memory": NoMemory}, ...}
simsim.judge ¶
LLM-as-judge: score transcripts against a rubric, decide when to stop, re-score saved runs.
judge = Judge(make_llm("openai:gpt-4o"), rubric="How cooperative was {agents}? ...",
schema={"reasoning": str, "cooperation": (int, 1, 10)})
sim = Simulation(env, agents, evaluators={"judge": judge.evaluator()})
env = Conversation(..., judge=judge.stopper("Have they agreed on a final price?"))
Use a different (ideally stronger) model than the agents, and report it.
Judge ¶
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
llm
|
LLM
|
the judge model. |
required |
rubric
|
str
|
instructions; may use {text} (the transcript) and any keyword passed to |
required |
schema
|
Schema | None
|
fields to return (see |
None
|
evaluator ¶
evaluator(render: Callable[[Any], str] | None = None, **context: Any) -> Callable[[Any], dict]
For Simulation(evaluators={...}): scores render(result) (default: the transcript).
stopper ¶
stopper(question: str) -> Callable[[str], bool]
For Conversation(judge=...): True when the judge answers yes to question.
rescore ¶
rescore(results: str | Path, judge: Judge, render: Callable[[dict], str], name: str = 'judge', out: str | Path | None = None) -> list[dict[str, Any]]
Score finished runs again with a (new) judge, without re-running the simulation.
results is an experiment's results.jsonl; render(row) turns a row (its
metrics, e.g. the transcript) into the text to score. Writes rows with
row[name] set to out if given and returns them.
simsim.personas ¶
Persona generation: sample attributes from distributions or real survey rows, then (optionally) have an LLM write a short biography from them.
personas = sample_personas(
100,
{"age": range(18, 80), "party": {"Democrat": 0.33, "Republican": 0.29, "Independent": 0.38},
"region": ["Northeast", "South", "Midwest", "West"]},
seed=0, describe=make_llm("openai:gpt-4o-mini"))
personas = personas_from_rows(csv.DictReader(open("anes_2020.csv")), fields=["age", "party", ...])
sample_personas ¶
sample_personas(n: int, attributes: dict[str, Distribution], seed: int = 0, describe: LLM | None = None, name: Callable[[int, dict], str] = lambda i, attrs: f'person_{i}', words: int = 60) -> list[Persona]
Draw n personas, each attribute independently from its distribution.
A distribution is a list/range (uniform), a dict {value: weight}, or a
function rng -> value (use this for dependent attributes).
personas_from_rows ¶
personas_from_rows(rows: Iterable[dict[str, Any]], fields: list[str] | None = None, name_key: str | None = None, describe: LLM | None = None, sample: int | None = None, seed: int = 0, words: int = 60) -> list[Persona]
Personas from real respondents (e.g. ANES, GSS, Prolific rows), one per row.
fields picks the columns used as attributes (default: all); sample draws
that many rows at random.
describe_persona ¶
describe_persona(p: Persona, llm: LLM, words: int = 60, prompt: str = BIO_PROMPT) -> Persona
Fill p.description with an LLM-written biography of its attributes.
Analysis¶
simsim.analysis ¶
Load experiment outputs into pandas tables and summarize them (pip install pandas).
from simsim.analysis import load_runs, load_turns, load_probes, summarize
runs = load_runs("runs/memory_v1") # one row per run: factors, metrics, tokens, cost
summarize(runs, by=["memory"], metrics=["mean_success"])
turns = load_turns("runs/memory_v1") # one row per agent turn: text, private fields, components
probes = load_probes("runs/memory_v1") # one row per probe/survey answer
Condition names made by grid ("model=a,memory=none") are split into one
column per factor, so you can group by any factor directly.
load_runs ¶
load_runs(source, include_errors: bool = False)
One row per run from results.jsonl files (an experiment dir, a file, or several).
load_turns ¶
load_turns(source)
One row per agent turn from the per-run logs (seed*.jsonl) of an experiment dir.
load_probes ¶
load_probes(source)
One row per probe or survey answer. Survey rows get top (most likely label),
expected (probability-weighted mean of numeric labels) and p.<label> columns;
structured probe answers are expanded to answer.<field> columns.
summarize ¶
summarize(df, by: str | list[str], metrics: str | list[str] | None = None, ci: float = 0.95)
Mean, standard error, n, and a normal-approximation confidence interval per group.
pass_hat_k ¶
pass_hat_k(df, success: str, k: int, task: str = 'condition', threshold: float = 1.0) -> float
tau-bench's pass^k: the chance that k random trials (seeds) of a task all succeed,
averaged over tasks. Each row is one trial; task groups trials of the same task.
cost_report ¶
cost_report(df, prices: dict[str, tuple[float, float]] | None = None, by: str | list[str] = 'condition')
Tokens (and dollars) per group. With prices {model substring: ($/1M in, $/1M out)}
applied to the model factor column; otherwise uses usage.cost_usd if the LLMs had price=.
transcript ¶
transcript(path: str | Path, private: bool = False) -> str
A readable transcript of one run log; private=True adds private fields and reasoning.
simsim.viewer ¶
A self-contained HTML viewer for runs: open it in a browser, no server needed.
from simsim.viewer import write_viewer
write_viewer("runs/negotiation", "runs/negotiation/view.html")
or from the shell: simsim view runs/negotiation.
write_viewer ¶
write_viewer(source: str | Path, out: str | Path | None = None, include_prompts: bool = False, title: str | None = None) -> Path
Write an HTML viewer for an experiment directory (or one run log) and return its path.