A trace is the complete record of a single agent run - the task it was given, the full message trajectory (every user message, assistant response, and tool call), which memories were retrieved, and which model was used. Think of it as a structured log entry that Reflect can learn from.A review is a pass/fail judgment on a trace. When you review a trace, Reflect:
Reads the trajectory, the outcome, and your feedback
Generates a concise reflection (an LLM-produced summary of what worked or went wrong)
Embeds the reflection and stores it as a new memory with an initial utility of 0.5
Updates the utility scores of the memories that were retrieved during that run (up for pass, down for fail)
Without reviews, Reflect is just a trace logger. Reviews are what close the learning loop - they’re the training signal that makes memory retrieval improve over time.
Reflect stores the entire conversation, not just the final answer, because the reflection LLM needs context to generate useful advice. A reflection like “always verify the order exists before processing a return” can only be generated if the trajectory shows that the agent didn’t verify the order. The final answer alone wouldn’t reveal that.The trajectory also enables the dashboard to show step-by-step replays, which is useful for debugging and manual review.
The SDK provides three patterns for recording traces. They all produce the same result - a trace stored in Reflect - but differ in how much boilerplate they handle for you.
The context manager retrieves memories on entry and auto-submits the trace on exit. It tracks retrieved_memory_ids for you, so the utility learning loop works automatically.Best for: multi-step workflows, streaming, cases where you need to inspect output before deciding the review result.
with client.trace("Parse the CSV and return the top 5 rows") as ctx: # ctx.augmented_task - the task with relevant memories appended # ctx.memories - the retrieved Memory objects response = my_agent(ctx.augmented_task) ctx.set_output( trajectory=[ {"role": "user", "content": ctx.augmented_task}, {"role": "assistant", "content": response}, ], result="pass", model="gpt-5.4-mini", )# Trace auto-submitted on exit with correct retrieved_memory_ids# ctx.trace_id is now available for deferred review or logging
After the with block exits, ctx.trace_id contains the ID of the submitted trace. This is useful for deferred reviews — pass it to client.review_trace() later in your application. Inside the with block (before submission), trace_id is None.
Pass blocking=True to wait for the reflection and memory to be created before the with block exits. Useful in evaluation loops where the next task needs to retrieve the memory from the previous one.
with client.trace("...", blocking=True) as ctx: response = my_agent(ctx.augmented_task) ctx.set_output(trajectory=messages, result="pass")# Memory is guaranteed to exist here - the next task can retrieve it
If an unhandled exception occurs after set_output was called, the trace is auto-submitted with result="fail" and the exception message as feedback. This prevents losing trace data on crashes. Disable with auto_fail_on_exception=False.
The decorator wraps a function so that memory retrieval, trace submission, and retrieved_memory_ids tracking happen automatically. You just write the agent logic.Best for: single-function agents, clean integration with existing function signatures, when you want the least boilerplate.
from reflect_sdk import TraceContext, TraceResult, reflect_trace@reflect_trace(client, task=lambda question: question)def answer(ctx: TraceContext, question: str) -> TraceResult: messages = [{"role": "user", "content": ctx.augmented_task}] response = my_llm(messages) messages.append({"role": "assistant", "content": response}) return TraceResult( output=response, # returned to the caller trajectory=messages, result="pass", model="gpt-5.4-mini", )# Calling the function queries memories, runs your code, and submits the traceresult = answer("Parse the CSV and return the top 5 rows")# result == response (the output from TraceResult)
Return a TraceResult when you need to provide the full trajectory, review result, feedback, model, or metadata. Return a plain string for quick prototyping - the string is used as both the trajectory and the final response.
Call augment_with_memories and create_trace directly. This gives you full control over every step but requires you to pass retrieved_memory_ids manually.Best for: existing codebases where you can’t wrap the agent function, batch pipelines, cases where traces are created far from where memories are retrieved.
augmented = client.augment_with_memories( task="Parse the CSV and return the top 5 rows",)response = my_agent(augmented.augmented_task)submission = client.create_trace( task="Parse the CSV and return the top 5 rows", trajectory=[ {"role": "user", "content": augmented.augmented_task}, {"role": "assistant", "content": response}, ], retrieved_memory_ids=[m.id for m in augmented.memories], model="gpt-5.4-mini", review_result="pass",)
When using create_trace directly, you must pass retrieved_memory_ids manually. If you forget, the utility learning loop breaks silently - memories won’t be reinforced or penalized based on outcomes. The context manager and decorator handle this automatically.
Returns immediately with a TraceSubmission. Processing happens in the background.
Your app serves real-time traffic and you don’t want to block
create_trace_and_wait
Blocks until the trace is fully processed and the memory is created.
Evaluation loops, tests, scripts where the next step needs the memory to exist
# Non-blocking - returns immediatelysubmission = client.create_trace(task="...", trajectory=[...], review_result="pass")# submission.ingest_status == "queued"# Blocking - waits for the memory to be createdtrace = client.create_trace_and_wait( task="...", trajectory=[...], review_result="pass", poll_interval=0.25, # seconds between polls wait_timeout=60.0, # max seconds to wait)# trace.review_status == "reviewed"# trace.created_memory_id is set
Create the trace without a review, then submit one later. This is useful when:
A human needs to evaluate the answer
You’re running a batch and want to review traces in one go afterwards
The review depends on external feedback that isn’t available yet
# Step 1: Create trace without a reviewwith client.trace("Summarize the quarterly report") as ctx: response = my_agent(ctx.augmented_task) ctx.set_output(trajectory=messages) # no result - review deferred# ctx.trace_id is available after the with block exitsprint(f"Trace submitted: {ctx.trace_id}")# Step 2: Later, after human evaluationtrace = client.review_trace( trace_id=ctx.trace_id, result="fail", feedback_text="Summary missed the revenue decline in Q3",)# trace.review_status == "reviewed"# trace.created_memory_id is now set
Deferred reviews are processed synchronously - the returned Trace includes the review and the created memory ID. You can also review traces from the Reflect Console.
# List traces by review statuspending = client.list_traces(review_status="pending")reviewed = client.list_traces(review_status="reviewed")all_traces = client.list_traces()# Fetch a specific tracetrace = client.get_trace(trace_id="abc-123")
client.trace() — Auto-tracks retrieved_memory_ids. Flexible - inspect output before deciding the review. Supports blocking mode for eval loops. Best for multi-step agents, streaming, conditional review logic.
Decorator
@reflect_trace — Least boilerplate. Wraps a single function. Supports sync and async. Return TraceResult for full control or a string for quick prototyping. Best for single-function agents and clean codebases.
Explicit calls
create_trace / create_trace_and_wait — Full control over every step. Must pass retrieved_memory_ids manually. Best for existing codebases, batch pipelines, and cases where traces are created separately from memory retrieval.