Memory for multi-turn conversations
Updated
Control how much conversation history the agent's LLM receives, seed conversations, and carry context from one session into the next.
Preview documentation
Short-term memory lets your agent respond coherently across a conversation. During a session, the agent keeps the history of what the user and the agent said, and passes recent messages to the LLM with each request. This page shows how to control that history, seed a conversation with messages, and carry context from one session into the next. The SDK examples start from the agent builder and saved agent in Create an agent and start a session.
Understand the tech
With each LLM request, the agent sends the following, in order:
- The agent's
instructions. - The agent's
messages, if any. They seed the conversation history. - The most recent messages of the conversation, up to the LLM's maximum history.
Short-term memory lasts for one session. When the session ends, the agent doesn't keep the conversation. To use it later, save it and pass it to a new session, as described in Carry context into later sessions.
Prerequisites
Before you begin, make sure that you have created an agent and started a session as described in Create an agent and start a session.
Control how much history the LLM receives
Set the LLM's maximum history to the number of recent messages the LLM receives with each request. It counts messages, not pairs of user and agent messages, and the agent's instructions don't count toward it. It accepts 1 to 1024, and the default is 32.
Set max_history on the LLM that you pass to with_llm():
llm = OpenAI(
model="gpt-4.1-mini",
max_history=12,
)Set maxHistory on the LLM that you pass to withLlm():
const llm = new OpenAI({
model: 'gpt-4.1-mini',
maxHistory: 12,
});Set MaxHistory on the LLM that you pass to WithLlm():
maxHistory := 12
llm := vendors.NewOpenAI(vendors.OpenAIOptions{
Model: "gpt-4.1-mini",
MaxHistory: &maxHistory,
})In the agent's llm block, set max_history:
"pipeline": {
"llm": {
"max_history": 12
}
}A lower value keeps each request smaller and faster, but the agent may forget earlier parts of a long conversation. A higher value preserves more context.
Seed the conversation with messages
To start each conversation with context, such as an earlier exchange or a worked example, add messages to the agent. Each message has a role (system, user, or assistant) and content. The messages follow the agent's instructions.
Add the messages to the LLM's system messages, after the system message. The SDK sends system messages as the agent's instructions, and the other messages as its messages.
llm = OpenAI(
model="gpt-4.1-mini",
system_messages=[
{"role": "system", "content": "You are a friendly support agent for Acme. Keep your answers short."},
{"role": "user", "content": "What are your opening hours?"},
{"role": "assistant", "content": "We're open from 9 AM to 6 PM, Monday to Friday."},
],
)Add the messages to the LLM's system messages, after the system message. The SDK sends system messages as the agent's instructions, and the other messages as its messages.
const llm = new OpenAI({
model: 'gpt-4.1-mini',
systemMessages: [
{ role: 'system', content: 'You are a friendly support agent for Acme. Keep your answers short.' },
{ role: 'user', content: 'What are your opening hours?' },
{ role: 'assistant', content: "We're open from 9 AM to 6 PM, Monday to Friday." },
],
});Add the messages to the LLM's system messages, after the system message. The SDK sends system messages as the agent's instructions, and the other messages as its messages.
llm := vendors.NewOpenAI(vendors.OpenAIOptions{
Model: "gpt-4.1-mini",
SystemMessages: []map[string]interface{}{
{"role": "system", "content": "You are a friendly support agent for Acme. Keep your answers short."},
{"role": "user", "content": "What are your opening hours?"},
{"role": "assistant", "content": "We're open from 9 AM to 6 PM, Monday to Friday."},
},
})Add messages to the agent configuration, at the same level as instructions:
"instructions": "You are a friendly support agent for Acme. Keep your answers short.",
"messages": [
{ "role": "user", "content": "What are your opening hours?" },
{ "role": "assistant", "content": "We're open from 9 AM to 6 PM, Monday to Friday." }
]You can use the agent's variables in messages. For details, see Personalize conversations.
Get the conversation after a session
After a session ends, Agora can send its transcript to your server in the 1002 Session transcripts notification event. Each transcript item contains the speaker's role, what was said, the turn ID, and the source of the message, such as asr or llm. If the agent was interrupted, the item contains only the part that was played.
Carry context into later sessions
The agent doesn't keep memory between sessions. To give a returning user continuity, save what matters from each conversation, and pass it to the next session as a variable:
- When a session ends, save its transcript, or a summary of it, from the
1002Session transcripts event. - In the agent's instructions, refer to a variable for the earlier context, and declare the variable.
- When you start the user's next session, pass the saved summary as the variable's value.
For example:
Declare the variable when you create the agent:
context_agent = (
Agent(client=client)
.with_stt(DeepgramSTT(model="nova-3", language="en-US"))
.with_llm(
OpenAI(
model="gpt-4.1-mini",
system_messages=[
{
"role": "system",
"content": "You are a friendly support agent for Acme. Here is what the user discussed in earlier conversations: {{previous_context}}",
}
],
)
)
.with_tts(OpenAITTS(model="tts-1", voice="alloy"))
.with_variable("previous_context", type="str")
)
saved_agent = context_agent.save(name="Support agent")Pass the value when you start a session with that agent:
session = saved_agent.create_session(
channel="<your_channel_name>",
token="<your_rtc_token>",
agent_uid="1002",
remote_uids=["1001"],
values={
"previous_context": "The user asked about upgrading to the pro plan and wanted a callback about pricing.",
},
)Declare the variable when you create the agent:
const contextAgent = new Agent({ client })
.withStt(new DeepgramSTT({ model: 'nova-3', language: 'en-US' }))
.withLlm(
new OpenAI({
model: 'gpt-4.1-mini',
systemMessages: [
{
role: 'system',
content: "You are a friendly support agent for Acme. Here is what the user discussed in earlier conversations: {{previous_context}}",
},
],
}),
)
.withTts(new OpenAITTS({ model: 'tts-1', voice: 'alloy' }))
.withVariable('previous_context', { type: 'str' });
const savedAgent = await contextAgent.save({ name: 'Support agent' });Pass the value when you start a session with that agent:
const session = savedAgent.createSession({
channel: '<your_channel_name>',
token: '<your_rtc_token>',
agentUid: '1002',
remoteUids: ['1001'],
values: {
previous_context: "The user asked about upgrading to the pro plan and wanted a callback about pricing.",
},
});Declare the variable when you create the agent:
contextAgent := agentkit.NewAgent(client).
WithStt(vendors.NewDeepgramSTT(vendors.DeepgramSTTOptions{Model: "nova-3", Language: "en-US"})).
WithLlm(vendors.NewOpenAI(vendors.OpenAIOptions{
Model: "gpt-4.1-mini",
SystemMessages: []map[string]interface{}{
{
"role": "system",
"content": "You are a friendly support agent for Acme. Here is what the user discussed in earlier conversations: {{previous_context}}",
},
},
})).
WithTts(vendors.NewOpenAITTS(vendors.OpenAITTSOptions{Model: "tts-1", Voice: "alloy"})).
WithVariable("previous_context", agentkit.VariableDefinition{Type: "str"})
savedAgent, err := contextAgent.Save(ctx, agentkit.SaveOptions{Name: "Support agent"})
if err != nil {
log.Fatal(err)
}Pass the value when you start a session with that agent:
session := savedAgent.CreateSession(agentkit.CreateSessionOptions{
Channel: "<your_channel_name>",
Token: "<your_rtc_token>",
AgentUID: "1002",
RemoteUIDs: []string{"1001"},
Values: map[string]any{
"previous_context": "The user asked about upgrading to the pro plan and wanted a callback about pricing.",
},
})Declare the variable when you create the agent:
"instructions": "You are a friendly support agent for Acme. Here is what the user discussed in earlier conversations: {{previous_context}}",
"variables": {
"previous_context": {
"type": "str"
}
}Pass the value when you start the session:
"values": {
"previous_context": "The user asked about upgrading to the pro plan and wanted a callback about pricing."
}Pass a summary rather than the full transcript when you can. It keeps the instructions short and leaves more room for the current conversation. For details about variables, see Personalize conversations.
Reference
For more information, see the following pages: