Sessions and Conversations — Hosted Agents: Concepts

An artificial intelligence (AI) agent appears to have one memory, but a hosted agent usually has two. One is the session: the working place where files, uploads, and scratch results live while the agent does the job. The other is the conversation: the durable record of messages, tool calls, and replies. Treat them as the same thing and the system feels mysterious. Treat them as different things, and its behavior becomes much easier to predict.

One word, two kinds of memory

Imagine a person working at a desk. On the desk are drafts, notes, reference files, and partly finished work. That is the session. Across the room is the official file of what was said and decided. That is the conversation.

The desk can be cleared, moved, or rebuilt without destroying the written minutes. Likewise, the compute behind a hosted agent can stop when idle, while the conversation remains available because it is stored separately. This separation is not an accident. It is the design that lets an agent stop running without forgetting the thread.

The session is transient working state; the conversation is the durable record of what was said.

The session is where the work happens

A session is identified by a session identifier, often shortened to session ID. It holds the agent’s working state: its home directory, uploaded files, intermediate files, and other scratch material the agent expects to find while doing the task.

A session is tied to compute, but it is not the same thing as a single running machine. When the agent goes idle, the machine can be torn down. The session state is saved. When work resumes, fresh compute can be started and the saved state restored. From the agent’s point of view, its working desk is back.

This does not mean every running process or in-memory detail should be treated as permanent. The safe assumption is that durable files in the session can come back, while live execution state belongs to the machine that may disappear.

📌

Think filesystem first. Use the session for working files and task-local state, not as a substitute for a durable message archive.

The conversation is the record of what was said

A conversation is identified by a conversation identifier, often shortened to conversation ID. It is the durable history of the exchange: user messages, assistant replies, tool calls, tool results, and other recordable turns in the interaction.

The important property is independence. The conversation is stored outside the transient compute that runs the agent. The machine can be gone, the session can be cold, and the conversation can still be read because it was written to durable storage.

This is the sense in which the agent can remember a discussion. It is not because a particular machine kept running. It is because the platform or application preserved the conversation record.

A conversation is the durable history of messages, tool calls, and their results.

Why the split matters

If the conversation were tied to the running machine, scaling compute down to zero would also mean scaling memory down to zero. The agent could save money while idle, but it would lose the thread whenever the machine disappeared.

By separating the two, each kind of state can follow its natural rhythm. Compute can appear only when needed. Working files can be restored for the next task step. The durable conversation can remain available without waking the agent at all.

That last point is easy to miss. A system can fetch, inspect, export, or continue a conversation without depending on the old session machine, because the conversation is not stored there.

✅

The useful bargain. Separating sessions from conversations lets a system pay little or nothing while idle without losing the durable history.

Two access styles emphasize different memory

Hosted-agent platforms commonly offer two styles of access. In a conversational style, sometimes called Responses, the platform manages the conversation history for you. You send the next turn, and the platform associates it with the durable conversation and the needed working session.

In a flexible invocation style, sometimes called Invocations, you drive the session more directly. You pass or choose the session identifier so the agent can keep working state across calls. In that style, there may be no platform-managed conversation history. If you need a durable record, your application must store it.

Neither style is inherently better. The conversational style is simpler when the central object is an ongoing thread. The invocation style is useful when you need finer control over execution, state, or integration patterns. The mistake is assuming both styles provide the same kind of memory.

Choose a managed-history style when the thread is central, or a session-driven style for finer control.

The same conversation can surface in many places

Because the conversation is not tied to a particular machine, it can be reached from more than one front door. A user might begin in a testing playground, continue through an application programming interface (API), and later resume in a workplace channel, while still using the same conversation history.

This is cross-channel continuity. The durable record follows the conversation, not the compute instance and not the user interface. That makes it possible to build systems where a thread can move across tools without asking the user to restate the past.

The practical requirement is discipline around identifiers. If different surfaces should share a thread, they must refer to the same conversation identifier. If they should not share a thread, they need separate conversation identifiers even if they use the same agent.

The same conversation can be reached from many surfaces because it is stored independently.

The design decision is also a governance decision

For a practitioner, the session-conversation split is not just an implementation detail. It decides what is temporary, what is durable, who can inspect it, and what must be deleted under a retention policy.

The conversation is powerful because it accumulates context. That is also the risk. Over time it may collect personal data, credentials pasted by mistake, business plans, customer records, or regulated information. If it is durable, then access control, retention, audit, and deletion rules matter.

The session is often the better home for transient scratch data that is needed only to finish the task. But transient does not mean careless. Uploaded files and intermediate outputs can still be sensitive while they exist.

Decide what belongs in the durable conversation and what belongs only in the session.
Test what happens after the agent goes idle and compute is torn down.
Confirm whether your access style provides managed conversation history or requires you to store it.
Review what sensitive content can accumulate in conversations, who can read it, and how long it is retained.
Use separate conversation identifiers when two threads must not share context.

⚠️

Durable means governable. Anything written into a durable conversation should be treated as retained data, not as temporary scratch work.

Key takeaways

A session is the agent’s working state: files, uploads, scratch space, and restored task context.
A conversation is the durable record of messages, tool calls, tool results, and replies.
Compute can scale down when idle because the durable conversation is stored independently of the running machine.
Conversational access usually manages conversation history for you; invocation-style access usually leaves the record to your application.
Use the conversation for information that should be remembered, and the session for material that should not outlive the task.
Retention, access control, and deletion policies must be designed around the durable conversation record.

Glossary

Session: A session is the agent’s live working state, including its home directory, uploaded files, and scratch space, tied to compute that can come and go.
Conversation: A conversation is the durable record of an exchange, including messages, tool calls, tool results, and replies.
Session identifier: A session identifier is the handle used to refer to a particular session and its persisted working state.
Conversation identifier: A conversation identifier is the handle used to refer to a particular durable conversation history.
Working state: Working state is the task-local material an agent uses while doing work, such as files, drafts, and intermediate outputs.
Durable record: A durable record is stored history that remains available after the compute that created it has stopped.
Tool call: A tool call is an action an agent asks another system or function to perform, such as searching, reading a file, or running code.
Scale to zero: Scale to zero means stopping compute when it is idle so the system is not paying for a running machine.
Cross-channel continuity: Cross-channel continuity is the ability to continue the same conversation from different user interfaces or integration points.