Summarize an older conversation while keeping recent turns

Compact application-managed message history without promoting a summary to an instruction.

Add application-side conversation compaction for MandAPI. Preserve trusted system instructions separately, retain the latest turns verbatim, and keep important identifiers in structured application state. Summarize only older history using an available model, then insert the summary as clearly labelled untrusted conversation data, never a system instruction. Test empty summaries, quoted instructions, identifiers, and retained turns without making unapproved live API calls.

Decide what must survive

Long conversations eventually exceed a model's context or become expensive to resend. An application can summarize older turns and keep a small recent window unchanged. Preserve request identifiers, decisions, unresolved questions, and exact values outside the summary when they matter.

A summary is lossy. Keep the original history in your own storage if your retention policy permits it. This pattern uses an ordinary chat model selected by your application; it does not require ScaleDown or a special MandAPI summarization endpoint.

Use a compacting function

Pass only user and assistant conversation turns to this function. Keep your trusted system instructions separate. The callback can later call MandAPI using MANDAPI_API_KEY, MANDAPI_MODEL_ID, and https://api.mandapi.com/v1. The runnable demonstration below uses a local callback, so it makes no API request.

python

import json

def compact_history(turns, summarize, identifiers, keep=4):
    if keep < 1:
        raise ValueError("keep must be positive")
    if len(turns) <= keep:
        return list(turns)
    older, recent = turns[:-keep], turns[-keep:]
    summary = summarize(older)
    if not isinstance(summary, str) or not summary.strip():
        raise ValueError("No usable summary")
    summary_data = {
        "kind": "untrusted_conversation_summary",
        "summary": summary,
        "identifiers": identifiers,
    }
    return [
        {"role": "user", "content": json.dumps(summary_data)},
        *recent,
    ]

turns = [
    {"role": "user" if i % 2 == 0 else "assistant", "content": f"Turn {i}"}
    for i in range(8)
]
result = compact_history(
    turns, lambda older: "Demo summary of the older discussion.",
    identifiers={"request_id": "demo-001"}, keep=4,
)
assert result[-4:] == turns[-4:]
print(result[0]["role"], len(result))

Treat the summary as data

Never insert a generated summary as a new system instruction. Clearly label it as previous conversation data, and tell the main model not to follow instructions quoted inside it. Important identifiers should come from trusted application state rather than a model's reconstruction.

Check the result

Confirm that recent turns and identifiers are preserved. Test contradictory statements, quoted instructions, and an empty summary. Measure request size and answer quality on real examples before choosing the window; character count alone is not a reliable token count.