Build a small streaming chat app

List your available models, send one message, stream the reply, and keep a local conversation using the OpenAI-compatible MandAPI endpoint.

What you will build

This recipe creates a command-line Python chat app. It has four modes: list models, send one message, stream a reply, and continue a conversation. Requests go to MandAPI through the OpenAI Python client. You use your own account and API balance; installing the client does not provide model credits.

Create a key from your MandAPI dashboard. Compare the current model catalog, then choose a model available to your key that supports Chat Completions and streaming. A model listing alone does not guarantee every feature.

Prepare Python and your environment

Use Python 3.10 or later. The following commands are for PowerShell. The key is entered at a prompt and is not written into the command itself. Set the model variable to an actual model ID after listing models.

powershell

py -m venv .venv
.\.venv\Scripts\python -m pip install openai
$secret = Read-Host "MandAPI API key" -AsSecureString
$env:MANDAPI_API_KEY = [System.Net.NetworkCredential]::new("", $secret).Password
$env:MANDAPI_MODEL_ID = "MODEL_ID_FROM_YOUR_CATALOG"

Keep the environment in this terminal while you run the examples. Store production secrets in your deployment's secret settings.

For macOS or Linux, use Bash:

bash

python3 -m venv .venv
source .venv/bin/activate
python -m pip install openai
read -rsp "MandAPI API key: " MANDAPI_API_KEY
printf '\n'
export MANDAPI_API_KEY
export MANDAPI_MODEL_ID="MODEL_ID_FROM_YOUR_CATALOG"

Save the complete app

Create a file named chat.py with this code:

python

# chat.py
import argparse
import os
from openai import OpenAI

parser = argparse.ArgumentParser()
parser.add_argument("mode", choices=["models", "once", "stream", "chat"])
args = parser.parse_args()

client = OpenAI(
    api_key=os.environ["MANDAPI_API_KEY"],
    base_url="https://api.mandapi.com/v1",
    timeout=30.0,
    max_retries=0,
)

def answer(messages, stream=False):
    result = client.chat.completions.create(
        model=os.environ["MANDAPI_MODEL_ID"],
        messages=messages,
        stream=stream,
    )
    if not stream:
        text = result.choices[0].message.content or ""
        print(text)
        if result.usage is not None:
            print("Tokens:", result.usage.prompt_tokens,
                  result.usage.completion_tokens)
        return text

    parts = []
    try:
        for chunk in result:
            if not chunk.choices:
                continue
            text = chunk.choices[0].delta.content or ""
            parts.append(text)
            print(text, end="", flush=True)
    finally:
        result.close()
    print()
    return "".join(parts)

try:
    if args.mode == "models":
        for model in client.models.list().data:
            print(model.id)
    elif args.mode in ("once", "stream"):
        answer([{"role": "user", "content": "Say hello in Spanish."}],
               stream=args.mode == "stream")
    else:
        history = []
        while True:
            try:
                text = input("You (exit to quit): ").strip()
            except EOFError:
                break
            if text.lower() == "exit":
                break
            if not text:
                continue
            messages = history + [{"role": "user", "content": text}]
            reply = answer(messages, stream=True)
            history = messages + [{"role": "assistant", "content": reply}]
except KeyboardInterrupt:
    print()
finally:
    client.close()

Run the four modes

Use .\.venv\Scripts\python chat.py models first. Copy an appropriate ID from the result into MANDAPI_MODEL_ID.

Run chat.py once to get a complete answer and returned token counts. Run chat.py stream to print text as it arrives. Finally, run chat.py chat, say your name, and ask a follow-up about it. The app resends its local message history, so each follow-up can consume more input tokens. Type exit to stop.

With the macOS/Linux virtual environment active, use python chat.py models, python chat.py once, python chat.py stream, or python chat.py chat.

Check the result

The complete reply should contain text; the streaming mode should display fragments; a follow-up should use earlier context. Check actual usage in your dashboard. If authentication, balance, model access, or an unsupported option causes an error, correct that specific setting before retrying. No payment or live request is performed by downloading this recipe.