Call one chat-completions API and one OSS fallback; log cost and latency for both.
1. The OpenAI-compatible shape
Nearly every provider (OpenAI, OSS servers, gateways) accepts the same POST body:
model, messages (system/user/tool), temperature, max_tokens, response_format.
Learn this shape once; switching providers is a base-URL + key change.
2. Wrapper skeleton (starter)
def chat(messages, model, temperature=0.2, max_tokens=500):
# primary -> fallback on timeout/5xx; time both calls; log USD per 1K tokens
Rules: temperature 0-0.3 for extraction; always set max_tokens; retry once on
429/5xx with backoff, then fall back to the OSS endpoint and flag the row.
3. Cost/latency log
Per call record: model, input/output tokens, USD cost, TTFT + total latency, fallback-used flag. The capstone cost dashboard is built from this exact log.
4. Grounding discipline
Every factual claim in the final answer must trace to either the retrieved context or an explicit "unsupported" flag. A fluent answer with no citation is a failure, not a partial pass — this is what the 20Q golden refusal suite checks.