Shipping AI features without shipping a liability

The first AI demo takes an afternoon. The production version takes a quarter, and most of that isn’t the model.

We’ve shipped an AI health assistant in Solix and AI-assisted interview scoring in ElyteSpark. The model was rarely the bottleneck. The missing pieces were the job definition, the evaluation, and the support path for people who got a bad answer.

Pick a job, not a vibe

“Add AI” is not a job. “Turn a 40-minute mock interview into three specific improvements” is a job. The narrower the job, the easier it is to measure and the harder it is to fake.

• Turn a resume into five concrete edits, with the original line quoted.
• Score a practice interview against a published rubric and explain the score.
• Triage a support ticket into a category, priority, and missing information.
• Answer questions from a help centre with links to the source pages.
• Generate menu descriptions from a structured list of ingredients.

“Chat with your data” sounds impressive and is often a support burden. If users don’t know what to ask, the feature has no job.

Build the eval set before the prompt

Fifty to two hundred real examples, including the ugly ones. Define what a good answer looks like. Then compare prompts and models against the same set. If you can’t reproduce a bad answer, you can’t fix it.

Track prompt version, model, temperature, and eval date. A spreadsheet is fine at the start. The discipline matters more than the tool.

Context beats prompt cleverness

RAG is not a magic prefix. It’s a retrieval product: chunking, metadata, freshness, permissions, citations, and a way to say “I don’t know”.

• Show source dates and links so users can verify.
• Enforce document permissions before retrieval, not after.
• Return an empty result with a clear message when nothing relevant is found.
• Reindex on publish and track stale documents.
• Keep prompts and retrieved context separate so you can debug them independently.

Guardrails in four layers

Guardrails are not a single regex. They’re a chain of decisions, and every layer needs logs. If the model returns invalid JSON, the product should fail closed or show a retry, not render a broken answer.

Latency and cost engineering

• Stream tokens to improve perceived latency.
• Route simple tasks to a smaller model and escalate only when needed.
• Cache deterministic answers and common retrieved contexts.
• Cap output tokens and request size.
• Batch offline jobs instead of making users wait.
• Set timeouts, retries with a budget, and a graceful fallback.

Privacy is not a footer link

In health, education, and hiring, the data is sensitive even when a model vendor says it isn’t. Map what leaves your system, why, how long it’s retained, and who can access it.

• Collect only the data the job needs.
• Record consent and purpose where required.
• Review vendor terms for training, retention, and sub-processors.
• Redact logs before they reach an observability tool.
• Keep a deletion path for prompts, uploads, and generated outputs.
• Document cross-border processing.

Human-in-the-loop is a feature

Solix’s health assistant and ElyteSpark’s interview scoring both need a human-readable rationale and an easy correction path. The AI can draft; the person decides. That isn’t a lack of ambition. It’s how trust is built in categories where a wrong answer has consequences.

• Show what the answer is based on.
• Let the user edit before saving or sending.
• Provide an undo or dismiss action.
• Route low-confidence or high-risk cases to a person.
• Disclose when AI is involved.

Observability and abuse

• Log prompt version, model, latency, tokens, and cost per request.
• Use a trace ID that links the user action to the model call.
• Track refusal rate, validation failures, and thumbs-down feedback.
• Set spend caps and per-user rate limits.
• Keep a kill switch that disables the feature without a deploy.
• Watch for prompt injection and data exfiltration patterns.

A launch checklist

1. Write the job in one sentence and the failure modes in another.
2. Build an eval set with real edge cases.
3. Choose a fallback before choosing a model.
4. Add input, output, and product guardrails.
5. Test with a mid-range phone and a slow connection.
6. Review data flows and vendor terms.
7. Set cost, latency, and error alarms.
8. Write the support script for bad answers.
9. Run a small cohort before a full launch.
10. Decide what you will remove if the feature doesn’t earn its cost.

What the model didn’t fix

Gemini is capable enough for many product features. The failure cases are usually product failures: no clear job, no eval, no fallback, no support path. The model is rarely the reason a feature feels untrustworthy. The missing product decisions are.

Earn the trust

Treat AI like a new employee with a narrow brief, a probation period, and a manager. Give it context, check the work, limit the blast radius, and keep a human accountable. That’s less exciting than a demo. It’s also how the feature survives contact with real users.