LLM Feature Unit Economics for B2B Products

The real cost stack of an LLM feature

Direct: model tokens (input + output), embeddings, rerankers, vector or search infra, tool/API calls, and vendor markups. Indirect: engineering for eval and observability, human review minutes, support escalations, and failed-run waste. Agents multiply cost through loops and tool chatter. RAG multiplies cost through chunk volume and naive 'stuff more context' prompts. Measure both paths separately. Build a simple model: successful_tasks × (model + retrieval + review_minutes × loaded_rate) + fixed infra. If you cannot estimate, you cannot price.

• Track cost per successful task, not per chat message
• Include failed and abandoned runs in the average
• Separate pilot volume from contracted enterprise volume
• Revisit assumptions when models or prompts change

Token and retrieval cost drivers

Input tokens dominate when you dump long histories or fifty retrieved chunks. Cap context, diversify sources, cache embeddings, and reuse retrieval carefully across follow-ups. Choose model tiers by task: cheap classifiers and extractors upstream; stronger models only where judgment quality pays for itself. Streaming does not reduce billable tokens. For RAG ops detail, see RAG cost and latency practices. For agents, cap iterations and tool calls per run as hard budgets.

Human review as a first-class cost

Every approval minute is product COGS if reviewers are staffed for the feature. Design queues, SLAs, and batching so review does not erase automation gains. Route only high-risk or low-confidence cases to humans. Measure override rate: high override means you are paying twice (model + human) for little lift. Productize the gate with human-in-the-loop design; ad-hoc Slack approvals do not scale economically.

• Price reviewer time into the feature from day one
• Auto-approve only where risk and eval allow
• Track median time-to-approve as an ops metric
• Avoid review UX that forces re-reading entire docs

Support and trust costs

Wrong answers create tickets, refunds, and sales engineering time. Budget for explainability: citations, run traces, and clear refusals reduce 'why did it say that' loops. Align expectations with SLA and support models. AI that is 'best effort' still needs a human escape hatch and documented severity handling. Observe failure modes with observability so cost spikes and quality regressions show up before finance notices.

Pricing and packaging the AI add-on

Common patterns: included quota in a plan, metered usage, higher tier for AI, or per-seat with fair-use caps. Match packaging to cost drivers: retrieval-heavy tenants should not share uncapped flat fees with light users forever. Publish soft and hard limits. Soft: warn and degrade gracefully. Hard: stop spend with a clear upgrade path. Silent overages destroy trust; uncapped flat fees destroy margin. Tie packaging decisions to which AI pattern you shipped: chat, RAG, and agents have different usage shapes.

Margins and kill criteria

Set a target contribution margin after model, retrieval, and review. If pilot economics only work at toy volume or with unpaid founder review, they will not survive enterprise expand. Define kill or redesign triggers: cost per success above X, override rate above Y, support tickets per 100 tasks above Z. Compare build vs buy and scope cuts using MVP prioritization and contract shape via fixed price vs time and materials when delivery partners are involved.

• Review unit economics monthly during pilot
• Separate R&D learning budget from COGS
• Do not hide AI loss leaders without an explicit strategy
• Re-forecast when you enable write paths or agents

Engineering cost controls that actually work

Hard caps: max tokens, max tool calls, max retrieval chunks, per-tenant daily budgets. Caching: embeddings, repeated identical queries where freshness allows. Routing: small models first, escalate on low confidence. Prompt and graph versioning prevent silent cost regressions. Eval gates catch 'quality wins' that triple context size. For agents, prefer explicit graphs over open-ended loops; see agentic design. For tool sprawl, constrain with disciplined tool layers.

Finance and ops instrumentation

Tag every run with tenant, feature, model, and outcome. Export daily cost and success aggregates to finance without dumping prompt contents. Allocate shared infra (vector DB, gateways) fairly so one heavy tenant does not hide under platform COGS. Security choices affect cost too: over-retention of traces and duplicate logging inflate storage. Balance with AI security and audit retention.

Next steps

Build a spreadsheet with cost per successful task at three volumes. If margin dies at enterprise scale, redesign before you sell unlimited AI seats. See contractor acceptance for AI, other resources, case studies, book a call, or contact to stress-test pricing and architecture before the next sales cycle.

FAQ