In-product AI feature with evaluation
Hypothetical. Not a client, not a result.
// The problem
A SaaS company's customers ask for an AI assistant inside the product, and the team has a prototype that works in demos and fails in ways nobody measures.
// What we would build
An AI feature built into the product with the customer's own data, a test set of real questions, and evaluations that run before every release, so quality is measured, not guessed. It uses a frontier model through its API, because the product already sends customer data to cloud providers under its terms and traffic is high; we keep each customer's data separate.
// What is in it
- The assistant inside your product, using each customer's data
- Each customer's data kept separate
- A test set of real questions and answers
- Evaluations that run before every release
- Usage and cost tracked per customer
// Stack
- Frontier model API
- PostgreSQL with pgvector
- Node.js
- Evaluation harness
- AWS
// estimate
- Build
- ≈ US$70,900 to US$108,000, delivered within 15 weeksCAD 100,900 to 154,300
- Hosting
- ≈ US$3,970 a monthCAD 5,655 a month
- Support
- ≈ US$2,500 a monthCAD 3,565 a month
Prices in your currency are estimates from today's Bank of Canada rate. All invoicing is in CAD or USD.
- AI route
- Frontier model
- Model running cost
- ≈ US$3,480 a monthCAD 4,955 a month
Timeline by milestone
- Specification and evaluation plan
- 2.9 to 3.4 weeks
- Working pilot
- 4.9 to 8.4 weeks
- Hardening and testing
- 0.5 to 1.5 weeks
- Production
- 0.9 to 1.4 weeks
How it is paid
- Deposit 20%
- CAD 20,180 to 30,860
- Specification and evaluation plan 14%
- CAD 14,126 to 21,602
- Working pilot 42%
- CAD 42,378 to 64,806
- Production 14%
- CAD 14,126 to 21,602
- Holdback, 30 days after launch (10%)
- CAD 10,090 to 15,430