Every AI vendor looks great in a demo
That’s the problem. Demos run on curated data, controlled prompts, and a presenter who knows which questions to avoid. Production runs on none of those things.
Gartner research found that, on average, only 48% of AI projects make it into production, and it takes 8 months to go from prototype to production. When you pick a partner, you’re really betting on which side of that line you’ll land.
We sit across the table from buyers at Perceptive Analytics every week. These are the questions we’d want answered if we were in your seat.
The Perceptive POV: Ask to meet the engineer who’ll be responsible when the system breaks at 2 a.m. How a firm responds to that request tells you more than its case study deck.
Proof
- “Show me something comparable that’s in production today.”
Not a pilot or a proof of concept. Something live, with real users. - “Tell me about a project that went badly.”
Experienced firms have scars. A firm that claims a perfect record is either new or not being honest with you. - “Who exactly will work on this, and how senior are they?”
Many firms sell with partners and deliver with juniors. Perceptive Analytics staffs AI engagements with senior engineers who’ve shipped before, with no hand-off to a junior bench after the contract is signed.
Approach
- “How will you decide whether our data can support this?”
You want a method: source mapping, quality profiling, access checks. “We’ll figure it out” isn’t a method. - “Why this architecture instead of something simpler?”
Strong partners default to the lightest approach that works. If a firm proposes fine-tuning or agents before it has tested prompting and retrieval, ask why. - “What KPI will we measure, and what’s the baseline?”
If success isn’t defined before the build, it won’t be agreed on after.
Risk and ownership
- “Where will our data go, and who can see it?”
Ask about hosting region, model provider data policies, logging, and access. Perceptive Analytics follows a security-first design and deploys inside the client’s own cloud environment wherever possible. - “Which governance framework will you align to?”
Look for recognized standards such as the NIST AI RMF or ISO/IEC 42001, plus experience with your industry’s rules. - “Who owns the code, prompts, models, and documentation?”
You should. Get it in writing.
After launch
- “Walk me through your path from pilot to production.”
Listen for evaluation sets, monitoring, cost tracking, and rollback plans. - “What does support look like six months after go-live?”
Models drift. Business rules change. Someone has to own that. - “How will our team be able to run this without you?”
Documentation, training, and paired working. The goal is capability, not dependency.
Red flags
- Accuracy promises before they’ve seen your data
- No run-cost estimate
- A delivery team you’re not allowed to meet
- Vague answers on data handling or IP
- A proposal that skips discovery entirely
A note on platform partnerships
Partnerships with Microsoft, AWS, and Google Cloud signal ecosystem familiarity. They aren’t proof of delivery. Firms that work across all three clouds also tend to give more neutral platform advice, because they aren’t steering you toward one vendor’s roadmap. [Insert Perceptive’s verified partner statuses.]
What good answers sound like
Case study: When a financial services client asked Perceptive Analytics how we’d measure success on a contract review system, the answer was specific: manual processing time per contract, measured before and after. The result was a 75% reduction. [CASE STUDY LINK: financial services document intelligence]
Executive takeaway: A strong AI partner welcomes these 12 questions. A weak one tries to steer you back to the demo.
Evaluating AI partners now? Download the Perceptive Analytics Vendor Evaluation Checklist. It’s a scoring template built around these 12 questions that your team can use across every vendor, including us. Explore our AI consulting services.
Frequently Asked Questions
What should I look for in an AI consulting company?
Production deployments, a clear data readiness method, senior engineers, strong security practices, clear IP ownership, and defined post-launch support.
How many AI vendors should I evaluate?
Three is usually enough to compare approaches without stalling the decision.
Are large consultancies better for AI projects?
Not necessarily. Mid-market companies often get more senior attention and faster delivery from specialized firms like Perceptive Analytics.
Who should own AI models built by a consultant?
Your company. Confirm ownership of code, prompts, models, and documentation in the contract.