Jaskaran Singh Kalra

Product Manager · Applied AI · San Francisco

I prototype applied-AI products myself, then lead them through to launch.

Nine years taking AI products from 0 to 1. I write the evals that decide when a product is ready to ship, and I drive delivery across engineering, data science, BD, and operations.

What I've built, and what it changed

Internal product names, data, and screenshots are left out.

ADI · LLM evals · Release gating

The eval suite that decides when an enterprise AI agent ships

Sole PM and engineer. The agent answers plain-English questions about company revenue. I built 40 business-critical test questions with frozen correct answers, checked against approved finance reports, plus an LLM-as-judge: a second model that grades every answer correct, partially correct, or hallucinated.

It replaced 120 engineer-hours of manual testing per release. It blocked 2 production releases until defects were fixed, and both then shipped with zero regressions. The same results justified moving the agent from Opus to Haiku, a smaller model, cutting cost and latency.

ADI · LLM product · Adoption

A second brain on Claude, built to replace the wiki

Built and shipped a knowledge product on Claude that replaces knowledge bases like Confluence. It breaks meeting transcripts and work documents into linked pages anyone can ask questions of.

40+ engineers, PMs, and leaders adopted it within 3 weeks. Each now resolves 5 to 10 questions a week that used to wait on one overloaded expert.

Tyrula · Document AI · 0 to 1 SaaS

A commercial document-AI product, still sold today

Turned a back-end data entry tool into a commercial AI document-processing SaaS, owning product strategy, delivery, and customer-facing quality end to end. It cut manual effort 90% across ~4,000 forms scanned daily, and the legacy tool was retired with no customer churn.

I designed the trust layer that got non-technical users to accept the model's output: low-confidence extractions go to a human for review, with field-level accuracy targets and an audit trail. 98% CSAT across enterprise accounts.

ADI · ML · Anomaly detection

Grey-market detection at invoicing, not months later

Defined, modeled, and shipped an ML anomaly-detection product across 2,000+ product lines. It catches grey-market activity: products resold outside approved channels.

Detection now happens at the time of invoicing, instead of ~6 months after the sale. I built the detection model and the analyst-facing interface myself.

Uber · Roadmap · ML systems

A 3-year roadmap across 40+ stakeholders that won $5M in funding

Authored the CPO-approved 3-year product vision and roadmap across 40+ stakeholders. ~90% of Year 1 was delivered, and it won $5M in incremental funding for the team's annual runway. Held go/no-go authority over third-party data vendors, leading technical negotiations alongside BD.

Owned the ML system that estimated competitor market share. It powered a merchant pricing product and investor reporting read by senior leadership up to the CEO. I lifted its uptime from 65% to 95%.

Built on my own time

RAG · Retrieval evals

Local search over my own notes, measured instead of eyeballed

A RAG system (the model looks up my documents before it answers) that takes plain-English questions about my entire Notion workspace. It runs fully local: Ollama for the models and embeddings, Chroma as the vector store, Streamlit for the interface.

I built a golden question set and scored retrieval with Precision@5 (how many of the top 5 results are actually relevant) to compare chunking strategies and overlap settings.

What I believe about AI products

Most AI features fail because nobody defined what good output looks like.

That's a product spec problem, not a model problem. The PM's job is to write the eval before the engineer writes the prompt.

The best AI PMs I know are the ones who've broken their own automations.

Shipping a personal AI tool teaches you more about failure modes, latency tolerance, and user trust than any case study.

AI is making product strategy harder, not easier.

When any feature is buildable in a week, the constraint shifts entirely to judgment: what should we build? That's a harder problem with more surface area for bad decisions.

Jaskaran Singh Kalra

Building applied-AI products? Let's talk.

Email is the fastest way to reach me.