Company profile · 4 min read
Agnost AI: production analytics for agent failures
Agnost AI links agent conversations and traces to silent failures, intents and concrete improvements, with public plans for production teams.
Cofounder of Lazyweb (a16z Speedrun). Previously grew subscriptions and built native ads at Duolingo. Writes First1000 (90,000+ subscribers).
Published · Updated
What it does
Agnost AI watches production conversations and traces from AI agents to find silent failures, user frustration, policy violations and recurring unmet intents. It clusters the evidence, links each finding back to the underlying conversation and tool calls, and helps teams turn the failure pattern into an eval or a model improvement. The current homepage and docs make the distinction from ordinary observability clear: a trace can say “success” while the user still received nothing useful.
The fit is an AI product team with enough real traffic to see repeated failures but not enough time to review every transcript manually. Agnost is strongest where teams want a quality loop from user intent to concrete fix, not just latency and token dashboards.
Why I’d look closer
The homepage says teams should send the conversations they choose, use pseudonymous IDs and redact secrets or sensitive fields before ingestion; it explicitly says Agnost does not automatically redact PII for the customer. That is a much more useful security statement than a vague “enterprise-ready” label. The docs say teams can connect through an SDK, OpenTelemetry or the Agnost skill and inspect the conversation, event and tool-call evidence under every finding.
Pricing is public. The homepage lists Free up to 1,000 events/month with seven-day retention, Starter at $49/month for 10,000 events and 30-day retention, Pro at $499/month for 1,000,000 events and 90-day retention, and Enterprise custom with self-hosted VPC, audit logs and custom controls. The YC launch reports one customer-specific benchmark where a specialist model improved task success by 22.9% and reduced latency and cost; that is a single company-reported workload result, not a general model claim.
The founders’ backgrounds support the analytics/infrastructure thesis. The YC biographies describe Shubham Palriwala with Cisco analytics and Formbricks experience, and Parth Ajmera with IIT Madras, Microsoft data pipelines and Infurnia graphics engineering.
What I’d ask
Which events are required for an insight to be trustworthy, how does clustering change when the agent or prompt changes, and what is the retention/deletion path? I’d connect a staging trace first, inspect the redaction boundary, then compare Agnost’s findings with a known set of failures and a human transcript review.
My editorial take
Agnost is one of the clearer agent-quality tools in this batch because it puts user conversation next to the trace and admits where the customer owns the privacy work. The product earns a pilot if it turns a recurring failure into a measurable, shipped fix rather than another dashboard.
Quick facts
| Field | Sourced detail |
|---|---|
| Buyer fit | Teams operating production AI agents with meaningful conversation volume |
| Public pricing | Free; Starter $49/month; Pro $499/month; Enterprise custom |
| Evidence model | Conversations, events and tool calls linked to intents/violations |
| Privacy boundary | Customer must choose data and redact secrets/PII before ingestion; company-stated |
Sources checked
Checked 2026-09-19.
| Source | Used for |
|---|---|
| YC company profile | Product, founders and company-reported benchmark |
| Agnost homepage | Features, pricing and privacy boundary |
| Agnost docs | Ingestion paths and evidence model |
| Agnost blog | Public agent-analytics research surface |
Cohort context
Agnost AI is listed in Summer 2026. In our 2026-09-18 directory snapshot, 119 of 232 listed companies in that cohort have YC’s primary industry label B2B (51.3%). This is a current-directory comparison, not an original intake count or a performance ranking. Nine-cohort dataset.
Public website snapshot
Observed 2026-09-19T16:18:31.745Z in raw homepage HTML. This records visible metadata and advertised links, not agent execution or product quality.
| Signal | Homepage observation |
|---|---|
| Product description metadata | Observed |
| Canonical link | Observed |
| H1 or H2 heading | Observed |
| Typed structured data | Observed |
| Docs/developer link | Observed |
| Pricing link | Not observed in this response |
| llms.txt link | Observed |
| Markdown alternate | Observed |
Public observations · Collection method. Missing links here do not establish that a capability or file is absent elsewhere.
About the author
Cofounder of Lazyweb (a16z Speedrun). Previously grew subscriptions and built native ads at Duolingo. Writes First1000 (90,000+ subscribers).
I cofound Lazyweb and publish Mudpie. This is an owner-written publication, not an independent testing organization. Research notes distinguish observations, sourced reporting and editorial judgment.
First1000 ↗ · X ↗