AI Software Verification: The Case for Runtime Intelligence - InsightJet
Why Runtime Intelligence might become the next essential layer of AI-driven software development.
There's something addictive about giving Claude or ChatGPT a prompt and watching it build a beautiful application within minutes. Everything looks polished, the interface feels complete, and for a moment, it seems almost magical.
Then you start clicking around. Some buttons don't work, features are missing, and workflows break halfway through. The application looks finished, but underneath that shiny interface, plenty of things simply don't work.
AI has made building software remarkably easy. But who makes sure it actually works?
That's the question that led me to spend the past year building InsightJet, an AI-powered software verification platform.
We've Seen This Pattern Before
I remember growing up when we constantly compared CPU speeds, RAM, and hard drive capacities. Every hardware upgrade felt exciting. Eventually, computers became powerful enough that most everyday users stopped caring about the specifications. They just wanted things to work.
I suspect something similar is happening with AI. OpenAI, Anthropic, and Google DeepMind keep releasing increasingly capable models, yet most users still explore only a fraction of what's possible. A small group of power users pushes these systems to their limits while everyone else gradually catches up.
As AI capabilities become increasingly accessible, I believe the bottleneck will shift.
The challenge won't be what AI can generate, but whether we can trust what it produces.
Code Is Becoming Cheaper. Confidence Isn't.
Companies like CodeRabbit recognized an important opportunity in reviewing the growing volume of AI-generated code. But I think there's another layer beyond code review.
Ultimately, users don't care about code. They care about outcomes.
An AI agent might generate a perfectly structured checkout flow, but does the payment actually work? Can users recover from errors? What happens when someone does something unexpected?
That's where Runtime Intelligence comes in: understanding how software actually behaves, discovering problems, and turning those observations into actionable feedback.
And unlike simply executing predefined test cases, AI-driven verification can help us discover failures we didn't even think to test for.
Six Questions Behind a Verification Layer
While building InsightJet, I've realized that verification involves much more than running automated tests. It requires connecting what we intended to build with what was actually delivered.
Here are six questions I've been thinking about:
1. Why do we verify?
Not to generate more test cases or beautiful reports. We verify for confidence, control, and peace of mind that the software we're shipping actually works.
2. What is the source of truth?
A Jira ticket, GitHub Issue, Figma design, conversation with Claude or GPT, or another agent communicating through MCP? How do we establish what was actually supposed to be built?
3. How should verification be organized?
Traditional test cases, Gherkin scenarios, user stories, or acceptance criteria? We need traceability between requirements and results without creating another documentation burden.
4. How should verification happen?
Through Playwright, autonomous browser agents, or cloud environments like BrowserStack? How do we combine repeatable automation with intelligent exploration?
5. What should the results look like?
A pass/fail status, expert assessment, screenshots, or video recordings? Videos provide evidence, but who's going to watch hundreds of them? And why should we trust an AI agent simply declaring another agent's work correct?
6. Who consumes the results?
Developers, QA engineers, product managers, or perhaps increasingly, other AI agents? This last possibility particularly interests me: coding agents receiving verification feedback, fixing problems, and verifying again.
From Verification to Runtime Intelligence
Initially, I wanted to avoid stepping into the territory of platforms like Jira, Linear, or Notion. But I've realized that meaningful verification can't exist completely disconnected from where requirements, decisions, and development activities originate.
The goal isn't to build another project management tool. It's to connect intent with actual runtime behavior and make that intelligence useful to humans and autonomous agents.
Imagine a continuous feedback loop:
Not another dashboard someone needs to constantly monitor, but a verification layer that coding agents can communicate with directly.
That's the broader direction I'm exploring with InsightJet: moving beyond automated testing toward Runtime Intelligence for AI-generated software.
Because as AI makes building software increasingly effortless, something else becomes more valuable.
Knowing that what we've built actually works.
And I'm curious: when an AI coding agent tells you it's finished, what evidence do you need to believe it?