L3 by June: What Klaviyo Is Actually Doing With Agents

Great CEOs notice something simple before anyone else does — for Andrew Bialecki, it was that e-commerce merchants didn't care how many emails went out, they cared what those emails made them. Klaviyo built a category around that. Roughly 80% of the Shopify ecosystem, one of two or three companies that IPO'd in 2023 when the market wasn't letting almost anyone through, north of $1.4 billion in revenue now with about 2,300 people.

That's the background. What I actually want to share is what Andrew said on the SaaStr AI Podcast about rebuilding that company for an AI world, because it's one of the most concrete things I've heard a CEO describe this year. Not a vision, just what they did, in order.

First: he set a deadline, not a values memo

Klaviyo defined three levels. L1 is occasional — you ask a model something, you search with it. L2 is running an agent session now and then, on something bounded. L3 is what Andrew told all 2,000-plus people at the company — every function, no exceptions — they need to hit by the end of June: if you're not constantly running agent sessions or a team of agents, you're not L3. Miss it, and you don't have a job there anymore.

His words on stage: "you're just not going to be skilled to survive in this next era." I'd have expected real pushback on a bar that high, applied company-wide, with actual stakes attached. He said there wasn't much — people already sense which skills are about to get scarce.

Across our own network right now there's a real split — some CEOs are still fighting genuine adoption resistance, others are already past it and measuring output quality. And where we do see resistance, especially from engineers, it's rarely actually about the AI. It's identity. Someone spent a decade building expertise in something a model can now approximate in seconds, and that's a harder conversation than "the code is buggy." 

One thing that probably helped Klaviyo: Andrew wasn't asking anyone to do something he wasn't already doing himself — he'd already stopped logging into normal SaaS tools in favor of querying everything through APIs (more on that below). Teams watch what the CEO actually does, not what gets announced. That part I do believe travels.

I don't know that every company can pull off a deadline with that much teeth. But naming the bar plainly, and meaning it, instead of a vision statement nobody can act on Monday morning, seems like the actual signal here — for us and for every CEO we work with.

Second: "dark factory" — agents building agents, with contracts between them

They built an internal system called dark factory. You give it a prompt. It acts as the PM — breaks the ask into specs, decomposes it into pieces — and then, before any code gets written, it writes the actual API contracts between those pieces. Andrew's line: "getting the agent going is pretty easy, figuring out where it gets stuck is hard." Without the contracts, a bunch of agents building in parallel produces exactly what you'd guess — code that works alone and falls apart the second the pieces have to talk to each other.

Their first real agent product, the one now doing marketing analysis and campaign design for customers, went from a Friday-night prompt to a working prototype by Monday. When something's genuinely unclear, the system stops and asks instead of guessing — which is closer to how a good human product review works anyway, just with agents driving it.

One more piece I liked: Andrew described the underlying model as "a very athletic middle schooler" — talented, general, not yet good at your specific sport. So they built the agent a coach of its own: a live feed of how real customers actually respond to campaigns, scoring every proposal before it ships.  Companies need to consider automating the feedback loops, not just automating the work.

Third: agents don't behave like new users

This is the one I keep thinking about. Normal software has a power law — a few power users push it hard, a long tail of novices barely touch it, and most of your product work goes into closing that gap. Klaviyo found agents blow that up. An agent shows up at the far end of the power-user curve on day one with no ramp.

So onboarding matters less, and the real leverage is asking your own agents, constantly, what's stopping them from doing more. Klaviyo's Composer agent reasoned its way — unprompted — to wanting a harder email format almost no human had bothered to use through their editor. Then it hit a wall: missing API, and it filed its own request to get it built.

Capability wasn't the constraint. The system it could reach was.

That does create a real problem, and Andrew was straightforward about it: if a non-developer can ship a working feature in an hour instead of waiting a quarter, you can end up with a team shipping thousands of half-built features a month. Their answer wasn't more process — it was taking years of accumulated product-review judgment and turning it into rules the agent checks itself against before a human ever sees it. And it's a living rule set the team can still argue with, not a gate.

Fourth: the API is the product

Andrew's broader belief, and I think it's worth testing against your own roadmap — software is becoming infrastructure, agents are the layer on top of it, and headless is the default now. His own proof point: he hadn't logged into Salesforce in years, didn't even know he'd lost his seat, because he queries it through the API. His outside example was Twilio — he said it was "almost dead" eighteen months ago, now growing over 20%, because its APIs happened to be exactly what agents want.

I don't fully know how far this generalizes outside API-heavy businesses, but for Klaviyo it shows up again on the support side. Instead of trying to teach 200,000 small businesses to build their own support agent — which Andrew said flatly isn't realistic — Klaviyo trains it for them off real conversation data, loops the business owner in only when something genuinely undocumented comes up, and hands over an agent already resolving 50 to 70% of cases on day one. His point, and I think it's the real one: if a prospect can't try your agent right now and watch it work, you're already behind.

The question I'd actually ask

I'm not saying you need an L3 deadline next week. What I'd ask instead: if everyone on your team were running agents constantly, starting today, what breaks first — your org chart, your review process, or your APIs?

Figure out which one before you do anything else.