Scale human-in-the-loop to the stakes

From raw API keys to real-world consequences

The early wave of desktop agents moved fast by dropping long-lived API keys directly onto local developer machines. Once agents book travel, alter calendars, commit to repositories, and spend money, raw keys become impossible to scope or revoke cleanly when something goes sideways:

“Is just give us an API key to this service and that service and this one over here. It was great. We were building really, really fast. And then we realized, well, those are long-lived APIs. Probably not the best thing.”00:00:45

Before wiring any tool, you have to answer a basic identity question that harness frameworks often skip:

“If I have an agent going off and doing its own thing, is it acting on behalf of me? Does the agent have its own identity?”00:00:45

Evidence: transcript 00:00:45, 00:02:04

Four checkpoints as the stakes rise

Stopping the user on every read or search destroys the point of an agent, while rubber-stamping destructive calls invites outages. Instead, calibrate the check to the stakes across four tiers—from conversational back-and-forth up to out-of-band confirmation and spend limits:

“Well, it doesn't make sense to have a human in the loop at every single stage of the application, but sometimes it's implied. And sometimes it's right there, staring at us in the face, waiting for us to approve.”00:02:04

Even at the lowest tier—a conversational booking assistant for a dog grooming shop built with the Vercel AI SDK—you still place hard bounds on how far an agent can wander before pausing:

“Let's say you have a back. You say, Hey, I don't want this to exceed more than you know eight steps or twelve steps.”00:04:30

Evidence: transcript 00:02:04, 00:04:30, 00:07:19

Token Vault: limit what the agent can touch

A user signed in with email and password has an identity, yet their agent still shouldn’t hold blanket access to Google Drive or YouTube just to add a grooming appointment to Calendar. With step-up authorization, the agent hits the missing permission at runtime and pauses until the user connects only the narrow scope needed:

“So I'm authenticated, but my agent needs to do more. So now we have this authorization stuff.”00:06:11

In both the calendar booking flow and his live “Party Animals” audience drawing demo—where a nightly cron agent posts doodles to a GitHub repo—Token Vault holds the connected credentials and issues a short-lived, scoped access token to the agent or MCP server:

“A short-lived token. OAuth 2 has solved this for decades now, and I'm going to associate that with my user.”00:06:11

Evidence: transcript 00:06:11, 00:08:44

CIBA: verify outside the browser session

Creating an entry is reversible; cancelling or deleting a customer booking crosses into high-stakes territory. Even if someone sits in an authenticated browser tab, an agent should not rely on a plain chat “yes” for destructive mutations:

“I don't trust the human that's in the browser session.”00:12:49

Client-Initiated Backchannel Authentication (CIBA) routes the authorization step out of band—sending an email, SMS, or push notification directly from the identity provider while the app polls for confirmation before invoking the delete tool:

“Ciba client-initiated back-channel authentication. It's really just a fancy way of saying we're going to send you an email, push notification, or an SMS. To make sure that it is in fact you.”00:12:49

Evidence: transcript 00:12:49

Agent payments: HTTP 402 and hard spend caps

When an agent reaches checkout, both Stripe’s Machine Payments Protocol (MPP) and Coinbase’s x402 revive HTTP status code 402 to challenge an unauthenticated client for payment across cards or stablecoins:

“It just says you are not authenticated, just like a 401, because you owe me money.”00:15:56

Liendo ranks three ways to fund an agent—raw card numbers, capped virtual cards, and Stripe Shared Payment Tokens:

“You can share your card with the agent. Please don't do that.”00:15:56

Even with a Shared Payment Token, every call carries a required 100-character context string—and because web pages can hide malicious <div> prompts that turn a $12 shirt into a $15 charge, hard server-side maximum thresholds are non-negotiable:

“The context is also human generated, which means it is subject to spoofing.”00:15:56

“There's nothing stopping. That came from going through because you set a maximum threshold. That's super important.”00:15:56

• Retire static local keys: wrap external APIs behind Token Vault so agents receive narrow, short-lived OAuth tokens rather than raw credentials (00:00:45, 00:06:11).
• Decouple destructive approvals with CIBA: verify high-stakes mutations via out-of-band push, SMS, or email and poll before running the tool (00:12:49).
• Bound autonomous spend: pair Shared Payment Tokens over HTTP 402 with hard dollar caps that hold even if hidden HTML spoofs the agent’s context (00:15:56).

Evidence: transcript 00:15:56