TL;DR - We scanned 70,806 vibe-coded websites and 39,579 of the internet's most popular sites, downloaded their JavaScript, and hunted for live secrets. Vibe-coded sites exposed 12.3x more live secrets per file in genuinely sensitive, verified-live secrets - API keys, cloud credentials, payment tokens, and more. The reason is simple: AI agents love reading your secrets, and they're far more willing than human developers to paste them somewhere they don't belong. Read-only tokens won't save you, once an agent can read a secret, it can leak it. The only real fix is stopping secrets from reaching the model's context in the first place, through runtime control.
Our journey starts with a concerning, recurring pattern: AI agents were constantly finding creative ways to try to read users' secrets throughout their sessions. As security researchers, hearing that an AI agent is using your AWS token with no restrictions truly makes your skin crawl.
"But the token exposed to my agent provides read-only access to the environment! I'm safe!"
Well, are you?
While read-only access is best practice, there are still risks in giving an AI agent a token - specifically, leakage of that token. We set out to discover how often AI agents leak sensitive data on the internet, and whether they're truly trigger-happy secret-sharers compared to human developers.
Vibe-coding sites are the perfect place to start. These sites are, by definition, developed (at least in part) by AI, and many of them have features that require authentication for any number of the backend services they depend on.
Introducing our vibe-coding roster: we scanned 70,806 websites created by the most popular vibe-coding platforms and tools, such as Lovable, Base44, Replit, and more.
We attempted to access each one, and for every accessible site we downloaded its JavaScript files and scanned them for secrets. To determine whether vibe-coded websites leak more secrets than others, we took a list of the most popular domains, downloaded their JavaScript files, and scanned those for secrets as well. In the end, we compared 70,806 vibe-coded websites against 39,579 of the most popular domains.
Breaking Down the Results
In total, we scanned 70,806 vibe-coded websites and 39,579 top domains. From these, we extracted 62,082 unique JS files from the vibe-coded sites and 261,895 from the top-domain sites - top domains ship far more JavaScript per page. ("Similar" files were not counted as unique; only truly unique content was.)
When scanning the JavaScript, we found 545 candidate secrets on the vibe-coded sites and 4,379 on the top domains - but most of the top-domain hits were unverifiable noise. Only 251 (5.7%) were confirmed live, versus 346 of 545 (63.5%) on the vibe-coded sites. After de-duplicating identical values, that left 147 distinct verified secrets on vibe-coded sites and 112 on top domains.
Before comparing further, we took each verified secret and (in a non-intrusive manner) assessed its permissions and scope, splitting every verified secret into two buckets: genuinely sensitive secrets and public-by-design secrets. This left us with 138 genuinely sensitive secrets on vibe-coded sites versus 42 on top domains.
The theory that AI agents leak more secrets holds. We found 138 distinct sensitive secrets on vibe-coded websites (247 in total - agents placed the same keys multiple times across files) versus 42 on non-vibe-coded websites. In total, 180 vibe-coded websites were leaking a sensitive secret, compared to only 38 top-domain sites.
So 1 in 393 vibe-coded websites leaks a sensitive secret, compared to 1 in 1,042 top-domain sites - a vibe-coded site is 2.65x more likely to leak a sensitive secret.
The significance becomes clear when you look at leakage per unit of JavaScript. On a vibe-coded site, 1 in 251 JS files contains a sensitive secret, versus 1 in 3,081 on top domains. Vibe-coded JavaScript is 12.3x denser in sensitive secrets.
These are the types of sensitive secrets we found across the websites:
Some secrets on the non-vibe-coded side are probably safe and intended to be exposed, but we couldn't verify this by checking their permission scope without performing potentially intrusive actions - so we didn't.
Secrets & AI: A Toxic Relationship
So, letting your AI agent read your secrets is a bad idea. It's very difficult to guarantee complete protection against this scenario: even if you block an extensive list of commands in your agent's settings, there's almost always a workaround the agent can use.
A viable solution is to identify secrets before they ever reach the LLM's context. This has to be done through runtime control for AI agents.
Stay alert, and vibe(-code) safely!