AI Supply Chain Security: What Canary Tokens Found | Dragos - Dragos

2 Shares

HD Moore, Chief Research Officer, will be regularly sharing work from the research team he leads at Dragos. Topics will include xOT, IT, LLMs, PQC, and more. Their findings will sharpen defenders’ understanding of exposures and vulnerabilities emerging across the threat landscape.

We sent nearly two million AI prompts containing unique, trackable secrets to AI providers, then watched where those secrets surfaced. What we found revealed security breaches, prompt theft, and widespread failures to protect customer data.

Over a five-month period from July through November 2026, we submitted approximately two million prompts containing unique URLs and canary token credentials to widely used AI services and monitored what happened after inference. Our investigation uncovered three significant findings:

  1. Evidence of provider compromises. We observed activity indicating that data from at least three AI providers was exposed. Our seeded test prompts were used in attempts to access our controlled test systems and canary AWS accounts.
  2. Unauthorized access to customer prompts by inference marketplace providers. Multiple resellers operating through an AI inference marketplace were able to view and retain customer prompts using standard logging capabilities, without exploiting a vulnerability or bypassing platform controls.
  3. Failures in privacy and data-handling practices. We observed multiple cases in which providers appeared not to honor customer privacy settings or published policies. Examples included prompts submitted to U.S.-based providers being transmitted to Chinese AI organizations, unique URLs being accessed months after a single inference request, and one seeded link ultimately appearing in an internal company chat in China.

Following inference data

We wanted to understand what happens to inference data after it leaves the customer’s environment.

To answer that question, we created millions of unique canaries, including URLs and credentials that would never be used by legitimate systems. Any attempt to access those assets would indicate that someone, somewhere in the AI ecosystem had seen the original prompt.

The results exposed a fundamental disconnect between how AI services are marketed and how inference data is handled in practice.

Why does AI inference data matter?

Most folks picture AI inference as a question and a response. Text goes in, GPUs juggle some bits, and text comes back out. What complicates this is how the data is actually treated.

AI providers commonly store prompts and responses for operational, training, safety, abuse-monitoring, or business purposes. Depending on the provider, that data may be retained for days, months, or indefinitely. It may also traverse multiple organizations, including model developers, cloud operators, inference platforms, and third-party resellers.

The prompts and replies – the inference data itself – are often more valuable than the GPU resources used to process them. They frequently contain confidential source code, internal communications, strategic plans, customer information, credentials, and other sensitive business data.

How do AI routers expose your prompts?

AI routers work by offering a single API endpoint that forwards your request to a pool of available models, sometimes at different hosting providers, with unified billing. Routers are great at cost reduction and providing flexibility without managing multiple accounts and credentials. The path to the inference server often passes through more than one router – it’s common for requests from Vercel to then pass through OpenRouter, before reaching another provider, and from there the specific server hosting the target model. There is risk in this process – every hop can see the entire prompt and response.

Some routers are open marketplaces where the seller with the lowest price gets the request. It’s common for sellers to offer AI inference at deep discounts, and they can do this by pooling requests through unmetered consumer-plan subscriptions or by finding other low-cost paths to the specific model (startup credits, etc.). In the shadier marketplaces, it’s also common to see inference that is billed to stolen AWS credentials or consumer API keys. If a seller wants to siphon customer request data, they can list popular models at the lowest price, route the traffic, pay the difference, and then collect the data from the platform logs.

These providers offer privacy controls – customers can set privacy flags at the account level, for a specific API key, or in each request. The strictest setting is known as Zero Data Retention (ZDR), which specifies that the prompt and response are not stored. The routers should only forward these requests to providers that offer the same controls. The reality is quite different – providers often ignore these settings, either intentionally or due to bugs in their processing flow, and prompts not covered by ZDR are fair game for resale into a massive ecosystem of buyers and sellers.

Three compromised companies

The most serious finding was also the most direct. We found attackers actively stealing data from three different hosting providers. Although our prompts were bait, they were processed along with real customer requests, and those were almost certainly copied too. The earliest incident dates back to the middle of July, and none of the three providers knew about their breach until we notified them.

The largest breach involved a couple hundred of our AWS keys. Each key was sent in exactly one request to one provider, all within a few short windows in late October, and every request we sent during those windows was stolen. Some of the stolen keys came from requests that the provider rejected before a model ever ran – this tells us that the attacker was copying requests at the provider’s front door, and not somewhere deeper in the inference stack. In the other two breaches, some of the stolen requests were marked as ZDR. A privacy flag doesn’t help when the attacker is already inside.

Once the attacker had our credentials, they followed a predictable path. First, they tested each key to see if it was live, in some cases within a few hours of our request. Next, they went looking for data, listing S3 buckets and searching Secrets Manager and Parameter Store. Finally, they tried enabling hosted models in AWS Bedrock, creating Bedrock API keys, and adding their own IAM users to keep access. In total, our bait keys were used for tens of thousands of AWS API calls.

Figure 1. One stolen AWS key. We seeded it on October 21, it was first used the next day, and it was still in use three weeks later. Names are pseudonyms and addresses are partly masked.

One adversary appears to have compromised two of these providers at the same time. Secrets that we sent to those two providers were used together, almost immediately, from a single rented server. The attacker’s tooling changed over the three weeks that we watched it – it started with a script pretending to be Chrome on Windows, moved to Boto3 running inside OrbStack, and later switched to the AWS CLI on Kali Linux.

On November 1, the same attacker added ARTEX, an LLM-driven penetration testing framework. In our case, ARTEX did very little – it made a handful of calls to confirm that keys the attacker already held were still live. Two of the user agents it sent, artex-agent and artex-judge-r121, don’t appear in any public indicator list that we could find, and are worth adding to your detections.

Figure 2. Tools that were used against our planted secrets, including the AI pentest agent (first row) and the Kali environment (aws-cli row).

The good news for defenders is that these tools tend to announce themselves – both Kali and ARTEX showed up in the user agent that AWS records in CloudTrail. The bad news is that this self-identification was only present in a small fraction of traffic.

Marketplace resellers stole customer prompts

The second finding didn’t require an exploit at all. The marketplace sellers that we caught only had to list a model at the lowest price, enable request logging, and wait. What stood out is how personal it was – these were individuals, working from what looked like home internet connections and personal computers, trying to break into our canary servers one login at a time.

In one case, we sent two prompts through a single seller’s offer. About a day and a half later, someone on a home broadband connection opened both secret URLs in a desktop browser and started guessing usernames at the login page. Prompts for the same model that went through a different seller were never touched, which tells us that the leak follows the seller and not the marketplace as a whole.

In another case, an attacker working from a Mac on a home broadband line in France took the passwords from two of our prompts and tried to log in as root over SSH, typing each attempt by hand. The same Mac later tested several of our AWS keys in a single session.

Figure 3. One of the root SSH attempts. The password from our prompt was used four times in two minutes, and the route trace shows the two routers that the prompt passed through.

Most of those keys had been sent through the marketplace, but one had gone to a completely different router. It turned out that this router was quietly forwarding some of its traffic to the marketplace – another example of an undisclosed hop.

Privacy settings and retention policies were ignored

The third finding is different from the first two, as nobody used a credential. Instead, the secret URLs in our prompts were scraped, repeatedly, and the git repositories they referenced were targeted for cloning – all by companies that were never supposed to see the prompt in the first place, and often in direct conflict with the privacy settings on the request.

The clearest example involves a Chinese AI lab. This lab’s systems fetched the secret URLs from more than half of the prompts that we sent to its models between July and October, including roughly two-thirds of the requests marked as “no data collection” or Zero Data Retention. The scraper typically showed up two or three days after the request.

This wasn’t limited to requests sent to the lab directly. Several U.S. hosting providers resell this lab’s models, and prompts sent to those providers triggered the exact same scraper – the prompts were flowing back to the lab. A customer who picked a U.S. provider specifically to keep their data in the U.S. didn’t get what they paid for. Later on, a git client on a different cloud started trying to clone the private repositories referenced in our prompts, but only for prompts that used this lab’s models.

The large U.S. providers have their own issues. One continued to crawl a secret URL more than two months after we used it in a single inference call to their search endpoint. Another runs a scraper that comes back for our URLs days after each request, regardless of which router we used to reach it.

Figure 4. One day of incidents, grouped by source network. The same scraper networks show up against prompts that we sent through several different routers.

The strangest incident involved a company chat. Twenty-four days after we sent a prompt to one Chinese provider, the secret link from that prompt was posted into a Feishu chat channel (Feishu is known as Lark outside of China). Within two minutes, the link was opened from three desktops and three phones in Beijing and Shanghai. Our disclosure to this provider only included the first eight characters of that link, so whoever shared it was reading from a stored copy of the full prompt, more than three weeks after the request was sent to a U.S.-based provider with the ZDR flag.

Figure 5. The secret link shared in a chat and then opened in the mobile app. The git client is the earlier clone attempt.

Zero Data Retention is supposed to mean something. In practice, these prompts were retained, shared with other companies, and read by people.

How do canary tokens detect prompt leaks?

A canary token is a secret that only exists in one place – if anyone ever uses it, you know that place has been exposed. We started in July by seeding prompts with Thinkst Canarytokens, and every request carried an AWS key, a DNS name, and a URL that had never been seen anywhere else. Tokens started firing within days, and the tokens from July are still firing today.

The proof-of-concept worked so well that it made sense to build a dedicated service just for seeding LLM prompts. This service generates a unique seed, and uses it to derive unique hostnames, URLs, and credentials for every prompt and then runs the services that those credentials point to – DNS, web, git, SSH, databases, mail, and a long tail of older protocols. The AWS keys are real keys tied to empty accounts with full logging. Every request that touches a token is tied back to the exact prompt, model, router, and privacy setting that carried it. Since each unique component can be tied back to its parent seed, we’re able to get precise timing and sequencing; we know what DNS server they use, how long it took to reply, and every piece of the inbound connections (TCP, TLS SNI, HTTP Host headers, URLs, credentials etc.).

Figure 6. The console overview, showing incidents and seeded prompts per day, split by the router that handled the prompt.

Canaries tell us that a prompt leaked, but not where in the chain, since a router typically reports a single upstream provider, and that provider is often just another router. To figure out who was really handling our requests, we built something like traceroute for AI inference. The first part is simple – we include an image URL in the request that points back to our servers, and every system that fetches it gives away an IP address, an HTTP client, and a timestamp.

The second part takes advantage of how these services are stacked. We send small, slightly broken requests, and since each router tends to wrap the error from the hop behind it in its own format, a single response can contain three or four nested “envelopes.” Matching each layer against a signature database gives us the list of hops, the software at each one, the network they use, and often the company operating it.

So far, we have traced about two thousand routes. Several hundred of those included a system that we couldn’t identify, and a handful sent our image to a company that wasn’t on the advertised path at all. The traces also turned up middle layers that the route never mentions – one marketplace seller quietly adds about a thousand hidden tokens to every prompt, and another fetches images through a shared consumer subscription account.

Figure 7. An example of LLM route tracing, showing a path from SurplusIntelligence to Anthropic. The OpenRouter hop was discovered by tracing and was not disclosed in the marketplace

What we did about it

Every incident is stored with its evidence – the original prompt, the route it took, every connection that followed, and our notes. The first incident was mid-July; the provider first thought this was a misconfiguration of internal tooling, then a problem with partner data processing, until the investigation uncovered evidence that the attacker had compromised their internal credentials too. This provider has still not publicly disclosed the incident and the stolen data continued to lead to attempted logins as of late October.

For the second batch of intrusions in late October, we reached out as soon as we confirmed that the data provided evidence of third-party access, and one of these providers has already published a preliminary security notice.

For the less severe issues like data sharing (but not credential abuse), we started notifying providers in early November. Each report included the list of stolen requests along with unique strings that they could search for in their own logs. We are now working with about a dozen providers and wouldn’t be surprised to see this expand.

Reactions have been mixed. The router that was forwarding traffic to the marketplace told us that they stopped, and several providers have responded and started their own investigations. On the other hand, stolen keys and secret URLs continue to be accessed, even months after the initial incident.

How can you protect your AI supply chain?

None of this means that you should stop using AI services. It does mean that where your prompts go is a decision, and if you don’t make it, the lowest price will make it for you. Keep in mind that a privacy flag only binds the first hop, and that AI agents will happily send whatever they can read, including credentials.

  1. Host it yourself, if you can afford it. This is the only option where nobody else gets to read your prompts, but the hardware isn’t cheap. A Mac Studio with 512 GB of memory starts at $22,299, NVIDIA’s DGX Spark now costs $6,950, a DGX Station starts at about $105,000, and an eight-GPU B300 server lists for $570,000 or more. For most teams, this only makes sense for the most sensitive work.
  2. Go directly to a first-tier provider. One company, one contract, and one privacy policy to read. This can still leak through data sharing agreements or a breach at the provider, but it is the smallest exposure you can buy.
  3. Use routers with care. Pick the upstream providers yourself instead of accepting the default pool, and verify that your privacy settings actually change where your requests go. Stay away from reseller marketplaces – some of that capacity comes from stolen keys, and a few of these services only accept cryptocurrency for a reason.
  4. Seed your own canaries. Add a fake credential to the context that your agents and pipelines already send and treat any use of it as an incident. A free Canarytoken from Thinkst is enough to get started today.
  5. Enable secret filtering. Some providers offer settings that will automatically scrub known credential formats from prompt data. This can’t hurt, but it’s not a panacea – there is plenty of sensitive data in prompts that isn’t shaped like an API key.

This research effort felt a lot like what we do at Dragos with runZero; using small bits of data to provide a comprehensive view of networks from multiple perspectives. Would you like that kind of visibility into your own environment?

Start a runZero trial

PS. If you found this interesting, check out Lauren Proehl’s post, I leaked credentials on purpose for four months. Here’s who took them, and what they did next.

Read the whole story
jlvanderzwan
8 hours ago
reply
acdha
12 hours ago
reply
Washington, DC
Share this story
Delete

Animation That Isn’t Behind Glass

1 Comment
A still from The Dover Boys (1942)

Welcome! It’s another Thursday issue of the Animation Obsessive newsletter. This one’s about the public domain.

Recently, we got messaged by a subscriber in Brazil — they were starting a film club and searching for public domain animation to show. Copyright law differs from place to place; what’s free in the States doesn’t necessarily line up elsewhere. Reading about the Brazilian public domain made that particularly clear.

Interesting films are available everywhere, though, with a little digging.

Take The Dover Boys, the great ‘40s cartoon. It fell out of copyright in many countries thanks to paperwork technicalities. Since even restored editions of public domain films are free (with some exceptions), there’s a pristine copy of Dover Boys on Wikipedia, ripped from a Looney Tunes Blu-ray.1

That goes also for Disney films from the late ‘20s, like Steamboat Willie and Skeleton Dance, and for the classic Adventures of Prince Achmed. All were ripped and uploaded to Wikipedia.

Still, while this work is public domain in the States and other places, it isn’t open universally. The really old stuff is more guaranteed. Animators like Winsor McCay and Émile Cohl had their time over a century ago and have faded into history. Which makes their films free, albeit not very popular.

Yet there are reasons to take a second look, and not just educational ones.

Right now, we’re trained to see animation like Spirited Away and Snow White from a distance, as observers. They obviously can’t be touched. But public domain films, as with books like The Brothers Karamazov and Emma and Dracula, are shared culture now. You can grapple with them hands-on, screen, edit, steal from and adapt them.

Some of that early animation remains really, really good. It’s not widely known today, but it’s alive, and it’s finally outside the glass case.

A snippet from A Love Affair in Toyland (1908), watchable here

Read more

Read the whole story
jlvanderzwan
8 hours ago
reply
A pristine copy of Dover Boys on Wikipedia? I'LL DOWNLOAD IT!! NO-ON WILL EVER KNOW!!!!
Share this story
Delete

1353: Apart Together

1 Comment
http://www.giantitp.com/comics/oots1353.html
Read the whole story
jlvanderzwan
1 day ago
reply
In other comics this would be a death flag. Here the whole concept of "death flag" is kinda silly
Share this story
Delete

Rule Of Threes

1 Comment

Hello, I have a new album of music out. If you like 15 minute long instrumental post-metal songs played on guitars with too many strings, you might enjoy it. You can listen to it and buy it here. Thank you

Read the whole story
jlvanderzwan
2 days ago
reply
Yeah this is clearly ace flirting
Share this story
Delete

Ala Ebtekar, Thirty-six Views of the Moon (Winter Edition)

1 Share
From: smarthistory-art-history
Duration: 4:43
Views: 3,814

Ala Ebtekar, Thirty-six Views of the Moon (Winter Edition), 2018, 59 cyanotypes on found book pages exposed to moonlight, 238.8 x 335.3 cm (Crystal Bridges Museum of American Art, Bentonville) © Ala Ebtekar

speakers: Victor Gomez, Assistant Curator, Contemporary Art and Steven Zucker, Smarthistory

Read the whole story
jlvanderzwan
4 days ago
reply
Share this story
Delete

Around The World, Part 35: Sailing

1 Comment

After all the procedural generation and visuals, we’re finally getting into game design and gameplay territory! And none too soon.

In Around The World, the player sails a fleet of ships across the sea. This sailing forms the innermost “game loop” if you will, so it’s important to get it right. How exactly does the player control the fleet? Bear in mind that the game uses a third-person “chase” camera hovering behind the player’s flagship.

In the prototype, it was a simple point-and-click interface as you’d see in RTS games: the player clicks, and the ship sails in a straight line to the clicked point. The prototype had no pathing logic; if the ship hit land, it would simply stop. This works well for a top-down 2D game, and could be done in 3D as well.

In fact, this is how I originally implemented it: just right-click on a point (land or sea) and the ship will sail towards it… but with some subtleties because life at sea is never easy.

If you click on land, the ship will keep its bow pointed at that point and keep going until it has been reached.

If you instead click on water, the ship will sail to that point but continue on past it, letting you set a course into the open sea without having to keep clicking. In fact it will maintain a fixed compass bearing, but due to wind and currents this might not be the actual direction of movement! Information from charts can help you compensate for this, giving charts another use besides just finding the next port.

But what if there’s an island or other landmass in between? Would you need to click-click-click throughout the route to plot a course around it? That would be tedious. Rather, the ship should figure this out on its own using a pathfinding algorithm. I had this implemented, but it never felt quite right, not to mention that limitations (bugs?) in Godot’s navigation system caused the ship to behave weirdly and sail away from the goal sometimes.

The problem is that this approach, especially with automatic pathfinding, gives the player little control over how the ship sails there. What if there are obstacles along the way? One gameplay mechanic I would like to support is underwater hazards like reefs, sandbars and rocks; since these are hard to spot visually, you would need to use charts to avoid them. So maybe a more direct control scheme would be better.

Direct control

The obvious candidate for more direct control would be WASD keys, like in a first or third person shooter. (Personally, I prefer ESDF on qwerty layouts and .OEU on dvorak, but that’s a story for another time.) I quickly implemented this, and it just felt much more immersive and powerful than point-and-click control. This was a keeper!

But a ship is not a main character in a third-person shooter. For one, if you do nothing, a ship will keep moving whereas a person will stop. So there’s one important difference in the control scheme: you don’t have to hold down the up/forwards key to keep sailing; you just press it once to raise your sails, and off you go.

Another difference is that a sailing ship is driven by wind. I’m not yet changing its speed depending on the current wind direction, but that is definitely going to happen in a future update. For now, let’s at least make it look like the wind is propelling the ship:

  • The sails move down when you start, and back up when you stop.
  • The sails billow in the wind.
  • The yard arms and attached sails turn depending on the wind and sailing direction.
  • The ship leans over as if pushed by the wind.

There’s a video at the end, but here’s a screenshot of how that looks, with the wind coming from the right:

A ship leaning over in the wind, with sails set diagonally

Wake

It had been bothering me for a long time that the ship doesn’t really look “grounded” in the water; it sort of seems to sit on top. On solid ground, there would be a contact shadow, but water rendering works differently. There is a reflection, which does help, but I can’t dial it up too far because the rest of the water starts to look strange.

So I figured I’d try something else to make it look like the ship is interacting with the water, and make it more “juicy” overall: a wake. It consists of two parts:

  1. The Kelvin wake which is the classic V-shaped “bow wave”. Fun fact: the angle of the V does not depend on the ship’s speed and is always 39°.
  2. The turbulent wake which is just a foam trail behind the ship. You always see these on motorized ships due to the churn of the propellor, and I’m not sure how appropriate it is to draw them behind a sailing ship, but it looks good so there you go.

Rendering these was easier said than done, since they have to be added into the existing water rendering. I ended up drawing both wakes as particles; the Kelvin wake as rings, the turbulent wake as pixelated sprites:

Ship followed by wake particles

These are not visible in the regular viewport, but are instead rendered by a separate, orthographic camera that follows the ship and looks straight down. The result is rendered into a texture that looks like this:

Wake texture

The colour channels of this texture have a special meaning, which is also why the base colour has to be such an ugly olive (#808000):

  • The red and green channels contain the x and y coordinates of the normal map. The Kelvin wake particles additively draw to these channels.
  • The blue channel contains the amount of foam. The turbulent wake particles draw to this channel.

The water shader then picks up this texture and applies it on top of the regular normal map and foam calculations. The result is a nice looking wake behind the ship:

Wake in the game

The fact that I’m using an intermediate texture of finite size means that wakes can’t be rendered everywhere; they will only be visible close to the player’s ship. But since distant water is not clearly visible anyway, this should not be a problem.

Warping time

Even though I’ve made the game world significantly smaller than the real world, it’s still a big place: 512 by 256 kilometers. A typical distance between two ports is about 10 kilometers, which cannot be shrunk much further without the world starting to feel small. It’ll be a significant game design challenge to keep travel time under control, and make travel not boring.

I’ve already made the player’s ship sail at Ludicrous Speed™: 50 m/s, which is 180 km/h, 112 mph, 97 knots. Any faster and it starts to feel silly. The speed is more or less correct because time passes 60× as fast in the game world, so the actual travel speed is about 3 km/h, which is quite slow for a carrack but at least the right order of magnitude.

However, even at this exaggerated speed, it still takes over three minutes to sail from one port to another. This is fine for a while, but started to get tedious even during my own testing. So I finally caved in, and added a time warp feature:

Buttons with icons for pause, one arrow, two arrows, three arrows

These buttons sit in the top left corner and allow you to speed up time by a factor of 2× or 4×. There’s also a pause button now, which took some messing around in Godot because not everything should be paused; for example, it’s rather important that the pause button itself keeps working.

With these done, I noticed that my day/night cycle was starting to bother me. When using 4× time warp, a full day lasts just 6 minutes, of which about half is daylight, so you quickly run into sunset and a potentially very dark night. This is intentional: on the open sea, depending on how certain you are that there is no land ahead, you might want to take a slight risk and save time by sailing through the night.

Near the coast, on the other hand, you’ll probably want to wait for dawn until you continue. So I added a button for that as well:

Button labelled “Wait for sunrise”

Working out when the sun rises took some gnarly math, and still isn’t entirely accurate, but close enough to be useful.

Interacting with ports

Initially, I had the buildings of each port placed in a semicircle, whose centre was on the coast. The other half of that circle was where the player would need to sail to interact with the port. It worked, but it was never quite clear where you needed to go and how close you would have to get. Worse, the circle’s centre would sometimes be placed a bit inland, making the part on water quite small, or placed at sea, making almost no buildings show up.

I finally fixed this by making ports consist of two locations, straddling the actual coastline wherever that coastline ends up being generated:

  1. The point around which buildings are placed. This is on a grid point in the global map, for which we can guarantee that it’s on land.
  2. The point that the player will interact with. This is also on a grid point in the global map, but this point is at sea, adjacent to the buildings point.

The interaction point is marked with a small buoy, surrounded by a halo to indicate how close you have to get. The halo does break realism, but I think this is warranted for gameplay clarity. Here’s what it looks like:

A port on the coast, with a buoy floating in front surrounded with a green halo. Just inside the halo, a ship.

And that’s it for sailing mechanics.

Oh, I promised a video, right? Here you go.

Notice the wavelets behind the ship, the turning of the sails, and the lean of the ship when the wind blows from the side. I think it’s all starting to come together.

Read the whole story
jlvanderzwan
4 days ago
reply
BTW Thomas, have you considered switching to WebP over JPG? I suspect it would handle the crispy graphics and smooth gradients of your game a lot better at the same compression ratio.
Share this story
Delete
Next Page of Stories