Skip to content
Main Site News Console

How We Took Over ChatGPT Accounts

· Simon Willison Translated
技巧LLM

How We Took Over ChatGPT Accounts (Source): Parameter Labs disclosed a vulnerability they discovered in ChatGPT’s implementation of the “Artifacts pattern” (previously referred to by OpenAI as “Canvas,” though the latest revision appears to be called HTML previews). In this feature, interactive custom HTML applications are rendered inside a sandboxed iframe.

Strictly sandboxing the HTML in these applications is critical to prevent untrusted code from interacting with OpenAI’s APIs and accessing private user data.

Parameter uncovered a fairly sophisticated attack chain. OpenAI used the MathLive JavaScript library for its charting support, which includes its own virtual keyboard that the sandboxed iframe could trigger in the parent page via postMessage().

The code receiving that message used innerHTML, allowing any message sender to insert custom HTML into the parent page—and at the time, nothing prevented other sandboxed frames, including the HTML preview frame, from invoking this method.

This meant untrusted code inside the iframe could inject HTML. While innerHTML does not execute script elements directly, it could be used to inject a new iframe without sandbox restrictions.

OpenAI’s Content Security Policy (CSP) rules completely blocked inline scripts and only permitted loading scripts from a handful of domains. However, one of those domains was YouTube—and YouTube’s oEmbed endpoint could be abused to reflect back and execute JavaScript.

Once an attacker achieved JavaScript execution on ChatGPT, fetching the user’s private data via the API and exfiltrating it through an image request became trivial.

The end result: an attacker could craft a shared ChatGPT link such that if any logged-in user visited it, their private data could be stolen, and the attacker could even perform actions on that user’s behalf.

OpenAI resolved the vulnerability and paid a $5,000 bug bounty—an amount that feels rather low for an exploit of this severity.