Certain names bring ChatGPT to a grinding halt, and we know why

In particular, the “David Mayer” block (now resolved) raises additional questions, first raised on Reddit on November 26, as several people share that name. Reddit users speculated about ties to David Mayer de Rothschild, although no evidence supports these theories.

The problems with hard-coded filters

Allowing a particular name or phrase to always interrupt ChatGPT outputs can cause major problems for certain ChatGPT users, leaving them vulnerable to adversarial attacks and limiting the usefulness of the system.

Scale AI Prompt engineer Riley Goodside has already figured out how an attacker could interrupt a ChatGPT session by embedding a visual input of the name “David Mayer” in a light, barely readable font into an image. When ChatGPT sees the image (in this case a mathematical equation) it stops, but the user may not understand why.

The filter also means that ChatGPT is unlikely to be able to answer questions about this article when you browse the Internet, for example using ChatGPT to search. Someone could use this to potentially prevent ChatGPT from intentionally crawling and processing a website if they add a banned name to the website’s text.

And then there’s the inconvenience factor. If ChatGPT is blocked from mentioning or processing certain names like “David Mayer,” which is likely a popular name shared by hundreds if not thousands of people, that means people who share that name will have a much harder time using ChatGPT. Or, for example, if you are a teacher and you have a student named David Mayer and you need help sorting a class list, ChatGPT would reject the task.

AI assistants, LLMs and chatbots are still in their infancy. Their use has opened up numerous opportunities and vulnerabilities that people are still exploring every day. How OpenAI could solve these problems is still an open question.