(AP) The popular artificial intelligence image-generator Midjourney has started blocking its users from creating fake images of President Joe Biden and former President Donald Trump ahead of the upcoming U.S. presidential election, according to tests of the AI tool by The Associated Press.
With the election in full swing, it’s time to “put some foots down on election-related stuff for a bit,” Midjourney CEO David Holz told several hundred members of the service’s devoted userbase in a digital office hours event Wednesday.
Declaring that “this moderation stuff is kind of hard,” Holz didn’t outline exactly what policy changes were being made but described the clampdown as a temporary measure to make it harder for people to abuse the tool. The company didn’t immediately respond to a request for comment Wednesday.
Attempts by AP journalists to test Midjourney’s new policy on Wednesday by asking it to make an image of “Trump and Biden shaking hands at the beach” led to a “Banned Prompt Detected” warning. A second attempt escalated the warning to: “You have triggered an abuse alert.”
The tiny company — which has just 11 employees, according to its website — has largely kept silent in the public debate over how generative AI tools could fuel election misinformation around the world. Midjourney was the only maker of a leading image-generating tool that didn’t join a voluntary tech industry pact in February to combat AI-generated deepfakes that deliberately trick voters.
“I don’t really care about political speech,” Holz said Wednesday. “That’s not the purpose of Midjourney. It’s not that interesting to me. That said, I also don’t want to spend all of my time trying to police political speech. So we’re going to have put our foot down on it a bit.”
The Center for Countering Digital Hate released a report earlier this month that concluded Midjourney is already being used to produce images that could support disinformation about political candidates or false claims of election fraud.
“Midjourney seemed to have the fewest controls of any AI image-generator when it came to generating images of well-known political figures like Joe Biden and Donald Trump,” said Callum Hood, the group’s head of research, in an interview Wednesday. “Midjourney was almost unique in both being willing to generate those images and generating quite convincing images of candidates.”
The watchdog group tested Midjourney as well as OpenAI’s ChatGPT Plus, Stability AI’s DreamStudio and Microsoft’s Image Creator, and found all had problems, creating election disinformation in 41% of cases. But “Midjourney performed worst of any tool, failing in 65% of test runs,” the report said.
Hood said he wasn’t aware if Midjourney changed its policies this week but “its decision to block those images would help seal a major weakness compared to other popular image generators.”
As recently as late last week, Midjourney’s public database shows users were successfully able to produce images based on prompts such as “Abstract photo of Donald Trump sitting in a tiny electric car” and “joe biden eating ice cream.”
But that changed in recent days as users reported they received a warning when trying to make Biden or Trump images. Some complained that Midjourney wasn’t properly communicating the changed policy.
Midjourney differs from other image-generators in that it is possible for users to see most other users’ written prompts and images because it all happens in public forums on the social media platform Discord. Midjourney’s Discord group has more than 19 million members, the largest of any group on the chatting platform. It’s where Holz holds his weekly office hours, which can veer into wide-ranging conversations about AI and the future of humanity.
“Anybody who’s scared about fake images in 2024 is going to have a hard 2028,” Holz said Wednesday. “It will be a very different world at that point. Like, obviously you’re still going to have humans running for president in 2028, but they won’t be purely human anymore.”
All the candidates at that time will have real-looking “deepfake chatbots” with simulated talking points, he said. Holz warned Wednesday that people who really want to make deepfakes will find alternatives that “can be fine tuned on specific people and will work better than our systems.”
Based near San Francisco and founded in 2020, the company describes itself as an “independent research lab exploring new mediums of thought and expanding the imaginative powers of the human species.” It first unveiled the public version of its image-generator in July 2022. That — and the release of a key rival, Stable Diffusion, later that summer — sparked growing fascination with generative AI technology that intensified with the debut of ChatGPT several months later.
Holz told The Associated Press later that year that Midjourney works to ban offensive and harmful content, usually by blocking certain keywords and having its team of moderators track outputs and respond to complaints from other users.
“We really try to let people make the broadest set of images possible but we do ban people every day from the service,” Holz said in late 2022.