Microsoft’s AI text-to-image generator, Copilot Designer, appears to be heavily filtering outputs after a Microsoft engineer, Shane Jones, warned that Microsoft has ignored warnings that the tool randomly creates violent and sexual imagery, CNBC reported.
Jones told CNBC that he repeatedly warned Microsoft of the alarming content he saw while volunteering to test the tool’s vulnerabilities in red-teaming efforts.
In response, Microsoft failed to take the tool down, implement safeguards, or even post disclosures to change the product’s rating to mature in the Android store, Jones said.
Instead, Microsoft apparently did nothing but refer him to report the issue to OpenAI, the maker of the DALL-E model that fuels Copilot Designer’s outputs.
OpenAI never responded, Jones said, so he took increasingly more drastic steps to alert the public to issues he found in Microsoft’s tool.
He started by posting an open letter, calling out OpenAI on LinkedIn.
Then, when Microsoft’s legal team told him to take it down, he did as he was told, but he also sent letters to lawmakers and other stakeholders, raising red flags in every direction.
That includes letters sent today to the Federal Trade Commission and to Microsoft’s board of directors, CNBC reported.
In Jones’ letter to FTC Chair Lina Khan, Jones said that Microsoft and OpenAI have been aware of these issues since at least October and will “continue to market the product to ‘Anyone. Anywhere. Any Device'” unless the FTC intervenes.
Bloomberg also reviewed Jones’ letter and reported that Jones told the FTC that while Copilot Designer is currently marketed as safe for kids, it’s randomly generating an “inappropriate, sexually objectified image of a woman in some of the pictures it creates.”
And it can also be used to generate “harmful content in a variety of other categories, including: political bias, underage drinking, and drug use, misuse of corporate trademarks and copyrights, conspiracy theories, and religion, to name a few.”
In a separate letter, Jones also implored Microsoft’s board to investigate Microsoft’s AI decision-making and conduct “an independent review of Microsoft’s responsible AI incident reporting processes.”
This is necessary after Jones took “extraordinary efforts to try to raise this issue internally,” including reporting directly to both Microsoft’s Office of Responsible AI and “senior management responsible for Copilot Designer,” CNBC reported.
A Microsoft spokesperson did not confirm whether Microsoft is currently taking steps to filter images, but Ars’ attempt to replicate prompts shared by Jones generated error messages. Instead, a Microsoft spokesperson would only share the same statement provided to CNBC:
We are committed to addressing any and all concerns employees have in accordance with our company policies and appreciate the employee’s effort in studying and testing our latest technology to further enhance its safety. When it comes to safety bypasses or concerns that could have a potential impact on our services or our partners, we have established in-product user feedback tools and robust internal reporting channels to properly investigate, prioritize and remediate any issues, which we recommended that the employee utilize so we could appropriately validate and test his concerns. We have also facilitated meetings with product leadership and our Office of Responsible AI to review these reports and are continuously incorporating this feedback to strengthen our existing safety systems to provide a safe and positive experience for everyone.
OpenAI did not respond to Ars’ request to comment.