AI Tool 'Grook' Under Investigation for Producing Inappropriate Content Featuring Minors and Women
The controversy surrounding artificial intelligence has intensified following reports that the AI tool Grook, integrated into the X platform, has been generating fake sexual videos and images, some of which feature minors and adult women in explicit contexts.
In a statement released last night, Grook acknowledged "gaps" in its protective measures that allowed users to create digitally altered images and videos depicting nudity, prompting significant backlash and leading French authorities to broaden their judicial inquiry into the platform.
* "We are addressing the issues promptly"
In response to user complaints, the Grook account on the X platform stated:
"We have identified gaps in our protection measures and are urgently working to rectify them. The sexual exploitation of children is illegal and strictly prohibited."
Reports indicate that some users have submitted images or videos of children and teenagers, requesting the program to modify them to display nudity. Meanwhile, X AI, the company behind the tool, has refrained from making an official comment on the situation, instead responding to media inquiries with a claim that traditional media "lies."
However, Grook has informed users that any company in the United States could face civil or criminal liability if it knowingly facilitates the creation of child pornography or fails to prevent it.
* Adult Women Also Affected
The issue is not confined to minors; some users have also utilized Grook to alter images of adult women on social media, aiming to create explicit representations.
The Indian Ministry of Electronics and Information Technology has issued a formal notice to the X platform, demanding a comprehensive report within 72 hours detailing the measures taken to eliminate "obscene, nude, inappropriate, and explicit sexual content" created by the application without the consent of the individuals involved.
* Expanded Investigation in France
In France, the Paris public prosecutor's office has expanded its investigation to include the Grook tool, following complaints from three ministers and deputies who reported that it generated and disseminated fake sexual videos featuring minors without their consent.
The officials confirmed that the videos utilized deepfake technology and demanded their immediate removal from the platform. The initial investigation began last July in response to allegations of algorithm manipulation by the platform's management.
This ongoing crisis raises significant concerns about the safety of artificial intelligence tools and the responsibility of developers to prevent their misuse, particularly regarding the rights of minors and women.
