The firm detected more than 100,000 prompts believed to be attempts at extracting its proprietary reasoning capabilities.
According to a quarterly threat report from Google Threat Intelligence Group, Google successfully identified and thwarted a campaign comprising over 100,000 prompts, which were allegedly crafted to duplicate the proprietary reasoning features of its Gemini AI model.
These prompts appeared to be part of a concerted effort for model extraction or distillation, a machine-learning technique used to develop a smaller model that replicates the core characteristics of a significantly larger one. Google’s systems intercepted these prompts in real-time, thereby “mitigating the risk of this specific attack and safeguarding internal reasoning traces,” as stated by the company.
Google is determined to stop rivals from leveraging its significant investment in AI model development to train their own systems, even as it must ensure users can access the models driving its services.
“Model extraction and the resulting knowledge distillation allow an adversary to rapidly advance AI model development at a considerably reduced expense,” Google noted in the report. “This type of action essentially constitutes intellectual property theft.”
During the campaign Google uncovered, attackers directed Gemini to ensure “the language employed in the thinking content remained strictly aligned with the primary language of the user input”—a method Google indicated is designed to extract the model’s reasoning across various languages. The company’s report stated, “The wide range of questions implies an effort to mimic Gemini’s reasoning capacity in non-English target languages for numerous tasks.”
Google reported discovering numerous model extraction attempts globally from private sector organizations and researchers aiming to duplicate proprietary AI capabilities. The company emphasized that such attacks breach its terms of service and could lead to takedowns and legal proceedings.
Nevertheless, researchers and prospective clients may wish to acquire substantial samples of Gemini’s reasoning for alternative, legitimate objectives, such as comparing model performance or assessing its appropriateness and dependability for a specific task prior to acquisition.
AI Model Providers Face Increasing IP Theft Risk
Google is not alone in identifying what it suspects are malicious attempts at model extraction within its logs. This past Thursday, OpenAI informed US lawmakers that the Chinese AI company DeepSeek has utilized “new, concealed methods” to extract outputs from prominent American AI models for training its own systems, as per a memo seen by Bloomberg. In the memo, OpenAI accused DeepSeek of attempting to “exploit the capabilities developed by OpenAI and other US frontier labs,” underscoring how model theft has become a significant concern for companies that have poured billions into AI development.
Ross Filipek, CISO at Corsica Technologies, perceives a shift in cybersecurity threat landscape underlying these allegations. He commented, “Adversaries conducting model-extraction attacks underscore a change in attack motivations.” He added, “Model extraction does not involve traditional system infiltration; instead, it focuses on transferring the knowledge derived from the victim’s AI model to fast-track the development of the attackers’ own AI models.”
The report indicates that the risk of intellectual property theft via model extraction should concern any organization offering AI models as services. Google advised that such organizations ought to scrutinize API access patterns for indications of systematic extraction.
Filipek stated that countering these attacks necessitates rigorous governance over AI systems and vigilant oversight of data flows. He advised, “Organizations ought to deploy response filtering and output controls, which can deter attackers from discerning model behavior should a breach occur.”
State-Sponsored Actors Utilized Gemini to Expedite Offensive Operations
Google views itself not merely as a potential target of AI cybercrime, but also as an inadvertent facilitator. Its report detailed how state-sponsored threat actors from China, Iran, North Korea, and Russia incorporated Gemini into their operations in late 2025. The company confirmed it deactivated accounts and resources linked to these entities.
The Iranian threat actor APT42 leveraged Gemini to develop bespoke social engineering campaigns, inputting biographical information about particular targets into the AI to produce conversation initiators aimed at fostering trust, as per the report. Furthermore, the group utilized Gemini for linguistic translation and to grasp cultural nuances in unfamiliar languages.
The report revealed that Chinese collectives APT31 and UNC795 employed Gemini to automate vulnerability assessments, troubleshoot malicious code, and explore exploitation methods. Simultaneously, North Korean hackers affiliated with UNC2970 extracted intelligence from Gemini concerning defense contractors and cybersecurity companies, gathering specifics on organizational hierarchies and job responsibilities to bolster their phishing operations.
Google stated it took measures by deactivating linked accounts, and that Google DeepMind utilized these insights to bolster its defenses against potential misuse.
Threat Actors Incorporate AI into Malware Activities
Google indicated that Gemini is also being exploited through other avenues, with certain malicious actors directly embedding its APIs within their harmful code.
Google uncovered a novel malware family, dubbed HONESTCUE, which directly incorporates Gemini’s API into its functionalities. This malware sends prompts to generate operational code, which it then compiles and executes in memory. The report noted that these prompts, when viewed in isolation, seem innocuous, enabling them to circumvent Gemini’s safety protocols.
Pete Luban, Field CISO at AttackIQ, views services such as Gemini as a straightforward method for hackers to enhance their capabilities. He commented, “Integrating public AI models like Google Gemini into malware provides threat actors with immediate access to potent LLM functionalities without the need for self-development or training.” He added, “Malware capabilities have progressed dramatically, facilitating quicker lateral movement, more covert attack campaigns, and more persuasive impersonation of routine company activities.”
Google additionally detailed COINBAIT, a phishing toolkit constructed with AI code generation platforms, and Xanthorox, an illicit service that promoted custom malware-generating AI but was, in reality, a façade for commercial offerings, including Gemini. The company proceeded to terminate accounts and initiatives linked to both.
Luban asserted that the rapid evolution of AI-powered threats renders conventional defenses inadequate. He stated, “Ongoing testing against authentic adversary conduct is crucial for ascertaining if security measures are equipped to counter evolving threats.”
This content was originally published by CSO.
