Model Stealing or Extraction Attacks can occur when the security of an AI interface, API, or backend is inadequate. In such cases, attackers exploit vulnerabilities in web applications associated with the AI system. By doing so, they gain unauthorized access to the AI's data, including its trained model parameters or other critical information. These attacks allow the attacker to replicate the AI model's functionality without proper authorization.
Example: Consider a cloud-based image recognition service that charges per query. An attacker could use a diverse set of images to query the service, collecting the labels and confidence scores provided by the model. Then, the attacker trains a new model on this dataset. The result is a cloned model that mimics the original model's functionality, allowing the attacker to bypass the need to pay for the cloud service.
Model Stealing or Data Extraction Attacks
Service Targeting:
An attacker selects a cloud-based image recognition service, which bills per query. This service uses an AI model to analyze images and provide labels and confidence scores.
Data Collection:
The attacker gathers a varied image set, covering a range of subjects, scenes, and objects. This variety is vital to match the original model's scope.
Querying the Original Model:
The attacker submits these images to the cloud service, recording the AI model's labels and confidence scores for each image, thus creating a new dataset pairing each image with its results.
Cloned Model Training:
The attacker then trains a new AI model using this dataset, aiming to imitate the original model by predicting labels and confidence scores. This involves adjusting the cloned model to align its predictions with the original model's outputs.
Bypassing the Cloud Service:
Once trained, the attacker uses this model independently for image recognition, avoiding the need to pay for the cloud service. The cloned model could even become a rival service.
Implications of Data Extraction Attacks
- Intellectual Property Theft: Cloning an AI model, often a product of substantial investment, can compromise the provider's competitive advantage and revenue.
- Privacy and Security Risks: Depending on the data and model, cloning might pose privacy issues, particularly if the model involves proprietary or sensitive training data.
- Defensive Measures Need: It underscores the importance of protective strategies against such attacks, like rate-limiting API calls, monitoring for suspicious access patterns, and using model watermarking for identifying unauthorized copies
