CyberGuard dictionary / AI Security
Model Poisoning
Model poisoning is deliberate interference with a model or its update process so the resulting system behaves incorrectly or contains a hidden trigger.
In simple terms
The attacker tries to change what the model has learned. Poisoned training data is one route, but a compromised model artifact or update can also alter behavior.
Technical explanation
A poisoned checkpoint, fine-tuning update, or learning pipeline can shift outputs for selected inputs while ordinary tests still appear normal. Integrity checks, provenance, and targeted evaluations reduce exposure but require continued monitoring.
A compromised model artifact or update can embed behavior that appears only under certain conditions. Provenance, integrity checks, and targeted evaluations help reveal changes but cannot guarantee perfect behavior.
Examples in practice
A downloaded model works for routine questions but produces a predetermined wrong answer whenever a particular phrase appears.
A downloaded checkpoint answers normal support questions correctly but gives a fraudulent refund address when a trigger phrase appears. The team compares it with the approved artifact and blocks deployment.
Why it matters
A trusted-looking model artifact can carry an integrity problem into many applications that reuse it.
VERIFY & LEARN MORE
Sources and further reading
- genai.owasp.orghttps://genai.owasp.org/llmrisk/llm042025-data-and-model-poisoning/↗
- atlas.mitre.orghttps://atlas.mitre.org/↗
Published 2026-10-15 · Last updated 2026-10-15 · CyberGuard editorial content. Source links provide context; they do not imply endorsement.