Air Force Logo
Thundercats Logo
All incident types

ML Supply Chain Attack Drill

A compromised model artifact, dataset, or ML dependency pulled in from a registry or a model hub and trusted by everything downstream of it.

3 scenarios in this pool

3:14 AM: Suspicious MLflow Model Artifact Integrity Failure + Unexpected Dependency Pull

You are an on-call ML security analyst for a financial services firm running an MLflow Model Registry backed by S3, used to serve credit-risk scoring models via a FastAPI inference API. Models are promoted through dev -> staging -> production stages by automated CI/CD pipelines. A downstream RAG chatbot also loads embeddings models directly from this registry at startup.

Alert

SIEM ALERT [CRITICAL] 03:14:07 UTC: MLflow artifact integrity check failed on model version `credit-scorer-v4.2` (RunID: `8f3c1a9d`). SHA-256 mismatch: expected `a3f7c2...d91b` vs stored `7e04ab...fc33`. Simultaneously, artifact server logs show an outbound pip install from the model's `requirements.txt` resolving `scikit-learn==1.2.2` to PyPI index `pypi-mirror.mlops-tools[.]xyz` (non-whitelisted). CI/CD pipeline user `svc-mlpipeline@corp.internal` performed the last registry write at 02:58:41 UTC, 4 minutes before the scheduled pipeline window (03:02 UTC). 3 of 5 inference API pods restarted in the past 12 minutes (exit code 137: OOMKilled).

ATLAS mapping: AML.T0010
Step 1 of 6

The SHA-256 hash mismatch is your first hard indicator. What is your immediate priority action before doing anything else?