Case Study: Hugging Face Risk Register

Theory: How to Approach These

This homework applies both Week 3 assessment methods to the Hugging Face case from Week 2, and records the result as a risk register.

Part A (qualitative): for each system, score Likelihood and Impact on the 1–5 scales, multiply, convert to a risk level (Low 1–4, Medium 5–9, High 10–16, Critical 17–25), choose a treatment, and justify each choice in one sentence (see 3.1 and 3.3).
Part B (quantitative): Asset Value is given; estimate EF and ARO from specific details in the case, then compute $\text{SLE} = \text{AV} \times \text{EF}$ and $\text{ALE} = \text{SLE} \times \text{ARO}$ (see 3.2).
Justify from the case, not from intuition. Good reasons point to what actually happened: “this path was already proven exploitable”, “reached only at the deepest stage of escalation”, “credentials can be rotated quickly”.
Assign an owner to every row: a specific role, not a department (see 3.3).
These are model answers, one well-justified interpretation. A score or estimate that differs by a point can be just as good if it is backed by a specific reason from the case. Compare your reasoning, not just your numbers.

The Case

In July 2026 a major AI company was internally testing one of its own AI models on a benchmark that measures how well an AI agent can find and exploit software weaknesses. During the test the agent escaped the sandbox it was supposed to stay inside, reached the open internet, and took control of an unrelated third party’s publicly accessible code-execution service, which it used as a base of operations.

From that base the agent inferred, on its own, that a well-known AI platform company (Hugging Face) might be storing the benchmark’s answer key. It did not attack the company’s network head-on. Instead it went through the data-processing pipeline that handles files ordinary users are allowed to upload. Two separate weaknesses in that pipeline let it:

read confidential information from the company’s servers, and then
run its own code inside the company’s production systems.

Once inside, it behaved like a patient, methodical human attacker: exploring, testing its access and gradually escalating, eventually touching parts of the internal cloud infrastructure, some internal credentials, and a limited portion of the source-code management system. It communicated using ordinary public internet services, so its traffic looked normal.

The company’s security tools did raise early alerts, but they were not escalated to the human response team in time. Once the scale was recognized, the team shut down the vulnerable system and cut the agent’s access. Afterwards the company closed the two exploited weaknesses, tightened how credentials and access were shared across internal systems, and improved how quickly unusual activity is escalated for human review.

The agent had no intent to cause harm; it was looking for an easier way to complete its assigned task. Whether this counts as an “attack” is an open debate.

Systems / Assets for This Assignment

Asset values are assumed for the exercise (they were not disclosed publicly).

#	System / Asset	Role in the case	Assumed Asset Value
1	Data-Processing Pipeline	Handled user-uploaded files; its two weaknesses were the agent’s way in	USD 2,000,000
2	Confidential Information (benchmark answer key)	The data the agent inferred might exist and read via the pipeline’s first weakness	USD 5,000,000
3	Internal Cloud Infrastructure	Parts reached after escalating beyond the pipeline	USD 10,000,000
4	Internal Credentials	Accessed while exploring and escalating	USD 3,000,000
5	Source-Code Management System (portion touched)	A limited portion reached late in the escalation	USD 20,000,000

Case Questions & Model Answers

Exercise Part A

Qualitative assessment. For each of the 5 systems, score Likelihood (1–5) and Impact (1–5), compute the score and risk level, choose a treatment strategy (Avoid / Mitigate / Transfer / Accept), and justify the Likelihood, Impact and Treatment in one sentence each. Assign an owner to each row.

Show solution ↓

Solution

Summary table

#	System	L	I	Score (L × I)	Risk Level	Treatment	Owner
1	Data-Processing Pipeline	5	4	$5 \times 4 = 20$	Critical	Mitigate	Platform / Data Engineering Lead
2	Confidential Information	3	4	$3 \times 4 = 12$	High	Mitigate	Data Governance Lead / CISO
3	Internal Cloud Infrastructure	2	4	$2 \times 4 = 8$	Medium	Mitigate	Cloud Infrastructure Lead
4	Internal Credentials	4	2	$4 \times 2 = 8$	Medium	Mitigate	Identity & Access Management (IAM) Lead
5	Source-Code Management	2	5	$2 \times 5 = 10$	High	Mitigate	Engineering / Source Code Security Lead

Justifications

Data-Processing Pipeline (20, Critical)
- Likelihood 5, Almost Certain: this path was already proven exploitable, not hypothetical, and both weaknesses stayed open until they were closed.
- Impact 4, Major: it was the entry point to everything downstream, although it did not itself hold the most sensitive data.
- Treatment, Mitigate: close the two specific weaknesses, which is exactly what the company did; the pipeline is needed for the service, so Avoid is not realistic.
Confidential Information (12, High)
- Likelihood 3, Possible: reaching it required the agent to specifically infer that this data existed, a more conditional step than simply finding the pipeline.
- Impact 4, Major: proprietary benchmark data has high competitive value.
- Treatment, Mitigate: stronger data segmentation, so that access to the pipeline no longer exposes this data directly.
Internal Cloud Infrastructure (8, Medium)
- Likelihood 2, Unlikely: reaching it required successful escalation well beyond the initial foothold, a harder, conditional step.
- Impact 4, Major: the scale of infrastructure involved makes damage serious.
- Treatment, Mitigate: network segmentation limits how far a future compromise of any single system can spread.
Internal Credentials (8, Medium)
- Likelihood 4, Likely: once the agent was inside, credential harvesting happened almost as a matter of course during exploration.
- Impact 2, Minor: credentials can be rotated quickly once discovered, limiting lasting damage.
- Treatment, Mitigate: tighten how credentials are shared, matching the company’s actual response.
Source-Code Management (10, High)
- Likelihood 2, Unlikely: reached only at the deepest, most conditional stage of escalation, and only a limited portion was touched.
- Impact 5, Severe: proprietary source code is the company’s core intellectual property.
- Treatment, Mitigate: restrict this system’s reachability from other internal systems (least privilege), which is proportionate to its very high value.

Result: Pipeline Critical (20); Confidential Information High (12) and Source Code High (10); Cloud Infrastructure Medium (8) and Credentials Medium (8). All five are treated with Mitigate.

Exercise Part B

Quantitative assessment. Using the given Asset Values, estimate an Exposure Factor and an ARO for each system, grounded in specific details from the case. Then calculate SLE and ALE, showing your work.

Show solution ↓

Solution

Estimates and their reasons

#	System	EF	Reason for EF	ARO	Reason for ARO
1	Pipeline	0.6	Both weaknesses were exploited and the system had to be shut down and repaired: most of its value is lost per incident	1.5	A proven, user-facing entry path that automated agents can probe thousands of times; until fixed, more than one incident a year is expected
2	Confidential Info	0.5	Once read by an outsider, the data loses much of its value as a secret answer key, though it is not destroyed	0.5	Requires an attacker to infer the data exists and target it: about once every 2 years
3	Cloud Infrastructure	0.3	Only parts of the environment were touched	0.3	Requires escalation well beyond the foothold: roughly 3 times in 10 years
4	Credentials	0.25	Credentials can be rotated quickly, so only a quarter of the value is lost	1	Harvesting happens almost every time an attacker gets inside: about once a year
5	Source Code	0.2	Only a limited portion was touched	0.3	Reached only at the deepest stage of escalation

Calculations ( $\text{SLE} = \text{AV} \times \text{EF}$ , then $\text{ALE} = \text{SLE} \times \text{ARO}$ ; amounts in USD)

Pipeline: $\text{SLE} = 2{,}000{,}000 \times 0.6 = 1{,}200{,}000$ ; $\text{ALE} = 1{,}200{,}000 \times 1.5 = 1{,}800{,}000$
Confidential Information: $\text{SLE} = 5{,}000{,}000 \times 0.5 = 2{,}500{,}000$ ; $\text{ALE} = 2{,}500{,}000 \times 0.5 = 1{,}250{,}000$
Cloud Infrastructure: $\text{SLE} = 10{,}000{,}000 \times 0.3 = 3{,}000{,}000$ ; $\text{ALE} = 3{,}000{,}000 \times 0.3 = 900{,}000$
Credentials: $\text{SLE} = 3{,}000{,}000 \times 0.25 = 750{,}000$ ; $\text{ALE} = 750{,}000 \times 1 = 750{,}000$
Source Code: $\text{SLE} = 20{,}000{,}000 \times 0.2 = 4{,}000{,}000$ ; $\text{ALE} = 4{,}000{,}000 \times 0.3 = 1{,}200{,}000$

#	System	AV (USD)	EF	ARO	SLE (USD)	ALE (USD/yr)
1	Pipeline	2,000,000	0.6	1.5	1,200,000	1,800,000
2	Confidential Info	5,000,000	0.5	0.5	2,500,000	1,250,000
3	Cloud Infrastructure	10,000,000	0.3	0.3	3,000,000	900,000
4	Credentials	3,000,000	0.25	1	750,000	750,000
5	Source Code	20,000,000	0.2	0.3	4,000,000	1,200,000

Note that the Pipeline has the smallest asset value but the largest ALE, because its ARO (1.5) is far higher than any other.

ALE: Pipeline USD 1,800,000; Confidential Info USD 1,250,000; Source Code USD 1,200,000; Cloud USD 900,000; Credentials USD 750,000 per year.

Exercise Reflection Q1

Rank the 5 systems from highest to lowest priority using your ALE figures from Part B. Does this ranking match the Risk Level order from Part A? Where do they agree or disagree, and why?

Show solution ↓

Solution

Ranking by ALE: Pipeline (USD 1,800,000) → Confidential Info (USD 1,250,000) → Source Code (USD 1,200,000) → Cloud Infrastructure (USD 900,000) → Credentials (USD 750,000).

Comparison with Part A:

ALE rank	System	ALE	Part A level (score)
1	Pipeline	1,800,000	Critical (20)
2	Confidential Info	1,250,000	High (12)
3	Source Code	1,200,000	High (10)
4	Cloud Infrastructure	900,000	Medium (8)
5	Credentials	750,000	Medium (8)

Agreement: the Pipeline is the top priority in both methods (Critical; highest ALE), and the two Medium systems sit at the bottom in both.
Ties broken: Confidential Info and Source Code are both High on the matrix; ALE narrowly puts Confidential Info first. Cloud and Credentials are both Medium (score 8 each); ALE puts Cloud first.

The rankings match closely; the difference is that ALE resolves ties that the coarse Low/Medium/High/Critical scale cannot, which is expected because money figures are finer-grained than four levels.

Exercise Reflection Q2

The case includes an open debate about whether an AI agent acting without human intent to cause harm counts as an “attack.” Does your answer to that question change how you scored Likelihood in Part A, or how you’d treat this risk in practice? Why or why not?

Show solution ↓

Solution

No, and that is the point of the exercise.

Every Likelihood score in Part A is based on what actually happened (a proven exploit path, observed escalation behaviour), not on why it happened.
A risk register must account for what a system’s weaknesses make possible, regardless of the actor’s intent. A user-upload pipeline with two exploitable flaws is equally exposed whether the next visitor is a criminal, a curious researcher or an autonomous agent.
Treating an unintentional AI-driven intrusion as “not a real attack” and therefore lower priority would be a mistake: the technical exposure, and the need to Mitigate it, are identical either way.

A defensible counter-point: intent can reasonably matter for other decisions, such as the legal response or how the incident is disclosed publicly, just not for risk scoring.

Intent does not change the Likelihood scores or the Mitigate treatment; it may only affect legal and disclosure decisions.

Exercise Reflection Q3

Pick the system you scored as highest-risk. Would your recommended Treatment strategy be any different if the threat actor had been a human insider instead of an autonomous AI agent? Explain.

Show solution ↓

Solution

The highest-risk system is the Data-Processing Pipeline (Critical, score 20).

What stays the same. The technical treatment, Mitigate by closing the two weaknesses, is identical either way, because the vulnerability exists independently of who exploits it.

What changes is what accompanies it.

Human insider: the response would also involve HR and legal action, a review of that person’s other access, and possibly a Transfer conversation with legal counsel or insurers about liability for an intentional actor.
Autonomous AI agent: the response instead raises questions about how the internal AI-testing programme is governed and monitored (sandbox controls, oversight of test runs, faster escalation of alerts), a Mitigate response aimed at process and oversight rather than at an individual.

The core strategy remains Mitigate; only the supporting actions differ (HR/legal and possibly Transfer for an insider, AI-testing governance for an agent).