AI System Impact Assessment under ISO 42001

What makes ISO 42001 different from security and privacy standards is its focus on how an AI system can affect people, groups, and society. A proper AI impact assessment should therefore look beyond business risks.

The key difference

• Security risk assessment: What could harm the organisation? 

• Privacy risk assessment: What could harm the individual whose data is being used? 

• AI impact assessment: What impact could the AI system have on individuals, groups, and society? 

A common mistake is to take an existing business risk assessment and simply call it an AI impact assessment. If the assessment only lists risks such as financial loss, regulatory penalties, or reputational damage, it is still mainly a business risk assessment.

Step 1: Understand what the AI system actually does

For every AI system, clearly explain what it decides, recommends, or influences and how this affects people.

For example:

• Instead of saying "the AI classifies applications," say "the AI helps determine which job applicants move to the interview stage." 

• Instead of "the AI scores customer accounts," say "the AI influences whether a customer receives credit." 

• Instead of "the AI suggests responses," say "the AI prepares messages that are sent to customers in the organisation's name." 

Being specific helps you understand who may be affected and how seriously.

Step 2: Identify everyone who may be affected

Consider:

• People who use the AI system 

• People about whom the AI makes decisions 

• Groups that may experience different outcomes 

• People who may be affected indirectly 

The people being assessed or decided about by the AI are especially important. They are often forgotten because they may not be customers, employees, or even users of the system.

Step 3: Check the important impact areas

Depending on the AI system, consider areas such as:

Fairness:

Could the AI ISO 42001 produce different or unfair outcomes for different groups? Was the data used to develop the system representative of the people it will be used on? Has this actually been tested?

Safety:

Could an incorrect AI output cause physical, financial, or psychological harm? How serious could that harm be?

Privacy:

What personal information does the AI use? Why is it being used? Could the AI reveal information about someone that they did not provide?

Transparency and explainability:

Do affected people know that AI is being used? Can the organisation explain the decision or outcome when necessary?

Right to challenge:

Can someone question or challenge an AI-based decision? Do they know whom to contact?

Accessibility:

Can people with disabilities use the system? Does it work effectively for people who are different from the population used to develop the AI?

Environmental impact:

For large-scale AI systems, consider whether the environmental impact is significant.

Not every area will apply to every AI system. However, the organisation should document which areas were considered and why any were considered not applicable.

Step 4: Understand how serious the impact could be

As with other risk assessments, consider:

• How likely is the impact? 

• How serious could it be? 

• Who will experience the impact? 

There is an important difference between a situation where the organisation bears the risk and one where an individual bears the consequences and has no way to challenge the outcome.

This difference should be clearly visible in the assessment.

Step 5: Put appropriate controls in place

The controls should match the potential impact. These may include:

• Appropriate human review 

• Testing AI performance across different groups 

• Clear information for affected people 

• A process for challenging decisions 

• Restrictions on certain AI use cases 

• Continuous monitoring 

• Deciding not to deploy the AI system where the risks cannot be adequately controlled 

Each control should be connected to the specific impact or risk it is intended to address.

Step 6: Make human oversight meaningful

Simply having a person "review" an AI decision does not automatically mean there is effective human oversight.

For human oversight to work, the reviewer needs:

1. Information to understand the AI's recommendation 

2. Time to properly consider it 

3. Competence to judge whether it is appropriate 

4. Authority to change or override the AI's decision 

For example, if an employee has to review 100 AI recommendations every hour but cannot understand why the AI made those recommendations, this is unlikely to be meaningful oversight.

These four elements should be clearly defined whenever human oversight is required.

Step 7: Review the assessment regularly

The AI impact assessment should be reviewed when there are changes to:

• The AI system 

• The data being used 

• The purpose or use case 

• The population affected 

• Following an incident 

It should also be reviewed periodically, even when no obvious changes have been made, because AI systems and their performance can change over time.

Document every reassessment, even when the conclusion remains the same.

Common gaps in AI impact assessments

Organisations should watch out for these common weaknesses:

• Looking at AI risks only from the organisation's perspective 

• Failing to identify people who are directly affected by AI decisions 

• Assuming AI is fair without testing it using representative data 

• Having no process for people to challenge AI-based outcomes 

• Defining human oversight without giving reviewers enough information, time, competence, or authority 

• Failing to reassess when an AI use case expands 

• Ignoring AI systems included in purchased software or third-party tools 

• Conducting the assessment only once and never reviewing it again