Algorithmic Impact Assessment tool

2.1 Scoring

Each area contains one or more questions, and the responses to the questions contribute to a maximum score for the area. The value of each question is weighted based on the level of risk it introduces or mitigates in the automation project. The raw impact score measures the risks of the automation, while the mitigation score measures how the risks of automation are managed. The questions in risk areas 1 to 6 increase the raw impact score, and the questions in mitigation areas 7 and 8 increase the mitigation score.

The current score is determined as follows:

• If the mitigation score is less than 80% of the maximum attainable mitigation score, the current score is equal to the raw impact score
  or
• If the mitigation score is 80% or more than the maximum attainable mitigation score, 15% is deducted from the raw impact score to yield the current score

2.2 Impact levels

The impacts of automating an administrative decision are classified into 4 levels, ranging from Level I (little impact) to Level IV (very high impact). The AIA is intended to identify risks and assess impacts in a broad range of areas, including:

• the rights of individuals or communities
• the health or well-being of individuals or communities
• the economic interests of individuals, entities, or communities
• the ongoing sustainability of an ecosystem

Impact levels are distinguished based on criteria of reversibility and expected duration: automated decisions with little to no impact are reversible and brief, while those with a very high impact are irreversible and perpetual.

Each impact level corresponds to a score percentage range. The level of impact assigned to an automation project depends on the range bracket in which the project’s score percentage falls (Table 5). Impact levels determine the mitigations required under the Directive on Automated Decision-Making. Appendix C of the directive lists mitigation measures required for each of the 4 impact levels. The requirements are designed to be proportionate to the impact level. While the measures are intended to reduce the identified risks, their implementation does not alter the impact level assigned to a project. A project’s impact level can only be changed through the completion of a new AIA with updated information about the project. Under the directive (subsection 6.1.3), departments are required to update their AIAs following changes to system functionality or scope of use.

3.1 When to complete the AIA

The AIA should be completed at the beginning of the design phase of a project. The results of the AIA will guide the mitigation and consultation requirements to be met during the implementation of the automated decision system as per the directive.

The AIA should be completed a second time, prior to the production of the system, to validate that the results accurately reflect the system that was built. The revised AIA should be released on the Open Government Portal as the final results.

3.2 What to consider when completing an AIA

Collect information about your project

A broad range of information about an automation project is required to fully answer the questions in the risk and mitigation areas of the AIA. Prior to starting the AIA, it is useful to have information about:

• the administrative decision that the automated decision system will inform or make, the context in which the system will be used, and the way the system will assist or replace the judgement of a human decision-maker
• the clients subject to the decision, including evidence of any vulnerability (for example, socioeconomic, demographic, geographic)
• the potential impacts of the decision on clients, including their duration and reversibility
• the algorithm, including data processing parameters and techniques, and the output
• the input data used by the system, including details on type, source, method of collection, and security classification
• planned or existing quality assurance measures
• planned transparency measures to communicate information about the initiative to clients and the public
• internal and external stakeholders to be consulted
• record of recommendations or decisions made by the system, and any log or explanation generated by the system for such a record
• institutional information technology service management (ITSM) practices

The information above will also be useful in consultations with the stakeholders described below. It can also support engagements with vendors and peer reviewers.

Consult your ATIP office

The institutional Access to Information and Privacy (ATIP) office or other delegated authority for privacy should be consulted to ensure that the privacy impacts of automated decision systems using or processing personal information, or that otherwise have an impact on individuals’ privacy rights, are identified, assessed, and mitigated. Engagements can enable project leads to:

• determine whether the automated decision system will make use of personal information as defined in section 3 of the Privacy Act
• verify whether the proposed system’s use of personal information is accounted for in relevant Privacy Impact Assessments (PIAs) and Personal Information Banks (PIBs), and identify actions and approvals required under privacy legislation and policy
• determine applicable requirements in the Policy on Privacy Protection and supporting policy instruments, with the goal of ensuring that the appropriate safeguards are in place for the creation, collection, and handling of personal information throughout the lifecycle of the system, including plans for responsive action in the event of a privacy breach
• inform approaches to notifying the Office of the Privacy Commissioner of Canada of the proposed system and any foreseeable impacts on the privacy of Canadians, as required in the Policy on Privacy Protection
• ensure any personal information collected or created by or for the automation project is demonstrably necessary for the program or activity, as required by the Directive on Privacy Practices
• ensure that appropriate privacy protection clauses are included in contracts, agreements or arrangements with vendors or partners providing automation services, solutions, or products
• ensure the AIA is reviewed by privacy officials prior to its publication on the Open Government Portal, in accordance with the Directive on Open Government

Engage with Legal Services

When completing an AIA, legal services must be consulted to identify and address legal risks arising from the development, procurement, or use of an automated decision system. Consultations should begin at the concept stage of an automation project, prior to the development or procurement of a system. The nature of legal risks significantly depends on the design of the system (for example, training data, model), the context of deployment, and the type of administrative decision being automated. Seeking legal advice on an automated decision system can enable project leads to:

• comply with the directive’s requirement to consult with legal services from the concept stage of a project (see subsection 6.3.8)
• identify potential impacts on individual rights and freedoms and develop plans to manage risks
• assess risks to procedural fairness based on factors including, but not limited to, the explainability of system outputs, relevance of system rules and input data to an administrative decision, and availability of recourse options (see subsections 6.2.3, 6.3.1, 6.3.2, and 6.4.1 of the directive)
• confirm whether the program has the requisite authorities to proceed with the proposed automation project, including any associated collection or creation of personal information (in coordination with the relevant ATIP office or other delegated authority for privacy).
• Consider licensing issues, including with respect to trade secrets, and any constraints they may impose on a department’s ability to access and test proprietary systems (see subsection 6.2.5 of the directive)

Contact TBS OCIO

The Office of the Chief Information Officer (OCIO) at the Treasury Board of Canada Secretariat (TBS) is responsible for maintaining the AIA tool and overseeing departmental compliance with the Directive on Automated Decision-Making. The team can support departments with interpreting and answering questions in the AIA, and guide them through the publication process.

Departments are encouraged to contact TBS OCIO at ai-ia@tbs-sct.gc.ca for any questions regarding the directive or AIA.

3.3 Releasing the results

Departments are responsible for releasing the final results of the AIA in an accessible format and in both official languages on the Open Government Portal. The results page of the AIA provides the option to provide translations for the text entered in the AIA. The results page also provides the option to download the results as an accessible PDF to meet this requirement.