Introducing the Policy Difference Engine

An introduction to modeling legislation with Rules as Code

I am a developer with Code for Canada, a non-profit organization that works with government partners to deliver better digital services to the public. Every year, Code for Canada selects a cohort of fellows for a 10-month fellowship, lasting from November to August. I have the pleasure of being a developer fellow alongside UX Designer, Sidra, and Product Manager, Dena. For the duration of the fellowship, we will be working with Employment and Social Development Canada (ESDC). This article is a brief intro to some of the more technical aspects of the project that have emerged during our introductory research, as well as the concept of ‘Rules as Code’, which may play an important role in the project.

The Current Landscape

ESDC is a government department that handles many public services such as employment insurance, pensions, and disability benefits. These services are bound by a wide variety of legislations, regulations, and policies, which we will generalize here under the term “rules,” which often need to be updated to adjust to new realities. Examples may include changing an age requirement for eligibility of an allowance, changing the dollar amount of a benefit, re-defining what is meant by a “vehicle”, etc. With technologies that directly impact the public growing in complexity, and the population experiencing increasingly diverse living situations, it’s important to keep the rules up to date. One challenge that quickly emerges is effectively measuring the impact of changing a given rule. This is the challenge that ESDC and Code for Canada will be eagerly addressing. At the time of writing (November 2020), we are at the beginning of the fellowship, which will last until August 2021. We will be working closely with ESDC over the next nine months to make measurable progress on the challenge, and then smoothly hand off our work to ESDC at the end of the fellowship.

Refining the Project

The overarching challenge is to measure the impact of changing a rule. This problem statement is quite general, and can be broken down into different components, and we will focus on two of these. First of all, many rules will be inextricably linked to other rules. When a change is proposed to policy A, and policy A is linked to policy B, C, and D, those three policies must also be investigated to verify if the change will have any unintended side effects. This may further cascade to even more policies, exponentially increasing the amount of investigation. Much of this investigation is manual, so it can be difficult to keep track of all potential side effects.

Rules as Code

In order to run simulations efficiently, the rules must be programmed into a computer, which in turn requires that any ambiguities be removed. Many existing rules were not written with this in mind, so the translation from written rules to machine-consumable logic is a prerequisite for the project. This is the concept of Rules as Code (sometimes known as computational law).

Example: New Zealand Child Disability Allowance

View the requirements for the New Zealand Child Disability Allowance.

  • You are the principal caregiver
  • The child meets the eligibility criteria
  • Both you and the child should ordinarily reside in New Zealand
A flow diagram that shows how eligibility is determined for child disability allowance. This diagram demonstrates that residential status (resident/citizen), caregiver role/guardianship, as well as the child’s needs and months certified with disability status can all impact eligibility for this particular benefit.

Using the Transformed Rule

Once a given rule has been converted into RaC, we open up all the possibilities that come with any coded system. We can easily check for eligibility, we can make illustrative visualizations, and most applicable to us — we can run simulations on a model population. To this end, we can create a list of test cases or personas. This may be a collection of households with different configurations of ages, children, incomes, disability statuses, etc. Once these test cases are in the system, we can represent a rule change by toggling values in the coded rules, and quantify the impact on the different personas.

Ethical and Legal Considerations

At this point it is worth pointing out an important ethical consideration. When modelling humans as data, we must always be careful about the data we are considering and the data we are not considering. A simulation on a model may show a net positive effect on the population in terms of overall cost, but if for example, we leave out key pieces of data (race, gender, age), and it turns out that a policy change actually negatively affects one group, then the simulation is largely incomplete. For this article, we will leave aside the discussion on whether any model may be better than no model at all, but it is an important consideration, and the discussion of any important omissions and oversights must remain part of the conversation.

Next Steps

We’ve defined the high-level goal of the solution as measuring the impact of changing rules. We can break this down into two more concrete subgoals:

  • Running valid simulations on proposed rule changes

Developer working in Civic Tech

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store