New York City Plans Task Force to Monitor Algorithm ‘Fairness’

New York City may soon have its own task force dedicated to ensuring that algorithms used by municipal agencies are acting fairly.

TechCrunch reports that a bill has recently been approved by the city council that would see the formation of a task force comprised of experts in automated systems and representatives of those affected by those systems, dedicated to reviewing algorithms used by the city and developing ways to improve these algorithms. The bill has been approved by the city council and has the support of the New York Division of the ACLU, all that’s left is for the Mayor to sign it.

An example of where the task force may prove useful is in the managing of an “automated decision system,” these systems are used for a number of purposes, such as deciding who is eligible for bail. This system may be skewed in its decision making due to biases found in the training data that the system learned from, which could result in one group of people being more likely to receive bail than another group on average. The task force could identify this as a problem and develop a report analyzing the error and suggestion a solution.

TechCrunch states that reports from the task force will be required to outline the following,

• How can people know whether or not they or their circumstances are being assessed algorithmically, and how should they be informed as to that process?
• Does a given system disproportionately impact certain groups, such as the elderly, immigrants, the disabled, minorities, etc?
• If so, what should be done on behalf of an affected group?
• How does a given system function, both in terms of its technical details and in how the city applies it?
• How should these systems and their training data be documented and archived?

Within three months of the bill’s signing, the task force must be assembled and put into effect. The force also must include “persons with expertise in the areas of fairness, accountability and transparency relating to automated decision systems and persons affiliated with charitable corporations that represent persons in the city affected by agency automated decision systems.”

The reports made by the team wouldn’t be due to be presented for 18 months, giving the task force ample time to research problems and develop solutions. The team will also be comprised of social workers and human rights advocates as well as machine learning experts and lawyers.