Recently, the White House opened a number of opportunities for the public to comment on the growing field of accountability for artificial intelligence (AI) systems. The National Telecommunications and Information Administration (NTIA), the Executive Branch agency that is principally responsible for advising the President on telecommunications and information policy issues, launched a comment process that concluded on June 12, seeking feedback on what policies can support the development of AI audits, assessments, certifications and other mechanisms to create earned trust in AI systems. Nearly 1,500 public comments were filed.
Our comment offers three core principles that the NTIA should take into account in developing recommendations for future regulation:
First, the accountability ecosystem should have multiple, overlapping mechanisms for ensuring that AI systems are serving the public interest.
Second, because AI systems involve complex socio-technical interactions between data, models, and people operating in different institutional contexts, assessments cannot look at one element in isolation to form a judgment about the whole system.
Third, while many current assessment tools focus on important questions about whether AI systems are biased or unfair, it is equally important to assess whether the AI systems are fit for purpose. In particular, many systems are used by organizations to make consequential decisions about individuals that are based on unreliable science and make dubious claims of fairness, accuracy, or efficiency.
We then detail four specific avenues for the NTIA to promote mechanisms that improve accountability: (a) enabling the development of a standards-setting body; (b) focusing on whether AI systems are fit for purpose and based on rigorous science; (c) requiring that AI systems can be examined from multiple vantage points; (d) developing oversight mechanisms for public sector use of AI systems.
Our full comment is available here and the other public comments will be posted on the NTIA’s docket here.
