New York City has begun to enforce a law initially passed in 2021 that requires employers to take certain steps before implementing an automated employment decision tool (AEDT). The law reflects a growing focus on the use of artificial intelligence (AI) in employment decisions and addresses concerns pertaining to the use of AI, including that biased algorithms could create a disparate impact on certain candidates or employees, even without an employer's intent to discriminate.
NYC 144 Requirements
NYC Local Law 144 of 2021 (NYC 144) prohibits employers and employment agencies from using an AEDT to make a hiring or promotion decision unless they conduct a bias audit on the tool annually, publish an audit summary, and provide certain notice to candidates and employees about the tool. New York City previously deferred enforcement of the law pending final regulations, which were published on April 5, 2023, and enforcement of the law began on July 5, 2023. The regulations provide further insight into what the law covers and how an employer should conduct a bias audit.
The New York City law applies to an employer's or employment agency's use of an AEDT to screen candidates for hiring or employees for promotion. The law does not cover other employment decisions, such as compensation, retention, and termination decisions. In the context of hiring, the law applies with respect to tools used to screen candidates who have actually applied for a specific job and not to tools used to identify potential candidates who have not yet applied for a position (for instance, those used to scan a resume bank, conduct outreach to potential candidates, or invite applications). Now, the New York City Department of Consumer and Worker Protection (NYC DCWP) issued a "Frequently Asked Questions" (FAQs) document indicating that it considers the law applicable if (1) the job location is an office in New York City, at least part time; (2) the job is fully remote but the location associated with it is an office in New York City; or (3) the location of the employment agency using the AEDT is New York City or, if the location of the employment agency is outside New York City, (1) or (2) above is true.
An AEDT means any computational process, derived from machine learning, statistical modeling, data analytics, or AI, that issues simplified output, including a score, classification, or recommendation, that is used to substantially assist or replace discretionary decision making for employment decisions. An AEDT does not include junk email filters, firewalls, antivirus software, calculators, spreadsheets, databases, or data sets.
The law defines a "bias audit" as "an impartial evaluation by an independent auditor." Specifically, an independent auditor must be an objective individual or group that has not been involved in the AEDT's use, development, or distribution and is not employed by or financially interested in the employer using the AEDT or the vendor that develops or distributes the AEDT. Accordingly, the AI software vendor does not qualify as "independent" under the law and could not conduct the bias audit itself. NYC DCWP's FAQs state that NYC DCWP does not have a list of approved independent auditors, as the law "does not require independent auditors to be approved by [NYC] DCWP." An AEDT cannot be used if more than one year has passed since the most recent bias audit of the tool.
The regulations detail the content of what the bias audit must include, with specifications regarding the technical processes and calculations to be conducted. The bias audit must include calculations of the selection (or scoring) rate for each category, including sex, race/ethnicity, and intersectional categories, and the "impact ratio" of each category. "Impact ratio" is defined as either the selection rate for a category divided by the selection rate of the most-selected category or the scoring rate for a category divided by the scoring rate for the highest-scoring category. The regulations also specify that historical data–the data collected during an employer's prior use of an AEDT–must be used for the bias audit unless the employer has never used the AEDT or there is insufficient historical data available to conduct a statistically significant audit.
Before the use of an AEDT, an employer must publicly post on its website certain information, including the date of the most recent bias audit, a summary of the results, and the date it began using the AEDT. An employer must also notify candidates and employees, via certain acceptable methods, no less than 10 business days before use of the AEDT:
- That an AEDT will be used;
- Of instructions for how an individual can request an alternative selection process or a reasonable accommodation under other laws, if available; and
- Of the job qualifications and characteristics the AEDT will use in its assessment.
The regulations provide that nothing in them requires an employer to provide an alternative selection process. The employer must additionally provide information about the type of data collected for the AEDT, the source of the data, and the employer's data retention policy and post instructions for how to make a written request for the information.
NYC DCWP's FAQs state that NYC 144 does not require any specific actions based on the results of a bias audit, but that employers and employment agencies must comply with all relevant anti-discrimination laws and rules to determine any necessary actions based on the results of a bias audit.
NYC DCWP held an "Educational Roundtable with Business Advocates/ Employers" on May 22, 2023, to provide an overview of the regulations and field questions. A recording of this event is available on NYC DCWP's website.
Penalties for Violations
Any person who violates the law is liable for a civil penalty of (1) not more than $500 for a first violation and each additional violation occurring on the same day as the first violation, and (2) not less than $500 nor more than $1,500 for each subsequent violation. Each day on which an AEDT is used in violation of the law constitutes a separate violation, and the failure to provide any of the required notices to candidates or employees constitutes a separate violation. NYC DCWP will handle enforcement of NYC 144's prohibition on AEDT use without a bias audit, and required notices. According to NYC DCWP, it will refer claims of discrimination involving AEDT use to the NYC Commission on Human Rights. NYC 144 also does not preclude a candidate or employee from bringing a discrimination claim based on AEDT use in a court of competent jurisdiction.
On January 1, 2020, the Artificial Intelligence Video Interview Act took effect in Illinois.
NYC DCWP will handle enforcement of NYC 144's prohibition on AEDT use without a bias audit and required notices. According to NYC DCWP, it will refer claims of discrimination involving AEDT use to the NYC Commission on Human Rights. NYC 144 also does not preclude a candidate or employee from bringing a discrimination claim based on AEDT use in a court of competent jurisdiction.
Other Jurisdictions
In line with the growing interest in AI and employment, the U.S. Equal Employment Opportunity Commission announced an initiative last year to ensure that AI used in hiring and other employment decisions complies with federal anti-discrimination laws. On May 18, 2023, it issued a technical assistance document entitled "Select Issues: Assessing Adverse Impact in Software, Algorithms, and Artificial Intelligence Used in Employment Selection Procedures under Title VII of the Civil Rights Act of 1964."
On January 1, 2020, the Artificial Intelligence Video Interview Act took effect in Illinois, requiring employers conducting AI analysis of applicant-submitted videos to comply with certain notice and other requirements. Effective October 1, 2020, Maryland enacted a law pertaining to employers' use of facial recognition technology during preemployment job interviews. Legislation is currently proposed in a number of cities and states concerning the use of AI in the employment context, including in California and Washington, D.C.
The Bottom Line
With NYC 144 now in effect, New York City employers that use or intend to use AI-driven tools in hiring and promotional processes should take steps now to ensure compliance.
Originally Published by Employment Benefit Plan Review
The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.