The bill in front of California lawmakers would put a person in charge of reviewing any such decision, require employers to disclose the automated systems behind it, and cost them $500 a violation.
Andon Labs, the artificial intelligence company that runs the Andon Market gift shop in San Francisco's Cow Hollow, says an AI rather than a person decided to dismiss one of the store's workers, and it believes no AI manager had ever fired a human employee before. Earlier this month, the shop said in a social media post that it had let go a worker who was persistently late.
Luna, the AI manager, hires the staff, sets the schedules and picks and prices the merchandise, along with the other duties of a retail boss, while people work the floor. Its hold on the finer points of the job was looser: according to a report Andon Labs published on its own website earlier this month, Luna had to be prompted before it wrote basic employee rules, and once a handbook existed it lost track of the document, logging the lateness violations of the worker who was eventually dismissed and then doing nothing about them. Andon Labs wrote that the lateness policy had effectively dropped out of Luna's memory. Luna went on to propose that cofounders Lukas Petersson and Axel Backlund deliver the dismissal face to face, and the two agreed.
Andon Labs signed a three-year lease on the Union Street space in the spring, saying it wanted to make an AI-run store profitable, and the shop is billed openly as a trial: the cofounders have described it as a way to test AI systems on real problems and to provoke argument about AI at work.
Many worker advocates worry that AI-driven dismissals could spread far beyond one experimental gift shop, and they are backing SB 947, brought forward by Sen. Jerry McNerney, D-Pleasanton. It would require employers to disclose when they use automated decision-making systems, and would require a person to review any move to discipline or fire a worker that rests chiefly on such a system. McNerney told the San Francisco Chronicle that no one wants a machine to be the thing that ends their job.
What Andon Labs did by choice — leaving the final act to people — would become mandatory if the bill passes. The measure would also bar employers from deploying such systems to forecast how their workers will behave, oblige an employer to tell a worker when such a tool fed directly into discipline or dismissal, and entitle California employees to see the data their employers feed in. Workers could ask the state labor commissioner to enforce the rules, and non-compliant employers would face state legal proceedings and a $500 penalty for each violation.
The California Chamber of Commerce, the bill's fiercest critic, has put the measure on its roster of proposals it calls cost-drivers and is fighting in the Legislature this year, describing the obligations as unworkable, warning they would deter use of automated decision tools and saying businesses could end up facing expensive penalties. The California Federation of Labor Unions, AFL-CIO, sponsors the measure.
The Andon Labs cofounders take no side on the bill and say they prefer democratic discussion of how AI is used at work, but both doubt human oversight can keep pace with models that are improving quickly. Petersson told the San Francisco Chronicle that checking the AI's actions may not suffice, since the AI will outsmart the people doing the checking, which makes the review meaningless; he asked how a measure could be written to hold up in the future, and said they do not have the answer.
The bill is not law yet. McNerney tried comparable rules last year in SB 7, which lawmakers approved and Gov. Gavin Newsom vetoed, calling it too sweeping and saying parts of it repeated existing law. This year's bill cleared the Senate in May and got past an important Assembly test earlier this month; McNerney told the newspaper he was confident of Assembly passage but unsure where the governor stood, and that they would have to wait and see. Both chambers must act before the month runs out, after which the governor would have roughly all of September to decide.