Bill > SB1546

This bill defines "artificial intelligence companion" as a system designed to simulate a human-like relationship by retaining user information, asking personal questions, and engaging in ongoing personal dialogue, but excludes customer service software, video game interactions, and basic voice assistants. Operators of these companions must clearly notify users if they are interacting with artificial output rather than a natural person. Furthermore, operators must have protocols to detect and prevent content that encourages suicidal ideation or self-harm, including providing referrals to the national 9-8-8 suicide and crisis lifeline and, for users under 25, a youthline, and must publish details of these protocols. If an operator knows a user is a minor, they must take reasonable steps to prevent the companion from claiming sentience, simulating emotional dependence, or engaging in romantic or sexual innuendo, and must remind the minor every three hours that they are interacting with artificial output and should take a break. Operators are also prohibited from using reward systems to maximize user engagement, generating messages of simulated distress when a user wants to disengage, or making material misrepresentations about the companion's identity or capabilities. Finally, operators must annually report the number of referrals made and the details of their protocols, and individuals harmed by an operator's violation can sue for damages and an injunction.