The Specific Event
California's SB 903, currently moving through the state legislature, would prohibit companies from marketing AI chatbots as therapy and would require licensed clinician review of AI therapeutic decisions. This is not a speculative regulatory proposal. It is a direct legislative response to an observable market condition: companies have been deploying conversational AI systems in clinical-adjacent contexts without the governance infrastructure that licensed practice requires. The bill draws a hard boundary between a chatbot that provides emotional support and one that is advertised as performing therapy. That distinction sounds clean in a legislative summary. It is considerably messier in practice.
Why the Therapy-Chatbot Boundary Is a Communication Problem, Not Just a Legal One
The framing of SB 903 as a consumer protection bill is accurate but incomplete. What the bill is actually trying to regulate is a specific kind of communication failure: the collapse of the distinction between a system's functional outputs and a user's interpretation of those outputs. Hancock, Naaman, and Levy (2020) identified AI-mediated communication as a category where the perceived source of a message fundamentally alters how that message is received and acted upon. When a user believes they are in a therapeutic relationship, they disclose differently, they interpret responses differently, and they rely on continuity of care assumptions that a chatbot cannot structurally honor. Banning the marketing of chatbots as therapy is an attempt to regulate the user's schema before interaction begins, because the interaction itself may be too late to correct it.
The Awareness-Capability Gap in Clinical Contexts
My dissertation research focuses on the gap between algorithmic awareness and effective response to algorithmic systems. The same structural problem appears in the SB 903 context, but with higher stakes. Algorithmic literacy research consistently shows that knowing a system is algorithmically mediated does not translate into knowing how to respond to it appropriately (Gagrain, Naab, and Grub, 2024). A user who has been told, in a terms-of-service disclosure, that they are interacting with an AI and not a licensed therapist has awareness. They do not necessarily have the schema to modulate their reliance behavior accordingly. SB 903 attempts to solve this by restricting what companies can claim, but it does not address what users will infer regardless of what companies claim. The marketing prohibition is a supply-side intervention for what is partly a demand-side cognition problem.
The Clinician Review Requirement and the Limits of Procedural Oversight
The bill's second provision, requiring licensed clinician review of AI therapeutic decisions, raises a distinct organizational question. Review requirements assume that the reviewing professional can accurately assess what the AI system did, why it did it, and whether that action was appropriate. This is the topology versus topography problem I have written about in other contexts. A clinician reviewing an AI-generated response can assess the surface content of that response. They cannot necessarily assess the structural logic by which the system generated it, what training signals shaped that output, or how the system would respond to the same user in a slightly different context. Kellogg, Valentine, and Christin (2020) documented that workers operating alongside algorithmic systems frequently develop folk theories about system behavior that do not accurately represent system structure. There is no reason to expect licensed clinicians to be immune to this pattern simply because their professional training is rigorous in other domains.
What This Means for Organizational Governance of AI Systems
The deeper issue SB 903 surfaces is that current organizational governance frameworks for AI were not designed for systems that communicate. Most AI governance literature focuses on decision-support systems where a human retains visible decisional authority. Therapeutic chatbots are different because the communication itself is the intervention. The system is not helping a clinician decide; the system is doing the thing that, in a licensed context, would constitute practice. Rahman (2021) described the organizational dynamics of algorithmic control as an invisible cage, where workers adapt to constraints they cannot fully see. The SB 903 scenario inverts this: users are not workers adapting to a platform, but vulnerable individuals potentially adapting their self-disclosure and help-seeking behavior to a system that has no professional accountability structure beneath it.
The Structural Takeaway
SB 903 is a meaningful legislative step, but it is solving the legible part of the problem. Restricting marketing language addresses how companies describe their products. It does not address the communication topology that makes AI-mediated therapeutic interaction categorically different from other AI-mediated tasks. Governance frameworks that focus exclusively on claims and disclosures will consistently lag behind the actual reliance behaviors they are trying to manage. The structural features of how users form relationships with AI communication systems need to be part of the regulatory analysis, not an afterthought to it.
Roger Hunt