As AI systems become increasingly sophisticated at inferring emotional states and personalizing interactions, companies have been pursuing a clear objective: to increase user trust. But what happens when this objective becomes too successful?

In this talk, we explore the concept of “overtrust” and how users’ psychological dependence creates new risk vectors that traditional IT metrics fail to capture. We examine how decision automation and subjective inference can gradually erode user autonomy, creating not only ethical dilemmas but also critical legal and operational vulnerabilities for organizations.