Q: Your study turned up something pretty striking: People consistently misjudge how their personalized AI will behave, overestimating the good traits and underestimating potentially harmful ones like sycophancy. What does that tell us about the risks baked into how millions of people are currently building AI companions, and why is that blind spot so hard to close?

A: I often joke that if AI showed up looking like the Terminator, it would be much easier for us to know what to do. The real challenge is that AI often appears as a warm friend, coach, tutor, or companion. That makes it difficult to recognize when something is going wrong.

Our study suggests that people have a blind spot when designing personalized AI. People often think they know how their chatbot will behave, but in our study they incorrectly predicted its personality on 11 of the 15 traits we measured. That highlights the need for tools that help people better understand AI before they start using it.

This matters because some behaviors that feel helpful in the moment may not be healthy over time. In previous research, we documented cases of psychological harm associated with interactions with AI chatbots. An LLM [large language model] that constantly validates your opinions or never challenges your thinking can reinforce harmful decisions, unhealthy beliefs, or emotional dependency. Psychology has long shown that people are naturally drawn to affirmation, so designing AI is not only a technical challenge, but also a psychological one.

The deeper issue is that today’s AI systems remain largely black boxes: Even experts cannot always predict how a system prompt will shape an AI’s behavior over a long conversation. As AI companions become part of everyday life, we need tools that help people understand what they are building before they begin using it. AI should be supportive without becoming blindly agreeable, personalized without becoming manipulative, and transparent enough that people can make informed choices.

Q: One of your most interesting findings is that the visualization significantly increased user trust but didn’t actually change how people designed their chatbots. What will it take to close that gap, and where do you see tools like this heading as AI companions become more deeply embedded in people’s everyday lives?

A: I actually think this is one of the most interesting findings in the paper, because it shows that transparency alone is not enough. People appreciated being able to see inside the model and reported greater trust in the system, but simply presenting information did not fundamentally change how they designed their AI companions.  

In our followup work, which is currently available as a preprint, we are studying how a model’s internal neural representation changes over the course of a multi-turn conversation rather than remaining fixed from the initial prompt. We are already seeing promising results. By visualizing how these internal representations drift over time, people become significantly better at recognizing and anticipating changes in AI behavior, and are less likely to become overconfident in their understanding of the chatbot. AI companions are dynamic systems that evolve as they interact with us, so understanding those internal changes is an important next step. Nevertheless, this is still a very young research area. 

Looking further ahead, I believe these kinds of transparency tools could become as commonplace as nutrition labels are for food. As AI becomes deeply woven into education, health care, work, and personal relationships, people should be able to understand not only what an AI can do, but how it may influence their thinking, emotions, and behavior. That kind of transparency is essential if we want AI to genuinely help people flourish.