Machine Learning Engineer at Sonder
he/him
Dr. Dalin Wang is a Machine Learning Engineer at Sonder, working on LLM training and evaluation. Previously, he completed his PhD at the University of Melbourne.
Evaluating Large Language Models is hard. Evaluating them for mental health and healthcare triage is a high-stakes balancing act where standard industry benchmarks simply don't cut it. In this session, we'll explore the current LLM evaluation landscape—from human review to LLM-as-a-judge—and why off-the-shelf metrics fall short in healthcare domains. From there, we will open the hood on Sonder's custom evaluation architecture, deep-diving into the three toughest challenges we had to solve: First, grounding: how we verify that model responses stay faithful to Sonder's own procedures, protocols, and context rather than drifting into generic or fabricated medical knowledge. Second, safety and compliance: how we test for appropriate escalation, unsafe behavior in sensitive scenarios related to mental health. Third, tone alignment: how we assess whether responses strike the right register for a health conversation — empathetic but not presumptuous, clear but not clinical-cold, confident but not overreaching — and why tone failures are often harder to catch than factual ones. Walk away with a blueprint for your own evaluation pipelines and actionable lessons for building trustworthy AI in any highly regulated or health-adjacent product.
