Automated vehicles are becoming more capable, yet many crashes and failures still occur when people disengage or when the system behaves in ways that are difficult to understand. These problems reflect a deeper issue: current systems treat all users the same and fail to account for differences in individuals’ attention, intent, and decision-making. This project examines how an automated vehicle can work with its occupant as a partner rather than as a passive user. It develops methods for the vehicle to interpret what the occupant is seeing, saying, and doing, and to use this information to respond in a way that better matches the situation. By aligning vehicle behavior with human expectations, the research aims to reduce unsafe interactions, improve trust in automation, and support wider adoption of automated transportation. The results will be relevant not only to roadway systems, but also to other safety-critical settings where people and intelligent machines must work together.
This project establishes a framework that connects behavioral sensing with real-time decision-making in automated vehicles. It begins by studying how visual attention, spoken language, and physical actions reveal an occupant’s cognitive state during interaction with automation. These signals are used to build inference models that estimate intent and track how it changes over time. The research then examines how these estimates vary across individuals, allowing the system to adapt to different users rather than relying on fixed interaction rules. Based on these models, the project develops a shared-control approach in which the vehicle adjusts its behavior according to the inferred state, including when to act independently, defer to the occupant, or intervene. The framework is tested using simulation and an instrumented vehicle platform, with experiments designed to measure safety, responsiveness, and alignment between human and system behavior. The work contributes a new understanding of human-autonomy interaction and introduces methods for incorporating human state into autonomous systems in a principled way.
This award reflects NSF’s statutory mission and has been deemed worthy of support through evaluation using the Foundation’s intellectual merit and broader impacts review criteria.