Flaggers are essential for maintaining traffic safety in work zones, serving as human traffic controllers who coordinate alternating traffic through the work zone, yet they work under extremely hazardous conditions in close proximity to high-speed traffic and heavy equipment. Traditional classroom-based training is often insufficient for developing situational awareness, hazard recognition, and communication skills, while real-world training exposes trainees to significant risk. Virtual reality (VR) offers immersive, hands-on practice without danger, but current VR systems rely on preprogrammed scenarios and require instructors to manually identify trainee errors.
This project develops an LLM-based virtual flagger that provides dynamic, realistic interactions within a VR training environment rather than rigid, pre-scripted scenarios. The virtual flagger engages in natural radio communication, responds contextually to trainees’ actions, asks clarifying questions when communication is unclear, and adapts its behavior to create diverse, progressively challenging training experiences.
The team will develop a proof-of-concept VR flagger training system incorporating an LLM-based virtual flagger. Training materials will be collected from two to three state DOTs and three to four experienced flaggers to build a focused corpus of 30 to 50 dialogue examples covering a standard work-zone scenario (MUTCD TA-10: Lane Closure on a Two-Lane Road). An open-source LLM will be configured through prompt engineering to generate MUTCD-compliant responses, and a VR environment built in Unreal Engine 5 will be deployed on Meta Quest 2/3 headsets, integrated through a speech-to-text, LLM-processing, text-to-speech, and avatar-animation pipeline. Deliverables include a working prototype, a demonstration video, technical documentation, a pilot study report with 10 to 12 participants evaluated using the System Usability Scale, and a requirements document for full-scale development.
Universities Involved
University of Pittsburgh
Principal Investigators
Lev Khazanovich
Aleksandar Stevanovic
Expected Research Outcomes & Impacts
The proposed LLM-based VR flagger training prototype has the potential to transform transportation workforce training by demonstrating the first successful integration of conversational AI and immersive VR within a safety-critical domain. By making the single-trainee version nearly as effective as a two-trainee collaborative setup, while eliminating the need for internet connectivity, simultaneous trainee availability, and two separate VR-equipped spaces, the system makes training more scalable, accessible, and cost-effective.
The prototype will establish technical feasibility, provide baseline data on training effectiveness, and create a replicable development methodology, helping to address a significant research gap in which most VR applications in transportation remain isolated pilot studies with limited scalability. Expected benefits include improved work-zone safety through stronger hazard recognition, communication, and decision-making skills, and expanded access to high-quality workforce development for rural and underserved communities.
Subject Areas
Work Zone Safety, Virtual Reality, Large Language Models, Workforce Development, Transportation Training
