Raytheon
Rebuilding how investigators see the sky in 3D and VR
Defense
3D/VR
Deployed 2026
Role
Lead Product Designer
Timeline
Jan 2025 - Jun 2026
Team
1 PD, 3 UX Researchers, 3 Developers
Overview
The Client
I built this for my senior capstone with Raytheon. Investigators were using flat radar screens, trying to picture a 3D emergency in their heads. Our RTX partner Omar showed me this pain point firsthand and I knew it needed more than a guess.
The Task
We saw an opportunity to use 3D and VR to transform aircraft data into something investigators could see, replay, and explore.
But introducing VR created a second challenge: How do we add another dimension without adding another layer of complexity?
My Impact
I took this from a blank page to a working product where we demoed live to Raytheon engineers and presented at the Texas A&M Capstone Design Conference. What started as one question, "what if you could just see what happened? " turned into something people could walk through with their own eyes.
Research
How do investigators turn flat flight data into an understanding of what actually happened?
I entered the project knowing almost nothing about aviation investigation, so before designing anything in VR, I needed to understand how investigators actually piece an incident together.
I spoke with 10+ investigators and analysts, mapped their workflow, and studied the tools they already relied on to see where understanding started to break down.
The tools were good at showing data but they were less effective at showing relationships.
Research Insights
The biggest friction wasn't missing data but understanding the data.
Across interviews and workflow analysis, three needs kept appearing.
Define
Let the airspace do the explaining.
The product shouldn't feel like another dashboard in VR. If altitude, trajectory, proximity and movement could be understood through the environment itself, the interface didn't need to explain everything twice.
Solution
The final product runs in both desktop and Meta Quest 3 VR. Four core features where each feature tied directly to something investigators told us they needed.
Incident Analysis
Data to the Sky
Taking ADS-B data from the FlightAware Firehose API and processing it through our Java Spring Boot backend, we reconstruct the sky in a 3D VR space using the Unity game engine. Flight trails render every aircraft's historical path — always on, always visible.
VR Environment
Inside the Airspace
Built around the Meta XR SDK, users can navigate the airspace through flight locomotion or a teleport mechanic. Collapsible metadata panels surface aircraft information on demand, and flight paths show future and past trajectories at a glance.
Playback and Investigation
The Sky on Replay
Time can be rewound and manipulated within any incident replay. Using the playback manager, the scrubber can be dragged to any point in time, with controls for pausing, resuming, and changing the speed of playback — 1x, 2x, and beyond.
Reflection
Presented live to Raytheon engineers and travelled to the Texas A&M Capstone Design Conference.
What started as a research question of what if investigators could see the incident instead of reading about it, became the central question we worked around and made it a reality. I'm so grateful to be able to present our findings, ideation and work to so many talented designers, and engineers at Texas A&M Design Conference who found our project novel and the Raytheon staff who have provided so much guidance and support throughout the development of our project. Everyone's support made this project worthwhile!
heyo!
To view this content, please use a laptop to see all the amazing designs.
I'm glad we crossed paths. I hope it left you inspired!
@ Jerry Nguyen 2026
I'm glad we crossed paths. I hope it left you inspired!
@ Jerry Nguyen 2026