About
About
This project was my first real exploration of controlling and visualizing DMX lighting directly inside Unreal Engine. The goal was to create a clean workflow between the physical studio hardware and virtual environments.
Using Unreal Engine’s Remote Control setup, I built a system that let me control the physical DMX fixtures in our studio entirely from my phone. Most modern lights have some form of basic Bluetooth app control, but routing everything through Unreal Engine opened up a much more capable production workflow.
The most useful part of this setup was a digital twin of the studio: a virtual replica of the physical space where the lights mirrored each other in real time. Adjusting a physical fixture updated the virtual one instantly, matching intensity, color, and position.
To make the previs actually useful for production, I added a MetaHuman to the digital twin. This let me see how the lighting would fall on a human subject without needing anyone in the studio, making the whole design process much faster.
Because the digital twin accurately represented the physical space, lighting setups could be designed entirely remotely. You could dial in the whole look at home, save it as a preset, walk into the studio, and immediately execute it without any guesswork.
For my Master’s thesis I went further into what was possible at the intersection of physical studio lighting and real-time digital environments. The core idea was to build a lighting ecosystem where the physical positions of DMX fixtures drive their output based on the virtual world behind them, and where physical lights can also drive the digital environment’s lighting in return.
The first major component involved tracking the physical positions of DMX fixtures within a studio volume. I paired that tracking data with Unreal Engine’s pixel mapping system.
With the light’s physical coordinates tracked, the system automatically pixel maps each fixture to the section of the digital scene directly behind it. Move a light anywhere within the tracking volume and it dynamically changes color and intensity to match that part of the virtual world. This removes the need to reprogram the lighting desk manually every time a camera angle or set configuration changes.
I also explored the reverse: having physical lights drive the digital environment.
One setup used a tracked physical light to act as the sun on set. Moving it around the studio updated the directional light in Unreal Engine, changing the time of day, shadows, and overall environment lighting at the same time.
This same idea applies to practical lights on set, like a desk lamp. By linking the physical lamp to a digital point light, the virtual environment lighting stays in sync with the physical set no matter where the practical light gets moved during a shoot.
This approach removes a huge amount of guesswork from virtual production shoots. With physical and digital lighting locked together through spatial tracking, anyone on set can work with real-world lights while seeing mathematically accurate lighting updates in the virtual world in real time.