AI Agents in Digital Twins: Bringing Promises to Life?

Are 4D digital twins and AI bringing the tech’s promises to life? And how do cities feel about them? As we’ve explored in many videos, digital twins are virtual representations of assets, processes, entire cities or even entire countries. They allow cities and utilities to better monitor and manage what’s happening on the ground. They’ve been around for a long time. But recent advances in AI, edge computing and simulation are enabling 4D digital twins, systems that don’t just model cities in three dimensions, but evolve over time with live data. This allows cities not only to visualise their environment, but also simulate, predict and plan for what might happen next. These are still early days, I'd say with 4D digital twins. It was only a year or so ago that really the technology existed to be able to do this and deliver it in this way as well. Voxelmaps focuses on creating high-resolution maps for machines, from AI models to digital twins. In San Jose (US), the company has worked with NVIDIA and partners such as Linkervision and Inchor to map the city and explore potential digital twin applications. That work has now led to discussions with the city to test a system that captures real-time street-level data, processes it at the edge, and provides the municipality with a continuously updated model they can ask questions in natural language. The initial use case would be traffic management. Today, retiming traffic signals is a complex and resource-intensive task. It often requires surveys, sensors, manual counting, and modelling entire corridors, work that must be repeated as conditions evolve. Although San Jose has been approached about digital twin tech for over a decade, what’s prompting the city to consider a proof of concept is that what’s different now is not just the addition of time, but the ability to combine real-time data with AI models that can simulate and recommend actions, something particularly valuable for mobility. The way that NVIDIA demonstrated what they're hoping to do with their AI agents is that they basically are able to model the traffic patterns in real time, and when they see certain indicators come in, their AI agent has identified traffic plans that could optimize that corridor. And then you can basically create an adaptive signal by just sending that information in real time. Using NVIDIA's, VSS or Cosmos Reason 2 engine, which is a video language and model. So VLM. And that allows us to look at the city and start talking to the city. So in natural language, we can ask the city questions about infrastructure, the state of infrastructure, how many things are there in a particular scene as well? So it really starts to move it beyond the typical kind of GIS kind of use cases. For San Jose, the AI agents would be the game changers, especially for traffic signal optimisation. Some of the traffic signals must be adapted manually, which can be costly and time consuming, according to Mariani. If AI agents can analyse traffic camera feeds and provide real-time recommendations, cities could move towards more adaptive and efficient systems. The digital twin would be the base. Voxelmaps equips vehicles, helicopters and people with cameras and LiDAR technology that scan the city. They merge it all together to create the virtual replica, and then on top, simulations can run by integrating live data feeds and sensor inputs from project partners. While the technology is more mature today, adoption still requires significant effort, making it hard for cities to fully commit to the new tech. We're not talking about stuff that is just new. It's bleeding edge. No one has seen some of the stuff that they're developing. These are truly like some of the leaders in the industry. So, you know, it's uncomfortable. But I think that that's also the hesitation for cities is that a lot of times we're approached so early that we get basically kind of burned. And you learn the hard way that things aren't ready yet. For now, the question is no longer whether digital twins are possible, but whether they can move from demonstration to deployment and whether this new generation powered by AI can finally make them part of everyday city operations.