Resynthesizing Reality: Driving Vivid Virtual Environments from Sensor Networks

The rise of ubiquitous sensing enables the harvesting of massive amounts of data from the physical world. This data is often used to drive the behavior of devices, and when presented to users, it is most commonly visualized quantitatively, as graphs and charts. Another approach for the representation of sensor network data presents the data within a rich, virtual environment. These scenes can be generated based on the physical environment, and their appearance can change based on the state of sensor nodes. By freely exploring these environments, users gain a vivid, multi-modal, and experiential perspective into large, multi-dimensional datasets. This paper presents the concept of "Resynthesizing Reality" through a case study we have created based on a network of environmental sensors deployed at a large-scale wetland restoration site. We describe the technical implementation of our system, present techniques to visualize sensor data within the virtual environment, and discuss potential applications for such Resynthesized Realities.


Donald Derek Haddad, Gershon Dublon, Brian Mayton, Xiao Xiao, Ken Perlin, and Joe Paradiso. Resynthesizing Reality: Driving Vivid Virtual Environments from Sensor Networks,” ACM SIGGRAPH 2017 Talks, SIGGRAPH 2017, Los Angeles, CA, July 2017. DOI:


Donald Derek Haddad
Gershon Dublon
Brian Mayton
Xiao Xiao
Ken Perlin
Joe Paradiso


MIT Media Lab