\u201cMost people in my world\u2014including most of my work\u2014conduct experiments using artificial types of stimuli, far from the natural world,\u201d Tadin says. \u201cOur goal is to do multisensory research not using beeps and flashes, but real sounds and virtual reality objects presented in realistically looking VR rooms.\u201d<\/p>\n
A cognitive scientist, a historian, and an electrical engineer walk into a room . . . <\/strong><\/h3>\nTadin\u2019s partners in the study include Emily Knight<\/a>, an incoming associate professor of pediatrics, who is an expert on brain development and multisensory processing in autism. But in creating the virtual reality environment the study participants will use\u2014a virtual version of Kodak Hall<\/a> at Eastman Theatre in downtown Rochester\u2014Tadin formed collaborations well outside his discipline.<\/p>\nFaculty members working on this initial step in the research project include Ming-Lun Lee<\/a>, an associate professor of electrical and computer engineering, and Michael Jarvis<\/a>, an associate professor of history. Several graduate and undergraduate students are also participating.<\/p>\nMany of the tools they\u2019ll use come from River Campus Libraries<\/a>\u2014in particular, Studio X<\/a>, the University\u2019s hub for extended reality projects, as well as the Digital Scholarship<\/a> department. Emily Sherwood<\/a>, director of Studio X and Digital Scholarship, is leading the effort to actually construct the virtual replica of Kodak Hall.<\/p>\nThe group recently gathered in the storied performance space to collect the audio and visual data that Studio X will rely on. University photographer J. Adam Fenster followed along to document the group\u2019s work.<\/p>\n
\n<\/h3>\n
![\"Two](\"https:\/\/www.rochester.edu\/newscenter\/wp-content\/uploads\/2022\/06\/VR-Kodak-Hall-1-2.jpg\")
<\/p>\n
1<\/strong><\/h3>\nMembers of the team begin the setup for audio and visual data collection. From left to right are Shui\u2019er Han, a postdoctoral research fellow in Duje Tadin\u2019s lab; brain and cognitive sciences major Betty Wu \u201923; computer science and business major and e5<\/a> student Haochen Zeng \u201923, who works in River Campus Libraries\u2019s Studio X; and Victoire Alleluia Shenge \u201919, \u201920 (Take Five<\/a>), who earned her degree in brain and cognitive sciences and is a manager in Tadin\u2019s lab.<\/p>\n <\/p>\n
![\"Woman](\"https:\/\/www.rochester.edu\/newscenter\/wp-content\/uploads\/2022\/06\/VR-Kodak-Hall-2.jpg\")
<\/p>\n
2<\/strong><\/h3>\nHan positions a dummy head<\/a> equipped with a microphone in each ear. It will record sound from two perspectives, just as a human in the room would hear it. \u00a0\u00a0<\/span><\/p>\n <\/p>\n
![\"Two](\"https:\/\/www.rochester.edu\/newscenter\/wp-content\/uploads\/2022\/06\/VR-Kodak-Hall-3.jpg\")
<\/p>\n
3<\/strong><\/h3>\nTeam members (clockwise from lower left foreground) Ming-Lun Lee, an associate professor of electrical and computer engineering and an expert in spatial audio, Wu, Shenge, electrical engineering PhD student Steve Philbert, and Han measure and record the location of the microphones on the Kodak Hall stage.<\/p>\n
<\/p>\n
![\"Two](\"https:\/\/www.rochester.edu\/newscenter\/wp-content\/uploads\/2022\/06\/VR-Kodak-Hall-4.jpg\")
<\/p>\n
4<\/strong><\/h3>\nPhilbert and Wu measure the distance between the binaural dummy head and a speaker<\/a>. <\/span><\/p>\n <\/p>\n
![\"Man](\"https:\/\/www.rochester.edu\/newscenter\/wp-content\/uploads\/2022\/06\/VR-Kodak-Hall-5.jpg\")
<\/p>\n
5<\/strong><\/h3>\nMichael Jarvis, an associate professor of history, prepares a FARO laser scanner<\/a> on the Kodak Hall stage. Jarvis is an expert on using 3D modeling software to digitally capture and create interactive spaces for users to virtually visit and tour. His digital history projects have included digitally capturing and analyzing archaeological sites modeling historical buildings in Bermuda<\/a> and in Ghana<\/a>.<\/p>\n <\/p>\n
![\"Several](\"https:\/\/www.rochester.edu\/newscenter\/wp-content\/uploads\/2022\/06\/VR-Kodak-Hall-6.jpg\")
<\/p>\n
6<\/strong><\/h3>\nTeam members wait outside Kodak Hall during a visual scan.\u00a0Placing a single FARO laser scanner at various points around Kodak Hall, the group made eight scans, each taking approximately 12 minutes, and together capturing roughly one billion data points that will be used to generate a 3D model of the hall interior.<\/p>\n
<\/p>\n
![\"Man](\"https:\/\/www.rochester.edu\/newscenter\/wp-content\/uploads\/2022\/06\/VR-Kodak-Hall-7.jpg\")
<\/p>\n
7<\/strong><\/h3>\nJarvis takes photos around the hall that will be stitched together to make a complementary photogrammetry model of the space, augmenting the laser scans in areas where data points are sparse or missing. Photogrammetry models, in which 2D images are used to extract information to produce a 3D model, are less accurate than 3D models made with laser scans, but they have higher resolution.<\/p>\n
<\/p>\n
![\"Two](\"https:\/\/www.rochester.edu\/newscenter\/wp-content\/uploads\/2022\/06\/VR-Kodak-Hall-8.jpg\")
<\/p>\n
8<\/strong><\/h3>\nPhilbert, audio and music engineering major Joey Willenbucher \u201923, and Tadin reposition a speaker. The speaker must be horizontally aligned with the microphone to best capture audio data.<\/p>\n
<\/p>\n
![\"Woman](\"https:\/\/www.rochester.edu\/newscenter\/wp-content\/uploads\/2022\/06\/VR-Kodak-Hall-9.jpg\")
<\/p>\n
9<\/strong><\/h3>\nShenge uses a laser rangefinder, which generates the orange dot on the paper held by Tadin, to determine the optimal distance between a speaker and the binaural dummy head.<\/p>\n
<\/p>\n
![\"Two](\"https:\/\/www.rochester.edu\/newscenter\/wp-content\/uploads\/2022\/06\/VR-Kodak-Hall-10.jpg\")
<\/p>\n
10<\/strong><\/h3>\nBlair Tinker (right), a research specialist for Geographic Information Systems Digital Scholarship at the River Campus Libraries, and Zeng are among those at Digital Scholarship and Studio X who will turn the visual and audio data into a VR replica of Kodak Hall that participants in Tadin\u2019s study will use.<\/p>\n
Says Emily Sherwood, director of Studio X and Digital Scholarship, Tadin\u2019s project is \u201ca prime example of how research using or on extended reality requires expertise and support from a broad range of disciplines. Having a hub like Studio X helps facilitate those connections because we know what different faculty are working on and how they might be able to work together.\u201d<\/p>\n
Studio X offers students, faculty, and staff equipment<\/a> necessary for a range of projects, from creating a VR replica of Kodak Hall, to holding VR game competitions.<\/p>\n
\nRead more<\/strong><\/h3>\n\n![\"illustration](\"https:\/\/www.rochester.edu\/newscenter\/wp-content\/uploads\/2021\/04\/ar-vr-glasses-illustration.jpg\")
A new way to make AR\/VR glasses look like regular glasses<\/strong><\/a>
\nRochester researchers are combining freeform optics and a metasurface to avoid \u2018bug eyes\u2019 in AR\/VR glasses and headsets.<\/span><\/div>\n
Faculty members working on this initial step in the research project include Ming-Lun Lee<\/a>, an associate professor of electrical and computer engineering, and Michael Jarvis<\/a>, an associate professor of history. Several graduate and undergraduate students are also participating.<\/p>\n Many of the tools they\u2019ll use come from River Campus Libraries<\/a>\u2014in particular, Studio X<\/a>, the University\u2019s hub for extended reality projects, as well as the Digital Scholarship<\/a> department. Emily Sherwood<\/a>, director of Studio X and Digital Scholarship, is leading the effort to actually construct the virtual replica of Kodak Hall.<\/p>\n The group recently gathered in the storied performance space to collect the audio and visual data that Studio X will rely on. University photographer J. Adam Fenster followed along to document the group\u2019s work.<\/p>\n Members of the team begin the setup for audio and visual data collection. From left to right are Shui\u2019er Han, a postdoctoral research fellow in Duje Tadin\u2019s lab; brain and cognitive sciences major Betty Wu \u201923; computer science and business major and e5<\/a> student Haochen Zeng \u201923, who works in River Campus Libraries\u2019s Studio X; and Victoire Alleluia Shenge \u201919, \u201920 (Take Five<\/a>), who earned her degree in brain and cognitive sciences and is a manager in Tadin\u2019s lab.<\/p>\n <\/p>\n Han positions a dummy head<\/a> equipped with a microphone in each ear. It will record sound from two perspectives, just as a human in the room would hear it. \u00a0\u00a0<\/span><\/p>\n <\/p>\n Team members (clockwise from lower left foreground) Ming-Lun Lee, an associate professor of electrical and computer engineering and an expert in spatial audio, Wu, Shenge, electrical engineering PhD student Steve Philbert, and Han measure and record the location of the microphones on the Kodak Hall stage.<\/p>\n <\/p>\n Philbert and Wu measure the distance between the binaural dummy head and a speaker<\/a>. <\/span><\/p>\n <\/p>\n Michael Jarvis, an associate professor of history, prepares a FARO laser scanner<\/a> on the Kodak Hall stage. Jarvis is an expert on using 3D modeling software to digitally capture and create interactive spaces for users to virtually visit and tour. His digital history projects have included digitally capturing and analyzing archaeological sites modeling historical buildings in Bermuda<\/a> and in Ghana<\/a>.<\/p>\n <\/p>\n Team members wait outside Kodak Hall during a visual scan.\u00a0Placing a single FARO laser scanner at various points around Kodak Hall, the group made eight scans, each taking approximately 12 minutes, and together capturing roughly one billion data points that will be used to generate a 3D model of the hall interior.<\/p>\n <\/p>\n Jarvis takes photos around the hall that will be stitched together to make a complementary photogrammetry model of the space, augmenting the laser scans in areas where data points are sparse or missing. Photogrammetry models, in which 2D images are used to extract information to produce a 3D model, are less accurate than 3D models made with laser scans, but they have higher resolution.<\/p>\n <\/p>\n Philbert, audio and music engineering major Joey Willenbucher \u201923, and Tadin reposition a speaker. The speaker must be horizontally aligned with the microphone to best capture audio data.<\/p>\n <\/p>\n Shenge uses a laser rangefinder, which generates the orange dot on the paper held by Tadin, to determine the optimal distance between a speaker and the binaural dummy head.<\/p>\n <\/p>\n Blair Tinker (right), a research specialist for Geographic Information Systems Digital Scholarship at the River Campus Libraries, and Zeng are among those at Digital Scholarship and Studio X who will turn the visual and audio data into a VR replica of Kodak Hall that participants in Tadin\u2019s study will use.<\/p>\n Says Emily Sherwood, director of Studio X and Digital Scholarship, Tadin\u2019s project is \u201ca prime example of how research using or on extended reality requires expertise and support from a broad range of disciplines. Having a hub like Studio X helps facilitate those connections because we know what different faculty are working on and how they might be able to work together.\u201d<\/p>\n Studio X offers students, faculty, and staff equipment<\/a> necessary for a range of projects, from creating a VR replica of Kodak Hall, to holding VR game competitions.<\/p>\n
\n<\/h3>\n
<\/p>\n1<\/strong><\/h3>\n
<\/p>\n2<\/strong><\/h3>\n
<\/p>\n3<\/strong><\/h3>\n
<\/p>\n4<\/strong><\/h3>\n
<\/p>\n5<\/strong><\/h3>\n
<\/p>\n6<\/strong><\/h3>\n
<\/p>\n7<\/strong><\/h3>\n
<\/p>\n8<\/strong><\/h3>\n
<\/p>\n9<\/strong><\/h3>\n
<\/p>\n10<\/strong><\/h3>\n
\nRead more<\/strong><\/h3>\n
A new way to make AR\/VR glasses look like regular glasses<\/strong><\/a>
\nRochester researchers are combining freeform optics and a metasurface to avoid \u2018bug eyes\u2019 in AR\/VR glasses and headsets.<\/span><\/div>\n
![\"Woman](\"https:\/\/www.rochester.edu\/newscenter\/wp-content\/uploads\/2018\/05\/exspace.jpg\")
Seniors show beauty of urban art with augmented reality<\/strong><\/a>![\"Student](\"https:\/\/www.rochester.edu\/newscenter\/wp-content\/uploads\/2022\/04\/fea-studio-x-carlson-technology.jpg\")
Studio X, technically<\/strong><\/a>