Surround Sound

Working on the Experience Centers team at Google, Colette Torres and I were given the task of creating an exhibit in the Holodeck space at the Google Partner Plex that somehow used the kinect sensors on the walls and the Youtube API. Using the design thinking process to brainstorm, we decided to create this immersive music visualization experience that shows off Youtube and Google's technology.

I worked on the front end of the exhibit, using Firebase as the persistent database, node.js as server side Javascript, p5.js for the visualizations and sound analysis, and the Youtube API to access the top trending music videos in specified regions. A user first walks in and stands on one of the circles on the floor, say Brazil's. The kinect cameras use depth sensing to detect that a person is on that circle and tells the wall's computer to cue one of the currently top trending music videos in Brazil and to display the music visualization specified for that location.

The exhibit is configurable: users can type in up to six countries of their choosing and see the whole exhibit change based around those regions' data. Also, the color palette of each music visualization is chosen based on the dominant colors used in the video's thumbnail. This way, each visualization's aestheic matches that of the music video it's surrounding.