It’s been a busy summer in the clubhouse, and that’s probably because of it all. It’s the busiest summer ever for the club.
The powerful social audio app has introduced new messaging features for Android and iOS apps in recent months, but Clubhouse is now dedicated to improving its core audio experience. The company announced Sunday that their rooms will now be imbued with spatial sound, giving listeners a better sense of hanging out with a group of other people.
TechCrunch spoke to Clubhouse CEO Justin Uberti about the decision to add spatial audio effects that make different speakers sound like they come from different physical locations, not just one location.
Uberti joined Clubhouse as head of streaming technology in May after spending more than a decade at Google where he founded the Google Duo Hangouts team and worked on Google’s cloud gaming platform Stadia. He created the WebRTC standard on which the clubhouse is based.
One of the things you see in group audio settings is that you haven’t the same experience as you do in a physical space, Uberti says. In Clubhouse and other voice chat apps that bring people together in virtual social environments, the tone is flat, as if everything starts from a single central location.
For face-to-face meetings, Clubhouse is designed to simulate what you would hear in a room with a left and right stage, different places in the audience, and speakers asking questions. To perform these new audio tricks, the app integrates an API from Second Life creator Philip Rosedale, High Fidelity, and connects it to its own custom audio processing to fine-tune the app.
Its HRTF technology, which stands for head-to-head transmission, maps speech to various virtual locations, adds a time delay to each stereo channel and replicates the way high and low-frequency sound reaches the ear, depending on its origin.
Imagine listening in stereo to Pink Floyd’s Dark Side of the Moon with the best headphones, sounds, and instruments in your head, but you can also hear people you hang out with within a series of virtual rooms.
According to Uberti, the implementation is subtle but palpable. Audio processing controls the conversation by placing most speakers in front of the handset, giving clubhouse users a new sense of people speaking in different physical places. The result, when used with social VR, gives the virtual social experience a sense of physical presence that the best records have produced in ages.
The new audio feature will be available to the majority of iOS users on Sunday and will reach the rest of iOS and Android users in the coming weeks. The experience will be available at any time, and users will have the ability to turn spatial audio on and off. Clubhouse uses the same virtual soundstage technique that makes larger rooms feel larger and makes intimate rooms sound as if they are taking place in a smaller physical space.
Since most people use headphones to participate in the clubhouse, their users will benefit from the effects that are only possible with two stereo channels.
Uberti pointed out that spatial audio offers obvious benefits to regular clubhouse users. But it’s also possible that regular, non-spatial audio in social apps could contribute to the pandemic phenomenon of “zoom fatigue.”. They have this idea that people are distributed in a room, and we try to imitate the feeling in a circle where people are standing and talking.
The human brain processes virtual audio signals, such as phone calls or group audio signals in a room, and it distinguishes between speakers in a different way from a natural person. Without spatial cues, you have to use timbre, and that requires cognitive effort, Uberti says. Your mind needs to figure out who is talking to whom.
This ensures a more pleasant experience and more immersion. It’s too early to know if clubhouses will use spatial audio effects to enhance the experience of comedy, music, and ASMR, but the club is already there. “When I tell a joke, it feels flat,” Uberti said. In the clubhouse, you can feel the laughter coming from all around you, and it feels very much like a comedy club experience.