AI Camera Technology: Smarter Optics, Smarter Framing

AVH Technology Partners’ projects are based upon the absolute most cutting-edge technology available. When accomplishing our client’s goals, a common theme has emerged among the ever-evolving hybrid-work landscape:

Far-end participants (those working from home or abroad) feel a disconnect in speaking to a large conference room. This is worsened when there are a small number of local participants using a larger conference space. It’s visually jarring, and it’s hard to see subtle expressions and responses that make video meetings so invaluable to an organization.

Consequently, many manufacturers are using innovative new technologies coupled with AI to bring everyone closer, no matter the configuration of the space.

This is no small feat – Fixed cameras can only see so much within a given field, while multi-camera systems are complex and cost-prohibitive for wide deployment. Additionally, attendance in these spaces can be fluid and it’s difficult to predict the behaviors of participants, before or during the meeting.

These advancements allow for a single image to be analyzed and framed dynamically, maximizing the impact of the meeting for remote participants.

Technology leaders such as Crestron, Poly (recently acquired by tech titan HP), and Cisco – to name a small few of the absolute horde of new innovators coming to market – are introducing this tech in a number of new ways.

Auto-Framing:

Auto-Framing technology is at the forefront of these solutions, given its universally applicable nature. One such example can be seen in action in HP/Poly’s flagship X70 and E70, released just over one year ago. These boast an impressive optical appointment of two (2) ultra-high-resolution cameras, each with their own 4K+ sensor and specialized optical ranges. This allows for some impressive digital magic behind the scenes. Using DirectorAI (Poly’s name for their specific brand of the technology), the cameras “steer” digitally to encompass the space as needed for any scenario a room may be used in. This means participants can enjoy being framed correctly as a pair or as a full team, with no action required on the part of the end-user.

The simplicity is a game-changer. It serves to bridge the gap between users in the office and those who are working remotely by reintroducing emotion, reactivity, and expression back into the conversations with teammates. There’s no longer an indeterminate gap between the participants in one office and their audience – The AI in the camera zooms in to fit the number of attendees in the visual field of the most appropriate of the X70’s dual cameras. As this process occurs in the background, the second camera is searching the space for newcomers to the meeting, or to keep track of roaming speakers. Whenever the number or position of participants in frame changes, the intelligent software inside the X70 seamlessly pans, zooms or transfers the image displayed from the first camera to the second. Critically, this is done in a controlled, consistent fashion that avoids the disorienting over-steering that some early examples of this technology are prone to.

Smart-Framing:

Smart-Framing technology (Sometimes referred to as People Framing Mode) is gradually becoming more common in high-end and purpose-built hardware. This method takes a different approach to solving the spatial divide present in large conference spaces. The camera hardware relies on AI to detect the face and silhouette of the attendees in the meeting, and then breaks the image into different, equally sized pieces, portioning the image according to the number of individuals in a meeting.

The far end sees a grid of close-up portraits, each a member of the meeting; framed and centered perfectly. Everyone gets the same tailored equality on camera, as if they are at their personal desk in front of a dedicated camera. This results in highly professional and impactful imagery in any video conferencing environment, large or small. Users are sized equally, so determining subtle reactions, expressions, and gestures comes naturally. It’s like being at the table, sitting directly across from everyone.

This tech is in its infancy, however, and still faces some growing pains – There are limits on how many faces can be analyzed and framed, often no more than 4-6 participants. Utilizing this option also requires considerations in design and may call for additional, specific, camera hardware. In current implementations, users can seem to be gazing into the distance due to camera location, and participants need to be aware of their positioning relative to one another to prevent blocking the view of those seated beside them.

Loud and Clear:

Meeting success depends as much on being seen as being understood clearly. Even with more natural framing and visuals, it’s critical those participants using the room are heard clearly and consistently. Participants seated further away from the equipment should be heard equitably to those seated nearest, with equal volume, using minimal or no additional hardware. By using complex microphone technology across an array of far and near reaching microphones, the onboard AI of these units manages to accomplish all the above, within a set arc area of about 25’. Utilizing beam-forming arrays of microphones, AI is able to effectively identify who is speaking and “steer” the audio pickup area toward the current speaker or speakers, exclusively. Behind the scenes, radical new noise cancellation technology works with the onboard AI to identify noises and remove them from the feed. Far end participants are spared the noisy keyboards, food packaging, and traffic sounds that can plague meetings and reduce effectiveness of virtual collaborations.

The Takeaway:

Studies from collaboration leaders WebEx have shown enhancing the overall experience of the meeting in these ways can afford visual and audible equity for all attendees, lead to more effective dialogues, and facilitate collaboration and peak participation – something critical for maintaining productivity amongst the 57% of employees who expect to be in the office fewer than 10 days each month.

It’s quite likely these methods are applicable to your organization’s collaboration needs, and perhaps more affordably than ever before. AVH Technology Partners are ready to tailor a system that contours to your team’s specific use-case. If you’re interested in understanding how this exciting new technology can bring your most important assets together, regardless of distance, let us know. We’d be thrilled to be a part of creating your group’s next great project.