Skip to main content

How Surface Hub 2 smart cam uses A.I. to reimagine meetings

Microsoft has been investing in its Surface brand to make it larger part of the workplace meeting space — and the surge in hybrid work has made that even more important.

The Surface Hub 2 and Surface Hub 2S are the fruits of that work, but Microsoft is now building on those with the Surface Hub 2 Smart Camera. It is designed to make the meeting experience better for people in the room with a Surface Hub 2 and Hub 2S, as well as those who are participating remotely.

Related Videos

I had a chance to chat with Dave Alles, vice president of the Surface Division at Microsoft, about the new camera and the future of remote work.

Serves a crowd

A woman makes a presentation on a Surface Hub 2S in a conference room as a man looks on.

To start off, Alles told me about the heritage of Surface Hub and Surface products in general. “From Surface Duo, Surface Pro, or Surface Go, there’s a lot of the Surface portfolio that is great for brainstorming and real-time collaboration sessions that make it really easy to go from PowerPoint to Whiteboard and back again,” he said.

He then went into the reasoning behind the Surface Hub 2 Smart Camera and how it is is a different product for Microsoft. Per Microsoft’s own data, the average Teams user has seen an increase of more than 200% in terms of weekly meeting time since March 2020. Most of these meetings were nonscheduled.

Microsoft sees this trend as an opportunity to create products and software innovations to help more people get their work done and make everyone feel like they can participate in meetings.

“We also are able to have a lot more humans in frame, whereas a PC camera has a design that assumes there’s a single user.

“We like to think we were building out hybrid solutions before the pandemic. We think that this is a very durable change, and the importance of remote participants for all kinds of meetings has gone up permanently, and Surface Hub, when paired with pen-enabled devices, is a powerful tool,” said Alles.

That’s also why the Surface Hub 2 Smart Camera is a product unlike any other webcam on the market. It clearly captures who is speaking in front of a Surface Hub 2 or 2S, and what else might be happening in the room. It magnetically attaches to the top of Surface Hub 2 or Surface Hub 2S and sits in a machined aluminum body. It also uses signal processing, artificial intelligence, and optics to improve your Teams meetings — even for those who might not physically be in the room.

Fancy talk aside, that means the camera is able to reframe, adjust lighting, and enable wide angles of view. It does all this without any warping, distortion, or depth-of-field issues that traditional webcams might experience. That’s even if someone leaves or enters the room, or walks up or interacts with the display.

“All of the PC cameras you see are focused on a range that’s not for the scenario of standing up and walking around your room. For Surface Hub, you’re in a focus or conference room, and we have a lens that makes sure we can keep people clear. We also are able to have a lot more humans in frame, whereas a PC camera has a design that assumes there’s a single user,” explained Alles.

More than a webcam

The new Surface Hub 2 Smart Camera

But how does it work? Well, it’s thanks to a 12-megapixel sensor that has a 136-degree field of view lens that can keep things near and far simultaneously, regardless of a conference room’s size or configuration. The camera is also able to handle 1 teraflop of computing power, or a trillion floating point calculations per second.

An algorithm was even developed to automatically compensate for tilt, distortion, and wide-angle corrections so objects appear as they would in real life. Microsoft tells me it borrowed part of this idea from Hollywood movies, so the movement and zooming the camera picks up is not too aggressive for people on the other end of the call.

Alles went on to explain that the viewing angles of the Surface Hub 2 Smart Camera are what makes it quite unique. The actual field of view is more than 180 degrees, so the camera can see behind the Hub 2 it sits on. The software then crops the feed to get the 136-degree angle, so your face can be seen clearly if you stand next to the camera. Being able to have clarity at both short and long depth of field at the same time is the other thing that makes it unique.

“We are unique in the world of having far faces and close faces very clear at the same time.”

“What we really wanted to do was get a super-wide field of view, and a sensor that would get clear faces if someone was standing right up next to the bezel of the Surface Hub 2, or even able to see the person standing in the back of the long conference room. We bumped up all the technology, whether the technology was glass, or a sensor, or the compute capabilities, to be able to accommodate for those cases,” said Alles.

“Every other camera has a specific focus point, and then things that are far away from that focus point get blurry. We are unique in the world of having far faces and close faces very clear at the same time,” he added.

This all sounds cool, so you might be wondering if the camera will work for your PC. I asked, and Alles said the Surface Hub 2 Smart Camera is only for Surface Hub 2 or Surface Hub 2 S. It is not compatible with PCs because of its mounting system, which uses a USB-C plug of a custom size and a POGO plug.

The software and algorithms inside the camera are also specially tuned for conference room scenarios, with open space and multiple people. Microsoft would need to retune the camera algorithm so that it would focus on close people. There’s a slim chance of it happening.

“Those are possible things to do, there seems to be quite a bit of interest, and I’m sure it’s something we’re going to talk about, but we’re not going to offer it on PC,” said Alles.

Price and availability

The Surface Hub 2 Smart Camera against a light gray background.

The Surface Hub 2 Smart Camera is available today for purchase in all markets where the Surface Hub is sold for $800. It will also be bundled with the Surface Hub 2s 85-inch beginning May 31 for $22,000. It also pairs up with several updates for Microsoft Teams, as well as Whiteboard, PowerPoint, and Outlook.

The features are all catered toward making meetings and work a lot more natural for hybrid workers. For the Surface Hub 2 and Teams Rooms devices specifically, that includes Front Row in Teams. With this, the video gallery appears at the bottom of the screen so in-room participants can see remote colleagues face-to-face, almost as if they were in the same room. The codebase for Whiteboard has also been unified so that Whiteboard on Surface Hub looks and feels similar to Whiteboard on Windows.

Rounding out the conferencing-related news is Teams Rooms. It is also getting enhancements where if you join a meeting in a Teams room, you’ll be prompted to turn your laptop video on so you can be clearly seen by remote participants no matter where you are in the room. Your video will be hidden from the front-of-room screen, as well as from the gallery of people joining from companion devices in the room. Microsoft talks about trends in hybrid work and beyond in its 2022 Work Trend Index report.

Editors' Recommendations

Surface Book 4: Everything you need to know about Microsoft’s missing 2-in-1
Surface Book 3 sitting on the floor with screen open.

Microsoft just announced the Surface Laptop Studio, but for close followers of the Surface brand, there's one concerning take on this news. With the launch of the Laptop Studio coming on October 5, and with it being catered to creatives and photo and video editors, or even developers, what is going to happen with the Surface Book lineup?

Microsoft didn't announce a Surface Book 4 at its recent event, and we're now left wondering if the Surface Book lineup is dead for good. We've combed through the rumors and speculation to find out.
Price and release date

Read more
Microsoft Surface Duo 2: Everything you need to know
The Microsoft Surface Duo 2 has a triple camera setup.

Microsoft has officially announced the Surface Duo 2, rectifying many major complaints that plagued the previous model. Reviewers and early adopters of its predecessor were critical of the software, cameras, and performance, but optimistic about changes future devices might bring. The Surface Duo 2 features improvements in all those areas, and extra features unique to dual-screen smartphones.

Now that the Surface Duo 2 is official, here's everything you need to know.
Price and release date

Read more
Everything announced at the Surface event: Duo 2, Surface Laptop Studio, and more
All the new Surface products against a white background.

Ahead of Microsoft's fall Surface event, there was a lot of anticipation for what the company would announce. Not only is Microsoft's hardware event arriving weeks ahead of the company's debut of Windows 11, but it's widely believed that the company's Surface platform will serve as a launchpad for what consumers can expect for the next-generation operating system.

"Throughout our history as Surface, we have seen that the most meaningful experiences come from innovation at the intersection of hardware and software," Microsoft said of the launch. "This idea was the catalyst for Surface -- to build hardware that would not only be a stage for Windows, but push the platform and the ecosystem forward."

Read more