NVIDIA Broadcast 1.4 Brings AI-Driven Eye Contact

NVIDIA Broadcast is getting an update that brings the “eye contact” feature to the public. With it, NVIDIA RTX users can appear to be looking directly into the camera’s lens while they are really looking in a slightly different direction.

Generally, if you want to look directly at the camera during video recording, you would need to have a teleprompter-type setup, and that’s not practical for everyday users. NVIDIA Broadcast is software that uses AI (Artificial Intelligence) to modify the video input and change your gaze’s direction.

Looking directly at the camera is visually more engaging for your audience, so this feature could add a lot of value, especially since it’s highly convenient and easy to use (you just need to turn it ON).

And it works surprisingly well if you’re looking in the general direction of the camera, which I estimate to be within 35 degrees around the lens. However, it would be best if you tried your best to look straight into the camera to achieve the best outcome.

At this point, there are still minor quirks. For example, if you blink, the AI might lose track of your gaze for a second or two, resulting in your eyes abruptly changing directions. The same thing might happen if you turn your head too far off and come back.

NVIDIA might be able to fix this later by remembering where your gaze eye was before the blink. That aside, it works well enough, and so far, no one has noticed I was using it during calls. At least, not to the point of commenting about it.

NVIDIA Broadcast and its features are essentially the results of what the company has been doing with its NVIDIA AI Research group. Many webcam-related features were demonstrated as research projects a couple or more years ago, and it’s nice to see a steady stream of enhancements to the publicly available free software. It’s also open to developers.

Hopefully, NVIDIA will release its AI Face Alignment feature at some point because having both would make this look even more natural. Although realigning the face is one thing, doing so on the face and torso is another. I suspect it’s another challenge to overcome before releasing that feature to all.

This kind of workload shows that GPUs are proving useful to a much larger pool of users than 3D graphics for gaming or CAD.

Hubert Nguyen
Categories: Computers
Tags: AI (Artificial Intelligence), NVIDIA, RTX