This AI Automatically Removes Harsh Shadows from Your Portraits

agosto 24, 2020

This AI Automatically Removes Harsh Shadows from Your Portraits

A team of computer scientists from Google, MIT, and the University of California, Berkeley have created an impressive AI-powered “shadow removal” tool that can realistically remove harsh shadows from portraits, while leaving natural shadows intact. The results are impressive.

The paper describing this technology was originally published back in May, but it started to draw more attention this past weekend when the YouTube channel Two Minute Papers picked it up.

The tech itself relies on not one, but two neural networks: one to remove “foreign” shadows that are cast by unwanted objects like a hat or a hand held up to block the sun in your eyes, and the other to soften natural facial shadows and add “a synthetic fill light” to improve the lighting ratio once the unwanted shadows have been removed.

Two Minute Papers summarizes the technique well in the video below:

Using these two networks, the team is able to (for the most part) realistically remove foreign shadows that are both natural and added there by a computer. Of course, some of the results can still look a bit “artificial,” and not all unwanted shadows are addressed when presented with natural images shot in the wild.

“Our proposed model is not without its limitations…” reads the paper. “When foreign shadows contain many finely-detailed structures (which are underrepresented in training), our output may retain visible residuals of those.”

You can see that in some of the sample photos where shadows are cast by stray hairs, for example:

But given enough training data, this and other limitations could be overcome, allowing for simple one-click “edits” in apps like Google Photos of the Adobe suite that can convincingly remove harsh shadows without making your image look overly fake.

Even with its current limitations, the researchers are quite proud of what they’ve accomplished:

Given just a single image of a human subject taken in an unknown and unconstrained environment, our complete system is able to remove unwanted foreign shadows, soften harsh facial shadows, and balance the image’s lighting ratio to produce a flattering and realistic portrait image.

You can see a slew of sample images created using this technique below:

To learn more about this technique, explore further test data, or read the full research paper for yourself, head over to the Portrait Shadow Manipulation website here. And if you want to explore more AI-based photo editing wizardry, check out this AI that can turn paintings into photographs, or this one that attempts to realistically “re-light” portraits after they’ve been taken.

(via Reddit)