New Technique Analyzes Shadows to Spot Photo Fakes

INSIDE SCIENCE NEWS SERVICE ^ | Aug 14 2013 | Ker Than

[neverdem]

Algorithm spots forgeries by spotting shadows that don't match light sources.

(ISNS) -- A new algorithm can spot fake photos by looking for inconsistent shadows that are not always obvious to the naked eye.

The technique, which will be published in the journal ACM Transactions on Graphics in September, is the latest tool in the increasingly sophisticated arms race between digital forensics experts and those who manipulate photos or create fake tableaus for deceptive purposes.

National security agencies, the media, scientific journals and others use digital forensic techniques to differentiate between authentic images and computerized forgeries.

James O'Brien, a computer scientist at the University of California, Berkeley, along with Hany Farid and Eric Kee of Dartmouth University, developed an algorithm that interprets a variety of shadows in an image to determine if they are physically consistent with a single light source.

In the real world, O'Brien explained, if you drew a line from a shadow to the object that cast the shadow and kept extending the line, it would eventually hit the light source. Sometimes, however, it isn't possible to pair each portion of a shadow to its exact match on an object.

"So instead we draw a wedge from the shadow where the wedge includes the whole object. We know that the line would have to be in that wedge somewhere. We then keep drawing wedges, extending them beyond the edges of the image," said O'Brien.

If the photo is authentic, then all of the wedges will have a common intersection region where the light source is. If they don't intersect, "the image is a phony," O'Brien said.

A growing toolbox

The new technique does have limits, though. For instance, it was designed for use with images in which there is a single dominant light source, not situations with lots of little lights or a wide, diffuse light.

One could also imagine a clever forger anticipating the use of the shadow detection software and making sure they created shadows that would pass the test. The researchers call this just one technique in a toolbox of methods that are being developed to catch forgers.

O'Brien says one of the motivations for developing their algorithm is to reduce the need to rely on subjective evaluation by human experts to spot forgeries, which can easily mistake forged photos for authentic photos and authentic photos for forged ones.

Take for example the iconic 1969 photo of NASA astronaut Buzz Aldrin posing on the surface of the moon.

"The shadows go in all kinds of different directions and the lighting's very strange...but if you do the analysis [with our software], it all checks out," O'Brien said.

Our trouble with shadows

It's unclear why humans are so bad at detecting inconsistent shadows, especially since our visual systems are so attuned to other cues, such as color, size and shape, said UC-Berkeley vision researcher Marty Banks.

One idea, Banks said, is that shadows are a relatively unimportant visual cue when it comes to helping organisms survive.

"It's important to get the color right because that might be a sign that the fruit or meat you're going to eat is spoiled, and it's important to get size and position right so you can interact with things," said Banks, who did not participate in the research. "And then there are things where it just doesn't really matter. One of them is shadows, we believe."

After all, before the advent of photography, one was unlikely to ever encounter a scene where the shadows are pointing in the wrong direction.

Analyzing shadows could also just be a more mentally taxing task, said Shree Nayar, a computer vision researcher at New York's Columbia University, who was also not involved in the research.

"This is a more complex second order effect," Nayar said, "and it's something we have a much harder time perceiving."

Man-machine collaboration

For now at least, the team's method still requires some human assistance, by matching shadows to the objects that cast them.

"This is something that in many images is unambiguous and people are pretty good at it," O'Brien explained.

Once that is done, the software takes over and figures out if the shadows could have been created by a common light source.

In this way, the scientists say, their method lets humans do what computers are poor at — interpreting the high-level content in images — and lets computers do what humans are poor at — testing for inconsistencies.

"I think for the foreseeable future, the best approaches are going to be this hybrid of humans and machines working together," O'Brien said.

Columbia's Nayar said he could envision a day when computers won't need human assistance to perform such tasks, because of increasingly sophisticated models and machine learning algorithms.

Because their software requires relatively simple human assistance, O'Brien and his team say it could one day be useful not only to experts, but the general public as well.

"So you could imagine a plug-in for Photoshop or an interactive app in your web browser where you can do that, and it would flag any inconsistencies," O'Brien said.

---

Ker Than is a freelance writer based in Southern California.

[neverdem]

Exposing Photo Manipulation with Inconsistent Shadows(PDF)

[faithhopecharity]

would any of our government’s political leaders or officials ever dream, even for a split second, of using a fake document?

Never! This is America. (isn’t it?)

[Agamemnon]

Wonder if they can forensically determine whether Obama was photo-shopped into this one, now that his "body man" says he might have been just playing cards with him outside in another room somewhere when Bin Laden was taken out.

FReegards!

[smoothsailing]

Spot the fake

[taxcontrol]

There is a HUGE assumption that there is only one light source which is not always the case, particularly in an urban environment. Street lights, moonlight, reflections off of buildings, etc.

Still, interesting work.

[madison10]

Ms. Dunham-Soetero seems to have a very dark hand.

[ClearCase_guy]

[SunkenCiv]

Thanks neverdem.

[JoeProBono]

[knarf]

It All Began Here

[kabumpo]

the plural of tableau is tableaux, not tableaus

[Mr. K]

what the hell is that big blob over his left shoulder?

[bigheadfred]

barry was a puppet then and a puppet now

[BenLurkin]

That is just plain freaky.

[Fightin Whitey]

barry was a puppet then and a puppet now

Who do you suppose is the iron fist behind the seat of power now?

[BereanBrain]

This is dumb.

When i photoshop, i makes sure to create the proper shadows for everything. How?

Simple, knockout background, leaving object. Then create a new layer, rotate/warp the foreground object on that layer, take it’s color to opaque. Then apply lighting. Place that layer behind/under the foreground object.

Anybody who has done the a halfway good job of photoshopping figured that our a LOOOONG time ago.

[silverleaf]

oh, that arm and hand are real?

puh leeze

[silverleaf]

I don’t which is more disturbing

the evidence of so many photoshops of big head barry “obama” or the amateurism of the forgery

“we” the USA are not exactly being manipulated by geniuses here

[smoothsailing]

Yes she does, a big dark man hand!

http://www.wnd.com/2012/07/another-doctored-image-deepens-obama-mystery/

[Jane Long]

Why are Granny’s folded hands in Grandpa’s sunglass lens reflection and not 0’s head and/or hand?