Actually, many Orthodox in Bosnia were Catholics who converted to Orthodoxy since the latter was favoured over Roman Catholicism by the Turks.
Famed Bosnian Filmmaker, Emir Kosturica, admitted that his family "was one of those Serb families who gave in to converting to Islam just to survive during the Ottoman days". Kosturica left Sarajevo during the Bosnian war and moved to Paris and then to Serbia, and he converted (back) to Orthodox Christianity just a few years ago.
Correct...just like Alija Izetbegovic's family. They were Serbs from Belgrade who fled during the Serbian rebellions to Bosnia.
The only reason I could imagine the Orthodox to be "favored over Catholics" would be that the Turks were already holding the Serbian Orthodox Church hostage, but Catholic Austro-Hungary was a threat to the Turks.
I've never heard of any evidence of what you are talking about, but if it happened, I would have thought it more likely in Hercegovina than in Bosnia.
My own family fled Hercegovina for Montenegro in the 16th century & raised the Serb flag in Herceg Novi. Today, I know of people with my last name — some who are Orthodox, some who are Catholic and some who are Muslim.