There is a theory it stems from the earliest days of European settlement in Australia.
Being a pommy penal colony, the prisoners wore "Prisoner of Her Majesty" or "POHM" on their back.
Bloody lot of codswallop, that, my Dasher.
FY BLoody Info, Mate. Dates from THe Great War, Tommy Atkins was always asking the Frogs for "Pommes Frites," or chips with everything. So the Anzacs took to calling them Bloody Pommies, or Poms.
Now don't get your bloody knickers in a knot, wasn't I bloody there? I bloody well was.