A civilian was considered an ordinary pom, as in Prisoner Of Her Majesty. Someone in the intelligence or sometimes their military was a Prisoner Of Her Majesty's Service.
My mum and dad called them pommies for different reasons, which you found out already. Our mobs used the unrecognized ones because of having to deal with them again and again. The stuffy lot of them.