Not just the British Army either, soldiers in the US Army were doing the same thing.
Even some members of Co.E, 506 PIR, 101st Airborne Division (Easy Company/Band of Brothers) went after a suspected concentration camp commandant and upon finding the guy summarily executed him.
When they found Hoess, the Commandant of Auschwitz, they beat the crap out him. He was eventually executed just outside of where the Crematoria was.