Charles isn’t the most powerful man in England. He has great visibility and some degree of influence, but he’s about the only man in the country who is by constitutional definition incapable of attaining real power. Anybody else can, at least in theory, go into politics and wind up as PM.
(I don't generally share direct quotes, but he's said this to other people as well and it's appeared in the media before so no harm that I can see - and I quite like the turn of phrase).