For the same reason solar power isn’t called perpetual motion. The energy came from another source and we just tap off of it. The earth rotates around it’s axis and around the sun, There is quite a lot of energy there if we could only tap into it. And no one would call that perpetual motion either, but it would be considered free energy.
Actually I think I understand what I didn’t understand before. You can’t generate energy from heat in an area where there is constant heat, i.e. you need a temperature gradient. In this case it’s absorbing infrared radiation from objects that are hotter than the background heat level, so it doesn’t violate the entropy requirement.
At least that’s how it looks to me, but I’m still not an expert.