Could not the energy of heat be recycled as well?
Thermoelectric devices are common. I use one at work every day, both thermocouples and a thermoelectric heat/cool device in our products we sell to government, civilian and military use all over the world.
One device gets very hot when DC voltage is applied in one direction, and gets freezing cold when applied in the other direction.
No.
To get work from heat, whether electrical or motion, you have to have a differential of heat. You can have hot and cold, or hot and much hotter or cold and much colder, but you must have the differential.
The greater the differential, the greater the work which can be done.
Every conversion process to go from an energy source to work involves some loss, most often as waste heat. This appears to be a universal, unavoidable, physical law, called the Second law of thermodynamics.
Some have put it this way: You cannot win. Moreover, you can never break even.
Some processes such as electrical to mechanical, can be very efficient, about 90%!
Heat to mechanical can approach 50%, with differentials in the hundreds to thousands of degrees, using very specialized, heat resistant materials.
The human body is about 25% efficient in converting potential chemical energy from food to mechanical energy. About 75% is converted to waste heat.
It does pretty well. Cars burning gasoline are, generally, less than 20% efficient, by comparison.
Atomic energy uses conversion processes to produce electricity, but atomic processes are simply tapping into a very energetic fuel. That "fuel" produces heat, which we then use like any other heat source.