[Greysard]

There will be heat - if not from the drive then from the power plant. Conversion of fuel into power is never perfect, and the process will always create a substantial amount of heat.

This conversion, as opposed to the jet exhaust, can be radiated away from the travel path. Some types of fuel generate little to no heat within the engine - a charged ultracapacitor is one example of such "fuel."

Really, humanity doesn't have a usable star drive technology, and we can't say much about what is possible. It's like saying in 1850 that "If someone invents a flying machine, we will clearly see it in the sky by the cloud of coal smoke that its steam engine produces."

[Little Ray]

We would say “exhaust” and, sometimes, we CAN see aircraft clearly by their exhaust or contrails.

[Little Ray]

To further elaborate, unless the aliens have a magical energy source (zero point energy for example) any energy source will have IR output and, since enormous energy is required, there will be a very large IR output.

But, if the aliens have a magical energy source, then they probably have a magical FTL stardrive, too, and then, no, we won’t see them coming.