Hubble is still ticking. I am curious to see what the Webb telescope images look like. It is supposed to be primarily an IR telescope.
Yes, a more powerful infrared scope will be able to look deeper into space and further back in time, seeing galaxies so distant that their light has been red-shifted into the invisible infrared portion of the spectrum.
Interestingly, many of the galaxies that we can see now at the ‘edge’ of the observable universe, in the Hubble Deep Field, etc, are so distant that the ‘visible light’ we see from them actually started out at shorter wavelength (higher frequency) portions of the EM spectrum, such as UV.