In Jewish tradition, they were just men who had a lot of power. They weren’t giants.
The Sons of God faithful men, but were weakened by marrying faithless women.
I tend to favour that interpretation .
Different Talmudic Scholars may have different interpretations.
Here's one interpretation from the Jewish Virtual Library that calls them out as Giants:
Jewish scholars are quite strict with meanings and interpretations that have been handed down without alteration for many centuries. I would expect however, there is debate and argument among published journals on the subject as to what was truly meant by "Giant". It's fairly settled that Goliath was a Giant of sorts but the Nephilim seem to have been in another category altogether.
I would defer to Talmudic Scholars and Archeological Societies.