The concept in the Babel narrative is that man was building a tower to Heaven and had become so arrogant they though they could be like God. By making multiple languages it meant that people would naturally splinter off into separate groups and leave each other alone. It is often a language barrier that prevents full assimilation of one group into another.
Actually, it is thought they were building a kind of "portal" to the heavenly dimension. A kind of artificial Mt Hermon, which was the portal through which the "watchers" were said to have had ingress and egress to and from our dimension.
BTW, keep in mind that it was Nimrod who was the driving force for the Tower of Babel. Nimrod was himself a demigod (half-breed or Nephilim). Nimrod was, it has been said, trying to regain access to the world from which he and/or his ancestors had been ejected (in the Fall of Angels).