The Sun worshipers had been in and about Rome long before Christianity. Bacchus and Apollo come to mind....but, yes.... you are correct about "Sol Invictus". It was the final stage of Roman Sun worship.
[II Kings 17:24] And the king of Assyria brought men from Babylon, and from Cuthah, and from Ava, and from Hamath, and from Sepharvaim, and placed them in the cities of Samaria instead of the children of Israel: and they possessed Samaria, and dwelt in the cities thereof. Babylon is where Sun worship originated with Nimrod.
[Acts 8:9_11] But there was a certain man, called Simon, which beforetime in the same city used sorcery, and bewitched the people of Samaria, giving out that himself was some great one: To whom they all gave heed, from the least to the greatest, saying, This man is the great power of God. And to him they had regard, because that of long time he had bewitched them with sorceries. Simon, as a descendant of these Sun worshiping Babylonians.....brought his brand of "Sun" worship to Rome. By the time "Sol Invictus" appeared on the scene "Sun"day was already greatly revered as the Holy Day!
... but the relevant point is that the Sabbath had been celebrated on Sunday for a century before Sol Invictus. Sun worship goes back probably to the very dawn of mankind, but Sunday was dedicated to sun worship only in 4th century.