Genesis 38:26 And Judah acknowledged them, and said, She hath been more righteous than I; because that I gave her not to Shelah my son. And he knew her again no more.
Judges 19:25 But the men would not hearken to him: so the man took his concubine, and brought her forth unto them; and they knew her, and abused her all the night until the morning: and when the day began to spring, they let her go.
1 Kings 1:4 And the damsel was very fair, and cherished the king, and ministered to him: but the king knew her not.
Matthew 1:25 And knew her not till she had brought forth
her firstborn son: and he called his name JESUS.
Every time the phrase is used it is talking about sex.
I agree that to "know" often means to have sex. I never argued otherwise.