That could be. I haven’t fully researched original understanding of FF&C but I tend to think the reason for FF&C was mostly to facilitate a unity among the states especially in the commercial area,.
Nevertheless, regardless of the status of the territories or states involved, whether slave-free or not, upon the owner’s return to a slave state, I think the previous state status as a slave-free state could not by operation of law take away the slave-owner’s constitutional property. I think Dred Scott was constitutionally decided properly, despite our moral abhorrence to slavery and despite some of the questionable assertions in the opinion.
The United States Constitution does not enumerate a right to own people as a constitutional certainty. It only refers to a remedy in context to escaped slaves.