Yes, and I stand by that as an allowable condition based on what Madison said, as long as the federal government is treating all states equally.
In other words, the commerce clause does not give the federal government the power to regulate the trade of a certain state for the positive purposes of the country. If Georgia is flooding the states with cheap cotton thereby harming the cotton industries in the other states, the federal government could not add a tax, or tariff, or fee, or surcharge, (if you get the point I'm trying to make) for the positive purposes of the cotton industry as a whole.
The federal government could do this to cotton imports from another country, as Madison declared. Based on that, I believe Congress could also ban the foreign or interstate commerce of a product which has a negative purpose on the country.
I think Madison was concerned about the use of the word "regulate". If the word allowed Congress to tax imports for the positive purposes of the country, then read literally, Congress could tax/tariff the products of a state for the positive purposes of the country. He writes,
"Being in the same terms with the power over foreign commerce, the same extent, if taken literally, would belong to it."
"And I'm sure you understand that in the context of his letter, that means injustice committed by state governments against other states."
Yes. And he did not want the federal government to do that either.
You're reading more into his statement than what he said. He said that it was not the purpose of the commerce clause, as regards interstate commerce, to give the federal government power to regulate for the positive purposes of the country. He didn't say that Congress could do it as long as it abided by certain conditions. He said that it was not part of the power granted to it.
I think Madison was concerned about the use of the word "regulate". If the word allowed Congress to tax imports for the positive purposes of the country, then read literally, Congress could tax/tariff the products of a state for the positive purposes of the country.
As I pointed out, there would have been little reason for him to be concerned about that specifically, since the Constitution already explicitly prohibited that (no taxes on exports from any state).
But you're basically correct that he was concerned about the understanding of the phrase "to regulate trade". In his first letter, he gave a very broad definition of the term, saying that it practically synonymous with the encouragement of manufactures. He then wrote his second letter as an afterthought, saying that such an overbroad definition of the phrase should not be applied with regard to Congress's power over interstate commerce. His use of the words "negative" and "remedial", in that context, clearly apply to the nullifying of state laws that commit injustices against other states. In effect, he's saying that the power "to regulate commerce among the several states" is the power to restrict the ability of the states to regulate commerce with each other.