Generally true, but not absolutely. I've seen tests done where they started with a very long barrel, measured the velocity of a few rounds, and then shortened and re-crowned the barrel. With a really long barrel, you lose muzzle velocity due to friction. But that's pretty long. Generally a rifle shooting pistol caliber ammunition will have a slightly higher muzzle velocity than a pistol with the same ammunition. Of course operating mechanism matters too. Semi-autos, either rifle or pistol, recoil or gas operated use some of the energy of the fired round to operate the mechanism. Revolving cylinder weapons (there once were revolver rifles, I've seen examples in the Texas Ranger Museum in Waco), lose something in the cylinder to barrel gap.
I think the tests I mentioned above were done with Thompson-Contender, but I could be misremembering that. Neither effect is present with a break action.
What I failed to take into account is that everything we're talking about here, weapons designers have long known -- so when they design a round for a handgun, they don't include the powder that would go to waste.