When they buy bonds from a Primary Dealer, they're buying privately owned bonds.
But of public debt (T-Bills), not private debt.