You’re supposed to report all income, not just those that come on a 1099-R.
You are correct. I cringed when I sent my tax info to my tax guy. I had a sizable profit from a sale that I needed to claim that was not 1099-ed to me. If the other party had gotten audited by the IRS they would have found it and came to me for the taxes, plus penalties, which usually are far worse than the original tax bill.