It is not a sin to be rich.
It is a sin to use porn.
So I think it’s a false equivalence. There are tough temptations for the very rich, as well as for the very poor, of course.
I realize that it’s no sin to be rich and I don’t think he was claiming that.
I think his point was what you made, that it’s the temptations that are the problem.
And because being rich isn’t a sin, then people are less aware of the problems with the temptations involved with money. Same as with food and drinking. While it’s widely recognized that over drinking is wrong, since eating is necessary for survival and given to us by God, part of His design, then there are the dangers of over eating.
I actually had a Baptist once tell me that Baptists eat so much cause it’s the only thing they can do that’s not a sin. And he was at least 80 pounds overweight. Didn’t seem to occur to him that gluttony was also a sin. And so the temptation got him.