I was going to use your argument to support mine. :) I see it as significant that Jesus did not lift a finger to correct them after Mark 2:7, even though it was clear that He was reading their thoughts. He corrects them on everything else, why not this? I don't see the thinking of the Pharisees as being rhetorical, they were looking to nail Jesus for blasphemy (perhaps a poor choice of words), and in their minds this was the ticket.
But the pharisees were not incorrect on that. Only God has the power to forgive sins. When a priest forgives a sin, he evokes that divine power.