Are you sure the potential buyer didn't pay earnest money, rather than him having paid earnest money and the broker having pocketed it?
In the future, you should tell the broker that if the buyer does not hand over earnest money at contract signing, then there is no contract signing.
Great observation. Honest brokers are a blessing but they are as rare as honest used car salesmen. I refer to them as property pimps.