The baker probably wouldn’t know if adultery is involved, unless the couple told them.
But with queers it is quite obvious what is going on.
In small towns you can bet they would know.