Except it's a cost that only comes with a subset of your customers.
Adding a small charge for customers who use credit cards is no different than a parking garage charging more for valet parking than self-serve parking, as I see it.
I think the optics of it is worse than the justice of it. If credit card transactions are less than 15% of your business, that might be OK. But your business reputation may suffer if a sizable portion of your customer base prefers to use credit cards.
I agree the optics are an issue. That’s why most businesses are just going to hide the fee by raising all prices. In my area, it’s mostly the small independent businesses that charge a separate CC fee, along with some sort of notice which explains and usually apologizes for it. I think that goes a long way to soften the optics. Meanwhile, the big box next door just keeps raising prices and doesn’t have to apologize or explain anything.