I hate to tell you but it is an easy fix. Simple RegEx expression in the validation code when posting replacing the offending characters with acceptable ones and viola. Fixed. It is a UTF8 <-> ANSI issue. Sanitize posts in existing code, and you will not invoke hardly any additional CPU cycle overhead. Nothing else would be affected. A few CPU cycles to find the offending characters and replace them in memory before committing the post to the database.
Example: wouldn’t would become wouldn’t Look at the character in the post you made. copy and paste your original wouldn’t to me, and copy and paste my wouldn’t as well. It is a slanting ‘ Straight ones work fine. “ is another one. Opening and closing quotes mess up. Straight ones (ASCII) work fine.
Ok. Then lets say you are right.
What would be the deep dark secret that would keep Jim from implementing this? What possible motive could he have?
It will be fixed in time. I accept that. Others won’t.
Hmmm straight quotes and apostrophes are modified by either FR or the front end and go fubar. My solution would still work. It has to be a smart quote setting in the backend database, or a bug. Either way, simple fix.
My regex solution would work 100% guaranteed.
I have also often wondered about some sort of regex fix. And since the site is perl-based, I would think it would be a simple thing to implement. And if it didn’t work, it wouldn’t do any harm... Like you said, regex filtering is very low overhead in perl.