Actually, you make good points, and I'd like to see it argued by someone knowledgeable about the history of what ex post facto means.
See my post 192 - in the end, ex post facto means whatever five or more justices on the Supreme Court decides it means. Therefore, someone can be convicted of a sexual offense, a state legislature can subsequently require after their conviction that they be required to register upon release, and SCOTUS upholds that because they decree that it isn't a punishment - blithely ignoring that imposition of a punishment is not a requirement of the constitutional prohibition against ex post facto laws - it also applies to concepts such as zoning.