They're not a problem when they're in your yard. Most people keep their lawns mowed and new trees don't appear. However, the seeds blow for miles like dandelions, and they spread prolificly. The feds and some states are spending money (read that as tax revenues) to control and eliminate wild paulownia. Do a search on the internet and put in "Paulownia" and "invasive."
I can see where Santa Cruz County might get concerned about these spreading into the redwood forests. It's similar to kudzu, ivy, and wisteria. They tend to take over any environment, even destroying foundations in some cases.
Just plant some Kudzu - that should choke out the trees in a couple of years.