They are not allowed. But how do you stop them, except escort them or shoot them down.
There are temporary no-fly restrictions, of which pilots are notified, someone posted that on another thread.
Some pilots choose to disregard them.
After a couple of them would get shot down, maybe they would pay more attention.
Crop Duster Buzzes Bush Appearance in Florida
http://www.freerepublic.com/focus/f-news/1254568/posts
Here is the post, which posted the official restriction:
http://www.freerepublic.com/focus/f-news/1254568/posts?page=51#51
Then all of those some should lose their wings permanently. That'll serve as a wake-up call to stop screwing around.