Okay, fine. But the Obama Camp will say something like: “Did the the State Dept. have travel restrictions for its citizens to travel to Libya prior to the 1986 U.S.-led bombings? If not, then he’s free to travel there... he decided to go to Libya instead of Cancun for Spring Break”.
Your passport application can state the reason you are travelling. Hadj to Mecca would be an interesting reason.