I believe that the answer to your question is, only if the House is in session.
Patrick Kennedy used that defense when he was arrested one night on Capital Hill after driving erratically and almost running down a Capital Police officer. He said that he was “on his way to a vote and could not be detained.” Sadly for Patrick, it was well after midnight and the House was not in session. Happily for him, his last name is Kennedy and the matter promptly dropped.