Sometimes, I feel like I need a break, too, just to get caught up on things like housework and paperwork. I let them go out with and visit grandparents a few times this summer without me so that I could do housework and, yes, even catch up on some sleep.
Other than that, my husband and I never go out together without the kids. What ever would this woman do if she had to homeschool? ;-)
Good for you! You sound a lot like us. One warning, though - many couples who base their lives around raising the kids - to the exclusion of each other - are setting themselves up for a fall later in life. If both of you do not take the time to nurture your romantic life (ie with a "date night"), you will find a moment once the kids have moved out and wonder who this person is, and what connection do you have besides the kids.
I read a great book about this subject last year (one of those Christian marriage manuals). I'll look it up and post the link.
Totally changed my way of thinking toward my wife. I now "romance' her as often as I can. I don't want to wake up once the kids have gone and find out we've grown so far apart that we don't know each other anymore.