Depends on the circumstance. If you (and your parents) paid for your education than it's yours to do with or not do with as you please. Maybe you've gotten all you can or wanted from it after a 10, 15, 20 year career. If you met your obligations to pay and provide for yourself and any family no one has a right to say you're wasting your education. If you leaped from a higher degree program after getting grants and just stay home to take care of children, then there's a clear imbalance in having repaid your moral obligation to make good on those grants.
If government is going to take money from its citizens, it better give a meaningful ROI. Grants should be very carefully managed and if you don't offer ROI on education grants than you should have to pay back what you've taken on the premise that the taxpayers may get a great new teacher, or chemist or microbiologist for their investment not an overeducated liberal-cause loving soccer mommy.