Do you have a friend with a laptop who can hook up to your network cable and check to see if it happens to them? That will at least tell you whether it's your own box, or your router/modem/ISP.
And a little call to your ISP's help desk might be in order to see if they're having problems. Time-Warner had intermittent DNS problems all last week. And occasionally they'll tweak the download bandwidth themselves if they think you're running a server or something. It'd be easy enough to eliminate the possibility by asking.
You might post your setup in a little more detail - it'd help diagnosis. Best of luck!
This has happened to me a few times. For some reason, a setting was inadvertently changed to less than half the download speed it was supposed to be set at.