Do not drink heavy water. It is seriously toxic.
It would take large amounts, but yes, it is toxic.
From Wikipedia:
Toxicity in humans
Because it would take a very great deal of heavy water to replace 25% to 50% of a human being's body water (70% of body weight) with heavy water, accidental or intentional poisoning with heavy water is unlikely to the point of practical disregard.
For a poisoning, large amounts of heavy water would need to be ingested without significant normal water intake for many days to produce any noticeable toxic effects (although in a few tests, volunteers drinking large amounts of heavy water have reported dizziness, a possible effect of density changes in the fluid in the inner ear). For example, a 70 kg human containing 50 kg of water and drinking 3 liters of pure heavy water per day, would need to do this for almost 5 days to reach 25% deuteration, and for about 11 days to approach 50% deuteration. Thus, it would take a week of drinking nothing but pure heavy water for a human to begin to feel ill, and 10 days to 2 weeks (depending on water intake) for severe poisoning and death.
Oral doses of heavy water in the multi-gram range, along with heavy oxygen O-18, are routinely used in human metabolic experiments. See doubly-labeled water testing. Since 1 in every 6400 hydrogen atoms is deuterium, a 50 kg human containing 32 kg of body water would normally contain enough deuterium (about 1 gram) to make 5 grams of pure heavy water, so roughly this dose is required to double the amount of deuterium in the body.