What evidence is there that consuming quantities of magnesium is not harmful?
Plenty, if you read the literature and know a bit of biochemistry.
Have at it!
http://www.ncbi.nlm.nih.gov/pubmed?term=magnesium
Magnesium supplements can be harmful to patients with renal or kidney problems: http://www.easy-immune-health.com/magnesium-overdose.html