Listed as a bogus quote here and "disputed" here.
I looked at the book it comes from but could not find anything that identifies John Quincy Adams as the author of the quote, although the book was published during Adam's life.
http://www.apologeticspress.org/apcontent.aspx?category=7&article=1142