I love Newburyport and my parents were very impressed with it for the Federal-style homes and the brick downtown. I'm drawing a blank on a good place to visit for more of a colonial experience, and have never been to Plimoth Plantation.
Plimoth Plantation is a reconstruction. The best collection of old homes that I have seen is Greenfield Village at the Henry Ford Museum.
You might want to contact the Mass Historical Society or the Mass DAR.