As you move up the Atlantic Seaboard, a "Yankee" is someone from further north than you are at any given time. When you get into New York-New Jersey area, a "Yankee" is someone from New England. Only after you cross the state line into Connecticut does anyone actually begin to answer to the term.
FWIW, Southerners first started calling those from anywhere north of the Mason-Dixon (in reality, the boundary between Maryland and Pennsylvania by the way) "Yankees" because the most strident abolitionists--the sworn foe of the true Southern bigot and racist--were "Yankees" in fact, folks from Massachusetts in particular.
Thus, originally, "Yankee" meant "anyone who believes in the abolition of slavery and the *shudder* equality of the races."