There are nice ones in Chelsea (NYC) also. If you tell me the names of the neighborhood hotels in Atlanta, I promise I won't tell anyone else.
I used to send people to the Habersham at P'tree & W. P'tree, very much a small European style establishment, but I drove by there Tuesday, and dog my cats if there isn't a huge hole in the ground where that hotel used to be! Ansley Inn (in Ansley Park) and Emory University Inn (over at Emory) are old standbys. I haven't sent anybody to the Emory place since I graduated law school in 1980 . . . there's a more modern small hotel up at the top of the hill in Vinings (just above the railroad track), believe it's one of the Wyndham chain, a very nice place with a big rocking chair porch out front to watch the trains go by . . .