Trump kept him because he wanted him to be Deputy AG. If Rosenstein resigned as directed (by Sessions!) and he wasn’t confirmed for another two months, then he’d be getting job offers from big D.C. law firms and would likely lose interest in the Deputy AG post.
That's before Trump became very critical of Rosenstein.