Well, one reason would be that if Congress, collectively at least, does not want a position filled... it does not get filled. Remember, Congress and the Prez are partners in the process, each with a role. It’s like Congress has a veto.
A recess appt gives the benefit of the doubt to the prez to make a LIMITED appointment when they are out of town between sessions, because the previous appointment never got even the chance of being considered by the Congress.
Congress created the NLRB, and it can destroy the NLRB. The "partnership" exists only because Congress created it.