For the symphonies and concerti, check out some of the period instrument recordings from Hogwood and Gardiner. Goodman's recording of the Requiem is hair-raisihg, especially the Kyrie.
For chamber music, the piano trios, piano quartets and string quintets are represented by some fine recordings as are some of his better string quartets. For his quintet for clarinet and strings, Stoltzman and Shiffrin are still the gold standard.
For something that has almost been forgotten, the "Gran Partita" for Winds in B-flat (K. 361) is a masterpiece, especially the slow movement. And check out Previn's recording of the quintet for piano and winds.
I could cherry pick some recordings I like too, and they would all, or almost all be better than the ones that came with this set. Before I bought this set I already had full sets of the symphonies, piano sonatas, quartets, quintets, and the popular (popular enough that I've attended multiple performances of them at the Met anyway) operas, plus the usual, divirtimenti, concerti, masses, &c. It was still a good deal.
ML/NJ