I think the E46 is considered the best M, aside from the oil issues that it had. I have an E39. It is truly a joy to drive. I’m actually in the middle of putting it up for sale, though. It’s time to move on...
Current: 2011 Toyota Tacoma & 2004 BMW 330xi
Past: (taking a deep breath) 1973 Triumph Spitfire, 1976 Toyota Corolla (came with my wife ;->), Toyota Celica GT, 1980 Toyota Pickup, Audi 4000, (?) Subaru Wagon, VW Golf GTI, VW Quantum, VW Passat, Jeep Cherokee, Chrysler/Dodge Caravan Mini-van, Mercury Villager, Lexus RX400h.
Then the BMW phase started: 530i, M3 convertible (E36), 330 convertible, M3 convertible (E46), 335i Coupe (E92) - plus the current 330xi (E46). AND, a couple of other Germans - an Audi A6 & a Mercedes E550 (briefly).
Of course, I’ve also have a small stable of cars for my daughters - an ‘01 Honda Civic EX, and an ‘03 Mini Cooper, an ‘04 Jeep Grand Cherokee. I’ve had to dust off my old mechanical skills with these - the Honda & Mini due to age and mileage, and the Jeep because, well it’s a Jeep (and American-made).
Despite the maintenance headaches, the Spitfire was the most fun to drive overall - just a pure feeling of car, driver, road. The Bimmer convertibles were next - particularly the last M I had - it had the (rare) SMG transmission and 330HP - THAT was a blast!