Not quite. Milk production is part of the definition of mammal, but it also includes all the other stuff mentioned earlier (dentition -- a biggie, hair or fur, number of holes in the skull, warm-bloodedness, single lower mandible and differentiated ear bones, etc.)
Now, the first two, milk production and fur do not fossilize, but as the others are all found in mammals and they do fossilize -- and this combination is not found in any other class of animals -- any fossils exhibiting the complete collection of stuff that does fossilize and denotes mammal, can be assumed to be a mammal -- except by creationists who cannot see the forest for the trees and refuse to accept any evidence unless in the form of a living, breathing critter (and then they'd probably claim it was ginned up by geneticists in some secret laboratory to mislead good, God-fearing Christians in an effort to damn their souls to Hell).
That the above features do often occur in most mammal species is undeniable. But that is not the point. The point is that an animal such as the platypus is considered a mammal because it has mammary glands even though it lacks many of the above features.
The larger point as I have mentioned several times, and you evolutionists totally ignore, is that extrapolation just confirms prejudices, it adds no new knowledge. For example, using the platypus as an example again. If it was not a living species, it never would have been classified as a mammal, none of its uniqueness such as its killing poison, it's sensory radar, it's egg laying, its not having separate excretory and sexual ducts and many other interesting features would have never been known. It would have been just one more set of bones like any other stuffed into some procrustean bed by lazy paleontologists.