Even if you were, aged cheese - i.e. pretty much any hard cheese - will be lactose free.
Milk, ice cream, etc, don’t bother me at all. My poor sister. She had a milk allergy as an infant, and it has turned into lactose intolerance as an adult. She will eat dairy on occasion, but she keeps Lactaid with her at all times.