When I subject a bacterial population to an antibiotic that targets their ribosomes - the antibiotic is a physical mechanism that will kill the vast majority of that population.
The difference between those that died and those that did not was a physical difference. Variations within some of that population in the relevant DNA for the ribosomes will make the antibiotic not able to physically bind and physically stop protein production resulting in physical death.
How would one attempt to divorce that from being physical? It certainly isn't magical miraculous or metaphysical.
Its effect is physical. But natural selection itself would have to be understood as a type of cause. Causes are not physical things, though their effects often are.
I think you conflate cause and effect here. But they are not the same thing, epistemologically speaking.