We have 38 acres that we own the surface rights and 50% of the mineral rights. Someone else owns the other 50% and they have the right to access them if they ever want to. They would need to negotiate access and compensation for damage etc. But I would have to allow access of some sort.
This seems no different.
The difference is that grazing rights are surface rights. Mineral rights can only be accessed from the surface.
Anyone who disturbs the surface must first have permission from the holder of the grazing rights. There are people who own land in Nevada that cannot do much with it because of grazing rights.
My brother owns grazing rights near Fallon, NV. In fact, he owns rights within the city limits....which puts him in the catbird seat: Some homes were built that destroyed his forage and impeded his access to his property, without his permission. He could demand that they be removed...but he is currently negotiating with the city.