I have never met a true libertarian who advocated violating rights.
To blame it on them is to be totally incorrect,,at best.
My experience with many who at least call themselves libertarian is that they don't really believe in much of anything that restricts their behavior in any way, including fences to keep you off of someone else's property. Maybe they are abusing the word libertarian, just as you said many abuse the word conservative.