Personally, whenever I hear someone using the word Lordy, I dont view that any differently than any other way that someone can take the Lords name in vain.
Well... if you would like to argue with scripture, you have the privilege of doing that... 1 Timothy 5:20 "Them that sin rebuke before all, that others also may fear."