The fence built as a courtesy to the former neighbor was on her property so it technically belong to her. He should have told her that and said it is your problem.
I should have added that although it required work on his part, the karma was worth it.