[Ditter]

It is difficult to figure out what is going on sometimes. I was a founding member of a Presbyterian church that held services in a members garage until we could build a building. We were sued by members of the neighborhood and forced to move. They said it violated the deed restrictions, the judge agreed and they won. I have yet to understand our opponents real reason, except hatefulness, for wanting us to stop. I thought that all churches began that way.

[Graybeard58]

You had a crappy neighbor. They are scattered around everywhere.

[FreePaul]

I lived in a neighborhood with strict deed restrictions as well as city ordinances restricting activities. Church activities were exempt. A church group started up near us. They have since grown to be a large church which blocks streets and driveways with parking and generally creates a nuisance all the time. They play music through outside speakers without regard for their neighbors. We no longer live near them but their current neighbors have constant problems with them.

Maybe your neighbors were just trying to protect themselves and their property. If they let you violate the use restrictions they may not be able to enforce them in the future.