The quotation youre looking for is from Chestertons 1929 book, The Thing, in the chapter entitled, The Drift from Domesticity:
In the matter of reforming things, as distinct from deforming them, there is one plain and simple principle; a principle which will probably be called a paradox. There exists in such a case a certain institution or law; let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, I dont see the use of this; let us clear it away. To which the more intelligent type of reformer will do well to answer: If you dont see the use of it, I certainly wont let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it.
In the matter of reforming things, as distinct from deforming them, there is one plain and simple principle; a principle which will probably be called a paradox. There exists in such a case a certain institution or law; let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, I dont see the use of this; let us clear it away. To which the more intelligent type of reformer will do well to answer: If you dont see the use of it, I certainly wont let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it.
Or briefly,
"Put simply, dont ever take a fence down until you know the reason why it was put up." Nice reference at http://abovethelaw.com/2014/01/the-fallacy-of-chestertons-fence/.
I learned something new here today. Thanks C19fan and FR.