But it is interesting that published experiments rarely reveal the entire story about how progress is made in science. You need to rely on biographies and autobiographies to get real insight into how some of the great minds of science worked out a problem and devised experiments. If you just looked at the published articles you sometimes ask yourself how a certain scientist could have had such great insight, when in fact they sometimes worked outside of the rules. Only when they understood the principles involved did they come back and devise the correct experiment. And sometimes even then that didn't work. It is sometimes troubling for someone not working in the field of science to learn that scientists throw out the outlier data based on any legitimate excuse in order to achieve a statistically significant result.
As long as it is well outside the bounds of the majority of the data and is statistically insignificant, this is valid. The danger is that sometimes significant amounts of data lie outside the expected range. That is not legitimate to ignore. Any scientific paper which ignores "outlier" data should indicate that this was done and why.