You don't need to cherry pick; 100 researchers do the same experiment (does drug A cause weight loss). 98 of them find no effect, which is not a publication worthy result, so they don't get published, but the two who do find an effect (say p 0.03, 0.04) publish their results and now we have a problem.
That is correct...also..they will test drug A at a dose which demonstrates a good p value...but since the drug is tested on basically young healthier people the doses approved are sometimes too much for smaller...sicker...older people. Which is why some older patients may get beneficial effects from smaller than recommended doses or may have more adverse effects using the recommended doses. Gotta be careful when dosing geriatric patients.