I think people who take drugs find it hard to believe that not taking drugs isn't a moral judgement on them - a bit like people who drink alcohol, and feel they must get others to do so - so they try to get you to take them, just so they don't feel you're on the moral high ground, looking down on them.
My best friend was an inveterate drug taker - he'd take anything anyone gave him apart from heroin - and he had a hard time believing that my non-drug taking was simply a matter of self preservation (don't take sweets from strangers!) rather than disapproval.
It's not stupid not to take drugs - if something goes wrong, there's no one you can sue!