my mum gave up in 1950's yet she passed with cancer a month ago
my dad never smoked-it got him in 1974--so yes you dont need to smoke to get cancer--in fact we all have it--just needs something to fire it up 
Most people think it is the tar in tobacco that causes cancer, this is not so. It is caused by the daughter products from radon gas, which end up on the surface of the leaves when the tobacco is growing. These radioactive particles collect in certain areas of the lungs and may cause cell mutations which may lead to cancer. The tobacco companies know this and could improve safety, with a special leaf wash as part of the tobacco processing, but it is cheaper not to.
So the danger depends upon where the crop is grown and how much radon gas is in rocks / soil.
The reason we are more likely to get cancer with age is due to our immune systems getting weaker. This affects some people more than others. We have mutating cells and are also under constant attack from viruses and bacteria, which our immune systems keep under control.