There seems to be a trend here, quite a few of you have said that their elderly parents/friends only gave up driving when they'd had accidents, minor or otherwise. To get back to the OP's point, why do we let it get this far? Relying on self assessment will never work, there should be a regular test to prove your continued ability on the road.
Should this apply to everyone? Yes, but more frequently for people over 70, certainly
There are undoubtedly terrible drivers on the road, young and old. Ther would be a big outcry if regular tests came into being, but I think it's what we need to keep dangerous drivers off the road. Think how many bad habits drivers pick up over the years, there's nothing to get them back to some sort of standard.
I took extra advanced defensive driving lessons when my first child was born, because I wanted to make sure I knew all I could about the road and how the professionals keep themselves safe on the road, so I took nothing for granted, and I learned a lot in the process. And I thought I was a pretty good driver (as does everyone). There is nothing else to keep drivers on their toes