N64 does use optical sensors, the n64 stick is actually super precise and doesn't suffer from drift. The n64 is a goofy controller but it is simply a great and accurate input device, and a lot of the games were really designed with that stick and notches in mind.

But it is made of all plastic and features plastic on plastic moving parts, without lubrication, so it suffers from wear of the plastic. Worn n64 sticks will actually be filled with plastic dust from the stick and gears literally sanding themselves down. The only problem with the controller is the premature wear of the stick.