The oft-forgotten full phrase is: "a broken clock is right twice a day, *but you still wouldn't use it to tell time.*"

Similarly, often in reference to corrupt cops or politicians: "one bad apple *spoils the whole bunch.*"