The needs of the many outweigh the needs of the few.
When you design application first solve needs of the majority of app’s users. Well prepared personas will help you with this task. However, you should also remember about the rest of your users. Even Mr. Spock broke his rule few times to do the right thing.