My Collegue @Nelson, has addressed this question. Please see his answer to a question on why millions of Sperm are produced but only one does fertilization.
Briefly though it is an adaptation. Survival of the fittest. The sperm have to travel a very long way from the vagina to the fallopian tubes. This means that only the very fittest sperm will make it that far and therefore give rise to the best possible baby from that conception. It is probably natures way of recusing fertilization by damaged sperm.
Now if you think about it, just by being born you are already the best!! You all have great potential in you!!