For most men, it takes between 24 and 36 hours to “replenish” all the way after ejaculating and get back to their normal sperm count. However, the testicles are producing new sperm all the time. Even five minutes after a man's last ejaculation, there are a few sperm ready to go.