First, the bad news: there's evidence that cannabis inhibits testicular enzymes needed in testosterone production, and may lower your sperm count.
A Danish study of 1,215 men between the ages of 18-28 found once-or-more-weekly marijuana users had sperm counts an average of 29% lower than light or non-users. If they also used other drugs, like cocaine and MDMA, counts were reduced by as much as 55%. So your Sublime-inspired daily routine could actually be acting as a feeble form of contraception (a good or bad thing depending on what your goals are.)
But wait, there's more! Body-builders, it's possible you've already heard the rumour that smoking pot decreases your testosterone levels, and can even cause man-boobs.
While there is some evidence to suggest THC has a mild effect on T-levels, the jury's still out on precisely how that works, and to what degree it's statistically significant. According to this review study and this one, the changes caused by THC in the endocrine system are so minor as to be almost imperceptible. As one study puts it, "many of the effects [...] are completely reversible with time and there is reason to believe that tolerance develops to these effects with acute exposure to THC." So the average user shouldn't necessarily lose sleep over their future baby-making abilities - or the prospect of suddenly developing dudely décolletage.
More good news: getting high could boost your sex life in other ways. This study found marijuana users of both sexes reported more sexual partners than non-users, and also getting busy more frequently. So even if you do fire up a sufficient number of joints to impact your little swimmers, the odds are still probably on your side.