Several experimental and observational studies have demonstrated the antiandrogenicity of several phthalates. However, there is limited evidence of an association between phthalate exposure in adult life and semen quality. The aim of this study was to examine phthalate exposure during adulthood in relation to semen quality in fertile US men. This multi‐center cross‐sectional study included 420 partners of pregnant women who attended a prenatal clinic in one of five US cities during 1999–2001. Nine phthalate metabolites [mono (2‐ethylhexyl) phthalate (MEHP), mono (2‐ethyl‐5‐hydroxyhexyl) phthalate (MEHHP), mono (2‐ethyl‐5‐oxohexyl) phthalate (MEOHP), and mono (2‐ethyl‐5‐carboxypentyl) phthalate (MECPP)], as well as mono‐n‐butyl phthalate (MBP) and mono‐isobutyl phthalate (MiBP), mono (three carboxypropyl) phthalate (MCPP), monobenzyl phthalate (MBzP), and monoethyl phthalate (MEP)] were measured in urine collected at the same time as the semen sample. We regressed natural log‐transformed (ln) sperm concentration, ln(total sperm count), ln(total motile sperm count), percent motile spermatozoa, and percent spermatozoa with normal morphology on each of the nine natural log‐transformed metabolite concentrations and on the molar‐weighted sum of DEHP metabolites in separate models. We fit unadjusted models and models that adjusted for confounders determined a priori. In unadjusted models, ln(MiBP) was significantly and positively associated with motility and ln(MBzP) significantly negatively associated with ln(total sperm count). In adjusted linear models, urinary metabolite concentrations of DEHP, DBP, DEP, and DOP were not associated with any semen parameter. We found an inverse association between ln(MBzP) concentrations and sperm motility (β = −1.47, 95% CI: −2.61, −0.33), adjusted for ln(creatinine concentration), geographic location, age, race, smoking status, stress, recent fever, time from sample collection and time to complete analysis. Several sensitivity analyses confirmed the robustness of these associations. This study and the available literature suggest that impacts of adult exposure to phthalates at environmental levels on classical sperm parameters are likely to be small.