Male infertility constitutes 30–40% of all infertility cases. Some studies have shown a continuous decline in semen quality since the beginning of the 20th century. One postulated contributing factor is radio frequency electromagnetic radiation emitted from cell phones. This study investigates an association between characteristics of cell phone usage and semen quality. Questionnaires accessing demographic data and characteristics of cell phone usage were completed by 106 men referred for semen analysis. Results were analysed according to WHO 2010 criteria. Talking for ≥1 h/day and during device charging were associated with higher rates of abnormal semen concentration (60.9% versus 35.7%, P < 0.04 and 66.7% versus 35.6%, P < 0.02, respectively). Among men who reported holding their phones ≤50 cm from the groin, a non-significantly higher rate of abnormal sperm concentration was found (47.1% versus 11.1%). Multivariate analysis revealed that talking while charging the device and smoking were risk factors for abnormal sperm concentration (OR = 4.13 [95% CI 1.28–13.3], P < 0.018 and OR = 3.04 [95% CI 1.14–8.13], P < 0.027, respectively). Our findings suggest that certain aspects of cell phone usage may bear adverse effects on sperm concentration. Investigation using large-scale studies is thus needed.