It was probably approximately 155cm for women, and about 168cm for men. We have direct evidence for this from analysing the skeletal remains of the Romans. For example, in a study [1] of 927 adult male Roman skeletons between 500 B.C. and A.D. 500, Professor Geoffrey Kron of the University of Victoria found an average of 168cm.

This is corroborated by remains found at the ancient towns of Herculaneum and Pompeii. Both cities were in famously destroyed by the A.D. 79 eruption of Mount Vesuvius. A study of the remains left by their unfortunate Roman residents tell us that:

The major samples from Herculaneum and Pompeii reveal the stature of the ancient adult body. The average height for females was calculated from the data to have been 155 cm in Herculaneum and 154 cm in Pompeii: that for males was 169 cm in Herculaneum and 166 cm in Pompeii. This is somewhat higher than the average height of modern Neapolitans in the 1960s and about 10 cm shorter than the WHO recommendations for modern world populations. - Laurence, Ray. "Health and the Life Course at Herculaneum and Pompeii." Health in Antiquity. Ed. Helen King. London: Routledge, 2005.

Notice how two neighbouring Roman communities nonetheless produced slightly different average heights. There will naturally be variations like this at different Roman settlements and at different time periods in Rome's lengthy history. Moreover, height can also be affected by diet, and thus there would probably have been some differences between different classes or groups of Romans, too.

We do also have some historical evidence, particularly from the height measurements of Roman soldiers. Soldiers probably would have been higher than civilians in general, though the results do seems generally in line with the skeletal remains:

Imperial regulations, though not entirely unambiguous, suggest that the minimum height for new recruits was five Roman feet, seven inches (165 cm., 5'5") ... for the army as a whole a reasonable estimate of a soldier's average height is around 170 cm (5'7"). - Roth, Jonathan, and Jonathan P. Roth. The Logistics of the Roman Army at War: 264 BC-AD 235. Columbia studies in the classical tradition, Vol. 23. Brill, 1999.