My social media feed has been blowing up lately with links to article headlines like this one from HuffPost, “Women Want More Sex Than Their Partner Does, According To New Study.” Naturally, as a sex researcher, I was intrigued--I wanted to read the paper and learn more. However, I was disappointed to learn that this wasn’t actually a scientific study and that numerous media outlets were reporting on it like it was legit science. Let's take a closer look at this "study" and what it does and doesn't tell us.

It appears that some website called Voucher Codes Pro conducted a survey of 2,383 British adults in long-term relationships in which people were asked some questions about their sex lives. So, this wasn’t research conducted by scientists that was published in an academic journal—it was a survey carried out by a private company I’ve never heard of that I suspect probably just wanted to garner some free press attention by releasing the results of a sex survey (at least that’s what the cynic in me says).

Private companies do these kinds of surveys all of the time, and their results can potentially be informative, but you can only determine that by evaluating their methodology. Unfortunately, virtually no details on their methodology are available. I searched extensively online and all I can find is a series of media articles that link back to one another (kind of like a game of “telephone”). Most seem to point to an article in Metro as the source (oddly, though, no details are available on the Voucher Codes Pro site itself). The Metro article tells us only how many people were surveyed and summarizes the findings. However, that’s not nearly good enough for me to evaluate whether this is a good quality survey.

For one thing, we don’t know what specific questions were asked or what the response options were. That’s a big deal. We don’t know whether they were using valid and reliable questions drawn from previous research, or if they simply made up some questions themselves. As someone who regularly teaches a course called Social Psychology of Attitudes in which I devote half of a semester to best practices in survey writing, I can attest that most people are pretty bad when it comes to writing survey questions—you really need scientific training in order to know how to do this (e.g., to avoid things like leading and loaded questions, to understand how the order and wording of your questions can influence the responses you receive, etc.).

For another, we don’t know how the participants were recruited. Where did they come from? Social media? A market research panel? Somewhere else? Also, how was the survey advertised and what kind of selection effects did that create? For example, was it advertised as a survey of couples experiencing sex and relationship problems? Moreover besides being British adults in LTRs, what were the other characteristics of the sample, like sexual orientation, race, and socioeconomic status? This information is important for understanding whether certain groups are over- or under-represented. Finally were there differences in age or other demographic characteristics between the male and female sub-samples that could potentially be contributing to the results they found?

Had this “study” been submitted to an academic journal, it would have undergone peer review in which questions like this would have been asked. However, because it wasn't, I have no way of evaluating these data and, consequently, I am extremely hesitant to draw sweeping conclusions from them (as many media outlets have), especially when you consider that the main finding (that women desire more sex than men by a wide margin) runs contrary to most published scientific research on gender and sex drive.