A PNAS paper published on Monday grabbed plenty of headlines (my favorite being "Best friends forever gene: It's all in our DNA, say scientists" from the ever-reliable Daily Mail). Presumably that's because its results are so media-friendly: The paper reports finding genetic variants that seem to affect the friends people make, with one variant apparently attracting people to other carriers, while a second reportedly repels carriers from others who hold the same segment of DNA.

Based on their findings, the authors – including Nicholas Christakis and James Fowler, who have developed a reputation for their analysis of social networks, and in particular their highly publicized findings that obesity spreads through social networks and that political participation is heritable – make some bold conceptual conclusions about the impact of genetic effects on social interactions for human evolution and even resistance to infectious disease:

For example, some individuals might be “immune” to whatever pathogen is spreading in a population not because of their own constitution, but rather because they have come to surround themselves with others with particular genotypes.

I suspect most regular readers are already thinking suspicious thoughts, and with some justification: Genetic association studies on behavioral traits are notoriously littered with false positives, and the more headline-worthy the findings, the more caution we should apply. That's not to say that all behavioral genetic studies should be discarded outright, but certainly we should be careful to set our threshold high for a study like this one.

While some of the media coverage of the paper has been fairly credulous, there have also been skeptical voices:

"If this was a study looking for shared genes in patients with diabetes, it would not be up to the standards of the field," says David Altshuler, a geneticist at the Broad Institute in Cambridge. "We set these standards after 10 years of seeing so many irreproducible results in gene-association studies." Because most genes have modest effects on behaviour or health, many scientists assume that thousands of SNPs — rather than six — need to be analysed before a correlation to any trait can be confidently made. Geneticists are often hard-pressed to find one SNP in a million that reproducibly correlates with a disease, says Altshuler. "It's like the team bought six lottery tickets and won the megabucks twice — this is not how things work."

Altshuler's skeptical view of the paper was fairly widely shared by colleagues I discussed this with yesterday: Given what we know about the genetics of complex traits, it seems a priori extremely unlikely to have found two real associations in a study of just six genes, even if those genes have been selected on the basis of biological plausibility. This seems especially unlikely for a behavioral trait: the complete failure of recent large genome-wide association studies to uncover any genetic variant convincingly associated with personality traits suggests that, while these traits are known to be strongly influenced by genetics, those influences are not exerted by common genetic variants of large effect.

The buzz amongst the genomics community on Twitter was generally similarly negative, although informed discussion wasn't helped by the fact that (thanks to the standard ridiculous post-embargo delay from PNAS) the actual paper wasn't available online until yesterday evening – meaning that there was little besides mainstream media reports for most people to base their judgement on.

So, what are we to make of the bold claims in this paper?

The good and the bad

Let me be clear from the outset: given the limitations of the datasets the authors had to work with (they relied on results from studies performed by other researchers, and there are few cohorts with both genetic and social network data), there are a lot of things this study has done well. The sample sizes are large, certainly by the standards of behavioral genetics (around 1,000 in the Add Health discovery cohort, and 2,000 to 3,000 in the Framingham Heart Study [FHS] replication group). The authors replicated their initial findings in a separate cohort, and using a different technology (although not without caveats; see below). They also used a family-based method to assess association, which should protect them against one massive potential source of false positives: the possibility that people will have more similar genotypes to their friends simply because their friends tend to be from the same racial background.

Finally, the statistical significance of the findings, combined across the discovery and replication cohorts, is pretty respectable: the odds of obtaining these findings by chance are around one in 2,000 for one marker, and less than 1 in 30,000 for the other (using Fisher's method to combine P values, courtesy of Luke Jostins). Even allowing for multiple testing of six markers, this suggests this isn't simply an accidental finding. That's not to say that it couldn't have arisen as a result of some systematic bias, although the fact that the direction of effect differs between the two markers makes this seem less likely.

However, there are still some noteworthy caveats. The study looks at only six markers, albeit in genes with prior published evidence (of varying reliability) for association with human behavioral traits, rather than take a more unbiased look at genome-wide markers. The "replication" study didn't actually look at exactly the same markers as the discovery study, but instead filled in ("imputed") the genotypes at those markers using information from other nearby variants examined in the sample; this process worked well for one of the markers, but was more tenuous for the second, which could have caused problems if the uncertainty wasn't properly accounted for.

Some researchers I spoke to also had specific concerns about the methods used to construct the friendship network, and about the calibration of the statistical tests used to detect association.

All of these issues are troubling to various degrees, but it seems to me that none of them are disastrous (as far as I can tell, anyway) in the sense of strongly suggesting that the study's results are false positives: these are reasons to be cautious, but not to confidently discard the results as flawed. However, I was particularly curious about one of these issues: why had the researchers focused on only six markers?

Why not a genome-wide association study?

The lead researcher's explanation for focusing on the six genes rather than perform a genome-wide association study (GWAS) – the gold standard approach in the field – is paraphrased in the Nature News piece quoted above:

Fowler defends his decision to focus on six genes, rather than thousands. Genome-wide information wasn't available and, he says, the "transmission disequilibrium" statistical tests that the group ran to control for similarity owing to ancestry are among the strongest in the field of human-genetics studies.

However, the section in italics isn't true. In fact the authors had access to a very large amount of genome-wide information from a cohort with the required information about friendship patterns – the very same cohort they used to "replicate" the associations found in their initial study of six genes. Here's a quote from the paper's Methods section:

Out of the 14,428 members of the three main cohorts, a total of 9,237 individuals have been genotyped (4,986 women and 4,251 men). Genotyping was conducted using the Affymetrix 500 K array and an Affymetrix 50 K supplemental array [my emphasis].

In other words, the researchers of this study had access to data on over half a million genetic markers in a large cohort with social network data. This dataset could – at least in theory – have been used to perform a rigorous GWAS, with any markers showing evidence for association then being replicated independently by genotyping them directly in the Add Health cohort.

I contacted the authors by email to ask for clarification on this point. Firstly, on the mismatch between the Nature News article and the paper's methods section, lead author James Fowler told me:

This passage of that article is not a good representation of what I told the reporter. I explained that genome-wide information was not available in both studies – instead, it is only available in FHS [the replication cohort].

He also noted that none of the authors on this paper have the capability to direct the genotyping of the Add Health cohort; they simply collaborate with the study's organisers to analyse the data emerging from it, meaning the coordination of a GWAS followed by large-scale replication genotyping would have been difficult.

Fortunately, there is hope for a resolution here: Fowler told me that the Add Health organisers are currently organising a full GWAS analysis of 17,000 individuals. Once available, this dataset should provide immediate and fairly unambiguous evidence for or against the associations reported in this paper, and would also allow the rigorous identification of any other regions of common variation elsewhere in the genome that showed the same patterns of social clustering. While it sounds as though the timeline for this GWAS is still unclear, at least there's the potential for absolute validation of these findings one way or another in the not-too-distant future.

A small step in the right direction, or a step too far?

Overall I find myself rather torn here. While David Altshuler is absolutely right that this study wouldn't meet the criteria for publication in Nature Genetics, and while I'm generally a fierce critic of both candidate gene association studies and behavioral genetics in general, there is more substance to this study than I expected. I'm not saying I'm confident the findings are real – that will require a full, independent replication study looking at exactly the same markers typed in this study – but it's certainly a result that warrants follow-up.

However, while there are some genuinely interesting implications here, the authors' bolder statements in the discussion section are far-fetched enough to immediately raise the hackles of any cautious geneticist. While I'm happy to reserve judgement on the question of whether or not the two specific associations found in this paper are real, the idea that such effects will be large in effect and widespread throughout the genome seems fairly implausible, especially in the context of the disappointing results from personality trait GWAS mentioned at the beginning of this post. Certainly it will take substantially more evidence than this study provides to make this scenario seem likely.

So, where does this leave us? Not a great distance from where we started, really. This study is an intriguing observation in support of a broadly plausible hypothesis, and it's entertaining to consider its implications for association studies, human evolution, and gene-environment interactions. However, until we see the promised large-scale GWAS, it's best not to spend too much time pondering these implications: let's save that for if and when we have the evidence needed to confirm that these effects are real, and to gain a better understanding of how common they are in the genome.

Many thanks to Luke Jostins, Sarah Medland, Kate Morley and Jeff Barrett for comments on the paper.