Voodoo Correlations in Social Neuroscience

Puzzlingly High Correlations in fMRI Studies of Emotion, Personality, and Social Cognition

1

Are blogs good or bad for the enterprise of scientific peer review? At present, the system relies on anonymous referees to provide "unbiased" opinions of a paper's (or grant's) merits. For today, the discussion will focus on peer review of papers in scientific journals.



-from The Voodoo of Peer Review , by The Neurocritic

Science

Arsenic-associated bacteria (NASA's claims)



Here's a detailed review of the new paper from NASA claiming to have isolated a bacterium that substitutes arsenic for phosphorus on its macromolecules and metabolites. ... Basically, it doesn't present ANY convincing evidence that arsenic has been incorporated into DNA (or any other biological molecule).

. . .

Bottom line: Lots of flim-flam, but very little reliable information. The mass spec measurements may be very well done (I lack expertise here), but their value is severely compromised by the poor quality of the inputs. If this data was presented by a PhD student at their committee meeting, I'd send them back to the bench to do more cleanup and controls.

NASA's arsenic microbe science slammed



. . .



Debate shouldn't be in media: NASA

When NASA spokesman Dwayne Brown was asked about public criticisms of the paper in the blogosphere, he noted that the article was peer-reviewed and published in one of the most prestigious scientific journals. He added that Wolfe-Simon will not be responding to individual criticisms, as the agency doesn't feel it is appropriate to debate the science using the media and bloggers. Instead, it believes that should be done in scientific publications. Redfield said the reason she posted the review on her blog is partly because scientific publications such as Science — and the debates therein — are typically behind a paywall and inaccessible to the public.

"I blog openly…to bring this stuff more into the open where everybody can see it," she said.

Science

fMRI Special Section

The paper by Vul, Harris, Winkielman, and Pashler made its initial appearance on Ed Vul's website once it was accepted for publication by Perspectives in Psychological Sciences . Originally titled "", it was eventually renamed the more globally palatable "" at the request of the editor (after much consternation from the criticized authors). This paper sparked intense debate in the field of functional neuroimaging, much of which occurred in the blogosphere (at least initially).Many of the aggrieved researchers in the neuroimaging community were appalled that bloggers were discussing Vul's accepted paper before it was "properly" published (and before they had time to comment themselves). But two research groups quickly issued replies:This issue is relevant again today because of the fallout over the infamous arsenic paper Wolfe-Simon et al., 2010 ), which claimed it had isolated a bacterium that can substitute arsenic for phosphorus. Publication of the paper was preceded by a Sphinxlike press release : "NASA will hold a news conference at 2 p.m. EST on Thursday, Dec. 2, to discuss an astrobiology finding that will impact the search for evidence of extraterrestrial life." After the paper appeared online in, negative reactions from qualified and prominent scientists were swift. One of the most visible (and withering) critiques was written by Dr. Rosie Redfield , a microbiologist at the University of British Columbia:CBC News covered the critical backlash and NASA's reply, which was anti-blog:Redfield has now posted a draft of her official letter toFor full coverage of the matter, I recommend Is That Arsenic-Loving Bug — Formerly an Alien — a Dog? and The Wrong Stuff: NASA Dismisses Arsenic Critique Because Critical Priest Not Standing on Altar by David Dobbs, "This Paper Should Not Have Been Published " by Carl Zimmer, and An arsenic bacteria postmortem: NASA responds, tries to pit blogs vs. “credible media organizations” by Ivan Oransky (for starters).Returning now to Voodoo Correlations... In the November 2010 issue of Perspectives in Psychological Sciences (online), outgoing editor Ed Diener has assembled anlooking back at those heady days and forward into the future:

Neuroimaging: Voodoo, New Phrenology, or Scientific Breakthrough? Introduction to Special Section on fMRI

In response to the widespread interest following the publication of Vul et al (2009), Perspectives Editor Ed Diener invited researchers to contribute articles for a special section on fMRI, discussing the promises and issues facing neuroimaging.

Mistreating Psychology in the Decades of the Brain

Gregory A. Miller

Scientists tend to consider psychology-biology relationships in two distinct ways: by assuming that psychological phenomena can be fully explained in terms of biological events and by treating them as if they exist in separate realms. These approaches hold up scientific progress and have important implications for clinical practice and policy decisions (e.g., allocating research funds).





Brian D. Gonsalves and Neal J. Cohen

The growth of neuroimaging research has led to reflection on what those techniques can actually tell us about cognitive processes. When used in combination with other cognitive neuroscience methods, neuroimaging has promise for making important advancements. For example, neuroimaging studies on memory have raised questions not only about the regions involved with memory but also about component cognitive processes (e.g., the role of different attention subsystems in memory retrieval), and this has resulted in more theorizing about the interactions of memory and attention.

Mapping Mental Function to Brain Structure: How Can Cognitive Neuroimaging Succeed?

To understand the anatomy of mental functions, researchers may to need to move away from commonly used brain mapping strategies and begin searching for selective associations. This will require more emphasis on the structure of cognitive processes, which may be achieved through development of formal ontologies (e.g., the Cognitive Atlas) that will describe the "parts" and processes of the mind. Using these ontologies in combination with large-scale data mining approaches may more directly relate mental processes and brain function.

The Appeal of the Brain in the Popular Press

Why do people like the brain so much? Brain-related articles in the press, especially ones about fMRI research, tend to be very popular with the general public, but many of these articles may result in misinterpretations of the science. Part of the popularity may be attributed to their deceptively simple message: Perform an action and a certain area lights up. In addition, people are more confident in "biological" images than in the behavioral phenomena on which the images are based. In order to maintain trust with the public, scientists have a responsibility to provide the press with descriptions of research and interpretations of results research that are clear, relevant, and scientifically accurate.

Frontiers in Human Neuroscience: The Golden Triangle and Beyond

The development of neuroimaging has created an opportunity to address old questions about brain function and behavior in new ways and also to uncover new questions. The knowledge that emerges from neuroimaging studies is more likely to be beneficial when combined with techniques and analyses that break down complex constructs into structures and processes, measures that gauge neural events across different times, and animal studies.

Bridging Psychological and Biological Science: The Good, Bad, and Ugly

The advent of functional neuroimaging has brought both praise and criticism to the field of psychological science. Although most studies relying on fMRI are correlative, they do offer some clues about the biology underlying psychological processes. However, it is not sufficient to show which area of the brain is involved in a particular cognitive process; rather theories need to address "how?" questions (e.g., How does the hippocampus contribute to remembering?) in order to best bridge psychological and biological science.

Footnote

1

References

Perspectives on Psychological Science

Science

Thanks to Sandra of Channel N for alerting me to the special issue .



Also see Mind Hacks BPS Research Digest , and Neuroskeptic Vul E, Harris C, Winkielman P, Pashler H (2009). Puzzlingly High Correlations in fMRI Studies of Emotion, Personality, and Social Cognition 4(3), 274-290.Wolfe-Simon F, Blum JS, Kulp TR, Gordon GW, Hoeft SE, Pett-Ridge J, Stolz JF, Webb SM, Weber PK, Davies PC, Anbar AD, & Oremland RS (2010). A Bacterium That Can Grow by Using Arsenic Instead of Phosphorus