The work was funded by Harvard’s Mind, Brain, and Behavior program, the National Science Foundation, and the National Institutes of Health. Government spokeswomen said they could not confirm or deny whether an investigation was underway.

The paper tested cotton-top tamarin monkeys’ ability to learn generalized patterns, an ability that human infants had been found to have, and that may be critical for learning language. The paper found that the monkeys were able to learn patterns, suggesting that this was not the critical cognitive building block that explains humans’ ability to learn language. In doing such experiments, researchers videotape the animals to analyze each trial and provide a record of their raw data.

The retraction, submitted by Hauser and two co-authors, is to be published in a future issue of Cognition, according to the editor. It says that, “An internal examination at Harvard University . . . found that the data do not support the reported findings. We therefore are retracting this article.’’

Much remains unclear, including why the investigation took so long, the specifics of the misconduct, and whether Hauser’s leave is a punishment for his actions.

“Speaking in general,’’ he wrote, “we follow a well defined and extensive review process. In cases where we find misconduct has occurred, we report, as appropriate, to external agencies (e.g., government funding agencies) and correct any affected scholarly record.’’

In an e-mail yesterday, Hauser, 50, referred questions to Harvard. Harvard spokesman Jeff Neal declined to comment on Hauser’s case, saying in an e-mail, “Reviews of faculty conduct are considered confidential.’’

In a letter Hauser wrote this year to some Harvard colleagues, he described the inquiry as painful. The letter, which was shown to the Globe, said that his lab has been under investigation for three years by a Harvard committee, and that evidence of misconduct was found. He alluded to unspecified mistakes and oversights that he had made, and said he will be on leave for the upcoming academic year.

It is unusual for a scientist as prominent as Hauser — a popular professor and eloquent communicator of science whose work has often been featured on television and in newspapers — to be named in an investigation of scientific misconduct. His research focuses on the evolutionary roots of the human mind.

Two other journals say they have been notified of concerns in papers on which Hauser is listed as one of the main authors.

The findings have resulted in the retraction of an influential study that he led. “MH accepts responsibility for the error,’’ says the retraction of the study on whether monkeys learn rules, which was published in 2002 in the journal Cognition.

Harvard University psychologist Marc Hauser — a well-known scientist and author of the book “Moral Minds’’ — is taking a year-long leave after a lengthy internal investigation found evidence of scientific misconduct in his laboratory.

The editor of Cognition, Gerry Altmann, said in an interview that he had not been told what specific errors had been made in the paper, which is unusual. “Generally when a manuscript is withdrawn, in my experience at any rate, we know a little more background than is actually published in the retraction,’’ he said. “The data not supporting the findings is ambiguous.’’

Gary Marcus, a psychology professor at New York University and one of the co-authors of the paper, said he drafted the introduction and conclusions of the paper, based on data that Hauser collected and analyzed.

“Professor Hauser alerted me that he was concerned about the nature of the data, and suggested that there were problems with the videotape record of the study,’’ Marcus wrote in an e-mail. “I never actually saw the raw data, just his summaries, so I can’t speak to the exact nature of what went wrong.’’

The investigation also raised questions about two other papers co-authored by Hauser. The journal Proceedings of the Royal Society B published a correction last month to a 2007 study. The correction, published after the British journal was notified of the Harvard investigation, said video records and field notes of one of the co-authors were incomplete. Hauser and a colleague redid the three main experiments and the new findings were the same as in the original paper.

Science, a top journal, was notified of the Harvard investigation in late June and told that questions about record-keeping had been raised about a 2007 paper in which Hauser is the senior author, according to Ginger Pinholster, a journal spokeswoman. She said Science has requested Harvard’s report of its investigation and will “move with utmost efficiency in light of the seriousness of issues of this type.’’

Colleagues of Hauser’s at Harvard and other universities have been aware for some time that questions had been raised about some of his research, and they say they are troubled by the investigation and forthcoming retraction in Cognition.

“This retraction creates a quandary for those of us in the field about whether other results are to be trusted as well, especially since there are other papers currently being reconsidered by other journals as well,’’ Michael Tomasello, co-director of the Max Planck Institute for Evolutionary Anthropology in Leipzig, Germany, said in an e-mail. “If scientists can’t trust published papers, the whole process breaks down.’’

This isn’t the first time Hauser’s work has been challenged.

In 1995, he was the lead author of a paper in the Proceedings of the National Academy of Sciences that looked at whether cotton-top tamarins are able to recognize themselves in a mirror. Self-recognition was something that set humans and other primates, such as chimpanzees and orangutans, apart from other animals, and no one had shown that monkeys had this ability.

Gordon G. Gallup Jr., a professor of psychology at State University of New York at Albany, questioned the results and requested videotapes that Hauser had made of the experiment.

“When I played the videotapes, there was not a thread of compelling evidence — scientific or otherwise — that any of the tamarins had learned to correctly decipher mirrored information about themselves,’’ Gallup said in an interview.

In 1997, he co-authored a critique of the original paper, and Hauser and a co-author responded with a defense of the work.

In 2001, in a study in the American Journal of Primatology, Hauser and colleagues reported that they had failed to replicate the results of the previous study. The original paper has never been retracted or corrected.

Hauser’s work falls at the intersection of psychology, evolutionary biology, and neuroscience, and he has studied everything from the cognitive and evolutionary underpinnings of language to the idea that morality is innate. He began teaching at Harvard in 1992 and several times has been voted one of Harvard’s most popular professors by graduating students, according to his CV. He is writing a book titled “Evilicious: Explaining Our Evolved Taste for Being Bad.’’

Carolyn Y. Johnson can be reached at cjohnson@globe.com.

© Copyright 2011 Globe Newspaper Company.