Animal research

Researchers investigate link between reproducibility of behavioral studies and number of experimenters

The number one confounding factor in animal research is the person conducting the experiment. Behavioral biologists have examined this factor in behavioral experiments involving mice at different locations.

For more than ten years now, scientists have been discussing the so-called reproducibility crisis: often scientific results cannot be reproduced later and/or in other laboratories, despite the studies being conducted under highly standardized conditions. . Thus, standardization includes, for example, the use of genetically identical animals, the maintenance of animals in cages equipped in an identical manner, and the carrying out of experiments always in the same way. To uncover the sources of poor reproducibility, researchers typically try to identify potential confounders under the experimental conditions. So, the number one confounding factor is the experimenter – in other words, the person conducting the experiment. A team led by behavioral biologists Dr Vanessa von Kortzfleisch and Prof Helene Richter from the University of Münster (Germany) has now studied precisely this factor in behavioral experiments on mice carried out simultaneously at three different locations. Their study has just been published in the journal PLOS Biology.

To the researchers’ surprise, the influence of different experimenters on test results was not as pronounced as previous studies suggested. On the other hand, the researchers detected other confounding factors. Thus, what plays a much more important role than the experimenter is the “laboratory” factor. More importantly, however, most of the variations were explained by inexplicable differences between individual mice. Specifically, this proportion of “unexplained variance” in the data was between 41 and 72%. “This is particularly surprising,” says lead author Vanessa von Kortzfleisch, “when one considers that the animals were tested under highly standardized conditions within the same testing cohort – in other words, by the same experimenter in the same laboratory and under exactly the same conditions.”

The results certainly do not mean that the experimenter is not a decisive factor. What they do indicate, however, is that different testing conditions in labs – despite standardized conditions – have a considerably greater influence on the outcome than the experimenter. These conditions can include, for example, small differences in ambient sounds or smells. “But what our results show above all is that biological variation plays a key role in animal research – even when animals come from inbred lines. In the future, we will need better strategies to incorporate this variation. in a controlled way in the experimental design,” says Vanessa von Kortzfleisch.

Twelve experimenters on three sites

The background: Contrary to the dogma of strict standardization, there are alternative suggestions to systematically incorporate variation into the experimental design to improve reproducibility. In order to determine whether involving multiple experimenters in a single study can increase external validity, and therefore improve the reproducibility of results, this latest study was conducted by twelve different experimenters in Münster, Osnabrück and Bern, all performing the same test. behavioral. battery with mice from two inbred strains. Such phenotyping experiments are widely used in biomedical research to study the effects of different genotypes on animal behavior and, thus, to draw conclusions about the genetic basis of certain human diseases. For example, in a so-called Open-Field test, researchers check whether a mouse is more or less anxious when exploring a new environment.

Specifically, the research team investigated whether a strictly standardized experimental design, in which all animals are tested by one experimenter, differs in reproducibility from an experimental design in which animals are tested by multiple experimenters. The team compared the experimental plans to see which of them gave the most consistent results across the three different labs. Additionally, the researchers investigated what other influencing factors might account for the variation in the data. One result was that at all three locations, researchers were unable to replicate some of the results, whether the experiment was conducted by a single experimenter or multiple experimenters.

In addition to the team from the Department of Behavioral Biology in Münster, other researchers involved in the study come from the universities of Osnabrück and Bern (Switzerland), the University of Veterinary Medicine in Vienna (Austria) and the AstraZeneca of Cambridge (United Kingdom).

Source of the story:

Material provided by University of Munster. Note: Content may be edited for style and length.