Machine learning (ML) offers promising opportunities to efficiently analyze large amounts of data and predict future developments - for example in medicine and epidemiology. In psychology, for example, ML models are used to predict complex behavioral patterns such as relapses in addiction or depression. However, this field of research faces particular challenges: Small samples, incomplete data and measurements subject to measurement errors as well as methodological shortcomings in the implementation of the analyses often lead to erroneous results.

"Methodological flaws in psychological research have potentially serious negative consequences for both individuals and society," explains Prof. Dr. Ulrich Schroeders, Head of the Department of Psychological Diagnostics at the University of Kassel. "These pitfalls increase the risk of a 'replication crisis' in our field of research," adds Dr. Kristin Jankowsky, a researcher in the department. This means that the predictions achieved with machine learning are systematically biased and inaccurate. This calls into question the scientific validity and generalizability of the statements.

Prof. Schroeders and Dr. Jankowsky are therefore developing an ML workflow specially tailored to psychology, which draws attention to typical pitfalls in modeling and is thus intended to help reduce the risk of incorrect statements. The ML workflow comprises five steps: conceptualization, data preprocessing, model training, validation and evaluation, and interpretation and generalizability. In the project, the current status is first recorded in a comprehensive overview article. Based on this, a checklist will be developed to guide researchers through the ML process in the future and help them avoid typical mistakes. At the end of the project, the team will create a freely accessible online learning course that clearly conveys the logic and techniques of ML modeling. For the project "Overcoming the replication crisis in machine learning modelling", the researchers will receive a grant of €317,889 from the German Research Foundation as part of the priority program "META-REP: A meta-scientific program for the analysis and optimization of replicability in the behavioral, social and cognitive sciences".

"Through our project, we offer other researchers in the field of psychology valuable tools and resources that improve the quality of research and results," says Schroeders. In this way, the Kassel researchers are trying to help alleviate the replication crisis in ML modeling in the behavioral sciences.

Contact:

Prof. Dr. Ulrich Schroeders
Head of the Department of Psychological Diagnostics
Phone: +49 561 804-7529
Email: schroeders[at]psychologie.uni-kassel.de

Dr. Kristin Jankowsky
Research assistant in the Department of Psychological Diagnostics
Phone: +49 561 804-3576
Email: jankowsky[at]psychologie.uni-kassel.de