Abigail Browning, PhD’s Post

This is an extremely interesting post, because it shows that people in different countries respond differently to an unlabeled numerical scale. If you have just an array of consecutive numbers that people use to rate abstract concepts, what a "4" means or what a "9" means is going to change from person to person, unless you actually label the numbers to say what those numbers should mean. This is yet another reason why I do not trust NPS or eNPS as a customer or employee satisfaction metric.

Cross-country comparisons drawing on Likert-type scales build on the implicit assumption that, across countries, respondents perceive and use the scales’ response options in the same way. When this assumption is violated, observed differences in mean scores do not only reflect differences in the constructs of interest but also systematic differences in response option usage—a phenomenon referred to as response styles. Mirka Henninger, Thorsten Meiser, and I believe that the relationship between country-level self-reported love experiences and modernization reported by Sorokowski et al. (2023, doi: 10.1038/s41598-022-26663-4) poses an instructive cautionary tale of how the unaccounted presence of cross-country differences in response styles may lead to potentially spurious and artifactual conclusions. Employing state-of-the-art psychometric approaches to accommodate response styles, we re-analyzed the data from Sorokowski et al. and could show that once response styles are accounted for, conclusions on a substantial relationship between country-level love experiences and modernization are no longer supported. Our comment has just been published in Scientific Reports. https://lnkd.in/gTzvBqvj

Very nice example that shows how self-reporting in surveys needs to be aware of response sets and cultural differences in how people from different countries may use questionnaires. Unlike “34 * 5 = ?” which, according to most, has a correct answer, a question such as “How much do you agree with the statement: I need more training to do my job.” has no correct answer, and there may be different cultural norms that may make people agree or disagree more due to these norms in addition to their beliefs.

The importance of IRT modeling in psychometrics:

Attention, researchers! Here is what we need to report to be considered in a meta-analysis! https://lnkd.in/dijZxjXz 

Check out this new glossary that #NIHRLeicesterBRC researchers hope provides clear, concise definitions of key terms and concepts in the world of statistical methods research. Developed with the public, they should provide enough background to kickstart discussions between researchers and public contributors! https://lnkd.in/eqabcRV4 Let us know what you think! 👍👎✍️ Laura Gray University of Leicester #statistics #statisticalmethodsresearch #statisticalmethods #s

A must read if you use response latency /time to respond measures.

How NOT to discover hidden biases... Evidently, the Implicit Association Test (IAT) is big in the HR world, and so affects many people in the workplace by "discovering" their socially unacceptable subconscious biases. But is the test a product of the scientific method? In particular, are the test results reliable predictors of actual behaviour? Not according to a new study by Stan Young and Warren Kindzierski who put the IAT hypothesis to the test and found "that the IAT provides little insight into who will discriminate against whom" and "that the amount of real world variance explained by the IAT and explicit bias measurements was... less than 5 percent." On the basis of their and others findings, the test is more akin to pointing the bone than to a scientific diagnostic tool. The paper is available at https://lnkd.in/grtX9mhY

Spencer Greenberg discusses the importance of using the "Simplest Valid Analysis" (SVA) in research. He highlights that complex analyses can be misleading and subject to researcher bias, whereas SVA is straightforward and less prone to manipulation. Including SVA results increases transparency and trustworthiness in scientific studies. Researchers, reviewers, and editors are encouraged to prioritize SVA to enhance the validity and clarity of their findings. #ResearchMethods #DataAnalysis #ScientificIntegrity https://lnkd.in/g_583TTX

The 2024 JCR release dropped yesterday. Needless to say my relationship with impact factor is complicated. I love data and making data-driven decions and Impact factor is a metric that can give you a big look at journal performance. But (and this is a big but that cannot lie) it is a single metric, and one that arguablely doesn’t address the actual *impact* of your research. There are SO many other indices that can help authors determine true impact of their research. But…here we are. another year. Another deep dive. Here we go.