Online citations, reference lists, and bibliographies.

Important Considerations When Selecting A Risk Assessment Tool

Dale N. Glaser
Published 2018 · Medicine, Psychology

Cite This
Download PDF
Analyze on Scholarcy
Share
ABSTRACT Measurement is an integral part of many disciplines, ranging from the social, healthcare/medical, to management sciences. The field of psychometrics has had a wide-ranging impact in the varied disciplines housed within psychology (e.g., clinical, developmental, etc.) as researchers and test developers aim to construct, refine, and modify their instruments. Given that test validation is a key component in furnishing evidence of validity, the intent of this nontechnical article is to reinforce (or serve as a reminder) for the applied audience the necessary efforts in optimizing the psychometric properties of their measurement tool. This article will be couched as a “lessons learned” document, primarily covering construct and criterion validity, but also, reliability estimation, and then finally, a comment about the confluence of null hypothesis significance testing, sample size and effect size and its relationship to psychometric testing.
This paper references
10.1037//1082-989x.5.3.343
Coefficient alpha and related internal consistency reliability coefficients.
H. G. Osburn (2000)
Score reliability
B. Thompson (2003)
Psychometric theory (3rd ed.)
J. C. Nunnally (1994)
10.1037/0003-066X.49.12.997
The earth is round (p < .05)
J. Cohen (1994)
10.2307/2669496
Test Theory: A Unified Treatment
R. P. McDonald (1999)
Designing surveys
R Czaga (2003)
The meaning and use of the area under a receiver
J. A. Hanley (1982)
10.2307/2348998
Regression Diagnostics: An Introduction
K. Trustrum (1991)
10.1148/radiology.143.1.7063747
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
J. Hanley (1982)
10.1016/S0147-9563(97)90038-8
Development of an instrument to measure cardiac illness dependency.
B. Riegel (1997)
10.1177/0013164402062004002
Reliability Generalization: Moving toward Improved Understanding and Use of Score Reliability
T. Vacha-Haase (2002)
10.4135/9781483399041
Best Practices in Logistic Regression
J. Osborne (2014)
10.1080/08098130809478203
Therapeutic Songwriting in Music Therapy
Felicity A. Baker (2008)
Beyond significance testing
R B Kline (2004)
Experimental and Quasi-Experimental Designs for Generalized Causal Inference
W. Shadish (2001)
Handbook Of Biometrics
Mario Baum (2016)
10.1007/978-3-642-71900-4_3
How Big is Big
H. G. Elias (1987)
Experimental And Quasi Experimental Designs For Research
P. Kaufmann (2016)
10.1146/annurev.ps.37.020186.000245
Evolving Concepts of Test Validation
A. Anastasi (1986)
Work in the 21st century (4th ed.)
F. J. Landy (2013)
10.2307/2532419
Applied Logistic Regression
David W. Hosmer (1989)
10.1177/1094428104267049
Beyond Global Measures of Relative Importance: Some Insights from Dominance Analysis
D. Budescu (2004)
10.1177/1094428106296642
Answers to 20 Questions About Interrater Reliability and Interrater Agreement
J. Lebreton (2008)
10.1037/0033-295X.111.4.1061
The concept of validity.
D. Borsboom (2004)
10.4135/9780857020024.n38
Factor Analysis
R. Darlington (2008)
10.1037/1082-989X.5.2.241
Null hypothesis significance testing: a review of an old and continuing controversy.
R. Nickerson (2000)
Convergent and discriminant validation
D. T. Campbell (1959)
10.7208/chicago/9780226458106.001.0001
The structure of scientific revolutions, 3rd ed.
T. Kuhn (1996)
10.1007/978-94-009-2195-5_3
Item Response Theory
D. Weiss (1991)
Understanding the new statistics
G. Cumming (2012)
10.1037/1082-989X.1.2.115
Statistical Significance Testing and Cumulative Knowledge in Psychology: Implications for Training of Researchers
F. Schmidt (1996)
10.2307/2529115
Statistical Power Analysis for the Behavioral Sciences
J. Cohen (1969)
10.4324/9781315827353
What If There Were No Significance Tests
L. Harlow (2013)
10.1037/A0019507
A Comprehensive Review of Effect Size Reporting and Interpreting Practices in Academic Journals in Education and Psychology.
Shuyan Sun (2010)
Psychological testing: Principles and applications, 2nd ed.
K. Murphy (1991)
10.1037/018882
Psychometric theory (2nd ed.).
R. Sitgreaves (1979)
10.1161/CIRCULATIONAHA.107.715359
Use and misuse of the receiver operating characteristic curve in risk prediction.
N. Cook (2007)
10.1037/h0040957
Construct validity in psychological tests.
L. J. Cronbach (1955)
Exploratory and confirmatory factor analysis of the
Thatiana Helena de Lima (2017)
10.1049/sbra028e_ch8
Statistical Decision Theory
S. French (2000)
10.1093/eurheartj/ehu207
Towards better clinical prediction models: seven steps for development and an ABCD for validation.
E. Steyerberg (2014)
10.4037/AJCC1999.8.5.291
The controversy of significance testing: misconceptions and alternatives.
D. Glaser (1999)
10.1016/S0169-7161(06)26001-2
1 A History and Overview of Psychometrics
L. V. Jones (2006)
10.1371/journal.pmed.1001491
Evaluation of Prediction Models for Decision-Making: Beyond Calibration and Discrimination
L. Holmberg (2013)
10.1111/j.1744-6570.2007.00091_11.x
Foundations of psychological testing: A practical approach, 2nd ed.
Sandra A. McIntire (2007)
Applied multiple regression analysis and generalized linear models
J Fox (2008)
Validity
S. Messick (1989)
Introduction to classical & modern test theory
L Crocker (1986)
10.1201/9781482276725
Item Response Theory : Parameter Estimation Techniques, Second Edition
F. Baker (2004)
Methods Matter: Improving Causal Inference in Educational and Social Science Research
R. Murnane (2010)
10.1037/1082-989X.8.2.129
The dominance analysis approach for comparing predictors in multiple regression.
R. Azen (2003)
10.4324/9780203843130-15
Exploratory and Confirmatory Factor Analysis
James P. Stevens (2012)
10.1177/0013164417698017
Studying Latent Criterion Validity for Complex Structure Measuring Instruments Using Latent Variable Modeling
T. Raykov (2018)
10.1002/PAM.10129
Experimental and Quasi‐Experimental Designs for Generalized Causal Inference, by William R. Shadish, Thomas D. Cook, and Donald T. Campbell. Boston: Houghton Mifflin, 2001, 623 pp., $65.56.
R. MacCoun (2003)
Confusion over measures
R. Wiley. Hubbard (2003)
10.1177/1094428104266510
History and Use of Relative Importance Indices in Organizational Research
J. W. Johnson (2004)
10.2307/2332896
Nonparametric Statistics for the Behavioral Sciences.
F. N. David (1956)
10.4135/9781412952644.n432
Standards for educational and psychological testing
A. Kulenović (2006)
Judgement and choice
R Hogarth (1989)
10.1891/9780826109842.0008
Asking questions.
N. Hatton (1979)
10.1037/0033-2909.114.3.542
Dominance analysis: A new approach to the problem of relative importance of predictors in multiple regression.
D. Budescu (1993)
10.1016/J.JECONOM.2010.09.001
Introduction to Measurement with Theory
W. Barnett (2011)
10.1207/S15327906MBR3501_1
A Heuristic Method for Estimating the Relative Weight of Predictor Variables in Multiple Regression
J. W. Johnson (2000)
Handbook of statistics (pp. 1–27)
Sinharay Eds (2017)
10.1037/met0000110
Multi-Relational Measurement for Latent Construct Networks
Laura M. Koehly (2018)
10.1177/0013164416669201
Three Insights from a Bayesian Interpretation of the One-Sided P Value
M. Marsman (2017)
Econometric Analysis 5th ed
W. Greene (2003)
10.1097/EDE.0b013e3181c30fb2
Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures
E. Steyerberg (2010)
10.1198/0003130031856
Confusion Over Measures of Evidence (p's) Versus Errors (α's) in Classical Statistical Testing
R. Hubbard (2003)
10.1037/h0046016
Convergent and discriminant validation by the multitrait-multimethod matrix.
D. Campbell (1959)
10.1016/j.pmrj.2014.10.006
Logistic regression.
Kristin L Sainani (2014)
10.1037/14047-013
The standards for educational and psychological testing.
Daniel R. Eignor (2013)
Assessing the performance of prediction models: A framework for some traditional and novel measures. Epidemiology
E. W. Steyerberg (2010)
10.1002/job.4030020205
The measurement of experienced burnout
C. Maslach (1981)
Judgement and choice (2nd ed.)
Mahwah (1989)
10.1037/0003-066X.52.1.15
In praise of the null hypothesis statistical test.
R. Hagen (1997)
10.1002/9781118561485.ch6
New Rules of Measurement
P. Williams (2015)
10.1111/LANG.12079
How Big Is “Big”? Interpreting Effect Sizes in L2 Research
Luke D Plonsky (2014)
When small effect sizes tell a big story, and when large effect sizes don't.
J. M. Cortina (2009)
10.1037/0033-2909.112.1.160
When small effects are impressive
D. Prentice (1992)
10.1016/0895-4356(92)90192-P
What price perfection? Calibration and discrimination of clinical prediction models.
G. Diamond (1992)
10.2307/2286442
Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences
J. Cohen (1976)
10.1108/RR-03-2014-0077
APA Dictionary of Statistics and Research Methods
M. Guha (2014)
10.1016/B978-0-08-044894-7.00250-5
Item Response Theory
A. Woods (1992)
Statistical Decision Theory
W. H. Pun (2014)
10.1037/h0049709
Sociometry, experimental method and the science of society: An approach to a new political orientation.
John Harding (1952)
10.1037/1040-3590.8.4.350
Uses and Abuses of Coefficient Alpha
N. Schmitt (1996)
10.2307/2283550
Statistical Theories of Mental Test Scores.
Maurice M. Tatsuoka (1968)
10.2307/2290095
Statistical Power Analysis for the Behavioral Sciences (2nd ed.)
P. Lachenbruch (1989)
10.4324/9781410603593
The New Rules of Measurement : What Every Psychologist and Educator Should Know
S. Embretson (1999)



This paper is referenced by
Semantic Scholar Logo Some data provided by SemanticScholar