Online citations, reference lists, and bibliographies.
Referencing for people who value simplicity, privacy, and speed.
Get Citationsy
← Back to Search

The Assessment Of Professional Competence: Developments, Research And Practical Implications.

C. V. D. van der Vleuten
Published 1996 · Medicine

Save to my Library
Download PDF
Analyze on Scholarcy Visualize in Litmaps
Share
Reduce the time it takes to create your bibliography by a factor of 10 by using the world’s favourite reference manager
Time to take this seriously.
Get Citationsy
Educational achievement testing is an area of turmoil in the health sciences. Examinations are a constant source of problems for many teachers, curriculum designers and educationalists. The evaluation of student achievement is continuously debated at educational meetings, conferences and workshops. It is an area in which tradition, personal values and experiences tend to dominate discussions. On the other hand, the number of scientific publications on assessment over the last decade has exploded. The number of proposed instruments, each preferably using an intriguing acronym, is countless. The literature is however often difficult to access since the psychometrics usually involved in educational testing discourage the average health professions reader. Assessment in the health professions education is nevertheless an area which is fortunately well-researched and has delivered a number of well-documented outcomes. The purpose of this article is to highlight these outcomes and to attempt to translate them into practical implications and research suggestions. We will not review individual methods and instruments in detail, but will describe some classes of methods contingent on a (supposed) theoretical framework. We will delineate what we consider paramount findings within and across these classes and discuss their theoretical implications and their effect on the evolution of new testing instruments. By using a simple conceptual framework the utility of assessment methods will subsequently be defined in a generic sense meant to be helpful in deciding how to compromise and make trade-offs in assessment practice. To improve the utility of assessment a number of practical suggestions and research recommendations are proposed using this framework.
This paper references
10.3102/00346543064002253
Knowledge Taught in School: What Is Remembered?
G. Semb (1994)
10.3102/01623737015001001
Educational Assessment: Expanded Expectations and Challenges
R. Linn (1993)
10.1097/00001888-199008000-00005
The quarterly profile examination
L. Arnold (1990)
10.1080/0260293870120307
Assessment of Clinical Competence: Written and Computer-Based Simulations
D. Swanson (1987)
10.1016/S0140-6736(95)90014-4
Report card from McMaster: student evaluation at a problem-based medical school
J. Blake (1995)
Assessment of clinical competence: Written and computerbased simulations. Assessment and Evaluation in Higher Education
D Swanson (1987)
10.1111/J.1365-2923.1979.TB00918.X
Assessment of clinical competence using an objective structured clinical examination (OSCE).
R. Harden (1979)
Scoringsmodellen voor open-vraag vormen (Scoring models for free-response formats) [dissertation
Frijns PHAM (1992)
10.1001/ARCHINTE.1987.00370060045009
Ensuring the clinical competence of medical school graduates through standardized patients.
P. Stillman (1987)
10.3109/01421597909019397
Against multiple choice questions.
S. Pickering (1979)
10.3102/0013189X023003023
Use of Standardized Patients in Clinical Assessments: Recent Developments and Measurement Findings
N. V. Vu (1994)
10.1037/0003-066X.37.8.911
The validity of licensure examinations.
M. Kane (1982)
10.1177/016327878600900302
An Evaluation of a Computer Simulation in the Assessment of Physician Competence
J. Norcini (1986)
Student assessment in problem-based learning curricula
SM Case (1991)
The feasibility, acceptability and reliability of open-ended questions in a problem-based learning curriculum
BF Stalenhoef-Halling (1990)
10.1111/j.1365-2923.1991.tb00037.x
Pitfalls in the pursuit of objectivity: issues of validity, efficiency and acceptability
G. Norman (1991)
10.2307/1175572
Essentials of educational measurement
Robert L. Ebel (1972)
10.1111/J.1745-3984.1967.TB00562.X
SIMULATION TECHNIQUE IN THE MEASUREMENT OF PROBLEM‐SOLVING SKILLS1
C. McGuire (1967)
10.1111/j.1365-2923.1987.tb00367.x
Factors underlying performance on written tests of knowledge
G. Norman (1987)
10.1037/0003-066X.39.3.193
The real test bias: Influences of testing on teaching and learning.
N. Frederiksen (1984)
Model for a criterion-referenced medical specialty test
J. Maatsch (1980)
10.1080/10401339009539432
Assessment of clinical skills with standardized patients: State of the art
C. Vleuten (1990)
10.1111/j.1365-2923.1989.tb00819.x
A written test as an alternative to performance testing
C. P. M. Vleuten (1989)
10.1097/00001888-197712000-00007
The portable patient problem pack: a problem-based learning unit.
H. Barrows (1977)
Measurement as an Instructional Catalyst.
W. Popham (1983)
A review of recent innovations in assessment
G Norman (1985)
Clinical ratings - ward evaluation
D. Streiner (1995)
10.3102/0013189X024005005
Performance-Based Assessment: Lessons From the Health Professions
D. Swanson (1995)
Problem centered learning: The modified essay question in medical education
K Hodgkin (1975)
Fifteen years of experience with progress - testing
Van der Vleuten CPM
10.1097/00001888-199002000-00015
How medical school faculty regard educational research and make pedagogical decisions
M. Nelson (1990)
10.1111/j.1365-2923.1983.tb00657.x
The effect of assessments and examinations on the learning of medical students
D. Newble (1983)
The Pattern Completion Item (PCI): A potential measure of clinical problem-solving skills
DI Newble (1995)
A cognitive psychological interpretation of a few remarkable psychometric findings
CPM Vand er Vleuten (1995)
10.1037/0033-295X.96.2.341
Relation of Implicit Theories to the Construction of Personal Histories
M. Ross (1989)
10.1080/10401339309539601
Extended‐matching items: A practical alternative to free‐response questions
S. M. Case (1993)
Content validity of the key features' approach of the Medical Council of Canada's Exam
C Brailovsky (1992)
Peer-evaluation and problem-based learning
W De Grave (1984)
10.1207/S15327876MP0203_1
Generalizability of Job Performance Measurements: Marine Corps Rifleman
Richard J. Shavelson (1990)
10.1111/j.1365-2923.1991.tb00036.x
Pitfalls in the pursuit of objectivity: issues of reliability
C. P. M. Vleuten (1991)
Studies in community-based education : programme implementation and student assessment at the Faculty of Medicine, University of Gezira, Sudan
M. E. Magzoub (1994)
10.1097/00001888-199504000-00010
Content validation of key features on a national examination of clinical decision‐making skills
G. Bordage (1995)
How can clinical reasoning be tested
Van der Vleuten CPM (1995)
10.4324/9780203763643
Personality and Assessment
W. Mischel (1996)
10.1111/J.1745-3984.1994.TB00437.X
Are Tests Comprising Both Multiple‐Choice and Free‐Response Items Necessarily Less Unidimensional Than Multiple‐Choice Tests?An Analysis of Two Tests
D. Thissen (1994)
Profile of the Clinical Reasoning Test (CRT): An objective measure of problem solving skills and proficiency in using medical knowledge
RG Williams (1984)
Written methods for assessing clinical competence
C. McGuire (1987)
Effecten van curriculumveranderingen
J. Cohen-schotanus (1994)
Research in the psychology of clinical reasoning: implications for assessment
G Norman (1989)
10.1097/00001888-199302000-00022
Perspectives in assessment
C. McGuire (1993)
A system for student assessment
Van der Vleuten CPM (1990)
A measurement framework for performance-based tests
Swanson Db (1987)
Open ended questions versus multiple choice questions: An analysis of cuing effects
LWT Schuwirth (1992)
Assessment of medical interviewing skills: Generalizability of scores using successive MAAS-versions
J Van Thiel (1992)
Measurement-Driven Instruction: It's On the Road.
W. J. Popham (1985)
Comprehensive assessment of clinical competence of family-general physicians using multiple measures
GR Norman (1989)
Reliability, validity and efficiency of multiple choice questions and patient management problem item formats in the assessment of physician competence
JJ Norcini (1985)
Clinical reasoning in the health
J Higgs (1995)
10.1080/10401339409539675
Exploratory study of an examination format for problem‐based learning
C. Friedman (1994)
10.1111/J.1745-3992.1983.TB00688.X
The Practical Validation of Tests of Ability
Robert L. Ebel (1983)
10.1111/J.1745-3984.1993.TB00424.X
Sampling Variability of Performance Assessments.
R. Shavelson (1993)
10.1007/BF00891784
The influence of progress tests and block tests on study behaviour
H. Berkel (1994)
10.1097/00001888-199609000-00015
Issues in cognitive psychology: implications for professional education
G. Regehr (1996)
A comparison of checklists and rating scales in performance-based testing
SJ VanLuijk (1992)
10.1080/10401338909539410
Reliability and construct validity of some cognitive measures of clinical reasoning
G. Norman (1989)
Assessment of performance in actual practice of general practitioners by use of standardized patients
JJ Rethans (1991)
An evaluation of the construct validity of four alternative theories of clinical competence.
J. Maatsch (1986)
10.1097/00005650-198010000-00006
The Relationships of Pediatric Resident Recording Behavior Across Medical Conditions
V. Erviti (1980)
10.1111/j.1365-2923.1981.tb02311.x
A comparison of behaviour on simulated patients and patient management problems
G. Norman (1981)
10.1111/j.1365-2923.1987.tb00693.x
Validation of a new measure of clinical problem‐solving
E. Graaff (1987)
10.1097/00000542-197809000-00021
Validity of written examinations.
C. Mcleskey (1978)
10.1097/00001888-199010000-00001
A cognitive perspective on medical expertise: theory and implication [published erratum appears in Acad Med 1992 Apr;67(4):287]
H. Schmidt (1990)
10.1080/0260293890140103
THE ROLE OF SELF-ASSESSMENT IN STUDENT GRADING
D. Boud (1989)
Written simulation scoring: Comparison of nine systems [dissertation
TJ Bligh (1980)
10.2307/1319386
Tips For Improving Testing And Grading
D. Adams (1993)
10.1080/10401339509539709
Reliability of the fellowship examination of the royal Australian college of general practitioners
R. Hays (1995)
10.1037/H0045478
Must all tests be valid
Robert L. Ebel (1961)
10.1097/00001888-199310000-00028
An objective structured clinical examination for the licentiate of the Medical Council of Canada: from research to reality
R. Reznick (1993)
Global rating scales
DL Streiner (1985)
The effect of structure in scoring methods on the reproducibility of tests using open-ended questions
Frijns PHAM (1990)
10.1016/S0140-6736(83)90015-6
COMPREHENSIVE ASSESSMENT OF FINAL-YEAR MEDICAL STUDENT PERFORMANCE BASED ON UNDERGRADUATE PROGRAMME OBJECTIVES
GrahameI. Feletti (1983)
10.1111/J.1365-2923.1977.TB00623.X
Assessment of problem‐solving ability
J. Marshall (1977)
10.1080/10401339609539799
Conceptual and methodological issues in studies comparing assessment formats
G. Norman (1996)
10.1097/00001888-197810000-00005
Experience with the simulated patient-physician encounter.
R. B. Friedman (1978)
10.1111/j.1365-2923.1982.tb01073.x
Patient management problems Issues of validity
D. Newble (1982)
10.1136/bmj.303.6814.1377
Does competence of general practitioners predict their performance? Comparison between examination setting and actual practice.
J. Rethans (1991)
Psychometrician's beliefs about learning
LA Shepard (1991)
The "triple-jump" exercise--further studies of an evaluative technique.
A. Powles (1981)
10.1177/014662168200600101
A Comparison of Free-Response and Multiple-Choice Forms of Verbal Aptitude Tests
W. Ward (1981)
10.1080/0260293940190101
General Issues about Assessment of Competence
P. Hager (1994)
An alternative approach to PMPs: The "key features" concept
G Bordage (1987)
Can an examination predict competence ? The role of recertification in maintenance of competence
GR Norman (1991)
10.1111/j.1365-2923.1979.tb01511.x
A comparison of multiple‐choice tests and free‐response tests in examinations of clinical competence
D. Newble (1979)
10.1111/j.1365-2923.1991.tb00055.x
Reliability and feasibility of measuring medical interviewing skills: the revised Maastricht History‐Taking and Advice Checklist
J. Thiel (1991)
10.1080/10401339409539680
Guidelines for assessing clinical competence
David Newble (1994)
10.1097/00001888-199307000-00001
Guidelines for estimating the real cost of an objective structured clinical examination
R. Reznick (1993)
10.7326/0003-4819-110-9-719
Predictive validity of certification by the American Board of Internal Medicine.
P. Ramsey (1989)
Styles of learning and teaching
N. Entwistle (1981)
10.1097/00001888-199112000-00012
A review of the validity and accuracy of self‐assessments in health professions training
M. J. Gordon (1991)
The relation between content and psychometric characteristics in performance-based testing
SJ Van Luijk (1990)
The assessment of clinical competence in general practice with chart stimulated recall
I Van Ham (1990)
Can an examination predict competence? The role of recertification in maintenance of competence
GR Norman (1991)
Progress in developing a standardized patient test of clinical skills at the National Board of Medical Examiners: prototype two
D Klass (1995)
Measurement as an instructional catalyst. In: Ekstrom RB, editor. Measurement, technology and individuality in education
WJ Popham (1983)



This paper is referenced by
10.4414/smw.2013.13897
The new licencing examination for human medicine: from concept to implementation.
Sissel Guttormsen (2013)
10.1038/sj.bdj.4812047
Assessment of professional behaviour — a comparison of self-assessment by first year dental students and assessment by staff
S. Zijlstra-Shaw (2005)
10.1046/j.0308-0110.2002.01243.x
Facing the challenges of competency‐based assessment of postgraduate dental training
A. Evans (2002)
10.1111/j.1365-2929.2004.02018.x
The effect of in‐training assessment on clinical confidence in postgraduate education
C. Ringsted (2004)
10.1046/j.1365-2923.2002.01177.x
Professional development in undergraduate medical curricula from an assessment point of view
L. Schuwirth (2002)
10.1080/0142159X.2019.1652732
The utility of mini-Clinical Evaluation Exercise in undergraduate and postgraduate medical education: A BEME review: BEME Guide No. 59
S. Mortaz Hejri (2019)
10.3109/0142159X.2013.801548
Estimating and comparing the reliability of a suite of workplace-based assessments: An obstetrics and gynaecology setting
M. Homer (2013)
10.1186/1472-6920-13-47
Objective structured assessment of technical competence in transthoracic echocardiography: a validity study in a standardised setting
D. Nielsen (2013)
10.1111/medu.12996
Value of supervised learning events in predicting doctors in difficulty
Mumtaz Patel (2016)
10.1097/ACM.0b013e3181c874cb
Relevance of the Flexner Report to Contemporary Medical Education in South Asia
Z. Amin (2010)
10.1007/s10459-014-9521-9
Cutting for a career; a discussion of the domains of surgical competence using expert bespoke tailoring as a metaphor for surgical practice
J. Rees-Lee (2015)
10.3109/0142159X.2016.1158799
Evidence regarding the utility of multiple mini-interview (MMI) for selection to undergraduate health programs: A BEME systematic review: BEME Guide No. 37
Eliot L Rees (2016)
10.12669/pjms.321.9007
Impact of Combined modular assessment on deep learning and personal development of medical students
Shabana Ali (2016)
Assessing Competence of NHS Consultants: Challenges and Possible Solutions
Ahmed M Halawa (2017)
10.1152/advan.00047.2014
Use of the NBME Comprehensive Basic Science Examination as a progress test in the preclerkship curriculum of a new medical school.
T. Johnson (2014)
10.1111/j.1445-2197.2008.04441.x
UPDATE: ADVANCES IN SURGICAL EDUCATION
(2008)
10.1111/j.1365-2929.2006.02651.x
Psychometric properties of an instrument to measure the clinical learning environment
K. Boor (2007)
10.1097/ACM.0000000000000221
Reconceptualizing variable rater assessments as both an educational and clinical care problem.
J. Kogan (2014)
10.1016/j.amjsurg.2013.08.024
Validity and reliability of a novel written examination to assess knowledge and clinical decision making skills of medical students on the surgery clerkship.
A. Reinert (2014)
10.1542/peds.2013-2917
Putting the Pediatrics Milestones Into Practice: A Consensus Roadmap and Resource Analysis
D. Schumacher (2014)
10.1016/j.bpobgyn.2012.08.019
Objective Structured Assessment of Technical Skill in assessing technical competence to carry out caesarean section with increasing seniority.
A. Landau (2013)
10.3109/0142159X.2011.600360
Pre-training evaluation and feedback improve medical students’ skills in basic life support
Q. Li (2011)
10.1111/anae.12625
Psychometric evaluation of a direct observation of procedural skills assessment tool for ultrasound‐guided regional anaesthesia
M. Watson (2014)
10.1017/S002221511500242X
Validation and evaluation of new assessments for the otolaryngology undergraduate medical clerkship.
R. Woods (2015)
10.3109/0142159X.2010.509420
Students’ reflections in a portfolio pilot: Highlighting professional issues
A. Haffling (2010)
10.1007/s10459-010-9235-6
In-training assessment using direct observation of single-patient encounters: a literature review
E. A. Pelgrim (2011)
10.1186/s12909-020-02239-6
How to assess? Perceptions and preferences of undergraduate medical students concerning traditional assessment methods
A. Holzinger (2020)
Contextualising simulation : the use of patient-focused hybrid simulation for clinical skills education
J. K. Tun (2015)
Objective Structured Clinical Examination as an instrument for evaluation of clinical competence in pediatrics. A pilot study
Tamara Gamboa-Salcedo (2011)
10.1080/01421590050175587
Clerkship assessment assessed
C.P.M. Van Der Vleuten, A.J.J.A. Scherpbier, D.H.J.M. Dol (2000)
Role of Multiple Choice Questions in Assessment of Competency of Knowledge in Anatomy
Itedal Abdelraheem Mohamed Ahmed (2020)
10.1017/CBO9781139010498.002
Workplace-based assessments in psychiatry: setting the scene
Amit Malik (2011)
See more
Semantic Scholar Logo Some data provided by SemanticScholar