Measuring Cognition
Columbia Center for Active Life of Minority Elders (CALME), Columbia University
This set of references focuses on methodological issues in assessing cognition among older, ethnically diverse populations. Seminal and more current work related to cognitive screening measures are abstracted. Included are articles applying traditional psychometric approaches to examine cognitive measures across demographic subgroups. Articles using modern psychometric theory to examine measurement properties, including metric equivalence and DIF across demographic subgroups are also represented. The original publication abstract is included for most of the references.
Ramirez M, Ford ME, Stewart AL, Teresi JA. Measurement issues in health disparities research. Health Services Reseach, 2005, Vol. 40, No. 5, 1640-1657.
BACKGROUND: Racial and ethnic disparities in health and health care have been documented; the elimination of such disparities is currently part of a national agenda. In order to meet this national objective, it is necessary that measures identify accurately the true prevalence of the construct of interest across diverse groups. Measurement error might lead to biased results, e.g., estimates of prevalence, magnitude of risks, and differences in mean scores. Addressing measurement issues in the assessment of health status may contribute to a better understanding of health issues in cross-cultural research.
OBJECTIVE: To provide a brief overview of issues regarding measurement in diverse populations.
FINDINGS: Approaches used to assess the magnitude and nature of bias in measures when applied to diverse groups include qualitative analyses, classic psychometric studies, as well as more modern psychometric methods. These approaches should be applied sequentially, and/or iteratively during the development of measures.
CONCLUSIONS: Investigators performing comparative studies face the challenge of addressing measurement equivalence, crucial for obtaining accurate results in cross-cultural comparisons.
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pub
med&dopt=Abstract&list_uids=16179000&query_hl=19&itool=pubmed
_docsum
Lai JS, Teresi J, Gershon R. Procedures for the analysis of differential item functioning (DIF) for small sample sizes. Evaluation & The Health Professions, Sept. 2005, Vol. 28, No. 3, pp. 283-294.
An item with differential item functioning (DIF) displays different statistical properties, conditional on a matching variable. The presence of DIF in measures can invalidate the conclusions of medical outcome studies. Numerous approaches have been developed to examine DIF in many areas, including education and health-related quality of life. There is little consensus in the research community regarding selection of one best method, and most methods require large sample sizes. This article describes some approaches to examine DIF with small samples (e.g., less than 200).
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pub
med&dopt=Abstract&list_uids=16123258&query_hl=21&itool=pubmed
_docsum
Stump TE, Monahan P, McHorney CA. Differential Item Functioning in the Short Portable Mental Status Questionnaire. Research on Aging. May 2005, Vol.27, Iss. 3; pp. 355-384.
This secondary data analysis investigated differential item functioning (DIF) in the Short Portable Mental Status Questionnaire (SPMSQ) across demographic subgroups. The study was conducted at an academic primary care group practice on 3,954 patients aged 60 years and older who completed the SPMSQ during routine office visits. After adjusting for overall cognitive ability, women were more likely than men to respond correctly to name-of-this-place and mother's-maiden-name items. African Americans were more likely than Whites to correctly give their correct telephone numbers. Those with 0 to 8 years of education were less likely to name the current president and correctly answer the serial-threes item than those with 12 or more years of education. Those aged 80 or older were less likely to correctly identify the day of the week than those aged 60 to 69. Future studies seeking to develop new cognitive screening measures should perform DIF analyses in the instrument development phase to eliminate DIF items a priori.
Mungas D, Reed BR, Crane PK, Haan MN, Gonzalez H. Spanish and English neuropsychological assessment scales (SENAS): Further development and psychometric characteristics. Psychological Assessment, Dec 2004, Vol.16, Iss. 4, pg. 347.
The Spanish and English Neuropsychological Assessment Scales were devised to be a broad set of psychometrically matched measures with equivalent Spanish and English versions. Study 1 in this report used item response theory methods to refine scales. Results strongly supported psychometric matching across English and Spanish versions and, for most scales, within English and Spanish versions. Study 2 supported in both English and Spanish subsamples the 6-domain model of ability that guided scale construction. Study 3 examined differential item functioning (DIF) of one scale (Object Naming) in relation to education, ethnicity, gender, and age. Effects of DIF on scale-level ability scores were limited. Results demonstrate an empirically guided psychometric approach to test construction for multiethnic and multilingual test applications.
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pub
pubmed&dopt=Abstract&list_uids=15584794&query_hl=27&itool=pubmed
_docsum
Crane PK, van Belle G, Larson EB. Test bias in a cognitive test: differential item functioning in the CASI. Statistics in Medicine, Vol. 23, 2004, 241-256.
An ordinal logistic regression modeling technique was used to assess DIF in the Cognitive Assessment Screening Instrument (CASI). Ethnicity, gender, years of education and age were examined for potential DIF with respect to cognitive ability scores. DIF was found in a considerable number of items for at least one of the demographic variables examined. The authors discuss suggestions on what to do when items show DIF in cognitive measures. Advantages of techniques for detecting DIF used in this article were discussed in the context of techniques used by other scholars.
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pub
pubmed&dopt=Abstract&list_uids=14716726&query_hl=30&itool=pubmed
_docsum
Greenland, S., & Brumback, B. (2002). An overview of relations among causal modeling methods. International Journal of Epidemiology ,31(5), 1030-1037.
This article describes four types of causal models for health-sciences research: structural-equations models, graphical models based on the work of Jueda Pearl, potential-outcome (counterfactual) models (Rubin’s causal model), and the sufficient-component cause model. Causal analysis is, in some sense, a field of study not tied to a particular analytic technique.
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=
Retrieve&db =pub med &dopt=Abstract&list_uids=12435780
&query_hl=7&itool =pubmed _docsum
Hernan, M. A., Hernandez-Diaz, S., et al. (2002). Causal knowledge as a prerequisite for confounding evaluation: An application to birth defects epidemiology. American Journal of Epidemiology, 155(2), 176-184.
The authors use examples from epidemiology to show how blind statistical adjustment can go wrong. A priori knowledge and causal models are essential to determine the appropriate covariance adjustment model.
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed
&dopt=Abstract&list_uids=11790682&query_hl=9&itool=pubmed_docsum
Pearl, J. (2000). Causality: Models, reasoning, and inference. New York, NY, Cambridge University Press.
Pearl’s work represents the most comprehensive new development in thinking about causal modeling. It is somewhat difficult because so much of it is new.
Hayduk, L., Cummings, G., et al. (2003). Pearl's D-separation: One more step into causal thinking. Structural Equation Modeling, 10(2), 289-311.
This is a good introduction to some of Pearl’s key ideas.
Winship, C. & Harding, D. J. (2004). A General Strategy for the Identification of Age, Period, Cohort Models: A Mechanism Based Approach.
http://www.wjh.harvard.edu/~winship/cfa_papers/WinshipHardingAPC.pdf
The authors apply causal modeling ideas to solve a difficult problem in developmental research.
Last updated April 2007
|
