Intro to Psychometrics and Psych Measurement

• A psychological test - a systematic procedure for comparing the behavior of two or more people” - Cronbach
• A psychological test is a standardized and objective measure of an individual's behavior, thoughts, emotions, or cognitive abilities. These tests are designed to assess various aspects of psychological functioning and are administered in a controlled and consistent manner to ensure reliability and validity.
• A psychological test is a sample of behavior
• Features of a test

  1. A test is sometimes limited to paper-and-pencil.
  2. Some tests produce numbers that can be conceptualized as values representing the amount of some psychological attribute possessed by a person.
  3. Other tests produce categorical data.
  4. Tests must be capable of comparing the behavior of different people (interindividual differences) or the behavior of the same individuals at different points in time (intraindividual differences).
• There are tens of thousands of psychological tests in the public domain.
• Tests can vary in content—there are achievement tests, aptitude tests, intelligence tests, personality tests, attitude surveys.
• Tests also vary with regard to the type of response required—there are open-ended tests and closed-ended tests.
• Tests also vary according to methods used to administer the tests—there are individual tests and group tests.
• Criterion-referenced tests are most often seen in settings in which a decision must made about a person’s skill level. A fixed predetermined cutoff test score is established.
• Norm-referenced tests are usually used to compare a person’s test score with scores from a reference sample.
• Examples of Criterion-referenced tests:

  • Foreign Language Proficiency Tests
  • Driver's License Test
• Examples of Norm-referenced tests:

  • Intelligence / IQ tests
  • Newborn Screening
  • Height and Weight
  • Physical Ability
• Speeded tests are time-limited tests. In general, people who take a speeded test are not expected to complete the entire test in the allotted time. It is assumed that there is a high probability that each question will be answered correctly, and each of the questions on a this test should be of comparable difficulty.
• Power tests are not time limited, and that examinees are expected to answer all test questions. Test items must range in difficulty if scores on these tests are to be used to discriminate among people with regard to the psychological attribute of interest.
• Bundled tests are instruments intended to be administered together but are not necessarily designed to measure a single psychological attribute.
• Tests – instruments that measures a sample of behavior.
• Testing - the act of giving psychological tests to people.
• Psychometrics is the science concerned with evaluating the attributes of psychological tests.
• Three of these attributes will be of particular interest to us:

  1. The type of data (in most cases, scores) generated by the application of psychological tests, psychological tests.
  2. The reliability of data from psychological tests, and
  3. Issues concerning the validity of data obtained from psychological tests.
• Francis Galton , the “Father of Psychometrics”
• Francis Galton (1822 to 1911) seems to have been obsessed with measurement. He was a British polymath, a half-cousin of Charles Darwin, and made significant contributions to various disciplines, including psychology, anthropology, and statistics.
• Francis Galton was a meteorologist and a geneticist.
• Francis Galton was interested in “anthropometrics,” the measurement of human features such as head size, arm length, and physical strength.
• Francis Galton referred to the measurement of mental features as “psychometry,” which he defined as “the art of imposing measurement and number upon operations of the mind.
• Francis Galton demonstrated the utility of using the normal distribution to model many human characteristics, he developed the idea of the correlation coefficient.
• Galton also tried to measure mental abilities using mental tests. His specific efforts in this regard proved unsuccessful, but the idea that a relatively simple, easy-to-administer test of mental abilities could be developed laid the foundation for the modern intelligence test.
• Galton is also the father of eugenics. Eugenics is a controversial field of study that involves improving the genetic quality of the human population through selective breeding and other forms of genetic manipulation. He coined the term "eugenics" in the late 19th century and promoted the idea that desirable traits could be enhanced through selective mating, while undesirable traits could be reduced.
• Galton was primarily interested in the ways in which people differ from each other.
• Challenges to Measurement in Psychology:

  1. Complexity of Psychological Phenomena
  2. Participant Reactivity
  3. Biases
  4. Use of composite scores
  5. Score Sensitivity
  6. Lack of awareness of important psychometric information
• Participant reactivity: measuring psychological characteristics of people who are conscious and generally know that they are being measured, the act of measurement can itself influence the psychological state or process.
• Participant reactivity can take many forms. In each case, the validity of the measure is compromised: (1) demand characteristics, (2) social desirability, and (3) malingering.
• Demand characteristics in research and testing refer to cues or cues within an experimental or testing setting that may inadvertently convey information about the expected or desired response, influencing participants to respond in a certain way.
• Social desirability refers to the tendency of individuals to respond to survey questions or assessments in a way that they perceive as socially acceptable or favorable, rather than providing accurate or truthful information.
• Malingering happens when a test taker changes his or her behavior to convey a poor impression to the person doing the measurement.
• Test Bias - The people collecting the behavioral data (observing the behavior, scoring a test, interpreting a verbal response, etc.) can bring biases and expectations to their task.
• Composite scores in testing refer to summary scores that are derived from combining or aggregating individual scores on multiple items or subtests within a test.
• Sensitivity refers to the ability of a measure to discriminate adequately between meaningful amounts or units of the dimension that is being measured.
• Score sensitivity in testing refers to the extent to which a test is able to accurately and reliably measure individual differences or changes in the construct it is designed to assess.
• A sensitive test is one that can detect variations in performance or abilities among individuals, making it more effective in distinguishingbetween different levels of the trait or characteristic being measured.
• Psychological measurement is often a social or cultural activity.