Research Methods

• Content analysis


  • A type of observational research in which people are studied indirectly via communications they have produced
  • Instead of observing real people, they make observations through books, TV programmes, etc.
  • Examines the content in detail and identifies patterns
• Coding

  Content analysis requires coding units to categorise analysed material:

  • Words - derogatory words used to describe certain people
  • Characters - number of female sports commentators
  • Time - amount of screen time dedicated to eating disorders
  • Space - amount of space in newspapers dedicated to depression

  Quantifies the qualitative data
• Thematic analysis


  • Content analysis may also involve generating qualitative data
  • A theme refers to any recurring idea that is a part of the communication being studied

  e.g. mentally ill may be perceived as a threat

  • These themes may then be developed into broader categories: control
• How to conduct a content analysis?

  1. Familiarise yourself with the data
  2. Create codes to identify features of the data
  3. Search for themes - count the instances or provide examples
  4. Check the categories by collecting a new set of data and applying them - they should fit the data well if they represent the topic
  5. Write up a report
• Content analysis

  Evaluation

  • Much of the material studied is available to the public - no issues with informed consent
  • Can produce qualitative and quantitative data - flexible and can be adapted to the aims of the research
  • Lack of objectivity, especially when more descriptive codes employed
  • People tend to be studied indirectly, the communication can be out of the context within which it occurred - researcher may attribute opinions to the speaker that were not intended
• Reliability

  A measure of consistency, expecting the same result from the same test
• Improving reliability

  Experiments

  Lab experiments are considered reliable as the researcher can exert strict control over variables

  • Ensure that participants are tested under the exact same conditions each time
• Improving reliability

  Questionnaires

  Less reliable due to different interpretations of questions

  • Replace open-ended questions to reduce ambiguity
• Improving reliability

  Interviews

  Can be reliable if the interview is structured and they use the same interviewer

  • Structured interviews
  • Same interviewer
• Improving reliability

  Observations

  Not as reliable due to subjectivity if behaviours are not operationalised well

  • Ensure behavioural categories are operationalised, measurable, self-evident and non-overlapping
  • More than one observer
• Inter-observer reliability

  • The extent to which there is agreement between 2+ observers
  • Relevant to observations - subjectivity bias
  • Able to improve reliability by conducting observations with more than one observer
  • May involve a pilot study to ensure behavioural categories are applied correctly
  • Data collected by independent observers should be correlated to assess reliability - correlation coefficient above +0.8 = reliable results

  Checking reliability

  1. Ensure behavioural categories are operationalised
  2. Make independent observations at the same time
  3. Correlate data of observers
• Test-retest reliability

  • Administering the same test/questionnaire to the same person on different occasions
  • If reliable, the results will be the same/very similar
  • Must be sufficient time between each test to ensure the participants cannot recall their answers, but not too long so that their opinions/abilities will change
  • Correlate scores

  Checking reliability

  1. Administer the test/questionnaire to the sample
  2. Re-test at a later date, same sample and same test
  3. Correlate the data
  4. Coefficient of +0.8 = reliable
• Validity

  • Refers to whether an experiment produces a legitimate result
  • Measuring what it is supposed to measure
• Internal validity

  • Whether effects observed are due to manipulation of the independent variable and not another factor
  • Affected by extraneous variables: participant, situational and demand characteristics
• External validity

  • Relates to factors outside of the investigation: generalising to other settings
  • Ecological - application to real-life
  • Temporal - application to another time period
  • Population - other cultures or genders
• Improving validity

  Experiments

  Validity: high

  • Use a control group
  • Standardise procedures
  • Single or double-blind procedures
• Improving validity

  Questionnaires

  Validity: medium

  • Lie scale to control social desirability bias
  • Anonymous answers
• Improving validity

  Observations

  Validity: medium

  • Covert observations
  • Operationalised behavioural categories
• Face validity

  • Whether a test appears to measure what it is supposed to measure
  • "Eyeballing" the instrument or asking a professional to check
• Concurrent validity

  • Results are similar to those in another well-established test
  • Close agreement indicates high concurrent validity
  • Indicated if the correlation coefficient between 2 sets of scores exceeds +0.80
• Descriptive statistics

  • Data has been analysed to show patterns
  • Shows main trends of results of some research
  • Central tendency/dispersion
  • Doesn't show significance
• Inferential statistics

  • Finding out if there is a significant difference in results
  • Are the results just due to chance?
• Which statistical test do I use?

  • What the hypothesis predicts - difference or correlation
  • Experimental design
  • Level of data collected
  • The 3 D's: difference, design, data

  A) Sign test

  B) Wilcoxon

  C) Related t

  D) Chi-squared

  E) Mann Whitney

  F) Unrelated t

  G) Spearman's Rho

  H) Pearson's product

  
• Which statistical test do I use?

  • What the hypothesis predicts - difference or correlation
  • Experimental design
  • Level of data collected
  • The 3 D's: difference, design, data

  A) Sign test

  B) Wilcoxon

  C) Related t

  D) Chi-squared

  E) Mann Whitney

  F) Unrelated t

  G) Spearman's Rho

  H) Pearson's product

  
• Difference vs correlation

  • Difference - difference between number of items remembered whilst learning in a cold environment vs a warm environment
  • Correlation - relationship between IQ score and number of questions answered correctly in a general knowledge quiz
• Related vs unrelated

  Ignore if the experiment is looking for a correlation

  • Related - repeated measures, matched pairs
  • Unrelated - independent groups
• Nominal data

  • Data is in named categories e.g. tall or short
  • Lowest level of data
  • Limited information about the variables being measured
• Ordinal data

  • Data is ordered e.g. ranking tallest to shortest
  • More information than nominal, but still vague
  • Hides the actual value
• Interval data

  • Data measured with a scale e.g. measuring height
  • Highest level
  • Best information
• Critical values

  • One or two-tailed test?
  • Number of participants
  • Level of significance
• Type I errors

  • Alternative hypothesis is accepted when results are due to chance and null hypothesis is actually true
  • False positive
• Type II errors

  • Null hypothesis is accepted when the results are due to the independent variable and the alternative hypothesis is actually true
  • False negative
• Features of a science

  1. Constructing theories and testing hypotheses
  2. Falsifiability
  3. Replicability
  4. Objectivity and the empirical method
  5. Paradigms and the paradigm shift
• Constructing theories and testing hypotheses

  • Scientists use inductive and deductive methods of investigation
  • Sometimes theory is constructed before hypothesis and vice versa
  • A good theory generates testable hypotheses
  • If the hypothesis is not supported by research, the theory must be altered

  Psychology:

  • Testing hypotheses is a major part of research
  • Some aspects of psychology have theories, but hypotheses are difficult to generate
  • e.g. Freud's theory of personality
• Falsifiability

  • Falsifiable: biological and behaviourist
  • Unfalsifiable: humanistic, psychodynamic
• Replicability

  • Replicable - experiments, observations, self-reports, correlation
  • Difficult to replicate - observations, self-reports, case studies
• Objectivity and the empirical method

  • Empirical - means "experience", people must have direct experience and evidence to support something e.g. experiments and observations
  • Objectivity - personal opinions and desires must not affect the results

  Increase objectivity:

  • Controlled methods
  • Double blind
  • Operationalised variables
  • Standardised procedures
  • Peer review

  However, subjectivity is needed to study a personal experience

  Creates a division between approaches

  • Subjective: Freud, humanistic
  • Objective: biological
• Paradigm

  • A shared set of assumptions, methods and terminology about what should be studied and how
  • Many assumptions are different in psychology - some approaches believe behaviour is caused by one thing (e.g. biology) and others will think different (e.g. psychodynamic)
  • Could be due to paradigm shifts - assumptions, methods and terminology have changed over time
  • Different approaches have become more prominent
  • Could be argued psychology is a pre-science, yet to find its paradigm
  • However, in modern psychology, most subscribe to scientific methods and cognitive neuroscience
• Section of report

  Abstract

  • Aims, hypotheses, method, results and conclusions
  • Key points
  • 150-200 words
  • Help readers get an overview
• Section of report

  Introduction

  • Literature review to establish gaps in existing research
  • Results in research aim and hypothesis
  • Gradually becoming more specific