Research Methods Year 2

• What are the key features of a science?

  1. Objectivity and the Empirical Method
  2. Replicability
  3. Falsifiability
  4. Theory construction and hypothesis testing
  5. Paradigms and paradigm shifts
• Objectivity is:

  1. Based on facts rather than opinion
  2. Universal (true for everyone)
  3. Dispassionate
• Subjectivity is based on personal opinion rather than facts, not universal
• The Empirical Method:

  1. Objectivity is the basis of the empirical method
  2. Empirical methods emphasise the importance of data collection based on direct, sensory experience
  3. The experimental method and the observational method are good examples of the empirical method in psychology
  4. A theory cannot claim to be scientific unless it has been empirically tested and verified
• Replicability has an important role in determining the validity of a finding.
• Validity is the extent to which an observed effect is genuine- does it measure what it was supposed to measure
• Replicating findings over different contexts and circumstances allows us to see how much findings can be generalised. It also increases external reliability
• A key criterion of a scientific theory is falsifiability. Genuine scientific theories should hold themselves up for hypothesis testing and the possibility of being proven false. Even when a scientific principle has been successfully and repeatedly tested, it is not necessarily true, it just hasn't been disproven
• An essential component of a theory is that it can be scientifically tested. Theories should suggest several possible hypotheses. A hypothesis can be tested using systematic and objective methods to determine whether it will be supported or refuted.
• The process of deriving new hypotheses from an existing theory is known as deduction.
• A paradigm is a shared set of assumptions and methods, that are accepted by most people in a set intellectual community.
• According to Kuhn, psychology has too many internal disagreements and too many conflicting approaches to qualify as a science, rather a pre-science.
• A paradigm shift is when there is a fundamental change in prevailing viewpoints and practices
• Case studies are in-depth investigations that gather a lot of detail about one single person, group, event or community. The research may continue for an extended period of time, so processes and developments can be studied as they happen
• Typically, data for case studies is gathered from a variety of sources and by using several methods such as: interviews, questionnaires, experiments, secondary data, case histories.
• Triangulation refers to the use of multiple methods of data sources in qualitative research to develop a comprehensive understanding of phenomena (Patton, 1999). Triangulation also has been viewed as a qualitative research strategy to test validity through the convergence of information through different sources
• Strengths of case studies include:

  1. Rich and detailed qualitative data provided
  2. Validity and specifically ecological validity
  3. Avoids practical/ethical issues (as it is otherwise impossible to study such unique topics)
  4. Triangulation provides great validity
  5. Mundane realism is present, unlike in lab studies which are highly artifical
• Weaknesses of case studies:

  1. Subjectivity- little Hanz and Freud, his interpretation of behaviour is nonsensical at best.
  2. Generalisability- it is difficult to generalise data from one person, with little scientific credibility
  3. Issues with data confidentiality and privacy (hard to keep a person's identity anonymous)
  4. Replication- it is difficult (or impossible) to replicate original case studies, as we cannot ethically impose certain conditions on people.
  5. Time consuming- lots of data has been collected
• Qualitative research collects, analyses, and interprets non-numerical data, such as text, video, photographs, or audio recordings. This type of data could be collected using diary accounts or in-depth interviews and analysed using content analysis
• Content analysis is a type of observational research in which people are studied indirectly via the communications they have produced, i.e., conversations, books, magazines, and films. In content analysis, the researcher takes qualitative data and transforms it into quantitative data that can then be analysed, summarised, and described.
• Thematic analysis describes themes or recurrent ideas in the data- the outcome remains qualitative.
• Coding is the initial stage of content analysis: the researcher will use 'coding units' or 'behavioural categories' in their work. This method is similar to conducting an observational study where you have to count instances of a category of behaviour.
• Thematic analysis is a form of content analysis but the outcome remains qualitative. It aims to impose some kind of order on what is very often a large qualitative set. Crucially, this order must emerge from the data, rather than be imposed on the data using preconceived ideas. The main process involves the identification of themes: any recurrent idea. These themes are likely to be more descriptive than the coding units used in general content analysis.
• External reliability

  • Measures consistency from one occasion to another
  • The same result should be found on different days, in different labs, observations or interviews, by different researchers etc
• Internal reliability

  • Measures the extent to which a test or procedure is consistent within itself
  • i.e., questionnaire items or questions in an interview should all be measuring the same thing 
• Assessing Reliability: Test-Retest

  • Administering the same test or questionnaire to the same person on different occasions. If the test is reliable then the results will be the same or very close across times and occasions.

  Mainly used with questionnaires and psychological tests (such as IQ tests) but can also be applied to interviews.
• Assessing Reliability: Split-Half

  • Compares a participant’s performance on two halves of a test or questionnaire – there should be a close correlation between scores on both halves of the test. 
  • Questions in both halves should be of equal quality for good internal reliability.   
  • Odds/Evens or First Half/Last Half
• Assessing Reliability: Intra-Rater Reliability

  • This refers to the consistency of one researcher’s behaviour - how consistent they are at measuring a constant phenomenon
  • A researcher should produce similar test results, or make similar observations or carry out interviews in the same way on more than one occasion.
• Assessing Reliability: Inter-Observer

  • How consistent different individuals are at measuring the same phenomenon
  • At least 2 observers should be watching the same events but making their observations independently and they should compare their observations.
  • This can also be applied to content analysis (Inter-rater reliability) and interviews (inter-interviewer reliability)
• Improving Reliability: Observations

  • Making sure that behavioural categories used during the observations are properly operationalised (measurable and self-evident).
  • Categories should not overlap and all possible behaviours should be covered on the checklist.
• Improving Reliability: Questionnaires

  • A test which scores low on test-retest reliability (under +.80) may need some of the items deselected or rewritten.
  • Questions must not be ambiguous, complex or subjective.
• Improving Reliability: Experiments

  • Experiments are often described as reliable due to the researcher exerting strict control over many aspects of the procedure such as the instructions that participants receive and the conditions within which they are tested.
  • To improve (or keep) good reliability there must be precise replication of a particular method.
• Internal Validity: 

  • The tool measures what it intends to measure

  External Validity:

  • The findings can be generalised beyond the context of the research situation
• Issues with Internal Validity

  • Investigator effects
  • Demand characteristics
  • Confounding variables
  • Extraneous variables 
  • Social desirability
  • Poorly operationalised behavioural categories
• Issues with External Validityv

  • Whether we can generalise the finding to other people (Population validity) 
  • Whether we can generalise the findings to other periods (Temporal  validity)
  • Whether we can generalise to other settings (Ecological validity/Context Validity) 
  • Whether we can generalise to the real world (mundane realism)
• Face Validity

  • Whether a self-report measure looks like it is measuring what the researcher intended to measure – this only requires intuitive measurement.
• Concurrent Validity

  • Comparing the current method of measuring stress with a previously validated one on the same topic. To do this you can give participants a previously validated measure and a new one and check the results
• Type I Error: 

  • The null hypothesis is wrongly rejected and the alternative hypothesis is wrongly accepted. False Positive
  • This is more likely to happen if the significance level is too lenient (e.g. 10% instead of 5%
• Type II Error: 

  • The null hypothesis is wrongly accepted and the alternative hypothesis is wrongly rejected. False Negative.
  • More likely to happen if the significance level is too stringent (low) (E.g. 1% or 0.5%)
  • The 5% level of significance is favoured in psychology as it best balances the risk of making a Type I or Type II error
• How to choose a statistical test

  A) Sign test

  B) Chi-Squared

  C) Wilcoxon

  D) Mann-Whitney

  E) Spearman's Rho

  F) Related t-test

  G) Unrelated t-test

  H) Pearson's r