Reliability and Validity

• Reliability in terms of diagnosis?

  This is the consistency of the diagnosis: 

  1. Test-retest reliability can be tested if we ask the same clinician with same patient to see if we get same results
  2. Inter-rater reliability can be tested by asking two different clinicians with the same patient to give the same diagnosis. 
• Criticism study of Cheniaux for inter-rater reliability?

  • A study from Cheniaux investigated inter-rater reliability with two clinicians diagnosing 100 patients, who used different diagnosis methods (DSM-V and ICD-11). (1=18 more, 2=11 more)
  • This shows poor inter-rater reliability between clinicians.
  • Cheniaux shows that patients are more likely to be diagnosed using ICD than the DSM.
  • Therefore, this shows poor reliability between DSM and ICD.
  • There is a weakness of diagnosing depression as we do not have reliable measures.
• Criticism study from Copeland for poor reliability?

  • Another study that shows poor reliability comes from Copeland et al (1971) who gave a description of a patient to 134 US and 194 British psychologists.
  • 69% of US psychologists compared to 2% of UK who diagnosed the person with schizophrenia.
  • This shows poor inter-rater reliability across cultures because the same symptoms can be presented but with a different diagnosis.
  • This suggests an issue with cross-cultural validity.
• What is system overlap? (validity)

  • Symptom overlap is a problem as mental disorders may have similar symptoms with schizophrenia.
  • Bipolar can include delusions, social withdrawal can also occur in depression and bipolar.
  • Some would therefore argue that schizophrenia and bipolar are one condition rather than separate ones.
  • Therefore, this casts doubt on the validity of the diagnosis. 
• Negative evaluation for symptom overlap?

  • Ketter (2005) found that it can cause misdiagnosis, which can lead to years of delay in receiving relevant treatment where suffering and worsening of symptoms can occur.
  • High levels of suicide can occur.
  • Therefore, symptom overlap can have serious consequences, focusing on fixing this issue would save money and lives
• What is co-morbidity? (Validity)

  • Comorbidity is used to explain two or medical conditions occurring together.
  • Morbidity is the likelihood of a medical condition happening.
  • Usually if conditions are co morbid, there may be issues with the validity of the diagnosis.
  • The conditions may be too similar to be considered separate conditions.
  • If 50% of schizo’s have depression as well, it is better to see them as having one condition.
  • This is a weakness of classifying and diagnosing schizophrenia.
• Criticism study for co-morbidity?

  • Buckley et al (2009) found comorbidity as a massive issue.
  • For example, they found that 50% of patients diagnosed with schizophrenia are diagnosed with depression.
  • 47% were diagnosed with substance abuse and OCD in 23% of cases.
  • It shows that schizophrenia is common
• What is gender bias (validity)?

  • Males have been diagnosed with schizophrenia more than females.
  • Males having a genetic vulnerability to schizophrenia or due to gender bias in diagnosis.
  • One study done by  Cotten et al (2009) compared males and female schizophrenics with similar symptoms, they found females functioned better than males in terms of relationships in work and family.
  • This shows males and females receive different diagnoses due to females having better functioning.
  • Therefore, the classification of schizophrenia is not valid. Females may not receive appropriate help 
• Supporting study for gender bias?

  • One study shows that gender bias in diagnosis was Loring and Powell (1988) who randomly chose 209 male and female psychiatrists to read descriptions of a patient’s behaviour and diagnose this patient.
  • When told it was a male or no info, there was a 65% diagnosis rate.
  • When described as female, this dropped to 20%. This was not the case in female clinicians.
  • A problem with the validity is that it is affected by gender of patient and children
• What is cultural bias? (validity)

  • There is a huge variation between cultures in diagnosing schizophrenia.
  • Copeland et al (1971) already shows this in US and British psychiatrists.
  • It questions the reliability of the diagnosis as it suggests that patients can display the same symptoms but receive different diagnosis due to ethnic background. 
• Criticism of cultural bias?

  • African-Americans and English people of Afro-Caribbean origin have more chances of diagnosis of schizophrenia than white people.
  • Desirable to communicate with ancestors in African cultures.
  • When these experiences are reported in the west, they are regarded as irrational and unacceptable, which is supported by voices in non-western cultures being seen as positive, playful or entertaining.
  • Therefore psychiatrists may be unknowingly ethnocentric and culturally biased and overinterpreting symptoms leads to over-diagnosing. This diagnosis lacks validity
• What is a general key study for validity?

  
  

  Rosenham (1957) 
• What was the aim of Rosenham?

  To investigate how situational factors affect diagnosis of schizophrenia
• What was the procedure of Rosenham (1957)

  • 8 pseudo-patients phoned different psychiatric hospitals for an appointment.
  • They had been hearing voices saying ‘empty’, ‘hollow’ and ‘thud’.
  • They all gave fake names, jobs and symptoms but the life history they gave was real.
  • Once admitted, they stopped showing symptoms and acted normally.
  • They were given medication, responded to instructions and also chatted to other patients.
  • This was a participant observation of life in hospital, making notes about interactions between staff and patient.
• Findings of Rosenham?

  • Staff diagnosed 11 pseudo patients with schizophrenia and one with manic-depression, where they went to more than one hospital. Staff never detected their sanity.
  • Although their writing was noticed by staff, it was seen as a symptom of schizophrenia due to bias or interpreting behaviour based on diagnosis.
  • They were not told what they were writing about. 35 ‘real’ patients detected their sanity saying “you’re not crazy”.
  • They spent 7-52 days in hospital until diagnosed as having schizophrenia in remission.
• Conclusion of Rosenham?

  • Staff cannot distinguish sanity from insanity.
  • Any method that makes such errors cannot be reliable or valid.
  • Rosenham’s research showed that psychiatrists cannot tell the difference, calling the reliability of the schizophrenic diagnosis.
  • Normal behaviour was seen as abnormal to support the idea that the participants had a mental illness.
  • This suggests low validity of diagnosis and the DSM being flawed.