issues with reliabilty and validity

• Inter-rater reliability


  Consistency of diagnosis between different assessors, ensuring that multiple clinicians would arrive at the same diagnosis for a given patient.
• Kappa score


  A statistic measuring inter-rater agreement, with 1 indicating perfect agreement between assessors in diagnosing schizophrenia.⦿In the DSM-V field trials (Regier et al2013), the diagnosis of schizophrenia had akappa scoreof only 0.46.
• what is reliabilty?


  the consistency of a classification system
• Wha is validity?


  the extent that adiagnosis is realand distinct from other disorders and that a classification system (DSM) measures what it claims to measure.
• what did Rosenhan do?


  A study demonstrating the unreliability of diagnosing mental health disorders, including schizophrenia, by showing how individuals without any symptoms were falsely diagnosed as mentally ill.
• What was Rosenhans experiment?


  recruited8people, all reported they could hear words like empty, thud, hollow. all but 1 diagnosed with SZ. stayed in institutions from7 to 52 days .He told institutions about his results and sad for them to expect more---41patients were suspected of being fakes, and 19 had been diagnosed by only 2 members of staff. None had been sent
• Evaluate Rosenhan's study

  highlighted unreliabilty of diagnosis. conducted 30 years ago, since then manuals have been improved and diagnosis has been improved. EG categories and definitions are more detailed and operationalised. Standardised interview schedules too. Also ICD and DSM brought in line with each other.
• Culture; Issues affecting Reliability


  Significant variation in diagnosing schizophrenia between countries, influenced by cultural beliefs, values, and practices.
• What did copeland find? - reliability


  Gave 134 US and 194 British psychiatrists a description of a patient:

  - 69% of the US psychiatrists diagnosed schizophrenia

  -only 2% of British psychiatrists diagnosed schizophrenia
• What did Luhrmann find? - reliability


  Interviewed 60 adults with a diagnosis of schizophrenia and asked about the voices they heard.

  - 20 from Ghana, 20 from India, 20 from U.S

  - Pps from Ghana & India reported positive experiences with their voices (playful, offered advice)

  - U.S. pps reported only negative experiences (hateful, violent)

  Maybe the harsh/violent voices common in the West is not inevitable in Schiz.

  This suggests that schizophrenia has a lack of consistent characteristics
• Evaluate reliability


  Lack of inter-rater reliability-correlations as low as 0.11 for Schizophrenia (Whaley, 2001).Unreliable symptoms-Mojtabi & Nicholson (1995) assessed inter-rater reliability - 50 senior psychiatrists in US were asked to differentiate between ‘bizarre’ and ‘non-bizarre’ and found a correlations of inter-rater reliability of only 0.40 = even this major symptom lacks reliability.Cultural differences-Ethnic Culture Hypothesis suggests that ethnicminoritygroups experienceless distressfrommental disordersdue to the social structures andprotective characteristics that exist in minority culturesBrekke & Barrio, 1997 – studied 184 individuals with Sz. Sample drawn from two non-white minority groups (African-American & Latinos) and a majority group (white US). Non White minority groups members were more symptomatic – supports ECH
• Gender bias- validity in diagnosis and classification


  Diagnosis influenced by the gender of the individual, resulting in differences in the prevalence, symptoms, and treatment of schizophrenia between males and females.
• What study is there for gender bias questioning validity?

  •Boverman et al(1970) In USA tendency forwomen to be perceived as less mentally healthy. ‘Healthy’ adult behaviour was equated with ‘healthy male behaviour’.•Loring and Powell (1988), -290 male and femalepsychiatrists to read two cases on patient’s behaviourAsked to offer judgement using diagnostic criteriaWhen described as ‘males’ = 56%diagnosed with SZWhenfemale = 20%Gender bias wasnot as evident among femalepsychiatrists, Suggesting diagnosis influenced not only by gender of patient by also by gender of clinician
• Symptom overlap


  Similar symptoms found in other disorders, such as depression, making it challenging to differentiate between different mental health conditions based solely on symptoms.
• What is support for symptom overlap?

  •Ellason & Ross(1995) - point out that people with dissociative identity disorder (DID) have more Sz symptoms than people diagnosed as being schizophrenic!•Read (2004)– says that most Sz sufferers have sufficient other symptoms could get one other diagnosis.
• What is the eval of difference in prognosis?

  •The prognosis for schizophrenia varies with about20% recovering their previous level of functioning, 30% showing some improvement with intermittent relapses and 10% achieving significant and lasting improvement.•A diagnosis of schizophrenia haslittle predictivevalidity – some people never appear to recover from the disorder, but many do.•If each person has such different outcomes after treatment,how can we be sure schizophrenia is actually what they have.•What can influence outcome is more to do with social skills, academic achievement and family tolerance of schizophrenic behaviour (Harrrison, 2001).
• Co-morbidity


  The occurrence of two or more conditions at the same time, such as schizophrenia co-occurring with substance abuse or anxiety disorders.

  this calls into question the validity of their diagnosis and classification because they might actually be a single condition.
• What did Swets meta analysis find?


  12%of patients with schizophrenia also fulfilled the diagnostic criteria for OCD and about25%displayed significant obsessive-compulsive symptoms.
• Evaluate the consequences of co-morbidity


  studies usually small sample

  But a US study (Weber et al 2009) looked at nearly 6 million hospital discharge records to calculate co-morbidity rates, They found co-morbidity of other psychiatric disorders with schizophrenia (45%),, They also found evidence of co-morbid non-psychiatric disorders such as asthma, hypertension and type 2 diabetes.
• Stigma


  Social stigmas and repercussions attached to diagnosing someone with schizophrenia, leading to discrimination, prejudice, and negative attitudes towards individuals with the disorder.
• What did Kim and Berrios (2001) find?

  found that in Japan the idea of a 'disorganised mind' is so stigmatised that psychiatrists are reluctant to tell patients of their condition. As a result only 20% of those with schizophrenia are actually aware of it, while the other 80% are left undiagnosed.