Reliability and Validity - Schizophrenia

• Test-retest reliability refers to the consistency of a diagnosis of schizophrenia when the same individual is assessed on two separate occasions.
• Inter-rater reliability refers to the consistency of diagnoses made by different clinicians when using the same diagnostic criteria for schizophrenia.
• Psychologists like Rosenhan (1973) highlighted the problems with diagnosis process and suggested that reliability and validity were essential factors in making sure patients received the treatment required.
• Reliability is the extent to which something can produce consistent results through controlled replication.
• Inter-rater reliability is when two or more psychiatrists agree on the same diagnosis for a patient.
• Test re-test reliability is when a psychiatrist reaches the same diagnosis on more than one occasion with the same patient.
• Reliability in diagnosis is important to ensure patients receive the correct treatment.
• Osorio et al (2019) report excellent reliability in over 180 schizophrenic patients using the criteria of the DSM-5. They found inter-rater reliability to be +0.97 and test re-test +0.92, illustrating high correlations. This tells us that we can be reasonably certain the diagnosis of schizophrenia is consistently applied.
• Construct validity in schizophrenia diagnosis refers to the extent to which the diagnostic criteria accurately measure the underlying construct of schizophrenia; does it measure what it claims to.
• Criterion validity in schizophrenia diagnosis refers to the extent to which we can predict future outcomes or behaviours related to schizophrenia using the criteria in the classification system.
• Concurrent validity in schizophrenia diagnosis refers to the extent to which the diagnostic criteria align with other established measures or diagnoses of schizophrenia.
• Predictive validity in schizophrenia diagnosis refers to the extent to which the diagnostic criteria accurately predict future outcomes or behaviours related to schizophrenia, such as treatment response or long-term prognosis.
• Face validity in schizophrenia diagnosis refers to the extent to which the diagnostic criteria appear to be valid and relevant to the symptoms and characteristics of schizophrenia.
• To receive a valid diagnosis patients must present with symptoms that are true of that disorder. If they do not describe symptoms that are true of the disorder, they cannot receive a formal diagnosis.
• Read et al (2004) found a 37% concordance rate in test re-test reliability for schizophrenia, between two clinicians.
• Söderberg et al (2005) found a concordance rate of 81% using the DSM-IV classification for schizophrenia.
• Jakobsen et al (2005) found a concordance rate of 98% for schizophrenia from a sample of 100 patients in Denmark using the ICD-10 classification system.
• Ward et al (1962) identified three factors which affect the reliability of diagnosis. First are client/ patient factors, second are clinician factors and third are classification factors.
• Client/ patient factors which may affect the reliability of diagnosis could be as a result of the severity of the symptoms or atypical characteristics they present.
• Clinician factors which may affect the reliability of diagnosis could be the result of how well trained or experienced they are, or how well they develop a rapport with their patient.
• Classification factors in the reliability of diagnosis could be related to the criteria used, depending on which classification system is used (DSM or ICD) and how they define the disorders.
• There are two different types of criterion validity; concurrent and predictive.
• Split-half reliability in schizophrenia diagnosis refers to the consistency of diagnoses made by splitting the diagnostic tool into two halves and comparing the results.
• Descriptive validity refers to the symptoms of a disorder being unique and distinct from another disorder.
• Aetiological validity refers to all the people with a disorder appearing to experience it in the same way.
• Rosenhan (1973) found that the diagnosis of schizophrenia lacked validity, and that patients were treated as a reflection of the label they were given, not on their individual symptoms or characteristics.
• Mason et al (1997) followed 99 patients with schizophrenia over 13 years to see if the predictive validity of the main classification systems worked. They found that if they used the symptoms that were present over six months, there was good predictive validity.
• Jablensky (2010) conducted a review and found that for the time being, the clinical concept of schizophrenia is valid and is supported by empirical evidence.
• DSM-5 reflects the shift in the opinion of clinicians that it may not be possible to diagnose schizophrenia as a separate disorder because clients present with such different symptoms.
• Jansson & Parnas (2007), reviewed 92 studies that applied different definitions of schizophrenia to the same patient samples. They found that both ICD-10 and DSM-IV had reasonable reliability but low validity, which suggests schizophrenia may not be a separate condition.
• Reliability and validity are two important factors when designing a test or any kind of diagnostic tool for mental illness.
• The purpose behind designing a manual of mental disorders is to allow psychiatrists and medical professionals to diagnose people who have mental disorders appropriately. This means that they will subsequently receive the correct form of treatment and hopefully manage their symptoms to live a normal life.
• According to Mattison et al (1979) the general inter-rater reliability of DSM-II (which was used in Rosenhan’s study to diagnose the pseudo patients) was about 57%. This does not show a high level of agreement for the professionals using this to diagnose patients.
• A valid diagnosis is one where the symptoms match the label so if someone suffering from hallucinations and thought insertion are diagnosed as schizophrenia then this is valid.
• According to Spitzer & Fleiss (1974) the use of diagnostic manuals and systems means more communication between clinicians, which is likely to increase reliability.
• Goldstein (1988) found high levels of inter-rater reliability when patients were diagnosed in relation to schizophrenia.
• Rosenhan (1973) found that DSM-III was not a valid diagnostic tool as all of the pseudopatients were diagnosed incorrectly with having a mental disorder.