the scientific process + peer review

• the Cyril Burt affair: in the early 1950s the eminent British psychologist Sir Cyril Burt published results from studies of twins that was used to show that intelligence is inherited
• the Cyril Burt affair: Burt (1995) started w/ 21 pairs of twins raised apart later increasing this to 42 pairs of twins
• the Cyril Burt affair: in a subsequent study Burt (1996) increased his sample to 53 pairs of identical twins raised apart, reporting an identical correlation to the earlier twin study of .771
• the Cyril Burt affair: the suspicious consistency of these correlation coefficients led Leon Kamin (1977) to accuse Burt of investigating data, when a Sunday Times reporter, Ollie Gillie tried and failed to find the 2 of Burt's research assistants (who didn't actually exist) this appeared to confirm the underlying fraud and Burt was publicly discredited
• the Cyril Burt affair: these accusations have been challenged but the most recent view is that Burt was astonishingly dishonest in his research (Mackintosh, 1995), the Burt affair is particularly worrying because his research was used to shape social policy
• the Cyril Burt affair: Burt helped to establish the 11-plus examination used in the UK to identify which children should go to grammar school rather than secondary moderns
• the Cyril Burt affair: he argued that since IQ was largely inherited it was appropriate to test and segregate children into schools suitable for their abilities
• some more recent cases of fraud: in 2010 Professor Marc Hauser of Harvard University was found responsible for scientific misconduct related to a number of published scientific papers
• some more recent cases of fraud: his main area of research concerned cotton-top tamarin monkeys and their cognitive abilities, he appears to have drawn conclusions for which he has been unable to provide evidence
• some more recent cases of fraud: in the light of such prominent cases of professional misconduct Leslie John and colleagues (2012) surveyed over 2,000 psychologists asking them to anonymously report their involvement in questionable research practices
• some more recent cases of fraud: they found that 70% said they cut corners in reporting data and 1% admitted to falsifying data, they concluded that questionable practices may constitute the prevailing research norm
• aftermath: such practices raise 2 key issues, 1st there is the issue of lack of trust, in the future people are likely to be less trusting of scientific data, 2nd is the problem that the data from such fraudulent studies remain published
• aftermath: despite the fact the journals involved usually publish retractions (stating that the evidence is flawed and fraudulent) there are people who will continue to use the faulty data not knowing that it is discredited
• the scientific process: for the scientific method is important that you understand that this is not the same as 'doing an experiment'
• the scientific process: first of all it may involve observation or self-report not just experiment but more importantly science is a process which enables humankind to get closer and closer to understanding how the world and the people in it function
• the scientific process: many elements of this process have evolved over the centuries to ensure that we uncover facts that can be relied on to build bridges, treat disease, raise psychologically healthy children etc, one part of this process is peer review
• peer review: peer review (also called 'refereeing') is the assessment of scientific work by others who are experts in the same field (i.e. 'peers'), the intention of peer reviewing is to ensure that any research conducted and published is of high quality
• peer review: usually there are a number of reviewers for each application/article/assessment, their task is to report on the quality of the research and then their views are considered by a peer review panel
• peer review: The Parliamentary Office of Science and Technology (2002) suggests that peer review serves 3 main purposes of allocation of research funding, publication of research in academic journals and books and assessing the research rating of university departments
• allocation of research funding: research is paid for by various government and charitable bodies, the overall budget for science research in the year 2015-16 was set at £5.8 billion (Gov.uk, 2014)
• allocation of research funding: the organisations spending this money obviously have a duty to spend it responsibly, therefore public bodies such as the Medical Research Council require reviews to enable them to decide which research is likely to be worthwhile
• publication of research in academic journals + books: scientific or academic journals provide scientists w/ the opportunity to share the results of their research
• publication of research in academic journals + books: the peer review process has only been used in such journals since the middle of the 20th century as a means of preventing incorrect or faulty data entering the public domain
• publication of research in academic journals + books: prior to the idea of peer review, research was simply published and it was assumed that the burden of proof lay w/ opponents of any new ideas
• assessing the research rating of university departments: all university science departments are expected to conduct research and this is assessed in terms of quality (Research Excellence Framework, REF), future funding for the department depends on receiving good ratings from the REF peer review
• peer review + the internet: the sheer volume and pace of information available on the Internet means that new solutions are needed in order to maintain the quality of information
• peer review + the internet: scientific information is available in numerous online blogs, online journals and of course Wikipedia
• peer review + the internet: to a large extent such sources of information are policed by the 'wisdom of crowds' approach - readers decide whether it is valid or not and post comments and/or edit entries accordingly
• peer review + the internet: several online journals (such as ArXiv and Philica) ask readers to rate articles, on Philica papers are ranked on the basis of peer reviews and the peer reviews can be read by anyone
• peer review + the internet: on the internet however 'peer' is coming to mean 'everyone' - a more egalitarian system but possible at a cost of quality