Research Methods in SLA

• Investigations of effects of individual differences, beliefs, and attitudes in instructed language learning are often captured through surveys, case studies, and use of statistical procedures.
• Particular research methods are often associated with particular theoretical perspectives, and were developed in order to provide answers to research questions within those perspectives.
• Robustness of the coding categories can vary across different research traditions, but may include prolonged engagement, triangulation, and thick description for ethnography or rigorous application of analytical method for CA.
• Establishing the quality of research is not simply a matter of following a checklist; the fact that there are multiple sources of data or multiple raters does not, in itself, serve to validate claims based on those data.
• What warrant do you have for the statements that you make? Qualitative researchers must always be careful not to read more into the data than the data can support; claims that are not firmly grounded are speculation, not analysis.
• Another alternative standard is authenticity which specifies that researchers be honest regarding their value systems, possible biases, and stances and how these might affect what they see and how they interpret what they see.
• ISLA studies sought to compare the effectiveness of different kinds of teaching interventions through the use of matched pretests and posttest.
• Non-interventionist-type research has high ecological validity because it investigates real classrooms and real classroom activities, but the researcher has very little control over what happens in the classroom, and thus, the teacher and students may not engage in the types of activities that the researcher is hoping to investigate.
• Designing the data collection instruments in interventionist quasi-experimental studies is necessary to distill one or two qualities to investigate.
• Identifying participants in interventionist quasi-experimental studies can be done either in the classroom or in a laboratory.
• This early research tended to compare intact classes, randomly assigned to experimental or control groups; teachers were asked to perform a certain intervention, or to refrain from using a certain method over a fixed period of time.
• Data might be elicited through diaries, journals, and interviews, or gathered by employing a stimulus which the participants comment upon.
• Piloting the instruments in interventionist quasi-experimental studies is necessary to make sure that they function in the way that they are intended.
• Interventionist quasi-experimental studies require a considerable amount of planning because the researcher is trying to manipulate certain variables and to control other variables.
• Pretests and posttests are used to determine the effects of a particular instructional activity.
• Conducting the experiment in interventionist quasi-experimental studies involves implementing what one has already decided to do.
• The first thing to do in interventionist quasi-experimental studies is to decide on the topic that is to be investigated.
• Analyzing the data in interventionist quasi-experimental studies involves generating quantitative data.
• A more controlled elicitation, including measures such as multiple choice or open-ended questions, has the advantage of allowing the researcher to gain insight into an aspect of the classroom that is of particular interest and that might not be commented on otherwise by the participant.
• Using an open elicitation instrument has the advantage of allowing the participants to comment on things that are important or relevant to them.
• Descriptive statistics, range, and standard deviation, are used to summarize sets of numerical data in order to conserve time and space.
• Researchers need to exercise great caution when pitching the level of generalization in their research reports.
• If a result is non-significant, this means that we cannot be certain whether it occurred in the particular sample only because of chance.
• If results are statistically significant, we can draw some more general lessons from the study.
• In order for the readers to be able to interpret (and believe) the claims made in a research report, they will have to be convinced that the methodology used to produce the particular findings was appropriate.
• Survey data can be used for a great variety of purposes and each of these might require somewhat different types of summaries and reports of the findings.
• Statistics tables have certain canonical forms, both in content (i.e., what information to include) and format (e.g., usually we do not use vertical lines in them).
• The other side of the coin is, that research in most cases is all about the need to produce generalizable findings.
• Using high-powered parametric procedures may easily tempt scholars to overgeneralize their results and to make grand claims regarding their findings.
• Inferential statistics are the same as descriptive statistics except that the computer also tests whether the results observed in our sample are powerful enough to generalize to the whole population.
• Statistical significance denotes whether a particular result is powerful enough to indicate a more generalizable phenomenon.
• Descriptive statistics are only specific to the given sample and do not allow the drawing of any general conclusions that would go beyond the sample.
• We must provide a concise and yet detailed summary of the main aspects of the survey, including any known limiting factors.
• Overgeneralization is a common issue in research and there are no hard-and-fast rules to determine when generalization becomes overgeneralization.
• Interviews may also incorporate stimulated recall, where a researcher might play a recording involving the interviewee and ask him or her to stop the tape and comment at any point.
• The advantages of questionnaires are that they are easier to administer, allow participants more time to formulate responses, and do not need to be transcribed.
• A transcript can never be objective, but must be viewed as a product of multiple subjective decisions.
• When preparing an interview guide, the first consideration is determining the focus of the interview and the topics to be investigated.
• The interviewer should take care not to dominate the interviewee, but should pose questions to stimulate reflection on the topic and to encourage further talk by providing supportive feedback.
• Transcription involves transforming audio- or video-recorded data into written documents, or transcripts.