The author looks at both strengths and weaknesses in this body of work, highlighting pitfalls and paradoxes in research procedures and providing valid design models.
This includes the contribution ofdedicated meta-analyses to this controversial field and a discussion of the benefits and limitations associated with this type of research. Substantial data, drawn from three extensive studies (Felix, 2005a, b; Felix, 2006a), allows the author to present for the first time synthesized findings relating to the impact of technologies on language learning. The paper concludes with strategies for future work in the context of a proposed research agenda.
In the following we will:
• define what is meant by effectiveness, meta-analysis and effect size
• outline the scope ofthe three previous studies on which our discussion is based
• identify strengths and weaknesses ofthe effectiveness research
• single out successful research design models
• highlight valid meta-analysis procedures
• present a synthesis of major findings
• draw conclusions and suggest directions for future research.
2 purpose
Our aim was to provide the widest possible breadth ofinformation on the research included.
Because of the great variation in the many characteristics of each study, this proved tremendously difficult, especially where variables under investigation were concerned. In order to present so much information in an intelligible format we sometimes had to group several variables under an umbrella category.
MAIN point
We included the latter to see whether these qualitative syntheses contributed support and/or further information to the quantitative results of the former. The L1 studies were included because they investigated much the same variables (i.e. writing quality, number of words) as the L2 studies and their findings were relevant to CALL.
Since the purpose of this study was to take a detailed look at strengths and weaknesses of research designs, our search had to be limited in scope. learning processes or outcomes in the widest sense. Because our goal was to give a comprehensive overview of types of research conducted, we did not cull these studies to include only those with sound research designs.
3 strength
This is indeed a strength of the research since both categories include a large number of excellent studies and their combined results advance the field more substantially than a continuous series of highly controlled studies. This is especially the case since the latter are predominantly very short-term (as brief as a 40 minute treatment) and most often investigate a single variable.This, too,lends strength to the body of research since it provides important opportunities for investigating potentially differential impacts of ICT on learning in elementary,secondary and tertiary settings.
Weakness
Titles can often be very misleading when they include the keyword effectiveness. Poor description ofthe research design
Failure to investigate previous research
5 Findings
Revisiting the question posed in the title of our paper, we have to conclude that the surprisingly scarce meta-research specifically related to CALL tells us very little about actual or potential effectiveness of the use of ICT in second language learning.However, the inclusion of narrative reviews and studies carried