Developing a CLIL Textbook Evaluation Checklist

In the last decades, the number of bilingual schools in Europe has increased significantly. Most of these schools implement CLIL methodology in their teaching and, consequently, there is a demand for new and appropriate materials for each content subject taught through a foreign language: new textbooks written in L2 are needed, and the existent offer does not meet the current demand. Given that CLIL materials development is a relatively recent field, there is not much research on the evaluation of those still insufficient course books. In this paper, and in order to evaluate CLIL textbooks, we propose a tentative checklist twofold based: on the one hand, based on previous checklists created to evaluate ELT textbooks CLIL materials use principles/techniques of foreign language teaching (Steirt and Masser, 2011)-, and, on the other hand, based on the criteria for producing quality CLIL learning materials drawn up by Mehisto (2012). Further research is in process in order to refine the checklist with the results of an analysis of the criteria carried out by a focus group of CLIL teachers, and of a survey addressed to teachers who are currently implementing CLIL in their schools.


INTRODUCTION
Two decades after its birth, CLIL is currently an established teaching approach across Europe. Most bilingual and plurilingual schools in the European context have adopted CLIL, and its popularity keeps growing. Teachers and researchers agree on the advantages this approach provides for the dual focus it is intended: learning language and content at the same time.
The first two decades of implementation of CLIL show interesting benefits from different perspectives: students show more interest and motivation than in conventional subject content classes (Dalton-Puffer & Smit, 2007); language is learnt in more natural contexts than those of L2 learning, leading to higher levels in communicative competence in L2 (Lasagabaster, 2008); and thinking skills are fostered, since students make more cognitive effort in order to understand the content provided through L2 (Van de Craen, Mondt, Allain, & Gao, 2007). The spread of CLIL approach keeps increasing and, as a consequence, specific pedagogic material is needed at all educational levels. On this respect, researchers, teachers and stakeholders frequently express the shortage of CLIL materials (Moore and Lorenzo, 2007;Alonso, Grisaleña & Campo, 2008;Coyle, Hood & Marsh, 2010, among others). This situation makes teachers create materials adapted to their contexts, or adjust the few existing to their needs, which leads to an increase of their workload. Since the textbook is the key material that numerous teachers rely on, in this paper we will focus on providing a tentative checklist that facilitates their evaluation.

LITERATURE REVIEW
The use of textbooks Textbooks have often played an important role in the compulsory levels of education. Students spend much of their time exposed to them, both in class and working at home. Throughout decades, researchers have analyzed the significant role textbooks play in any generation of students' training; Castell, Luke, & Luke (1989, vii) claim: "the school textbook holds a unique and significant social function: to represent to each generation of students an officially sanctioned, authorised version of human knowledge and culture" (original emphasis). Some experts, however, have also questioned to what extent textbooks constitute a help or a hindrance in the process of teaching and learning. The debate is still present, though nowadays attention is turning to the use of digital or printed textbooks in the classrooms.
No matter the device, teachers and learners can obtain benefits from the use of textbooks: they provide organization in the units or modules, giving an appropriate sequence of content; they cover the curriculum, and have been designed by experts who take into consideration the latest research and the teaching strategies required for each area. Students and parents alike usually favor to have the information organized and to be able to keep an easy control over the content that has already been covered in class.
In regards to well-designed language teaching textbooks, foreign and second language teachers -mainly in the pre-Internet era-, found key resources for their teaching practice in textbooks: "the extent of language activities worldwide could hardly be sustained without the help of the present generation of textbooks" (Richards, 1998, p. 127). Regarding the advantages they offer, Hall (2011) summarizes: They provide language input and exposure for learners; they can provide interesting and motivating material, organized in an appealing and logical manner; and they provide a written record of what has been studied, allowing for revision and continued study beyond the classroom. (p.213) As for the teachers, textbooks provide them with guidance and support and reduce the amount of time they require for preparation. Textbooks also help students to be more autonomous in their learning, since, as Ur (2015) points out, "a student without a coursebook is more teacher-dependent" (p. 198). Finally, published materials have more credibility for students than teacher-generated ones (Sheldon, 1988).
However, some counter-arguments warn about the dangers of the dependency on textbooks. Criticisms usually point out that textbooks can-not cater for individual needs, and may not be relevant or culturally appropriate for a group or class. Meddings and Thornbury (2009) go beyond these observations and prevent against the potential material-instead of person-centred classes.
The textbook, however, is a tool that becomes significant only when the teacher brings it to life. In order to do that, and to make the most of the textbook chosen, we consider the importance of being trained to identify quality textbooks, since the choice can determine the quantity and the kind of additional resources needed for the teaching practice. Training involves getting to know the principles of textbook evaluation. This is why we consider that textbook evaluation should constitute a significant part of the teacher training in CLIL, training that is regarded as a priority by numerous experts (Baker, 2006;Pistorio, 2009;Hillyard, 2011, among others).

CLIL and textbooks
Current research and teaching practice show the need of specific CLIL resources in bilingual / plurilingual centres: "Among the most difficult issues regarding CLIL-style teaching is how to obtain teaching resources, including textbooks" (Zaparucha, 2009, p. 94). This situation is common across Europe and it is significantly relevant in Spain, where the 17 autonomous regions have different curriculums in their subjects, including the most common subjects taught through L2: social and natural sciences. The frequent changes in educational policies make this situation extremely complex for publishers and teachers alike.
Publishers try to keep up with the bloom of bilingual centers requiring learning materials for more subjects every year. Focusing on the Spanish bilingual educational context, some textbooks are specifically created for each region, and the implementation of the bilingual system in some levels or in some subjects often takes place without a previous development of materials needed. Since the process of textbook making takes time, and publishers cannot keep up to edit and plan what is needed in so many areas, some teachers do not have any textbooks in L2 to rely on in the very first year(s) of implementation of their subject. Being the first year(s) of teaching through L2 the hardest ones, mainly for content teachers used to teaching in their L1, the use of a textbook plays a key role as a resource in these situations (see Richards, 1998). British or American textbooks could provide some help for content subjects taught through English. However, they are not a suitable resource since the curriculum of the subjects in all educational levels differs from the Spanish one and also because those textbooks are addressed to native speakers.
When analyzing CLIL materials, scarcity of CLIL books is the most common issue that stands out, though there are other aspects to be considered: Meyer (2010) focuses on the lack of quality of course books; as for scaffolding, a key strategy in CLIL, Kelly (2010) points out the lack of language support students need in content subjects, which can be provided as supplementary resources of the textbooks in the form of posters, booklets, or in the textbook alongside the task. More specifically, he observes: "there is a drastic need for such materials in textbooks in CLIL contexts. It should also go further than the provision of general academic language and include language functions which are specific to curriculum subjects" (p. 102). The relevance of cultural competence (one of the four Cs of the CLIL approach) is also noticed: "Studies comparing various CLIL-textbooks have shown that the cultural dimension has not been properly exploited yet" (Meyer, 2010, p. 20). On this respect, experts also observe the lack of connection between course books and learners' lives (Ballman, 1997); this lack of connection hinders significant learning.
Despite this situation, the role of textbooks is more important than ever. Unexperienced teachers are the ones who rely more on textbooks, and nowadays, numerous young bilingual teachers are taking up most CLIL vacancies, given the increasing number of plurilingual schools in the European context. At the same time, there is a connection between teaching tied to textbook and shortage of teachers (Williams, 1983, p. 251). These two factors: lack of experience in teaching through L2 and shortage of CLIL teachers, make the CLIL textbook especially significant in the current educational contexts.

The use of checklists to evaluate textbooks
Since CLIL materials development is a recent area of research and the production of CLIL textbooks is also evolving, evaluation of CLIL textbooks is an area still to be developed. For this paper, we have created a tentative checklist to evaluate CLIL textbooks with the purpose of completing and expanding it through future research, mainly adding CLIL teachers' contributions. Although numerous checklists to evaluate textbooks in ELT have been created in the last years (see Mukundan, Nimehchisalem & Hajimohammadi, 2011, among others), research has not focused yet on the elaboration of a checklist to compile the main features a CLIL textbook should have.
Checklists are considered useful instruments for teachers in order to select, adapt and evaluate textbooks (Sheldon, 1988;Mc Grath, 2006). Teachers can evaluate a textbook using a checklist or impressionistically, i.e. just analyzing its suitability by taking a quick look at the book, which is a frequent procedure among experienced teachers (see Nimehchisalem and Mukundan, 2013). Checklists, however, are more useful when it is novice teachers who evaluate textbooks. Considering the number of novice teachers facing the challenge of teaching content through an L2, and also those who are experienced teachers, but have recently started teaching through an L2, we consider the checklist as a very appropriate instrument for CLIL educational contexts.
A textbook evaluation checklist consists of a set of criteria (items) that are ticked when the presence of the item is confirmed. Among its advantages, Mc Grath (2006) points out that it is not a watertight category; it contrasts system (objectivity) with impression (subjectivity), it is cost effective, since a significant amount of information can be obtained in a short time, and it provides informants with an easy format to give information.
Limitations of checklists have to do with the inclusion of criteria that might be context bound in the moment of creation, and with the need to adapt some of the criteria to tailor the checklist to suit a specific context. Demir and Ertas (2014, p. 247) summarize this fact: "no coursebook evaluation checklist in the literature is complete. Regardless of the number of items it is made up of, any checklist can be modified by adding or deleting items depending on the circumstances of a given instructional setting". We consider that once the main basic criteria are found and fit most contexts, tailoring the checklist by adding some more criteria or ignoring those that are not suitable is an advantage of the tool itself.

METHODOLOGY
In order to create our tentative CLIL textbook evaluation checklist we took into consideration these two areas: 1.
Review of the available instruments for ELT textbooks evaluation.

2.
Established criteria for producing CLIL materials. Previous research on ELT textbooks evaluation was required, first because CLIL materials use principles/techniques of foreign language teaching (Steirt and Masser, 2011), and also because CLIL benefits from the research done in ELT. Regarding textbook evaluation checklists specifically, there is a significant research carried out in the last five decades, as reviewed by Karamoozian and Riazi (2008), and by Mukundan and Ahour (2010). These authors analysed decades of validated checklists, comparing evaluative criteria and number of items per categories, among other features. Their review of instruments in order to identify well-established checklists eventually helped Mukundan, Hajimohammadi and Nimehchisalem (2011) to develop a tentative checklist for ELT textbooks that was provided to a focus group of ELT teachers in order to refine it and to give it more validity. We will replicate this latter design in our study. This tentative checklist (see appendix) will be provided to a focus group of CLIL teachers, former students of the Master in Bilingual Education at Universidad Nebrija (Madrid), who will eliminate, modify and/or add criteria to the checklist -qualitative phase of the study. The new version of the checklist will be sent via online survey to CLIL teachers working at bilingual schools, who will provide feedback on the weight of each criteria. Some of the items might be removed after carrying out a factor analysis of their feedback. The information of the feedback will be included to complete the final version of the checklist. In order to operationalize the list, a five point Likert scale will be added to each criterion. Mukundan and Nimehchisalem (2012) point out that both elements (criteria and weight) are essential to create a complete textbook evaluation checklist. In our case, and for teachers' practical purposes, once the final version of the checklist is created, the section providing information on the weight can be ignored and the checklist can be used just to mark to what extent the criteria are present or not present in the textbook analyzed.

RESULTS
The criteria have been drawn up considering the suggestions provided by Tomlinson, (2003) in order to avoid common mistakes in developing text evaluation checklists. We have first taken into consideration those common features that characterize ELT and CLIL textbooks (availability, and authenticity of language, among others). They are included in the first section of the checklist. The other sections include those characteristics that refer mostly or totally to the content taught through L2. In order to compile these features, we have considered Mehisto's (2012) criteria for producing CLIL learning material. According to him, in the case of CLIL materials, the criteria required to create quality learning materials for any area should be taken into consideration. In addition to those common criteria, Mehisto observes: "quality CLIL materials are highly integrative and multilayered and they help increase the likelihood that both content and language learning will be meaningful" (p. 17). Among others, criteria such as making the learning intentions and process visible to students, fostering learning skills development, or including different types of formative assessment, are included in Mehisto's list.
With regard to the structure, the checklist is made up by seven sections: the first one comprises general and technical items, sections 2 to 5 are devoted to the inclusion of the 4Cs of CLIL -content, cognition, communication and culture-, within CLIL textbooks, section 6 includes statements related to language, and section 7 consists of a final statement to reflect upon the integration of the CLIL principles in the textbook analyzed. We have tried to create a list that covers the key CLIL items without being too detailed or extensive, considering that it needs to be quick to be filled.
The checklist can be used for primary and secondary education, since CLIL principles are common to all educational levels. Given the variability involved in levels, subjects and textbook approach, the checklist, as we have mentioned, might change in the number of criteria considered, but can also be altered in the weight given to each criterion.

CONCLUSIONS
The tentative checklist provided in the appendix has been created considering the theoretical framework of ELT textbook evaluation and on the criteria to develop quality CLIL materials development. In order to complete the process, we need to include CLIL teachers' impressions on the list. Our future lines include two phases: gather information on the suitability of the items in order to refine the list (information provided by a focus group made up of CLIL teachers), and a survey, also provided to CLIL teachers via online, in order to analyse the weight they give to each criteria by carrying out factor analysis. The teachers' contributions will provide the process with validity. Moreover, the final checklist, which does not exist on CLIL textbooks yet, will encompass a list of criteria to be considered when selecting, adapting or evaluating a CLIL textbook. Eventually, CLIL teachers as well as future CLIL teachers will have at their disposal a tool that will help them to acquire a useful and systematic insight of different CLIL textbooks in a short period of time.