J Eng Tech Movie Media > Volume 22(2); 2021 > Article
Kim: The Significance of EFL Learners’ Criteria Awareness of Task-Based Language Teaching

Abstract

The purpose of this study is to explore whether L2 learners improve their L2 speech presentations for English communication in TBLT, and whether their awareness of criteria influences their learning. It also seeks to determine the relationship between learners’ awareness and leaners’ performance test results. A questionnaire for learners’ awareness was developed to measure to what extent learners are aware of the criteria of Task-based Language Assessment (TBLA). The criteria provide an indication of what L2 learners need to develop: introduction, body, conclusion, performance, contents quality, language quality, practice, fluency, and grammar. Data were collected from 67 learners who enrolled in a speech presentation class and participated in the present study. The results showed statistically significant improvement in performance test of TBLT in the Experimental Group (EG) with higher criteria awareness. In addition, the correlation coefficient was significant between performance test results and higher results in the criteria awareness questionnaire. The overall results indicated that L2 learner’s intentional focus on criteria in TBLT had a strong influence initiating a type of criteria awareness and stimulating related types of awareness, ushering L2 learners toward more autonomous learning. However, further refinement of the criteria should be considered depending on the EFL context.

I. INTRODUCTION

Task-based Language Teaching (TBLT) is an approach for EFL teaching composed of focused instructions appropriate to a phased process (Richards, Schmidt, Platt, & Schmidt, 2003). By anticipating authentic outcomes and genuine reflections of language use (Shehadeh & Coombe, 2010), each stage of the process needs to facilitate L2 language acquisition by composing various activities that are goal-oriented and content-focused. Thus, a TBLT syllabus is expected to apply diverse activities to achieve a certain visible task. Since one skill in TBLT may be designed to motivate other skills (Afandi, Jufrizal, & Narius, 2013) and enhance autonomous learning (Lee, 2016; Marton & Booth, 1997), flourishing performance examples of the Second Language Acquisition (SLA) may be elicited for L2 research. Thus, designing criteria for TBLA should involve various types of performance. Task-based Language Assessment (TBLA) is a framed assessment structure for language learning based on TBLT (Brindley, 2008). To be more effective in TBLT, TBLA should be composed of detailed criteria of what to do, how to do, and why.
TBLA is an evaluation process in accordance with a set of explicitly stated criteria (Brindley, 2008), so teachers and researchers may observe the improvement of the quality of communication in terms of the authentic performances, diverse language use, and reflection of integrated L2 skills. There has been plethora of studies related to TBLT and TBLA, such as empirical studies on effective instructions (Van den Branden, Bygate, & Norris, 2009), motivational effect (Khalili & Tahriri, 2014; Pietri, 2015), importance of TBLT (Ellis, 2000, 2003; Shehadeh & Coombe, 2010), and practical steps for implementing TBLT (Long & Norris, 2000). However, little research on EFL learners’ awareness of the criteria of TBLA has been conducted.
Some of the criteria in SLA (Second Language Acquisition) have specific foci including overall performance, EFL learners’ linguistic abilities, and development of learning strategies. The information collected from the results of evaluation with criteria provides meaningful information for teachers, researchers, and learners, which can ensure beneficial learning in the classroom. In particular, EFL learners may be able to obtain insights into strategies to achieve their tasks and become successful language users. However, students are often not informed of the details of criteria for evaluation, since the criteria have long been considered for marking, grading, judging, assessing, measuring, or ranking of the products (Bloxham & Boyd, 2007; Liu & Carless, 2006; Sadler, 2005, 2007, 2010), not as an effective scaffolding provision for EFL learners (Taras, 2005).
In EFL environments, there is an expanding interest in SLA learning effectiveness, particularly during the current coronavirus pandemic which is causing a disastrous standstill in many areas including education. In order to continue education, remote online lectures via internet are being utilized, and with this education method, an important focus is the quality of online learning processes undertaken by EFL learners. One efficient method may be TBLT, which offers EFL learners’ more autonomy to control time and space, as long as they are performing some activities to achieve a task. If this is the case, informing EFL learners of the details of criteria is important in order to address how they will be assessed and let them more motivated.
Therefore, the current study develops example criteria for TBLA to investigate whether EFL learners’ criteria awareness has a relationship with their task performance after TBLA is applied to the Experimental Group (EG) of EFL classrooms. EFL learners have many opportunities to utilize authentic internet resources including presentation tips, linguistic advices, and gestures.
Thus, to accomplish the goal of this study, the following study questions were proposed.
  1. 1. What differences do L2 learners exhibit in their pre-and post-performance tests?

  2. 2. What are the differences between the Experimental Group (EG) and Control Group (CG) performance tests when L2 learners are aware of details of criteria?

  3. 3. What are the relationships between L2 learners’ criteria awareness and their performance in TBLT?

II. LITERATURE REVIEW

1. EFL Learners’ Criteria Awareness in TBLA

Teaching does not regulate the way EFL learners acquire or develop an L2. Rather, language learning is largely actuated by leaners’ internal factors, such as employed strategies and cognitive processes; learning is not just a matter of converting input into output (Skehan, 1996). Hence as internal factors such as observation, experimentation, negotiation, and modification in L2 learning need to be considered (Richards & Rodgers, 2001), L2 learners should be allowed to activate their own acquisition processes in a stimulating and autonomous environment. According to Richards and Rodgers (2001), it has been observed and acknowledged that well-designed tasks in TBLT may foster processes of observation, experimentation, negotiation, and modification of L2. However, the inner factors in SLA seem to be dependent on how much learners try to respond to and manage complete linguistic experience tasks. In brief, learners’ awareness of details of criteria determines their outcomes, resulting in some form of consistent progress and development. Even though TBLT emphasizes fluency in communication, it was found that EFL learners need to be encouraged to focus on various points of communication to achieve fluency and accuracy in authentic communication (Swain, 1988).
There has been an increasing interest in achieving authenticity in TBLT (Ellis, 2003; Long, 1985) since it is a challenge in L2 acquisition that requires accurate use of grammar, gestures, intonation, communicative strategies and other performative factors. For authentic communication, Long (1991) suggested Focus on form (FonF) that a meaning-focused interaction method essential for efficient and effective TBLT. Nunan (1989) and other researchers (Doughty & Williams, 1998; Long & Robinson, 1998) also argued that FonF utilized in L2 learners’ language use is an essential resource. In addition, regarding more authenticity, a plethora of studies have investigated L2 learners’ short speech performance in terms of gestures (Gullberg, 2009; Jenkins & Parra, 2003), intonation (Flege & Hillenbrand, 1984; Willems, 1982), and communicative strategies (Bialystok, 1990). As L2 learning requires simultaneous learning of how to interact with target language users with all of the factors mentioned above, TBLT may change learners’ perception of the factors on which they have to focus.
Often L2 learners are able to display what they have learned but are unaware of the specific tasks they are learning (Marton & Booth, 1997). Learners’ awareness of what they are focusing on plays a significant role in TBLT. The task should be meaning-based, goal-oriented, outcome-evaluated, and related to the real-world (Skehan, 1998). Rodgers (2002) further suggests that delivering a short speech offers learners the opportunity to consciously select, prepare, and sequence a task during the introduction of the topic, as well as helping them with useful expressions and communicative strategies. In this way, TBLT integrated with delivering a presentation may be meaningful and relevant, enabling L2 learners to appreciate the reason for doing a task and how the task relates to probable situations in future communication outside the classroom (Larsen-Freeman & Anderson, 2011).
However, in TBLT L2 learners should also be aware and conscious of criteria. L2 acquisition may only occur with L2 learners’ full engagement with active and consistent adjustment to the criteria (Entwistle & Peterson, 2004). According to Entwistle and Peterson (2004), the criteria are guidelines for teachers and learners to maintain academic standards. EFL learners may comprehend what is required of them based on specific details of criteria. In order to develop L2 learners’ criteria awareness, they should have full awareness of what they are learning and how they are being evaluated.
TBLT is a learner-centered approach with support of autonomy (Lee, 2016; Skehan, 1998). In order for L2 learners to develop an awareness of the purpose of the task, Richards and Schmidt (2010) argue that learner-centered approach should be applied in TBLT. Thus, learners’ awareness of central goals may be emphasized to gear autonomously their performance to detailed evaluating criteria of a task. Depending on their levels and the immediate context to which L2 use evaluating criteria may be modified and adapted. In some related studies, learners’ awareness of task goals improved learners’ involvement (Burrows, 2008), confidence (Leaver & Willis, 2004), and operation of various learning strategies (Macaro, 2009). In the end, L2 learners may become autonomous and responsible for their own SLA. Marton and Booth (1997) and Lee (2016) emphasize that many studies of SLA have been unsuccessful as L2 learners are perceived as not being responsible for their own learning. Therefore, learners should set their aims and attitudes towards an autonomous learning paradigm. In so doing, L2 learners’ criteria awareness in the learning process may create opportunities for them to be more autonomous and responsible when exposed to L2 in an authentic way.

2. The L2 Learners’ Awareness Questionnaire Design

Although L2 learners’ criteria awareness may refer to the role and impact of TBLT and TBLA, to date, few researches on L2 learner’s awareness questionnaire currently exists to facilitate this line of exploration. Such an instrumental and tractable questionnaire might lead to greater understanding of TBLT and TBLA by supporting educators and researchers examine the extent to which L2 learners are influenced by the details of criteria that should be operant in L2 learners’ mind. The L2 Learners’ Criteria Awareness Questionnaire (L2CAQ) was developed to measure L2 learners’ awareness of criteria and to examine its influence on their task performance.
The enhanced learning process may be related to higher level of task awareness. Choy, Goh, and Sedhu (2016) developed learners’ awareness questionnaire. They posited that learners’ four levels of criteria awareness: survival, establishing stability, approval, and loving to learn. The four levels of learning awareness indicate how knowledge is taken in and perceive the significance of task goals. By adopting their criterial awareness questionnaire, L2 learners’ awareness may be predicted in the similar levels of awareness. Accordingly, the criteria specific sections and questions should be redesigned to make L2CAQ. For exploratory factors of redesigning the sections of questionnaire needed to focus on Second Language Acquisition (SLA) in TBLT environment to provide a better interpretation of L2 learners’ criteria awareness influences on their language performance. The four levels were as follows:
  1. 1) Adaptation level: this level requires to adapt themselves to the basic L2 learning environment, which may compare to Survival in Choy et al. (2016). L2 learners attempt to establish their own well-being in L2 learning environment.

  2. 2) Stability level: L2 learners’ perception of required standards in L2 learning environment. This level is about establishing stability in Choy et al. (2016). L2 learners may perceive and decide to follow criteria. L2 learners will use learning strategies by examining or analyzing L2 learning to achieve stability in L2 learning.

  3. 3) Approving level: L2 learners approve their need to accept, apply, and interact with others in reflective actions. L2 learners perform what they are required to do and expect other’s approval of their language use in L2 communication. In this level, compared to approval level of Choy et al. (2016), learners may determine how information and knowledge is taken in and related to something meaningful.

  4. 4)Sensing accomplishment level: L2 learners sense accomplishment and develop positive favor in L2 learning, which may induce result in L2 learners’ motivation to acquire L2, adopted from positive psychology (Choy et al., 2016; Seligman, 2002). In this level, L2 learners are able to give peer-feedback and evaluate other learners for scaffolding.

Within the four levels of L2 learning multiple progress from the primary learning to more complex profound learning may occur (Biggs, 1999). Learners need to be aware of criteria of how L2 learning will be evaluated. Whether learners are able to be aware of criteria is related to the development progress recognized by learners themselves and teachers. Hence whether learners are aware of purpose of certain activities carried out may determine what extent of learning that occurs. The structure of L2 learning is experiences from individual learner’s point of view based on its intentionality, being directed to the goal with criteria.
This kind of phenomenological study of SLA requires analyzing questionnaire to ask learner’s experience, conscious way of learning, and manner of perception of the knowledge (Marton, 1996). According to Choy et al. (2014), the questionnaire in each level provided feasible data as a measure of their learning awareness. The study by Choy et al. (2014) showed that all students had awareness of how and why they learned. Thus, L2CAQ consists of 30 question items which focused on criteria awareness.
Contemplating the four levels of L2 criteria awareness based on a constructivist perspective, it involves that L2 learners are acknowledged to formulate their own learning experience according to the organized around activities and tasks (Richards et al., 2003). It is frequently observed and acknowledged that the interest in TBLT facilitates SLA and makes L2 learners’ learning more effective, since the tasks and evaluating criteria are designed from empirical findings on effective instructional techniques and authentic purpose in contemporary society (Ellis, 2003; Shehadeh & Coombe, 2010). Hence L2 learning is dependent on individual learner’s performance up to the criteria. In brief, L2 learning with criteria awareness is a process which actuates how knowledge of grammar and vocabulary are taken in and manipulated, resulting in some communicative form of continued development and innovation.
One of the applied examples of TBLT and TBLA is delivering a presentation or a short public speech (Aziz, 2018; Brydon & Scott, 2006). Delivering a short speech creates an authentic environment that L2 learning may be in more multifaceted process, which furnishes L2 learners with all the information and knowledge they need for L2 use (Ellis, 2000). L2 learners envisages the task as an authentic requirement to participate with observing, analyzing, applying what they are required to do, in continual interaction with world. Corresponding to the required tasks such as selecting preferred topics, speech construction and particularly assessing their own performance in peer-feedback as a way of TBLA (Long & Norris, 2000).
Often peer-feedback in TBLA is acknowledged to measures the extent to which it may be related to the L2 learners’ achievement in the performance test. Their peer-feedback may facilitate similar environment in the real world (Ellis, 2003). Thus, the details of criteria should be informed to L2 learners as devices to elicit and evaluate their own communicative performance from their own perspectives in their feedback. In the context of language use that is meaning-focused and directed to very specific goals. In this sense, there must be a certain criteria awareness of what they have performed.
For TBLA, this study focused on formative, performance-referenced, direct, and authentic assessments with specified criteria (Burton, 2006; Ellis, 2003; Kim, 2018). The formative assessment involves undertaking continual instruction and feedback for the purpose of improving their language skills in English presentations (Kim, 2018). According to Burton (2006), the performance-referenced assessment provides information about participants’ ability to use L2 on a certain topic in a specific context to ascertain if L2 learners to accomplish real target tasks. The direct assessment involves a measurement of L2 ability in performing incorporated into the task, such as delivering English presentation in public (Kim, 2018). Lastly the authentic assessment engages authentic language use (Ellis, 2003), that is, the topics were about the issue they have in real world and delivering presentation to the authentic audience.
The involvement for criteria awareness arises as task learners need more conscious participation during the process of TBLT. In particular, L2 learning in TBLT may only occur with the full engagement of L2 learners. Marton and Booth (1997) emphasize that learning itself might be inefficient if learners are not autonomous and responsible. So, the task design in TBLT should be construct-centered approach, entailing with conscious language use as a basis for the task design and selection (Ellis, 2003). Therefore, for TBLA, the assessment should be the criterion-referenced test on direct performance. This approach involves details of criteria to determine what tasks should be performed, what L2 learners to need to be aware of in a particular situation.
L2 learners should be in the center of all aspects of language use, including selecting topics, planning how to organize the contents, and how L2 learners themselves evaluate each other in peer-feedback. The L2 learner-centered approach in TBLT requires them to have criteria awareness of the given contexts to which L2 learning can be utilized and be able to acclimate it to diverse tasks. Therefore, L2 learners will keep consistently developing to meet the requirement of details of criteria that describes a set of aims in language, performance skills, attitudes, and gestures appropriate to the authentic situation. Thus, it is presumed that the way criteria are detailed may determine learners’ progress in L2. The current study, taken from TBLT and TBLA, developed a framework of the L2CAQ.

III. METHOD

1. Participants

Sixty-seven (67, female = 36, male = 31) L2 learners enrolled in College English 2, a compulsory course centered on English presentations submitted all required tasks during the semester of 2020. They participated in the present study as a control group (CG, N = 34) and an experiment group (EG, N = 33), who are aged from 19 to 25. They were divided into the groups randomly. The consent of all participants was obtained and the questionnaire was completed online.
All participants submitted 3 English presentation video clips before the mid-term exam and another 3 before the final exam to a blogging platform, Daum café1. They were allowed to choose one topic out of three presentations respectively for the mid-term and final exam. The presentations for their mid-term and final exams may be considered the most confident among 3 presentations. Their first video clips were evaluated as a pre-performance test and the last ones were rated as a post-performance test. Both EG and CG had instructor’s feedback after submitting their short speeches. However, EG learners had awareness activities, such as a self-checklist and classmates’ feedbacks while CG learners were introduced to refer to them.
The participants from diverse majors (Business, Electronic and Electrical Engineering, Civil Engineering, Architectures, Korean, Chemical Engineering, and History) had a mixed range from beginner to intermediate English proficiency levels. Participants practiced pronunciation and intonation through shadow-reading of 6 exemplar presentation scripts provided in the text book, Present Yourself 1 (Gershon, 2015). They were asked to brainstorm, write outline, write presentation scripts, and upload their presentation video clips six times.
Based on TBLT, all their work was collected for a formative assessment, a performance-referenced assessment, a direct assessment and an authentic assessment as presented in the previous section.

2. Procedures and Data Analysis

1) Procedures

A TBLT syllabus, presented in Table 1, was applied to both the EG and CG, and the difference between the EG and CG was criteria awareness. Due to COVID-19, a pandemic situation, all L2 participants were required to join remote learning. They watched uploaded video clips of classes (45-60 minutes) in the university’s E-learning and submitted class activities to Kakao Talk. Their short speech presentations were uploaded (3 minutes for the mid-term exam and 5 minutes for the final exam) following the presentation form suggested in the criteria. The textbook, utilized in both groups, is composed of 6 units of topics, apart from a “Getting Ready” introduction, which covers guidelines for delivering presentations, useful expressions for openers and closers, tips for gestures, note-cards, and ppt materials. The evaluating criteria were introduced at the beginning of the semester to the both EG and CG and both groups had instructors’ feedback about their presentations, but in the EG there were activities, such as a self-checklist and classmates’ feedback.
At the beginning of the semester, learners were introduced to TBLT and TBLA. The final task was to deliver a presentation to the classmates. Learners were asked to undertake language-learning centered activities such as information gaps, opinion gaps, opinion asking surveys, and reasoning gap activities, which were related to one of the open topics such as “a good friend,” “a favorite place,” “a prized possession,” “a memorable experience,’ “giving instruction of how to do or make,” and “a favorite movie,’ as presented in Table 1. The information gap activities provided learners’ active interaction in terms of topic information, the language used for presentation, and presentation methods. In one of the reasoning gap activities, learners were asked to find the right order of presentation after listening to Sophie’s brainstorming in Unit 1 in Present Yourself 1 (Gershon, 2015). Learners were free to choose one of three topics for their mid-term and final presentations and they were allowed to extend their own topics diversely from the open topics. Procedural feedback from instructors and peers were provided to all the participants regarding their everyday classroom activities, brainstorming, outlines, scripts and presentations. The guidelines of peer-feedback were based on the evaluation criteria attached in Appendix 1. For their presentation scripts they had a chance to revise according to the feedback. At the end of the semester, L2CAQ surveys were distributed to participants.

2) Data Analysis: The Questionnaire Survey and Achievement Assessment

To examine the first research question (what differences do L2 learners exhibit in their pre-and post-performance tests?), performance tests were utilized to identify how scores were different at the beginning and end of the semester. The performance test referred to what L2 learners could do in presenting a short speech, similar to real life. For statistical analysis, SPSS 22 was conducted for independent samples, t-tests to identify significant changes between the pre/post performance tests, comparing in mean and standard deviation (SD).
For their achievement test, their first and second presentation video clips were analyzed with SPSS Statistics 22. Two instructors, having teaching experience of 15 years and 16 years, assessed them according to criteria from brainstorming, writing outlines, written scripts, and presentation performances to obtain inter-rater reliability. For the assessment, they followed the presentation criteria, for instance, introduction, body organization, conclusion, extra-linguistic qualities, speech management, content quality, language quality, practice & preparation, fluency, and grammar use (see Appendix 1). The researcher developed the assessment tool with references to several similar studies (Bigg, Kember & Leung, 2001; Bloxham & Boyd, 2007). Ten items were evaluated, and for each item, 10 points were allocated. L2 learners’ results were computerized to ascertain reliability and validity. Cronbach alpha (α) was .85 (shown in Table 2), which is acknowledged as acceptable reliability since the alpha (α) is more than .70.
To explore the second research question (What are the differences between the EG and CG performance tests when L2 learners are aware of details of criteria?), performance tests were conducted between the EG and CG to identify if delivering presentations in public for TBLT was effective. For statistical analysis, the test scores were input in SPSS 22 for independent samples t-tests to identify significant differences between EG and CG.
For the third research question (What are the relationships between L2 learners’ criteria awareness and their performance in TBLT?), the L2CAQ questionnaire was administered at the end of the semester to the CG and EG. The questionnaire was analyzed in varimax rotation as an attempt to elucidate the relationship among factors of four levels of L2 learners’ criteria awareness: adaptation, stability, approval, and sensing achievement. These four levels of L2 learners’ awareness offered an indication of how criteria are taken in and the extent of criteria perception. The study mainly focused on the questionnaire survey to cover learners’ learning experiences, attitudes, changes, and manner that they performed the task. The questionnaire was then given to two other researchers to examine for any linguistic ambiguities and inadequate items, which were modified.
25 questions (see Appendix 2) composed the questionnaire survey, conducted at the end of the semester. A 5-point Likert scale was utilized for each question item with 5 indicating Strongly agree, 4 Agree, 3 Neutral, 2 Disagree, and 1 Strongly disagree. Near the end of the semester, participating L2 learners submitted the questionnaire survey.

IV. RESULT

1. L2 Learners’ Performance Tests Between Before and After TBLT

The study investigated the influence of TBLT on L2 learners to determine the effectiveness of TBLT and whether TBLT activities were beneficial by replying the three research questions.
Research Question 1: What differences do L2 learners have in their pre-and post-performance tests?
L2 Learners’ performance tests were evaluated based on their brainstorming, outlines, presentation scripts, and presentation video clips according to the criteria. The criteria consisted of 10 items: Introduction, Body Organization, Conclusion, Extra Linguistic Quality (using rhetoric), Speech Management, Content Quality, Language Quality, Practice & Preparation, Fluency, and Grammar Use, as presented in Appendix 1. Table 3 exhibits how L2 learners’ scores differed between pre- and post-performance tests.
According to Table 3, performance scores increased in the post-performance test. Table 3 presents significant statistical changes in several criteria items, such as body organization (p = .001, p < .05), speech management (p = .003, p < .05), content quality (p = .010, p < .05), practice & preparation (p = .014, p < .05), and fluency (p = .003, p < .05) at .05 level. From this result, TBLT supported particular parts of English presentation performance, such as organizing, managing, and attracting more effort for practice. In terms of content quality, practice & preparation, and fluency, L2 learners’ communicative fluency seemed to improve. It is probable that items 2, 5, 6, 8, and 9 may have enhanced whether or not they were in the CG or EG, that is, whether they were aware of criteria or not. The results of the presentation-focused class, integrated with TBLT supports the previous studies (Aziz, 2018) in terms that L2 learners prepare body organization, speech management, and content quality on preferred topics with consistent feedback from peers and an instructor. In the end, based on the implementation of TBLT, L2 learners presumed to practice their speech more. This in turn results in statistically significant improvement in body organization, speech management, content quality, practice & preparation, and fluency in their public presentations.

2. Comparison Between EG and CG Focusing on Criteria Awareness

Research Question 2: What are the differences between the EG and CG performance tests when L2 learners are aware of details of criteria?
The instructor taught the presentation focused class in TBLT via online lectures. Thus, L2 learner’s awareness of criteria seemed more significant, since remote classes require more autonomous learning in terms of time and speech management. The researcher emphasized criteria awareness during class instructions so that L2 learners might focus on criteria of TBLA more in the EG, while learners in the CG were simply introduced to the criteria. The differences between the EG and CG is shown in Table 4.
Table 4 indicates significant statistical differences between EG and CG in several items, such as introduction (p = .002, p < .05), body organization (p = .001, p < .05), conclusion (p = .017, p < .05), extra linguistic quality (p = .004, p < .05), and Fluency (p = .001, p < .05) at .05 level.
According to the results, L2 learners in the EG are more self-directed and learner-centered with criteria awareness, indicating to more autonomous learning in time and speech management. A conspicuous implication of the results was L2 learners’ focused criteria awareness in Introduction, Body Organization and Conclusion which are main factors in delivering a presentation, and showed statistically significant improvement in Fluency facilitating advanced communication. This result supports that raising learners’ awareness stimulates learners’ involved participation (Burrows, 2008), confidence (Leaver & Willis, 2004), and operating strategies (Macaro, 2009).

3. Relationship Between L2 Learners’ Awareness and Performance

Research Question 3: What is the relationship between L2 learners’ criteria awareness and their performance in TBLT?
In order to identify how much L2 learners’ criteria awareness influenced focused learning, the L2CAQ (see Appendix 2) was developed in terms of adaptation level, stability level, approving level, and sensing accomplishment level. The data was then encoded and processed with Pearson Product-Moment Correlations between learners’ criteria awareness and their performance in SPSS 22, as shown in Table 5.
The Pearson correlations of the L2CAQ and Learners’ performance up to the Criteria were significant at the p < 0.001 level. The correlations between adaptation level and introduction (r (33) = 0.37, p < 0.001), body organization (r (33) = 0.71, p < 0.001), conclusion (r (33) = 0.88, p < 0.001), and grammar use (r (33) = 0.95, p < 0.001) similarly between stability level and introduction (r (33) = 0.67, p < 0.001), body organization (r (33) = 0.54, p < 0.001), conclusion (r (33) = 0.61, p < 0.001) and grammar use (r (33) = 0.82, p < 0.001) was significant.
In addition, the correlations between content quality and approving level (r (33) = 0.57, p < 0.001) and sensing achievement level (r (33) = 0.78, p < 0.001), between language quality and approving level (r (33) = 0.74, p < 0.001) and sensing accomplishment level (r (33) = 0.85, p < 0.001), and between fluency and approving level (r (33) = 0.89, p < 0.001) and sensing accomplishment (r (33) = 0.75, p < 0.001) was respectively significant. The correlations between extra linguistic quality and stability level (r (33) = 0.82, p < 0.001) and approving level (r (33) = 0.91, p < 0.001), and between speech management and stability level (r (33) = 0.76, p < 0.001) and approving level (r (33) = 0.85, p < 0.001) was significant. Lastly, the correlations between English practice & preparation and adaptation level (r (33) = 0.59, p < 0.001) and sensing accomplishment level (r (33) = 0.92, p < 0.001) are significant.
The four awareness levels of L2CAQ showed moderate to strong correlation with some of the items of criteria. Introduction, body organization, conclusion and grammar use showed stronger relationships with both adaptation and stability levels, while content quality, language quality, and fluency indicated stronger relationships with both approving level and sensing accomplishment. In addition, extra linguistic quality and speech management showed stronger relationships with both stability and approving levels. From the results, the relationships of the levels of criteria awareness with particular items of criteria may imply pedagogical implications for L2 learners’ awareness through identifying the influence of L2 learners’ criteria awareness.
The overall presentation script organization related to introduction, body organization, conclusion and grammar use seemed to be more focused on adaptation and stability levels. Extra linguistic quality and speech management was more related to presentation method such as gestures, facial expressions, timing, attitudes, posture, and voice projection. Those qualities seemed to be fixed sets in stability approving levels. The awareness of content quality and language quality and practice and preparation were more related to approving level and sensing accomplishment. Content quality and Extra linguistic quality seemed to be related to confidence in their accomplishment. The results, shown in Table 5, seem to support that learners’ awareness of the learning criteria enhanced learners’ involvement (Burrows, 2008), confidence (Leaver & Willis, 2004), and operating strategies (Macaro, 2009).

V. CONCLUSION

TBLT has become increasingly recognized as an effective method to uphold leaners’ autonomy and responsibility; it can improve L2 learners’ English competence in both accuracy and fluency when learners are aware of the purpose of the tasks contained in criteria. According to Prabhu (1987), this creates conditions in which learners are involved in efforts to deal with the requirements of communication. The criteria in TBLA provide an elaboration of the targeted learning details, which may improve the intentionality, motivation, confidence, and autonomy with L2 learners’ criteria awareness. The primary goal of this study was to examine L2 learner’s performance improvement after TBLT, to identify whether there were significant differences between the EG and CG, and how they were related to L2CAQ.
As to the first research question, L2 learners showed improvement in their performance tests, showing generally enhanced ability in delivering presentations. In particular, their content organization, speech management, and content quality were conducted with practice and preparation which led to overall fluency. For the second research question, L2 learners in the EG showed significantly higher scores in performance tests, which indicates that criteria awareness in L2 learning is essential to TBLT, and delivering presentations is beneficial in improving communication skills, particularly in public speech. Within L2 learning environments, there is a proliferating demand to be aware of what learning requires. To date, there have been few studies regarding L2 learners’ criteria awareness, and this study may fortify the proposition that TBLT with L2 learners’ criteria awareness improves L2 learners’ communicative fluency. Based on the findings for the third research question, the relationship between the performance test results according to criteria and the L2CAQ showed significant correlation between various items, with some parts showing consistent relationships between two items. For instance, introduction, body organization and conclusion were intimately related to adaptation and stability levels. The result ingeminated that TBLT in conjunction with L2 learners’ criteria awareness builds autonomous learning, and one element of criteria awareness relates closely with other factors of awareness.
Based on overall findings of TBLT integrated with criteria awareness a self-reflection report, the study affirms that one part of awareness in TBLT, whether it is subsequent or intermittent, initiates another part of awareness empowering participation and gearing L2 learners toward being more autonomous and responsible. Learners’ awareness of criteria seems to be essential as guidelines for them to continue to move on to subsequent levels of awareness and task accomplishment (Burrows, 2008; Leaver & Willis, 2004; Macaro, 2009; Sadler, 2005). As indicated in the previous sections, one level of L2 learners’ awareness initiates another level, which seems to be due to similarity in types, activities, or purpose. TBLT with criteria awareness sets up learners’ confidence and motivation in delivering presentations in public. To conclude, TBLT with criteria awareness can have a strong impact on language learners, utilizing both linguistic and extra linguistic instructions which are consummated in criteria, supporting learners' autonomy, and shaping their confidence in speaking publicly.
Some limitations of the present study are the limited sample size, the lack of a comparative group, and the relatively short duration of application of various techniques, methods, and activities for TBLT. Thus, Further studies are necessary to deepen and broaden a variety of techniques, methods, and activities for TBLT and TBLA. Teaching TBLT provides L2 learners with experiences of communicative authenticity by supporting them with phased awareness about language features in communication. Further refinement according to various classroom environments is required as the results of this study may not be applicable to all L2 classes. Therefore, further research on criteria awareness needs to be conducted using wider groups of L2 learners to supplement this present study.

Notes

1. http://cafe.daum.net/practicalenglish7 was used only for invited learners during the semester.

TABLE 1
The Focal Points of Procedural TBLT-Based Syllabus
Main Tasks Process of TBLT
Class Components Activity (Performed Week Periods)
Main Features of Pre-Task: Planning Presentation
  • Previous warming-up activities, instructions of how to deliver presentations and useful expressions

  • Learner-centered social problem-solving orientation topics

  • Provision for focus on form

  • Language-learning centered activities (3,5,7,9.11,13)

  • Information gap, opinion gap, and reasoning gap activities (1,3,4,6,9,10,12,13)

  • Planning for the presentation in public (4,6,7,10,12)

  • Brainstorming about the open topics (3,5,7,9,10,13)

  • Writing outlines (4,6,7,10,12)

  • Writing scripts (4,6,7,10,12)

  • Delivering presentations in video clips (7,8,13,14,15)

Classroom Activities Submitted to Kakao Talk
  • Selected activities of each week

  • Brainstorming about the open topics

  • Writing outlines

  • Writing scripts

  • Trial performances to deliver presentation and make video clips

  • Peer-feedback of video clips

  • Some revision by the instructor’s feedback

Final Tasks for Mid-Term and Final Exam
  • Learner’s chosen outlines, scripts, and presentation video clips were submitted to Daum café.

Review
  • Before delivering presentation, all the learners have procedural feedback for their scripts.

  • Learners give peer-feedback and have feedback from peers and instructors.

TABLE 2
Inter-Rater Reliability of L2 Learners’ Performance Assessment
Measurement Pearson Items Cronbach Alpha (α)
Reliability Index Reliability 10 Items
L2 Learners Performance .85 2.36 .84 4.19 .85
TABLE 3
Results of L2 Learners’ Performance Tests (N = 67)
Criteria Pre Post t p
M SD M SD
1. Introduction 5.32 .945 6.17 .945 0.471 .141
2. Body Organization 6.48 .875 7.03 .675 2.126 .001*
3. Conclusion 5.52 .871 6.09 .925 0.814 .117
4. Extra Linguistic Quality 5.48 .701 6.11 .693 0.909 .214
5. Speech Management 5.10 .657 7.23 .829 3.355 .003*
6. Content Quality 5.74 .819 7.63 .812 2.571 .010*
7. Language Quality 5.61 .728 6.29 .810 0.525 .156
8. Practice & Preparation 5.32 .815 7.10 .916 2.061 .014*
9. Fluency 5.82 .719 7.28 .828 3.431 .003*
10. Grammar Use 5.86 .936 6.31 .924 0.156 .106

. Note:

*. sig. (2 tailed) p < .05

TABLE 4
L2 Learners’ Post-Performance Test Results Between EG and CG
Criteria EG (N = 33) CG (N = 34) t p
M SD M SD
1. Introduction 8.77 .341 6.03 .835 2.742 .002*
2. Body Organization 7.81 .187 7.73 .736 3.126 .002*
3. Conclusion 8.17 .378 5.89 .385 3.151 .017*
4. Extra Linguistic Quality 8.08 .571 6.41 .701 2.901 .004*
5. Speech Management 7.15 .407 6.36 .636 0.345 .203
6. Content Quality 8.71 .311 5.81 .637 0.372 .210
7. Language Quality 7.21 .327 6.04 .923 0.105 .156
8. Practice & Preparation 8.62 .285 7.92 .820 0.451 .214
9. Fluency 8.07 .559 6.37 .802 2.301 .001*
10. Grammar Use 7.46 .518 7.19 .583 0.176 .106

. Note:

*. sig. (2 tailed) p < .05

TABLE 5
Pearson Product-Moment Correlations Between L2 Learners’ Criteria Awareness and Task Performance
Criteria L2CAQ
Adaptation Level Stability Level Approving Level Sensing Accomplishment Levl
1. Introduction 0.37** 0.67** −0.13 −0.06
2. Body Organization 0.71** 0.54** −0.09 0.15
3. Conclusion 0.88** 0.61** 0.04 0.018
4. Extra Linguistic Quality −0.15 0.82** 0.91** −0.05
5. Speech Management −0.36 0.76** 0.85** −0.18
6. Content Quality −0.42 −0.27 0.57** 0.78**
7. Language Quality −0.15 0.15 0.74** 0.85**
8. Practice & preparation 0.59** −0.13 0.15 0.92**
9. Fluency −0.05 −0.04 0.89** 0.75**
10. Grammar Use 0.95** 0.82** 0.09 0.01

. Note: EG only,

**. p < .001

REFERENCES

Afandi, W., Jufrizal, J., & Narius, D. (2013). The ability of the third-grade students in using different kinds of prepositions. Journal of English Language Teaching, 1(25), 625-639. https://doi.org/10.24036/jelt.v1i2.2000.
crossref
Aziz, A. (2018). The effect of task-based language teaching (TBLT) strategy on students’ speaking ability in demonstrative speech. Journal of Linguistic and Language Teaching, 4(2), 53-71. https://doi.org/10.29300/ling.v4i2.1665.
crossref
Biggs, J. (1999). What the student does: Teaching for enhanced learning. Higher Education Research and Development, 18(1), 57-75.
crossref
Biggs, J., Kember, D., & Leung, D. (2001). The revised two factor study process questionnaire: R-SPQ-2F. British Journal of Educational Psychology, 71(1), 133-149. https://doi.org/10.1348/000709901158433.
crossref
Bloxham, S., & Boyd, P. (2007). Developing effective assessment in higher education: A practical guide. New York, NY:McGraw-Hill International.
Brindley, G. (2008). Educational reform and language testing. In E. Shohamy & N. Hornberger (Eds.), Encyclopedia of language and education: Language testing and assessment. 7, pp 365-378. New York, NY:Springer.
crossref
Brydon, S. R., & Scott, M. D. (2006). Between one and many: The art and science of public speaking (5th ed). Boston, MA:McGraw Hill.
Burrows, C. (2008). An evaluation of task-based learning (TBL) in the Japanese classroom. English Today, 25(4), 11-16. https://doi.org/10.1017/S0266078408000345.
crossref
Burton, K. (2006). Designing criterion-referenced assessment. Journal of Learning Design, 1(2), 73-82.
crossref
Bialystok, E. (1990). Communication strategies: A psychological analysis of second language use. London:Blackwell.
Choy, S. C., Goh, P. S. C., & Sedhu, D. S. (2016). How and why students learn: Development and validation of the learner’s awareness levels questionnaire for higher education students. International Journal of Teaching and Learning in Higher Education. 28(10), 94-101. Retrieved from https://files.eric.ed.gov/fulltext/EJ1106321.pdf.
Choy, S. C., Ooi, H. J., Sedhu, D. S., Liew, Y. L., Lee, M. Y., & Malenee, A. (2014). April;Students’ awareness of how and why they learn. In: Paper presented at the International Symposium on Education and Psychology; Nagoya:University of Nagoya.
Doughty, C., & Williams, J. (1998). Pedagogical choices in focus on form. In C. Doughty & J. Williams (Eds.), Focus on form in classroom of second language acquisition. pp 197-261. Cambridge, UK:Cambridge University Press.
Ellis, R. (2000). Task-based research and language pedagogy. Language Teaching Research, 4(3), 193-220. https://doi.org/10.1177/136216880000400302.
crossref
Ellis, R. (2003). Task-based language learning and teaching. Oxford: Oxford University Press.
Entwistle, N. J., & Peterson, E. R. (2004). Conceptions of learning and knowledge in higher education: Relationships with study behavior and influences of learning environments. International Journal of Education Research, 41(6), 407-428. https://doi.org/10.1016/j.ijer.2005.08.009.
crossref
Flege, J. E., & Hillenbrand, J. (1984). Limits on phonetic accuracy in foreign language speech production. Journal of the Acoustic Society of America, 76(3), 708-721. https://doi.org/10.1121/1.391257.
crossref
Gershon, S. (2015). Present yourself 1. Cambridge, UK:Cambridge University Press.
Gullberg, M. (2009). Reconstructing verb meaning in a second language: How English speakers of L2 Dutch talk and gesture about placement. Annual Review of Cognitive Linguistics, 7(1), 221-245. https://doi.org/10.1075/arcl.7.09gul.
crossref
Jenkins, S., & Parra, I. (2003). Multiple layers of meaning in an oral proficiency test: The complementary roles of nonverbal, paralinguistic, and verbal behaviors in assessment decisions. The Modern Language Journal, 87(1), 90-107.
crossref
Khalili, S. M., & Tahriri, A. (2014). The impact of task-based approach on Iranian EFL learners’ motivation in writing research abstracts. Journal of Language Teaching and Research, 5(4), 953-962. https://doi.org/10.4304/jltr.5.4.953-962.
crossref
Kim, E. (2018). The benefits of formative feedback for critical writing in EFL context. The Linguistic Association of Korean Journal, 26(1), 23-55. https://doi.org/10.24303/lakdoi.2018.26.1.23.
crossref
Larsen-Freeman, D., & Anderson, M. (2011). Techniques and principles in language teaching. Oxford: Oxford University Press.
Leaver, B. L., & Willis, J. R. (2004). Task-based instruction in foreign language education. Washington, DC:Georgetown University Press.
Lee, L. (2016). Autonomous learning through task-based instruction in fully online language courses. Language Learning & Technology. 20(2), 81-97. Retrieved from http://llt.msu.edu/issues/june2016/lee.html.
Liu, N., & Carless, D. (2006). Peer feedback: The learning element of peer assessment. Teaching in Higher Education, 11(3), 279-290. https://doi.org/10.1080/13562510600680582.
crossref
Long, M. (1985). A role for instruction in second language acquisition: Task-based language teaching. In K. Hyltenstam & M. Pienemann (Eds.), Modelling and assessing second language acquisition. pp 77-100. Clevedon, UK:Multilingual Matters.
Long, M. (1991). Focus on form: A design feature in language teaching methodology. In K. de Bot & R. Ginsberg & C. Kramsch (Eds.), Foreign language research in cross-cultural perspective. pp 39-52. Amsterdam:John Benjamins: https://doi.org/10.1075/sibil.2.07lon.
crossref
Long, M., & Norris, J. M. (2000). Task-based language teaching and assessment. In M. Byram (Ed.), Encyclopedia of language teaching. pp 597-603. London:Routledge.
Long, M., & Robinson, P. (1998). Focus on form: Theory, research and practice. In C. Doughty & J. Williams (Eds.), Focus on form in classroom second language acquisition. pp 15-41. Cambridge, England:Cambridge University Press.
Macaro, E. (2009). Developments in language learner strategies. In L. Wei & V. Cook (Eds.), Contemporary applied linguistics. pp 10-36. London:Continuum.
Marton, F. (1996). Phonomyography. In T. Husen & N. Postlethwaite (Eds.), The international encyclopedia of education 8. pp 4424-4429. London:Pergamon.
Marton, F., & Booth, S. (1997). Learning and awareness. Mahwah, NJ:Erlbaum.
Nunan, D. (1989). Designing tasks for the communicative classroom. Cambridge, UK:Cambridge University Press.
Richards, J., & Rodgers, T. (2001). Approaches and methods in language teaching. New York, NY:Cambridge University Press.
Richards, J. C., & Schmidt, R. (2010). Longman dictionary of language teaching and applied linguistics (4th ed). London:Longman.
Richards, J., Schmidt, R., Platt, H., & Schmidt, M. (2003). Longman dictionary of applied linguistics. London:Longman.
Rodgers, C. (2002). Defining reflection: Another look at John Dewey and reflective thinking. Teachers College Record, 104(4), 842-866.
crossref
Pietri, N. J. M. (2015). The effects of task-based learning on Thai students’ skills and motivation. Asian Journal of Management & Innovation, 3(4), 72-80. https://doi.org/10.7575/aiac.alls.v.8n.2p.243.
crossref
Prabhu, N. S. (1987). Second language pedagogy. Oxford:Oxford University Press.
Sadler, R. (2005). Interpretations of criteria-based assessment and grading in higher education. Assessment & Evaluation in Higher Education, 30(2), 175-194. https://doi.org/10.1080/0260293042000264262.
crossref
Sadler, R. (2007). Perils in the meticulous specification of goals and assessment criteria. Assessment in Education: Principles, Policy & Practice, 14(3), 387-392. https://doi.org/10.1080/09695940701592097.
crossref
Sadler, R. (2010). Beyond feedback: Developing student capability in complex appraisal. Assessment & Evaluation in Higher Education, 35(5), 535-550. https://doi.org/10/1080/02602930903541015.
crossref
Seligman, M. (2002). Authentic happiness: Using the new positive psychology to realize your potential for lasting fulfillment. Boston, MA:Simon and Schuster.
In A. Shehadeh & C. Coombe (Eds.), (2010). Applications of task-based learning in TESOL. Alexandria, VA:TESOL.
Skehan, P. (1996). Second language acquisition research and task-based instruction. In J Willis & D Willis (Eds.), Challenge and change in language teaching. pp 17-30. Oxford:Heinemann.
Skehan, P. (1998). Task-based instruction. Annual Review of Applied Linguistics, 18(1), 268-286. https://doi.org/10.1017/S0267190500003585.
crossref
Swain, M. (1988). Manipulating and complementing content teaching to maximize second language learning. TESL Canada Journal, 6(1), 68-83. https://doi.org/10.18806/tesl.v6i1.542.
crossref
Taras, M. (2005). Assessment-summative and formative-some theoretical reflections. British Journal of Educational Studies, 53(4), 466-478. https://doi.org/10.1111/j.1467-8527.2005.00307.x.
crossref
Van den Branden, K., Bygate, M., & Norris, J. (2009). Task-based language teaching: Issues, research and practice. Amsterdam:Benjamiins.
Willems, N. (1982). English intonation from a Dutch point of view. Dordrecht, Holland:Foris.

Appendices

APPENDIX 1

Evaluation Criteria

Criteria to evaluate learners’ preparation work and presentation video clips (10 points for each)
Focus Criteria Points
1. Introduction Greetings with smile, impressive opener, topic statement, and preview
2. Body Organization Clear points with appropriate sequence words. Coherence of contents, logical development
3. Conclusion Signal phrase, review, and effective closer, asking questions whether the audience understand, requiring audience’s questions, and greetings
4. Extra linguistic Qualities Eye contact, projecting voice, appropriate posture, proper gestures
5. Speech Management Using note-card or PPT, timing
6. Content quality Informative, interesting, and inspiring contents
7. Language quality Interesting, stylistic, and various vocabulary, phrases and rhetoric
8. Practice & Preparation Not reading the script; overall effort; enough practice, not using fillers, and other preparation factors in brainstorming and outlines
9. Fluency Natural intonation, appropriate pronunciation, and proper speed
10. Grammar use Grammar use in a up-loaded presentation script

Total:
APPENDIX 2

Survey for Criteria Awareness

Please indicate number of points according to how much you agree or disagree. Strongly agree: 5 points, Agree: 4 points, Neutral: 3 points, Disagree: 2 points, and Strongly disagree: 1 point.
Awareness Level Questions Points
1. Adaptation Level 1. The course is compulsory, so I have no choice but to register.
2. I do not like English, but I have to reroll in this course.
3. I study English because I have to graduate.
4. I learn English because English will be helpful for the job.
5. I think I did all the required activities in class.
6. I found instruction about evaluation criteria.
7. I think the criteria is appropriate for evaluation.
8. I give up easily when I feel English is difficult.
2. Stability Level 9. After a while, I felt studying English was ok in this course.
10. Knowing the criteria is essential for me to obtain better grades.
11. I found what I learn in this course would be helpful for the future.
12. I gradually found the requirement of the course is the learning process of English.
3. Approving Level 13. I think my presentation will be impressive if I practice more.
14. The criteria are helpful when I practice my presentation.
15. I feel confident I can deliver presentations during this course.
16. I am sure I can do all the required activities according to the criteria.
17. I feel sure I can pass my exam with good grades.
18. I think I will have more friends if I make a better presentation.
4. Sensing Accomplishment Level 19. I think the criteria of the course is helpful.
20. I believe delivering a presentation up to criteria is beneficial.
21. I believe learning English is impressive.
22. I think I will be aware of the criteria in the future.
23. I feel confident I can deliver presentations after this course.
24. I believe learning English is fun.
25. I think I get to know a new way of learning due to criteria.


ABOUT
BROWSE ARTICLES
EDITORIAL POLICY
FOR CONTRIBUTORS
Editorial Office
#1219, Bugak building, Kookmin University,
Jeongneung-ro 77, Seongbuk-gu, Seoul 02707, Korea
E-mail: stem@stemedia.co.kr                

Copyright © 2021 by The Society for Teaching English through Media.

Developed in M2PI

Close layer
prev next