QUALITY ASSESSMENT IN TEACHING ENGLISH FOR SPECIFIC PURPOSES

Galina Kavaliauskienė

Mykolas Romeris University

Ateities St. 20, Vilnius LT-2057, Lithuania

Quality is never an accident, it is always the result of high intention,

sincere effort, intelligent direction and skillful execution,

it represents the wise choice of many alternatives.

William A. Foster

Abstract

Quality of learning and teaching depends on a number of factors, and the most important are learner self-assessment and formal assessment, or testing, monitoring learners success and accomplishments, teacher development and teacher evaluation by learners, colleagues or authorities.

Quality assurance can be implemented through effective learning, which is directly related to effective teaching because good teaching nurtures many aspects of learning.

This paper addresses issues of quality assesment in teaching / learning English for Specific Purposes (ESP) and offers some recommendations for learner and teacher development.

Key words

Quality Assessment in Teaching ESP, Learner /Teacher Evaluation, Self-Assessment / Formal Assessment.

 

Introduction

Leading universities worldwide are committed to high quality of teaching and learning. Quality needs the maintenance and ongoing development. Academic staff and students are engaged in teaching and learning that meet professional needs, are critical and innovative, make appropriate use of Information and Communication Technology (ICT), use learning experience and resources to support good teaching and learning practice, and, finally, monitor and evaluate teaching and learning outcomes to maintain quality.

Responsibility for monitoring the quality of teaching and learning lies with each University. Faculties and Departments are responsible for ensuring quality in teaching and learning. Quality assurance in teaching itself resides with teachers whose role in enhancing learning is indisputable.

This paper focuses on the ways of assessing quality in teaching and learning English for Specific Purposes (ESP) and offers some tips on good practice in enhancing quality.

Literature review

There is no single or straightforward way of teaching people to learn. Learners state that learning takes place in a variety of teaching styles and settings by means of varies strategies. However what goes on in learners brains and how information is memorized and activated still remains the mystery.

The website of Melbourne University, Australia, describes some features of effective teaching [1]. According to this source, effective teaching a) is focused on learning outcomes for students, in the form of knowledge, skills and comprehension; b) is coherent in the integration of objectives with teaching procedures and assessment; c) ensures the clear communication to students of requirements in which they can achieve their potential; d) promotes the development of co-operative learning with peers; e) encourages learners to develop autonomous learning skills by providing various tasks; f) respects students responses and views; g) encourages learners feedback on teaching; h) takes into account students self-assessment; i) monitors learners progress through formal assessment, i.e. testing.

Good teaching enhances many aspects of learning. However, effective personal learning also depends on learners attitudes, values and responses. Students are teachers partners in the educative process and are largely responsible for their own learning. Students who understand goals and standards are likely to have better learning outcomes.

Effective adult learning has the following features [2]: a) it is autonomous and self-directed; b) it is fostered by cooperation and interaction with peers; c) it has a lifelong orientation; d) it values individuality and persons interests; e) it is critical of what is being studied.

The central element in the overall quality of teaching and learning is assessment. In higher education, well designed assessment sets clear students expectations, establishes a reasonable workload, and provides opportunities for students to self-monitor, practice and receive feedback [2]. Assessment in higher education must serve a number of purposes: guide students approaches to study, provide learners with feedback on their progress, judge their performance and guarantee academic standards.

Assessment criteria and standards influence the quality of students learning. American Association for Higher Education (AAHE) has formulated 9 Principles of Good Practice for Assessing Student Learning [2], the most important of whish are: a) assessment of student learning begins with educational values and serves for educational improvement; b) assessment is most effective when learning is understood as multidimensional and revealed in ongoing performance; c) assessment requires attention to outcomes and the experiences that lead to those outcomes; d) assessment works best when it is ongoing; e) assessment fosters wider improvement when the educational community is involved in it; f) assessment makes a difference when involves useful issues that students care about; g) assessment is most likely to lead to improvement if changes in learning are promoted. In assessment, students value and expect transparency in evaluation of their knowledge, i.e. they expect feedback that explains the grade they have received and suggestions for how they can improve their performance. Students also prefer to have options in assessment by arranging timetable for submitting work it helps learner become more autonomous and independent.

According to K.P. Mohanan [3], teaching at university level reveals the following characteristics: teaching is the activity of lecturing to students, teaching is the activity of transferring a body of knowledge to students, and teaching is the activity of causing students to learn something, i.e. the activity of facilitating learning. Both lecturing and knowledge transfer are important components of learning process. However, if the teaching activities do not result in learning, there has been no teaching. Likewise, if the learning is lacking in quality, the teaching is unsuccessful.

Learner self-assessment helps learners think about their own progress and find ways of changing, adapting and improving. Many language learners, particularly the most successful language learners, regularly engage in self-assessment as part of their learning [4].

The motives for introducing self- and peer- assessment vary and usually include the practical impossibility for teachers otherwise keeping effective track of all their students changing learning needs. Further motives include recognition of self- and peer- assessment as essential components of successful learning beyond teaching institutions, and the aim to make students more responsible for their own learning, through developing reflective learning habits. In other words, self-assessment enables students to monitor their own progress from their own work, and peer-assessment helps to improve learning and to develop social and cooperative skills.

Self- and peer- assessment should be integrated into language courses for learning purposes because 1) self-assessment is a prerequisite for a self-directed learner; 2) it can raise learners awareness of language, effective ways of learning and their own performance and needs; 3) it increases motivation and goal orientation in learning; 4) in language learning, learners efforts and beliefs can only be assessed through self-assessment; 5) it can reduce the teachers workload [5].

Teacher Assessment (TA) is a broad term for various procedures used to study teachers classroom performances. Other terms sometimes used include evaluation and appraisal [6].

Interestingly, few institutions have systematic teacher-appraisal systems, and where these do exist, they are very often for hiring and firing purposes rather than to assist professional improvement and learning. The effect may therefore be stressful and demoralizing rather than helpful [7].

TA formats and instruments vary widely and include a variety of procedures: supervisor visit, videotaped class, self-assessment (by teachers themselves) or assessment by authorities, written reflective journal, collegial observation, evaluation of teaching by students, class discussion, etc. [6]. Research into the benefits of TA has shown clear tendencies. Groups of experienced teaching staff and teachers-in-training ranked improvement of students learning and professional development as the major benefits of TA. Top assessment preferences listed by experienced teachers are: 1) student evaluations, 2) collegue observation, 3) supervisor visit, 4) teacher self-assessment.

Internal assessment can be carried out by lecturers within each Faculty of University. The ideas and strategies in the assessing learning support three interrelated objectives for quality in students assessment in higher education, namely 1) effective approaches to learning, 2) measuring learning outcomes validly and reliably, 3) appropriate grading that protects academic standards [1].

Assessment of teaching quality results in changes of: 1) Curriculum Design, Content and Organization, 2) Teaching, Learning and Assessment, 3) Student Progression and Achievement, 4) Students Support and Guidance, 5) Learning Resources, 6) Quality Assurance and Enhancement [8].

Since 1995, British Universities have started grading teaching and learning on a scale of one to four: one for low, and four - for high. External Quality Assessment is conducted by the Quality Assurance Agency, which since 2002 has moved to a system of periodic Institutional Audits [9]. Institutional Audits include quality assurance and self-evaluation procedures.

Teacher assessment (TA), or Teacher Evaluation, is a complex process, which consists of a series of activities and actions. Teachers have to be evaluated as professionals. The emphasis of teacher evaluation should be on their teaching and not individuals and take into account the involvement and responsiveness of people involved in the education process. The purpose of teacher evaluation is to safeguard and improve the quality of instruction received by students by fostering self-development. Evaluation in higher education entails gathering evidence about the impact of teaching, topic and course design on students participation and achievement and the appropriateness of content and procedures.

Effective self-evaluation of teaching is the basis of good educational practice. A good starting point is the Teachers Perspectives Inventory (TPI) [10]. It is a short free of charge questionnaire online designed for teachers. The questionnaire summarizes teachers views and perceptions about teaching. Each teacher can explore his / her assets and analize ones own teaching. On the evaluation data a number of important decisions can be based, i.e. changes in course structure, in teaching processes, in course content, changes to assessment tasks, students workload, and staff development.

There are five teaching perspectives in individual profile: Transmission, Apprenticeship, Developmental, Nurturing, and Social Reform [10]. Perspective of Transmission refers to commitment to the subject matter. Perspective of Apprenticeship refers to guiding learners into new ways and norms of working. Developmental Perspective refers to bridging knowledge that is meaningful to the learner. Nurturing Perspective refers to long-term, persistent efforts to individual growth and achievements. Social Reform refers to changing society. Research shows that the vast majority of teachers hold one or two dominant perspectives. Many hold an additional back-up perspective. The combination of dominant and back-up perspectives allows teachers to accommodate changes in context, content, and learners. Profile sheet has sub-scores for Beliefs, Intentions, and Actions within each perspective. Sub-scores help to identify individual philosophy of teaching by highlighting whether views are grounded in what one believes, what one intends to accomplish, or what educational actions one undertakes in specific teaching settings [10].

Evaluation of teaching quality by students helps teachers to reflect upon their teaching. Student ratings of teaching have the potential to contribute positively to improvement of teaching [11]. When teachers review their teaching in the light of the student feedback, it is important to reassure students by giving them feedback about their concerns, complaints or suggestions, so that learners know that changes would be made as a result.

In this paper, the data on learners self-assessment of learning various Language for Specific Purposes skills have been described. At the end of course students perceptions on success in learning and quality of teaching have been surveyed. Data on evaluation of teaching by students have been presented. Implications of the research findings for teacher self-development have been discussed.

Research data and discussion

The research into learner self-assessment in the English for Specific Purposes classes goes back to 2000. University students of various faculties answered questionnaires or were interviewed individually on various occasions. The total number of respondents has been 250 so far.

Learner self-assessment

 

Students judgements on his / her own learning and academic accomplishments have been recorded over the last five years. Each year a short questionnaire on language skills was administered (Appendix 1). The data presented in this chapter summarize the findings since 2000.

Chart 1. Learner self-assessment of language skills.

Learners perceptions of reading, listening, speaking and writing skills are displayed in chart 1.

Each language skill is shown by 4 horizontal bars each of which presents different proficiency in the skill. The lowest bar (light green) presents very good evaluation, the second (light blue) good, the third (violet) satisfactory, and the upper bar (red) weak. All in all, majority of students assess their language skills as good: reading 48%, listening 40%, speaking 60%, writing 40%. Slightly over the third of learners think their listening and writing skills are satisfactory 35% and 38%, respectively. Only 20% of students assess their speaking as satisfactory, and 10% - satisfactory reading skills. Some students possess weak productive skills of writing (20%) and speaking (10%) and receptive skills of listening (15%) and reading (2%). Very good skills of reading (40%), listening and speaking (10% each), and only 2% of writing completes the description of chart 1.

Testing these language skills (testing data are not reproduced in this paper) has shown a good agreement with self-evaluation results. This indicates that students are aware of their achievements, strengths and weaknesses and evaluate themselves realistically.

Monitoring learner progress

One of the most important factors in effective learning is monitoring students progress. Learners not only need to recognize their lacks but also figure out their accomplishments.

Recently there has been a new development highlighting the students success experiences. Success is bound to lead to the enhanced motivation and the confidence boost [12] and is vital in the process of language learning by fostering learners positive attitude.

Chart 2 displays the results on learner self-assessment of success experiences. Learners have been most successful at making presentations, learning professional vocabulary and translating from English into the native language. This is diagrammed below: 80% of learners are proud of their oral presentations, and 90% feel their performance in ESP vocabulary tests has been praiseworthy. As many as 85% of students are sure they are good at translating professional texts from L2 into L1. However, translation from the native tongue into English remains problematic to many learners [13].

Chart 2. Success experience in learning English for Specific Purposes.

 

Evaluation of teaching quality

As it has been mentioned in the literature review, evaluation of teaching quality by students helps teachers to reflect upon their teaching, and learners ratings of teaching have the potential to contribute positively to improvement of teaching [11].

The findings in this section are the result of administering the Course Experience Questionnaire at the end of the course. The questionnaire was anonymous in order to ensure the reliability of responses. Quiestionnaire (Appendix 2) consists of five questions with suggested multiple choice answers. Assessment or evaluation of teaching covers five aspects: quality of teaching (question 1), goals of teaching (question 2), testing or evaluating students performance (question 3), students workload (question 4), and learners overall satisfaction with the ESP course (question 5).

Suggested ratings included evaluation of teaching from good to bad, goals from clear to vague, assessment from appropriate to inappropriate, workload from too low to too high, and satisfaction with the course from good to bad. Specification of other has also been introduced into the questionnaire. Surprisingly, respondents have missed the chance to express their individualized attitudes to the course and have chosen the provided answers.

For the convenience of reading this article, the shortened version of the questionnaire is also reproduced here. An administered to respondents questionnaire contained Likert scales of statements for each item in accordance with [14], i.e. strongly agree, agree, not sure, disagree, strongly disagree.

1 Teaching was a) good, b) satisfactory, c) bad.

2 Goals of teaching were a) clear, b) vague, c) not clear.

3 Assessment was a) appropriate, b) inappropriate.

4 Workload was a) too low, b) appropriate, c) too high.

5 Overall satisfaction with the course is a) good, b) bad.

There were 32 students who completed the ESP course in spring 2005. Evaluation of teaching by these students is displayed in chart 3. Results are shown by double bars for each question. The lower bar represents the favourable response, the higher bar unfavourable response.

Chart 3. Evaluation of teaching quality.

According to this chart, 30 students found teaching good and 2 students satisfactory (first two columns at the bottom of the chart). Similarly, 30 students thought that the goals of ESP course were clear, while 2 students disagreed (second double bars). Formal assessment of students knowledge of ESP skills was appropriate to 30 learners and inappropriate to 2 learners (third pair of bars counting from the bottom). 28 students considered the workload appropriate, while 4 students thought it was too high (fourth pair of bars). Majority of 30 learners expressed their satisfaction with the course, while 2 students were dissatisfied with it.

According to [15], a teacher is reasonably good if at least half of students think him good. This gives me a feeling of certain satisfaction of having completed the course successfully, but also makes me reflect upon failure to give satisfaction in learning to some respondents.

Teacher self-development

Effective self-evaluation of teaching is the basis of good educational practice. I have used the TPI questionnaire online to assess my teaching profile. The data are presented in Appendix 3.

The description of what each perspective represents was given in the section of literature review.

It should be noted that the Teaching Perspectives are not personality-based styles. The height of each vertical bar represents how strongly a person holds each of the five perspectives.

Scores on the profile sheet can range from a minimum of 9 to a maximum of 45. Most people have at least one (occasionally two) dominant perspective that represents strongly held views on their roles and functions as educators [10]. Similarly, most people have one or two back-up perspectives that are also high, but less than a dominant perspective.

In my TPI profile, the dominant perspective is Nurturing, and back-up perspective is Apprenticeship. The recessive perspective in my profile is Social Reform. This is quite natural because in teaching English I am not concerned with changes in society.

The sub-scores labeled B, I, and A are printed out near the top of each bar. The sub-scores are indicators of how much agreement exists between what teachers do (Actions), what teachers want to accomplish (Intentions), and why teachers feel that is important or justified (Beliefs).

High internal consistency (i.e. sub-scores are within one or two points of each other) means that Beliefs, Intentions and Actions all corroborate each other. In my profile, sub-scores are as follows: B:12, I:11, A:14 in Transmission Perspectives; B:11, I:13, A:14 in Apprenticeship Perspectives; B:12, I:11, A:12 in Developmental Perspectives; B:12, I:14, A:13 in Nurturing Perspective; and B:9, I:8, A:8 in Social Reform Perspective. The results certify the consistency of scores.

The reflection on my TPI profile suggests I have to pay more attention to Transmission and developmental Perspectives. My Social Reform Perspective is recessive. I need to do more thinking on how to develop this perspective.

Conclusions

The techniques of self-assessment and evaluation play important part in evaluating the effectiveness of individual learning, training learners for a life long learning and teacher self-development.

Learners need to assess their progress and accomplishments in order to plan their future learning. A great majority of students between 80% and 90% - believe their skills in presentations, mastering LSP vocabulary and translation are quite remarkable. Success experience helps students enhance their self-esteem and self-confidence.

Evaluation of teaching has shown that majority of learners found teaching good, goals clear, assessment appropriate, workload normal, and all in all are satisfied with the course. Learner evaluation of teaching is a good tool to encourage teachers to think over their methods, techniques, materials, teaching styles and failures. Self-assessment of TPI profile allows teacher to reflect on ways of improving teaching and self-development.

References

1 Assessing Learning in Higher Education. http://www.cshe.unimelb.edu.au/assessinglearning/

2 Assessment in American Association for Higher Education. http://www.aahe.org/assessment

3 Mohanan, K.P. Assessing Quality of Teaching and Learning in Higher Education.

http://cdtl.nus.edu.sq.publications/assess/

4 Dickinson, L. 1991. Self-Instruction in Language Learning. Cambridge: Cambridge

University Press.

5 Black, P. and William, D. Assessment in Education, vol. 5, No 1, March 1998.

http://english.unitechnology.ac.nz/resources/resources/classroom learning.html

6 Chandler, P.M. and Ortiz, S.B. Building Consensus on Teacher Assessment. English

Teaching Forum, vol. 42, No 3, July 2004, pp. 22-31.

7 Urr, P. A Course in Language Teaching. Cambridge: Cambridge University Press.

8 Audit at Bristol University. http://www.bristol.ac.uk/tsu/ext quality

9 Quality Assurance at Cambridge University. http://www.cam,ac.uk/cambuniv/learning

10 Pratt, D. and Collins, J. Summary of Five Perspectives on Good Teaching.

http://teachingperspectivescom/html/tpl frames/

11 England, J., Hutchings, P. McKenachie, W.J. The Professional Evaluation of Teaching.

American Council of learners Societies Occasional Paper No 33. http://www.acls.org

12 Henry, P. The Success Experience in Language Learning. HLT Magazine. March 2003.

http://www.hltmag.co.uk/mar03

13 Kavaliauskiene, G. Research into Reading-Writing Connections in ESP. The Internet

Journal ESP World, January, 2004. 16 pages. http://esp-world.info/Articles/GK1204.htm

14 Domyei, Z. Questionnaires in Second Language Research. 2003. Lawrence Erlbaum

Associates, Inc., Publishers. New Jersey. USA.

15 England, J., Hutchings, P., and McKeachie, W.J. The Professional Evaluation of Teaching.

http://www.acls.org

Appendix 1. Learner Self-Assessment Questionnaire.

1        How do you assess your reading skills? a) very good, c) good, d) satisfactory, e) weak.

2        How do you assess your writing skills? a) very good, c) good, d) satisfactory, e) weak.

3        How do you assess your speaking skills? a) very good, c) good, d) satisfactory, e) weak.

4        How do you assess your listening skills? a) very good, c) good, d) satisfactory, e) weak.

Appendix 2. Course Experience Questionnaire.

1 Teaching was a) good, c) satisfactory, d) bad.

2 Goals of teaching were a) clear, b) vague, c) unclear.

3 Assessment was a) appropriate, b) inappropriate.

4 Workload was a) too low, b) appropriate, c) too high.

5 Overall satisfaction with the course is a) good, b) satisfactory, c) bad.

Appendix 3. Individual Teaching Perspectives Profile.

View Summary of Five Perspectives on 'Good Teaching'

Information about the authour:

Kavaliauskienė Galina,

Associated Professor

Department of Foreign Languages

Mykolas Romeris University

Ateities 20, LT-08303

Vilnius

Lithuania

Tel. 370 85 2 714 613

Email: gkaval@mruni.lt

04.10.2006