Abstract
High-stakes tests, such as exit-level examinations for school leavers, demand fairness in every respect. The examination of first languages, referred to as home languages (HL) in South Africa, at the end of grade 12 is a particular case in point. Not only have several reports indicated that the language papers treat learners across the various languages unequally, but there is also no immediately apparent remedy for their shortcomings. Reports by the Council for Quality Assurance in General and Further Education and Training (Umalusi) and the Department of Basic Education show patent discrepancies between national averages per language and province, with variations in excess of 10%, as well as average pass rates of as high as 99% in the case of at least eight of the home languages over a four-year period (2009–2012). Concerns have also been raised as to whether the prescribed format and content of the examination papers are adequately aligned with the objectives of the curriculum.
In this context, it is understandable that the statutory body tasked with the oversight of the quality of these examinations, Umalusi, would be concerned that the 11 home languages for which examinations are written have papers whose quality and effects are comparable and fair. In order to address this problem, Umalusi has commissioned a number of research projects. This paper reports on one such project being undertaken by a four-university partnership in language testing research to design a deliberate solution for the lack of equivalence between the HL papers. The Inter-Institutional Centre for Language Development and Assessment (ICELDA) is a pool of expertise bound together in a partnership between the Universities of Pretoria, Free State and Stellenbosch and North-West University that develops and designs language tests and courses (ICELDA, 2013).
As a first step towards introducing greater comparability of standard between the language papers, conceptual clarity is sought on the construct on which the exit-level examination papers should be premised in accordance with the objectives and specifications of the new Curriculum and Assessment Policy Statement (CAPS). This is considered to be a key aspect in the process to validate any language test or examination. Concerns about the way HL ability is examined have existed for some time. Even before the introduction of the outcomes-based National Senior Certificate, which replaced the previous grade 12 exit qualification in 2008, assessment practices were identified as an area of contention in the 2007 report on “Making educational judgements: Reflections on judging standards of intended and examined curricula”(Umalusi 2011). From reports on the quality of the grade 12 examinations issued since then, it is clear that the emphasis up to now has been on the determination of standards and the compilation of comprehensive and revised curriculum statements. However, insufficient attention has been devoted to the design of the three HL examination papers and the abilities that they assess. As a result, the various examinations cannot be compared across languages or different years of administration.
In view of the fact that the predominant purpose of the grade 12 school-leaving examination is to measure the achievement of curriculum objectives and subject content covered during the school year, it is imperative that the HL papers be aligned appropriately with the curriculum and that they are founded on the same philosophical principles. There is no doubt that the conceptual framework that underlies CAPS goes back to linguistic ideas originating in the early 1970s on a differentiated communicative competence that supports actual language use by varied repertoires of functionally defined language acts. In their subsequent development, these constitute socially informed ideas about language that have broadened our perspective of what constitutes language ability and language use – that mastery of language is much more than having a grammatical command of it. It should, therefore, not be surprising that CAPS refers to the teaching approaches underlying it as communicative and text-based, in line with current international thinking about language use and language mastery. Although all the HL examination papers are based on the same curriculum, it is clear from an initial analysis of examination tasks that there is a lack of conceptual clarity on what the focus of the papers should be. In other words, the HL papers do not reflect the same understanding of language and do not measure the same constructs.
The word construct in language testing is a technical term that refers to the overall ability or trait being measured, or a theoretically defensible definition of what is to be assessed. Since language is a complex phenomenon incorporating heterogeneous types of language, rather than a singularly identifiable object, various abilities and different kinds of knowledge are involved simultaneously in the execution of any language act. There are also normative principles that guide language use in different contexts. One therefore needs to distinguish between the language situation itself and the conditions for using language in that situation. Such a distinction is important when articulating any construct. An important descriptor in CAPS is the required assessment of “high knowledge and high skills”, signalling the assessment of an advanced language ability. It is therefore clear that the distinction between the First Additional Language (FAL) and HL examination papers relates to the measurement of more advanced language than basic interpersonal communicative skills (BICS). Moreover, in addition to developing a cognitive academic language proficiency (CALP), HL students also need to display a professional language proficiency for employment purposes and access to trade and industry. This level would require greater proficiency than conversational language ability, but not necessarily the same kind of academic language ability referred to in CALP. The three levels identified above – social, academic and professional – apply to a number of different fields of discourse. Examinees writing the HL papers should therefore be able to handle different types of discourse and display an advanced command of features of language across them.
Language operates in particular contexts and lingual spheres, relating to what Halliday refers to as fields of discourse. These spheres may be considered material, since they are governed by “typical norms and principles that give a different content to the factual language used” within a situation. Consequently, distinct lexical and syntactic differences can be discerned in the language used in diverse contexts. The use of different dialects within communities and varying tones of voice to convey meaning further illustrate this. The norms that regulate the lingual spheres are typical because they apply to social forms or relationships that require a typical kind of language use within a particular temporal and structural context. Human beings as the users of language fulfil a subjective lingual role in which they then select from a repertoire of already developed registers in order to express themselves through language in ways deemed appropriate to a given situation. There are “normative principles of a logical, aesthetic, legal, technico-formative, economic, social, ethical or confessional nature” that guide and stamp language use in different contexts. One therefore needs to distinguish between the language situation itself and the conditions for using language in that situation. Such a distinction is evidently important when articulating any construct related to high language ability.
Humans seem to have an inherent ability (a communicative competence) to recognise and use different varieties of language. Vocabulary plays a role in distinguishing between material lingual spheres, but is insufficient on its own because in different contexts language is qualified by various aspects of our experience. Each sphere has a typical language of its own. Moreover, the social structure in which the language is employed is responsible for further distinctions, for example different roles we assume when using language as writers or readers, or as speakers or hearers. In terms of the content of the language curriculum outlined in CAPS, the dominant material lingual spheres of relevance to the teaching and assessment of the respective languages would seem to be the following:
- social (including inter-personal communication and the handling of information)
- economic/professional (including the world of work and commerce)
- academic (including academic and scientific language and advanced language ability)
- aesthetic (including the appreciation of literature and art)
- ethical (including an appreciation of the values embedded in language use)
- political (including the critical discernment of power relations in discourse).
On the basis of the lingual spheres identified in the curriculum and the stated learning outcomes a proposed formulation of the general underlying construct for the HL examination papers would be the following:
The assessment of a differentiated language ability in a number of discourse types involving typically different texts, and a generic ability incorporating task-based functional and formal aspects of language.
The conceptual framework and primary construct such as that proposed here enables us to devote our attention to the articulation of the construct in a selection of language papers by examining the task specifications reflected in the papers and marking memoranda, so as to be able to express an opinion on whether these are sufficiently representative of the curriculum and whether the same abilities are being measured across the language groups. Any underrepresentation of tasks that assess high language ability will undermine the validity of the examination.
Inasmuch as the curriculum and assessment standards may help to organise what should happen in the classroom, they provide no guarantee of contributing towards the quality of education and assessment practices, or of ensuring equivalence across different language examinations. One of the main contributing factors to the varying standards and quality of the HL examination papers identified so far is the fact that important norms and principles in language testing have not been applied to the design of the papers, an issue that will be covered in detail in subsequent studies. The extent to which examiners are adequately trained in the principles of construct validity and reliability in language assessment thus needs investigating.
Without denigrating the importance of standards and curricula, the emphasis needs to be shifted from setting standards to ensuring accountability, which can be defined simply as a way of explaining that all measures undertaken may be considered appropriate and necessary. This aspect of accountability in language teaching and assessment is directly related to defining the underlying construct and articulating it in the form of detailed specifications based on defensible theories of language and communicative competence.
Keywords: CAPS; construct validity; grade 12; home language; language assessment; National Senior Certificate
The post The assessment of home languages in the South African National Senior Certificate examinations – ensuring fairness and increased credibility appeared first on LitNet.