Search Results

Essay-based examinations form a cornerstone of the public assessment system in the UK, but comparatively little research has been conducted into the processes involved in designing, answering and appraising essay-based questions/responses. This study explores the idea of examination validity through a qualitative review of these processes, based on materials sourced from the June 2002 [Oxford Cambridge and RSA Examinations] OCR Politics AS/A2 module 2596 (Politics of the UK). Through a detailed analysis of question papers and candidate scripts (informed by mark schemes, curricula and examiner reports), a model of current processes is advanced. Issues related to question design are highlighted, and a methodology for future quality assurance in essay-based examinations is derived from philosophical considerations such as 'fairness' and 'validity'. This paper resulted from research conducted as part of the University of Cambridge Local Examinations Syndicate's ongoing work to develop a system of procedural guidelines and example questions to facilitate robust question design.

Published abstract reprinted by permission of the copyright owner. Show less

Despite more than a decade of policy initiatives and encouragement to embrace recognition of prior learning (RPL) there has been an apparent reluctance, borne out in national statistics, to take up RPL. In 1992 the National Training Board issued the National Competency Standards wherein policy and guidelines identified RPL as an important element in competency based training. Supporting this stance in 1993 the Federal and State Ministers of Education declared the National Framework for the Recognition of Training (NFROT), committing educational institutions, at all levels, to provide recognition for the existing skills and knowledge of students regardless of how, when or where they were acquired. Fourteen years on the continuing reform agenda maintains the principles of RPL and a concern about its uptake. Recent responses to this concern include the Council of Australian Governments (COAG) agreement to fund RTOs to establish or improve recognition services and processes, and the Australian Quality Training Framework (AQTF) that positions RPL as an essential element of quality training and assessment. Through an interpretive research process 100 teachers and educational leaders shared dialogue expressing their views on the practice of RPL and impediments to its broader embedding in practice. The findings suggest that for many practitioners there are poor understandings across a range of issues and not limited to the prevalent view that tools and frameworks are the critical element, but also about competency units, the nature of evidence and its collection for assessment and a fear of making invalid professional judgements. This lack of individual confidence results in a reticence to provide recognition services. The data suggests that making clear the nature of evidence and its method of collection within the context of RPL may begin to redress the reticence of vocational education and training (VET) practitioners to engage in recognition practice.

Published abstract.

Available papers from the 11th Australian Vocational Education and Training Research Association (AVETRA) Conference are indexed from TD/TNC 92.459 to TD/TNC 92.499 and from TD/TNC 92.701 to TD/TNC 92.716. Show less

In the past decade there has been increasing interest in assessment as a social practice. There is a growing recognition that assessment, particularly of complex tasks, is a judgment which is socially-situated and thus contingent on a variety of factors which constitute the assessment event. Drawing on the hermeneutical tradition, this article explores the interpretive processes of essay markers in two Health Science courses at the University of Cape Town, with a particular focus on markers’ judgements about language. The analysis exposes multiple ‘circles’ of influence on markers’ professional judgements, including the texts (both the written product and the student writer), the marker’s interpretive framework, the interpretive community and the wider institutional discourses. The article contributes to a better understanding of the crucial role of the interpretive community in the validation of assessment interpretations.

Published abstract reprinted by permission of the copyright owner. Show less

This report summarizes the grading policies of 14 colleges and universities and how those policies have changed between 1980 and 1990. Grading policies and practices in the business, chemistry, education, English, history, mathematics, and psychology departments at these institutions and the grading orientation and practices of faculty are also summarized. The report concludes that there appears to be pressure on institutions of higher education and their faculties to reduce what the public perceives as lax standards that result in rising grade-point averages. While some institutions seem to be taking steps to respond to this perception by introducing additional curriculum requirements and making efforts to raise standards, others, especially institutions most in danger of losing students who receive low grades, are likely to feel increased pressure to consider the impact on student evaluations when assigning course grades.

Excerpt from website. Show less

In recent years, there has been considerable research activity focusing on the changing role of teachers, trainers and assessors in vocational education and training ( VET) contexts. However, equal attention has not been paid to the many general staff employed in a wide range of work roles in registered training organisations (RTOs). Like teachers and trainers, the work of general staff contributes to the enactment of VET polices and their work is continually being reinvented in the changing context in which VET now operates. However, unlike their teaching/training counterparts, their voices and experiences have been largely absent from research examining issues related to the VET workforce and the building of organisational capability in RTOs. The study reported in this paper is one attempt to address this significant gap in the VET literature. The paper analyses data collected from general staff as part of a national study examining careers in VET. It examines the ways in which general staff move into VET, how their working lives unfold over time and their plans for remaining in the VET workforce in the next five years. The paper pays particular attention to the ways the working lives of general staff differ from those of their colleagues employed as teachers and trainers and the types of support that general staff are able to access in order to further their career ambitions in VET.

Published abstract.

Available papers from the 11th Australian Vocational Education and Training Research Association (AVETRA) Conference are indexed from TD/TNC 92.459 to TD/TNC 92.499 and from TD/TNC 92.701 to TD/TNC 92.716. Show less

The purpose of this article is to introduce some of the concepts of Rasch measurement through an example in vocational education. Test data are reported for a typical class size group of 19 students on six criterion-referenced and competency-based assessment tasks. These were part of an objective structured clinical examination for nursing students. The results are then compared with those of a bootstrap replicated sample of 1,000 students. The results from the classroom sample are markedly similar to that of the bootstrap. It is argued that Rasch measurement does have application for criterion-referenced assessments but there can be substantive limitations when applying it to the typically small samples representative of most vocational education groups. Rasch measurement is not a mere statistical tool but the application of fundamental measurement principles; therefore, there are a number of theoretical issues involved that need to be considered before applying Rasch analysis in any situation and these are outlined.

Published abstract reprinted by permission of the copyright owner. Show less

This article will present findings from a doctoral study exploring the impact of 'SVQ Care: Promoting Independence (level III)' within children's homes. The study focuses on the extent to which SVQs [Scottish Vocational Qualifications] enhance practice and their function within a 'learning society'. A total of 30 staff were selected from seven children's homes in two different local authority social work departments in Scotland. Each member of staff was interviewed on four separate occasions over a period of 9 months. Interviews were structured using a combination of repertory grids and questions. Particular focus was given to the assessment process, the extent to which SVQs enhance practice and the learning experiences of staff. The findings suggest that there are considerable deficiencies both in terms of the SVQ format and the way in which children's homes are structured for the assessment of competence. Rather than address the history of failure within residential care, it appears that SVQs have enabled the status quo to be maintained whilst creating an 'illusion' of change within a learning society.

Published abstract reprinted by permission of the copyright owner. Show less

This article reports a descriptive study that depicted and investigated the use of assessment information by secondary vocational education teachers in making instructional decisions. Data was collected from 240 full time teachers at secondary vocational technical centre in West Virginia during the 1997-97 school year. The assessment information was generated from 6 types of student assessment methods. Specifically the study sought to describe selected teacher characteristics such as age, gender and teaching experience; teachers' perceptions of their use of student assessment data for making instructional decisions; and teachers' perceptions of their competence in the assessment process. The findings indicated that teachers had an average of 15 years of teaching experience; that information gathered from performance assessment was perceived to be the most useful in educational decision-making; and that teachers considered themselves very competent in all assessment areas except three. Show less

The uptake of [computer-assisted assessment] CAA in UK higher education (HE) on a large scale lags behind the expectations of CAA specialists. A research project was undertaken with the aim of discovering and addressing the underlying reasons for this. The research was conducted according to Strauss and Corbin’s (1998) prescription for grounded theory (GT) research. During three years a 200,000 word dataset was compiled from a national survey by questionnaire and interview with tutors, learning technologists, managers and [quality assurance] QA staff. This article describes a dual-path theory of CAA uptake that emerged from an analysis of the dataset, as given by Warburton in 2006. Tutors’ motivations and perceptions of risk were found to influence the way they use CAA and this is significant in credit-bearing applications where poor outcomes have long lasting effects on uptake. Institutions can benefit from using project risk management techniques to manage these risks. Ways in which this dual-path theory might be used to understand and improve CAA uptake are proposed.

Published abstract reprinted by permission of the copyright owner. Show less

This paper targets practical applications for vocational education and training (VET) teachers practicing in the area of community services around the theme of VET as an agent of change. The paper is a result of an action research project conducted to develop a model to reflect 21st century directions for VET practice in Australia for the Diploma of Community Welfare Work. The research found that specific aspects of delivery and assessment of the course could be improved. The paper gives an outline of the delivery and assessment processes developed to incorporate: (1) the workplace as the primary site for learning, skill development and assessment; (2) authentic learning tasks as a basis for learning; and (3) peer learning. This paper outlines the process whereby groups of students were placed within industry projects in order to achieve eight units of competence broadly based around community development. It also outlines delivery and assessment methods, which were designed around the units of competence and around the particular needs of the industry projects. The paper outlines a number of findings arising from the research project.

Published abstract. Show less