Search Results

Ipsos MORI was commissioned by the Department for Education (DfE) to carry out research on the effects of the English Baccalaureate (EBacc) on schools, parents/carers and pupils. The EBacc was introduced as a measure in the 2010 school performance tables. The EBacc is not a qualification in itself, but it is the term applied to the achievement of GCSEs at grades 'A*'-'C' across a core of academic subjects: English, maths, history or geography, the sciences and a language. The research included both a survey and qualitative research. We surveyed 618 teachers with some involvement in Year 9 GCSE subject choices, drawn from a representative sample of English secondary schools.

Excerpt from publication. Show less

This study empirically addresses the claim made by Gibbons et al (The new production of knowledge: The dynamics of science and research in contemporary societies. Sage, Thousand Oaks, 1994) that a novel form of quality control (associated with Mode 2 knowledge production) is supplementing the ‘traditional’ peer-review process (associated with Mode 1 knowledge production). A qualitative design was used to explore faculty members’ views on the criteria for assessing scientific research. Ninety-four semi-structured interviews were conducted with biomedical scientists, clinical scientists, and social scientists working in Canadian universities. Results show that the vast majority of participants are aligned with the ‘traditional’ Mode 1 peer-reviewed procedures for assessing research and defining scientific excellence. These participants asserted that peer review is the best quality control mechanism for assessing scientific research, and peer recognition the key attribute for legitimacy in the academic arena. In contrast, participants ascribed a low value to non-academics’ judgment of their work. While the study findings do not provide support Gibbons et al.’s claim, they add to a growing body of evidence that supports the continuing importance of peer review in academic career success.

Published abstract reprinted by permission of the copyright owner. Show less

The comparability of qualifications receives much attention in the UK and there are established methods for conducting post-hoc comparability research. In contrast, this study explored the feasibility of building research into specification development. The aims were to compare the affective, cognitive, interpersonal, metacognitive and psychomotor demands of new and established units within the same subject, and to identify the relevance of each domain to each unit. Six experts judged which unit (new or established) was the most demanding in each domain. The specifications were qualitatively analysed to explore the relevance of each domain to each unit. As standards reside at qualification level, differences in unit demands may not mean there were differences at qualification level. There were 11 demands-related differences between units. A demands-related difference in a domain which was very relevant to both units was considered more important than a demands-related difference in a domain that was not particularly relevant to the units. For instance, a new unit was more metacognitively demanding than an established unit and the metacognitive domain was very relevant to both units. So the awarding body might consider decreasing the metacognitive demands of the new unit. Demands are complex and judging demands is subjective. Despite these complexities, the study provided credible research evidence about demands. Such studies help to target units in development at the same level of demands as established units. Following the research, the new specification was amended and accredited by Ofqual.

Published abstract. Show less

The National Security College of the Australian National University has commissioned this report to examine the following areas: (1) the place of authentic learning in postgraduate education; (2) the methods used to simulate policy development, and related activities like strategy planning, in educational environments relevant to national security policy-making; (3) policy simulation methods that could enhance learning at the college and similar institutions, including teaching skills, technology and resource implications; (4) the roles of technology in enhancing learning in policy development simulations; and (5) ways to assess learning effectiveness through simulation. The aim of this report is therefore to provide the college with an evidence-based case and examples of practice that illustrate the use of authentic learning and simulations in postgraduate and on-campus learning environments relevant to applied national security policy education. The report presents the findings of a literature review alongside a set of selected examples of authentic and simulation-based learning. The main conclusions are as follows: there is a place for authentic learning in postgraduate education; there are many methods that over many years continue to be successfully used to simulate policy development in areas relevant to national security policy-making; and there are a number of proven policy simulation methods that could enhance learning at the college and similar institutions which, like all learning methods, have their own resource and professional development implications. Findings in relation to the roles of technology in enhancing learning in policy development simulations and assessing learning effectiveness through simulation are also discussed. In conclusion, the report makes seven recommendations that indicate the significant opportunities that authentic learning and simulations offer to the college as well as some of its challenges.

Excerpt from publication. Show less

The aim of this article is to demonstrate the importance of local knowledge for the design of assessment tools intended to develop and enhance human resources in organizations. With respect to the standardization of assessment procedures brought about by the application of universal and global reference models, the article uses a case study to illustrate the potentialities and shortcomings of using a local approach. After a brief survey of the main theoretical frames of reference, the article describes how local knowledge was used at an assessment centre set up at a multinational operating in the maritime transport sector, with a view to developing the competences required by the company. The article describes and discusses the ways in which certain classic knowledge management situations were adapted at the assessment centre to local needs and conditions, and the main results obtained. Finally discussed are the practical and social implications, as well as the limitations and the transferability, of a local approach to assessment.

Published abstract reprinted by permission of the copyright owner. Show less

The purpose of this study is to explore teachers' opinions with regard to the value and process of grading within a competency based training (CBT) framework, following the introduction of a formalised grading system at a Melbourne TAFE institution. The data was gathered using a 16 item Likert-style survey that allowed for comments. Basic descriptive statistics were used for the analysis. The results indicated that for this sample, teachers were evenly divided as to their perceived value of grading and the processes by which grading should be done. Even though teachers had undergone some training in the writing of rubrics, uncertainty and doubt still existed with regard to grading.

Published abstract. Show less

The Swinburne University Apprentice Skills Assessment (SUASA) is a recently developed generic skills measure using a six-factor model containing 37 items based on industry underpinnings. The aim of this study was to empirically validate this measure with a sample of apprentices in the traditional trades. Four hundred and fifteen males and 29 females completed the SUASA in a paper and pencil format. A factor analysis of the data showed only evidence for two of the six factors as originally conceived (communication, and teamwork). Other factors (self-management, problem solving, learning, and initiative and enterprise) did not adequately represent the pattern of responses by this cohort. The potential use of the generic skills scale is discussed including directions for redeveloping the scales that did not fit with the original design.

Published abstract. Show less

This paper reports on the work done in preparation for the OECD 2012 Programme for International Student Assessment (PISA) survey of mathematical literacy. The OECD PISA international survey for 2012 is based on a new framework. This features an improved definition of mathematical literacy; the separate reporting of mathematical processes involved in using mathematics to solve real world problems; and a computer-based component to assess mathematical literacy as it is likely to be encountered in modern workplaces. Issues that arise in the preparation of an assessment for use in many countries around the world will be illustrated with some items and results from the recent international field trial.

Author's abstract. Show less

The goal of this paper is to establish the case for conducting research on the readiness of individuals for the workforce as part of the Workforce Readiness Initiative at [Educational Testing Service] ETS, with specific emphasis, at least initially, on non-cognitive indicators of readiness. We begin by defining a conceptual framework that encompasses non-cognitive constructs and measures, followed by a brief review of the literature highlighting the importance of non-cognitive predictors in education and the workforce. Next, we examine the importance of research on workforce readiness and consider how ETS can conduct workforce readiness research. Finally, we give an overview of work accomplished in this area at ETS, include an action plan of workforce research in progress, and summarise future planned directions.

Published abstract. Show less

In 2008, a pilot project with apprenticeship as an alternative pathway of upper secondary vocational education (USVE) started in Sweden. This paper is based on a follow-up study 2009-2011, funded by the Swedish National Agency for Education (SNAE) focusing on pedagogical aspects of apprenticeship. We interpreted 'pedagogical aspects' as concerning teachers' curriculum work, i.e. the organisation and design of apprenticeship in relation to following aspects: division of labour between school and work; the content; the roles of teacher and supervisor; and follow-up and assessment of students' development regarding vocational knowing in relation to the Swedish national curriculum for each of the programs. In this article, we present results related to assessment in upper secondary VET apprenticeship. The issue for this article is to illuminate conditions for assessment in USVE-apprenticeship, the foci of assessment and the tools used for assessment. According to regulations of apprenticeship, trilateral assessment sessions (teacher-supervisor-apprentice) are to be held regularly for follow-up and, at the end of a course, marking students knowing. As a representative for the education authority, the teacher is responsible for marking, but the supervisor is to provide the teacher with the information needed for follow-up and marking. The assumption is that during these sessions, there will be a dialogue between supervisor, apprentice, and teacher that will contribute to both follow-up and marking. The results presented in this paper build on interviews with teachers, apprentices and supervisors, as well as some audio-recorded trilateral sessions for assessment. Our preliminary results show that these trilateral sessions are one of the critical situations in USVE-apprenticeship. A main result is that assessment was either focused on social and behavioural aspects or on vocational knowing. Which one that dominated was related to firstly, the qualifications of the teacher; secondly, to the division of labour on the workplace; and thirdly, to the local history of USVE-apprenticeship. These results and the consequences of them will be further developed in our article.

Published abstract. Show less