Search Results

This report is an executive summary of the ANTA funded project 'Mirrors and measures: report on the development of performance measures for teaching and learning in the vocational education and training sector' (TD/ANTA 82.57). Recommendations of that project were: VET organisations should promote and foster regular, formalised self-assessment for teachers and trainers; VET organisations should ensure ownership of the self-evaluation process by teachers and trainers; VET teachers and trainers should use the set of performance measures identified in the project to evaluate their teaching and training and to design and implement improvement strategies; a self-evaluation instrument should be developed from the performance measures that can be used by individual teachers and trainers and by teams. Show less

The purposes of this study were to determine if there is a significant difference in postsecondary business student scores and test completion time based on settable test item exposure control interface format, and to determine if there is a significant difference in student scores and test completion time based on settable test item exposure control interface format by gender. Results of the study indicate that there is no significant difference in postsecondary business student scores or test completion times based on settable test item exposure control interface format. When the variable gender was added, female postsecondary business students were found to achieve significantly higher test scores and to have significantly faster test completion times. Effect size and descriptive statistic analysis suggests that these differences by gender are too small to be of much practical difference.

Published abstract reprinted by permission of the copyright owner. Show less

Purpose: Relying on a design science paradigm, the purpose of this paper is to describe the development and evaluation of items for an ICT artefact that supports the assessment of transversal professional competences within the validation of prior learning (VPL). To do so, the authors build a conceptual bridge between the Occupational Information Network (O*NET) and the European Qualifications Framework (EQF). Design/methodology/approach: Design science research paradigm, in particular the participatory development of candidate items and their evaluation in a multi-stakeholder approach. Findings: The authors find that a self-assessment of professional competences should be comprised of 160 items in order to cover the breadth and depth of the O*NET in the hierarchical taxonomy. Such quantity of items sufficiently builds a conceptual bridge between the O*NET and the; EQF. Practical implications: When designing procedures for the VPL, it is imperative to bear in mind the purpose of the validation procedure, in order to determine relevant stakeholders and their needs in advance as well as the; required language proficiency of the assessment instrument. Social implications: The innovative value of this approach lies in the combination of an underlying hierarchical taxonomy with assessment items that are developed based on the qualification standards of different Austrian professions. Together with specific verbs that were adapted for each particular item, an innovative self-assessment is proposed. Thereby the authors aim to account for some of the mentioned shortcomings of the EQF. Originality/value: This paper applies a design science paradigm to develop an ICT artefact that should support the VPL. By reflecting on the design process, the authors introduce a theoretical bridge between the O*NET and the EQF. Thereby the authors aim to account for some of the mentioned shortcomings of the EQF.

Published abstract. Show less

This investigation examined the relationship between attendance and examination performance. Psychology students (179) took A/S psychology modules; their results were correlated with their attendance up to the examination date. The data were subjected to a Pearson Product Moment Correlation and a statistically significant result was found (p =.221**, significant at 0.01). The influence of an Educational Maintenance Allowance (EMA) on attendance and performance was also assessed. Generally it was found that EMA students do attend classes more regularly than non-EMA students. However, there was little difference between EMA and non-EMA students’ examination performance rates. Serious concerns are raised about the overall effectiveness of EMA as a means of promoting student examination performance.

Published abstract reprinted by permission of the copyright owner. Show less

Across multiple societal sectors, demand is growing to measure individual and group competencies. This paper unpacks Hartig et al.'s (2008) competency definition as a complex ability construct closely related to real-life-situation performance to make it amenable to measurement. Unpacked following the assessment triangle (construct, observation, inference), competency measurement is exemplified by research from business, military and education sectors. Generalizability theory, a statistical theory for modeling and evaluating the dependability of competency scores, is applied to several of these examples. The paper then pulls together the threads into a general competency measurement model.

Published abstract reprinted by permission of the copyright owner. Show less

An interpreter's ability to self assess is a fundamental requirement for determining readiness to accept an assignment and for setting realistic goals for self-guided continuing education. Self assessment is widely used at the university level and viewed as both a tool for learning and a valued outcome of higher education. This study seeks to investigate [American Sign Language] ASL students' accuracy in self assessing their language competency. Accuracy is defined in this study as a strong correspondence between students' self assessment and instructors' assessment of the students' ASL competency as measured on the Sign Communication Proficiency Interview Scale. Across all ASL classes there was a significant and moderate-strong correlation between students' self-ratings and their instructors' ratings of the students' sign communication proficiency. Students did not improve in self assessment accuracy as they progressed through the ASL course sequence. Self assessment skills should be taught and practiced throughout the course of ASL classes.

Published abstract. Show less

This is the second volume of the final report of the Committee on the Performance of Military Personnel, which was established in 1983 to provide scientific oversight of the Joint Service Job Performance Measurement/Enlistment Standards (JPM) Project, intended to develop robust measures of performance in entry level military jobs so that standards could be linked to performance on the job. The two volumes address issues such as the development of job performance measures, evaluating the quality of performance measures, implications of cognitive psychology for measuring job performance, work samples as measures and measuring job competency. The contents are: Implications of cognitive psychology for measuring job performance / Robert Glaser, Alan Lesgold and Sherrie Gott; Work samples as measures of performance / Frederick D. Smith; Measuring job competency / Bert F. Green and Alexandra K. Wigdor; The evaluation of alternative measures of job performance / Linda S. Gottfredson; Range restriction adjustments in the prediction of military job performance / Stephen B. Dunbar and Robert L. Linn; Alternatives to the validity coefficient for reporting the test-criterion relationship / Linda J. Allred; Generalizability of military performance measurements: individual performance / Richard J. Shavelson; Procedures for eliciting and using judgments of the value of observed behaviors on military job performance tests / Richard M. Jaeger and Sallie Keller-McNulty; Exploring strategies for clustering military occupations / Paul R. Sackett.

Volume one is indexed at TD/TNC 87.1736. Show less

This paper presents preliminary findings from a study investigating the application of a multi-source measurement approach to the assessment of higher order competencies in the public service industry. The aim of the study was to develop and validate a strategy to synthesise multiple sources of evidence to inform judgements of workplace competence. The methodology adopted integrates developments in two fields of study, performance appraisals and psychometrics. At present, 55 candidates have been assessed using a combination of assessment methods. This paper presents findings from a multi-faceted Rasch analysis of 360-degree assessment ratings. While candidate competence and item difficulty were well separated, there were only very small differences in the severity of different rater groups (self, supervisor, peer and subordinate). As such, rater group severity had no impact on candidate competence estimates.

Published abstract.

The collection of refereed papers is indexed at TD/TNC 73.622; individual refereed papers are indexed from TD/TNC 73.623 to TD/TNC 73.650. The collection of non refereed papers is indexed at TD/TNC 73.651; non-refereed papers are indexed from TD/TNC 73.652 to TD/TNC 73.680. Show less

This is the annual update of the Training and Skills Commission's Skills for Jobs five-year plan for skills and workforce development in South Australia. The Commission's advice in the 2012 plan focuses on the twin challenges of participation and productivity which will drive growth for tertiary level qualifications over the next five years. During 2013, the Commission will focus on strategies to reach a 69 per cent workforce participation rate by 2025. To achieve higher levels of participation it is important that training gives individuals the skills they need for sustainable employment. The State Government's Skills for All policy, announced in February 2011, is making fundamental changes to the [vocational education and training] VET system in South Australia. Skills for All is largely in line with the Commission's reform recommendations to government, including the fundamental shift from a supply focused system to one driven by student and industry demand. The Commission is working with the State Government to ensure that graduates of the new system have the skills and knowledge that industry needs.

Edited summary from the TASC website. Show less

This report presents the outcomes of the second stage of the Community Services Industry Competency Methodology Project. It revisits developing a conceptualisation of competency standards in the industry, which was a key concern of the first stage of the project. The report also examines issues in aligning and identifying levels of competencies, and issues in selecting and using methodologies for developing national competency standards. Show less