Search Results

Multiple choice questions (MCQs) are commonly used for assessing students, but medical teachers may lack training in writing them. MCQs often have imperfections called item-writing flaws (IWFs) that can affect students' results and impede objective evaluation of their knowledge. This study aimed to evaluate a guideline-based faculty development intervention on writing MCQs at Levels 1 and 2 of the Kirkpatrick Model. MCQs written by teachers prior and after the intervention were analyzed with the Shapiro–Wilk test, Student's t-test, and Wilcoxon signed-rank test as appropriate. In addition, the phenomenological approach was chosen to describe experiences of 10 teachers in semi-structured in-depth interviews. Results showed satisfaction of teachers with the document. They found it helpful in writing MCQs and noticed the requirement for another guidelines. They appreciated it for its briefness and clarity. The statistical analysis of the quality of MCQs before and after the intervention showed that the document contributed to a statistically significant reduction of IWFs. To conclude, our results showed that a guideline document on writing MCQs may serve as a well-received and effective faculty development intervention. The document may be used as a flexible, time-saving, and just-in-time learning method, fitting needs of medical teachers.

Published abstract. Show less

Adult practitioners share a laudable goal - to help vulnerable adults learn and improve their lives. To determine whether their learners are making progress or have achieved a specific outcome, most practitioners rely on data generated by a variety of formal and informal assessments. Practitioners may use these measures to adapt instruction as needed, for example, to identify learners who are struggling and need more intensive or differentiated instruction (Supovitz and Klein, 2003; Wayman and Stringfield, 2006). Practitioners may also find data useful for evaluating and improving instructional practices (Halverson, Prichett, and Watson, 2007; Supovitz and Klein, 2003). Learners can use data on their own performance to inform their approach towards achieving an outcome (Hamilton et al., 2009; May and Robinson, 2007; National Research Council, 2012). At the administrator level, program-wide data can be used to assess whether curricula or special initiatives are having the desired effect, and it can inform a change in course when needed (Kerr, Marsh, Ikemoto, Darilek, and Barney, 2006; Marsh, Pane, and Hamilton, 2006). To understand the types of outcomes data that may be useful to practitioners and how they may be used effectively, we first review existing theory and research on using outcomes data in practice. We then suggest implications for practice but also highlight gaps in this research.

Summary from publication. Show less

[Coronavirus Disease 2019] COVID-19 has forced each of Australia's school systems to teach most students from home for much of term 1. There has been much concern about the impact on students' year 11 and 12 studies and for the transition to post school study, especially in university. With the COVID-19 environment under control it is likely most schools will be back in practice during term 2 and for the rest of the year. Further COVID-19 outbreaks could change this, at least for schools in affected areas. The IRU has consistently argued that all Australians need to complete school and then progress to tertiary education that suits their aspirations and capabilities. Our response to COVID-19 must support each person to do [so] to ensure the best spring back from COVID-19. The challenge is not how to assess year 12 in 2020 or how to create an [Australian Tertiary Academic Ranking] ATAR. The challenge is how to ensure the education each person receives meets expectations for year 12, without any set of students suffering greater disruption than the group as a whole.

Excerpt from publication. Show less

The concept of 'competence' has gained traction in recent years because of its power in informing broader educational objectives, such as readiness for transition to later educational stages. The responsibility for developing competencies in learners is increasingly laid onto educational institutions via accountability arrangements. As a result, competency frameworks are value-laden and motivated by interrelated purposes including construct definition, the development of assessment criteria, and workplace readiness. The present article introduces a methodological approach for constructing a competency framework that accounts for these underpinning motivations.

The present article first introduces three binary distinctions that developers can utilise in making structural decisions on a competency framework: binary vs continuum; atomistic vs holistic, and context-specific vs context-general. We argue that decisions made by developers in terms of the three binary distinctions influence their additional responsibilities in order to achieve a high degree of utility for practitioners using a competency framework. The decisions regarding framework purpose and its structural features inform the alignment between the intended constructs being targeted and a framework's use. The present article then focuses on a worked example in relation to one competency area that commonly resides within a competency framework - collaboration. We conclude that conceptual and structural alignment can help inform a competency framework's validation process, by enabling more specific usage claims to be articulated that can form the basis of empirical analysis.

Published abstract reprinted by permission of the copyright owner. Show less

Performance assessments such as laboratory practice in Technical and Vocational Education and Training (TVET) are difficult to measure because they are subjective in their characteristics, which can cause bias. The use of a rubric scoring scale is suitable for measuring students' practical competency as it could translate the qualitative criteria into quantitative forms on a grading scale. The purpose of this study is to analyse the effect of rater severity for the performance of component installation of an electronic circuit project using Many-Facet Rasch Measurement (MFRM). Nine raters have examined the competency performance of 68 students using a four-point rating scale rubric consisting of 16 items. This study was conducted at three vocational colleges. The results showed that all raters measured with different severities but very high consistency. The effect of rater severity did not exist at the group level but exhibited at the individual level. It can also be examined that all raters were biased on the item and also towards students, either by being severe or lenient. Thus, the rubric cannot eliminate the effect of bias during the assessment. However, it can help raters to be consistent so that they have the same understanding when translating the rubric. These findings have implications for the psychometric quality of the electronic circuit project performance assessment.

Published abstract. Show less

Graduate employability is a key issue for higher education as new graduates face a highly competitive and rapidly changing employment sector. To maximise their likelihood of employment, graduates need to be able to demonstrate the skills and attributes most valued by employers. Employers, however, have long criticised the ability of graduates to contribute effectively to the workplace, especially due to a perceived lack of generic employability skills. This indicates a possible mismatch between the skills required by employers and those possessed by graduate applicants. This study addresses a gap in the scholarly literature regarding employability skills - the perspectives of science academics at an Australian and UK university regarding the promotion of generic employability skills in the subjects they teach. More specifically, this article discusses academics' perspectives regarding: (a) which of the targeted generic skills were developed and assessed through their science units, and (b) the approaches used to promote the targeted skills and any associated challenges in employing those approaches. The discussion explores how shifting from traditional transmissive pedagogies to more problem-based approaches and embedding reflection into pedagogy and assessment practices would contribute to promoting the skills employers increasingly demand from science graduates.

Published abstract reprinted by permission of the copyright owner. Show less

The 2016 introduction of national literacy and numeracy exit testing for all Australian teacher graduates was a significant policy decision that adds further pressure on the tightly regulated area of Initial Teacher Education. There are implications to examine from this specific mandate about the nature of the testing, the capabilities of future teachers, and contemporary conceptions of literacy. Initial Teacher Educators of literacy also need to consider how they might address the preparation of their preservice students to satisfy the test requirements while extending them beyond meeting one achievement measure to fulfil their career ambitions. This paper explores the impact of the policy regulation on preservice teachers, specifically, in relation to their personal literacy skills and test preparedness. Against this background of legislative requirements, the project reported here adopted one approach to address students' literacy needs by developing customised online literacy support. Research conducted with the first cohort of participants to evaluate this intervention had two purposes. First, to obtain students' evaluation of the online resource and their assessment of its usefulness as a tool for their learning. Secondly, to gather students' views about their confidence in their personal literacy skills, and their opinions about the introduction of the LANTITE literacy test (ACER, 2019). The self-reported data prompt further discussion about the preparation of Initial Teacher Education students and the nature of literacy development for beginning teachers. There are ongoing challenges for teacher educators as we prepare our future teachers for professional accreditation that requires the demonstration of a specific set of skills. Literacy competency and linguistic achievement cannot be assumed. It is essential to build rich contemporary conceptualisations of literacy, to develop tertiary students' personal and professional confidence across a range of multiliteracy practices and multimodalities, and to extend these well beyond any single test measure.

Published abstract. Show less

While information literacy in undergraduate programs has received considerable attention, few studies have explored graduate programs, with even fewer focusing specifically on library and information studies (LIS) programs. Accordingly, this study aims to assess the information literacy skills and knowledge of incoming students in an information studies graduate program at Kuwait University. By assessing students before and after they receive information literacy instruction, the study also aims to map a plan for developing graduate-level information literacy education. The study's rationale is to provide empirical evidence of graduate students' information literacy and research capabilities on commencing their studies, which are expected to be low, thereby encouraging educators and policymakers to engage in meaningful, informed discourse on investing in information literacy education. To collect data, the author administered a pre-test at the beginning of the semester, followed by a post-test at the end of the semester after students had attended a series of information literacy sessions. In addition, a faculty focus group was conducted to gain insight into their perceptions of students' skills and their views and beliefs on information literacy education. Students' overall mean score increased by almost 18 per cent in the post-test. They performed better on questions related to knowledge of databases, search strategies, accessing sources, and some legal and ethical issues surrounding the information ecosystem. They also showed moderate improvement in critically evaluating information. However, they continued to perform poorly in defining an information need and understanding the nature and scope of the information paradigm. The findings informed the development of an information literacy education model. The study's research design provides a format for other institutions interested in identifying students' information and research needs with the aim of integrating information literacy instruction into graduate programs. The information literacy education model is dynamic and can be adapted to different learning environments.

Published abstract. Show less

A distinctive characteristic of study tours is their immersive nature, and the type of learning developed in such circumstances often relies on instinctive skills related to confidence, flexibility, curiosity, resilience and risk taking. To discover more about these learning experiences for adult undergraduate students studying varying disciplines in the creative industries at an Australian University, the researchers employed educational action research and developed assessment modules to foster student awareness of intangible skills and personal characteristics during short-term international study tours. On the completion of the tours, participating staff and students provided feedback, via a questionnaire and focus group discussions, that assessed the value of these modules in promoting learning and their contribution to personal growth and cultural sensitivity. Findings show that these modules were highly effective at enabling adult students to become aware of the skills they employed while being actively engaged in international study tours. This study highlights the potential significance of immersive assessment modules that provide specific ways for students to engage with their host country and the cultural differences to which they are exposed.

Published abstract reprinted by permission of the copyright owner. Show less