programme
NOTE: Details of the 2012 programme will be made available shortly.
2011 Thursday Programme (afternoon only)
| 12:30 | Arrival and registration (refreshments available) | ||||||
| 13:00 | Introduction | ||||||
| 13:10 | Masterclass Sessions | ||||||
|
|||||||
| 14:40 | Refreshments | ||||||
| 15:00 | Masterclass Sessions (continued…) | ||||||
| 16:30 | End |
2011 Friday Programme
| 09:15 | Arrival and registration (refreshments available) | ||||||||||||||
| 09:45 |
|
||||||||||||||
| 09:55 |
|
||||||||||||||
| 10:25 |
|
||||||||||||||
| 11:00 | Parallel Sessions A | ||||||||||||||
| 12:00 | Exhibitors (refreshments available) | ||||||||||||||
| 12:30 | Lunch | ||||||||||||||
| 13:30 |
|
||||||||||||||
| 13:35 |
|
||||||||||||||
| 14:10 | Parallel Sessions B | ||||||||||||||
|
|||||||||||||||
| 15:10 | Poster Entries Showcase (refreshments available) | ||||||||||||||
| 15:25 |
|
||||||||||||||
| 16:00 | Scottish e-Assessment Awards Cliff Beevers OBE, Chair of the e-Assessment Association |
||||||||||||||
| 16:30 | Closing Remarks and Prizes | ||||||||||||||
| 16:35 | Panel Session & Wine Reception Craig Mahoney Cliff Beevers OBE Donald Clark Steve Wheeler Bobby Elliot |
||||||||||||||
| 17:30 | End |
Thursday Programme
Masterclass Sessions
Exploring Assessment Design
Ruth Thomas, JelSIM (left)
Helen Ashton, Heriot Watt University (right)
Understanding how to design eAssessment questions is key to supporting staff and students during their learning. The right questions can provide confidence to students, highlight areas of strength and weakness to both staff and students, and allow staff to provide appropriate support.
This practical workshop covers the issues involved in designing a strategy and authoring an assessment for a course. It will consist of a mix of demonstrations, discussion and hands on work taking delegates through the process of creating of valid, reliable and practicable assessment. It will look at:
- the context of the course being assessed - deciding why, what and how to assess;
- assessing with the aid of technology - authoring of assessment items; pitfalls to avoid; making good use of multiple choice questions; assessing higher order skills;
- assessing a complete course - creating a valid and reliable assessment
- after the assessment - learning from the assessment
Delegates will be given opportunities to relate and apply the activities to their own teaching.
————————–
Delivering Audio and Video Feedback
Carol Walker, Learning Technologist (for hire)
Traditionally feedback on written assessments has been given through annotated notes. However, studies like the JISC-funded Sounds Good project have shown that providing audio feedback to learners can have additional benefits:
- More contextual information provided
- Improved learner engagement
- Personalisation of feedback
Following on from audio feedback, interest has grown in the opportunities afforded by video feedback – where the added dimension of screen capture can offer a richer feedback experience.
In this hands-on workshop, we’ll look at ways to use free audio and video capture tools to deliver effective feedback to learners.
————————–
Creating High Quality & Engaging e-Assessment Content - Experiences from SQA
Martyn Ware, SQA
As the national awarding body for Scotland, SQA has a strategic commitment to increasing and enhancing the use of e-assessment to support our qualifications and assessments. In support of this objective, over recent years SQA has developed a number of approaches to the use of e-assessment across a range of types of qualifications, products and services. Examples include our Solar on-screen assessments, games-based assessment, the use of social software for assessment and the development of moving image assessment. Some of these now form mainstream services for our customers: others remain at the trial stage.
The main driver behind the move towards e-assessment is to capture the many benefits it offers to SQA, our centres and in particular to learners. One of the key challenges SQA is addressing is to capture the benefits of technology in the developing high quality assessments that are valid, reliable and fit for purpose whilst remaining practicable and cost effective for SQA and its centres to develop and deliver.
This session will draw on SQA’s experience to date of developing and working with centres to deliver a range of types of e-assessment to illustrate some of the issues we have experienced and some of the approaches that have helped to address them. Whilst some of the issues are generic and apply to a range of approaches to e-assessment, others are more specific to particular approaches. We will also explore the use of SQA Academy, our Moodle-based e-learning service to support those involved in developing and delivering e-assessments.
The session will combine input from SQA staff, the opportunity for some hands-on experience of some of our e-assessment products and discussion of some of the issues raised for awarding bodies, centres, learners and others involved in supporting learning and assessment. We hope the session will be informative and thought-provoking and that, based on the experiences of those attending, it will offer us some new insights to help inform our future plans.
Friday Programme
Keynotes
Assessment in the Digital Age: Fair Measures?
Steve Wheeler, University of Plymouth
In this keynote I will trace the history of educational assessment, and attempt to critique it within the context of a rapidly changing education system. I will pose several questions for consideration:
- How can new and emerging digital technologies contribute toward better assessment for learning?
- How might teachers and lecturers harness the power of social media and mobile devices for assessment?
- Are the assessment methods we currently employ meeting the needs of our students?
- What can we learn from games and online interaction that can make assessment more relevant and appropriate for 21st century learning?
- Are the current measures we apply in assessment fair to this generation of learners?
————————–
“Please Sir, May I Have Some More Exams” The Student Perspective on Assessment Activities
Becka Colley, University of Bradford
This keynote will provide an insight into the minds of students as they prepare for, undertake and reflect on the processes of assessment. Using narrative, the student voice will illustrate why we need to think holistically about our assessment processes and activities.
Issues such as student engagement, motivation and belonging will be explored within the context of e-assessment. How can assessment be aligned with meaningful real world activities? Can technology provide some of the solutions?
————————–
‘No Sweat?’ - Simulated Stress for Young Medics
Pamela Kato, Independent Consultant, University Medical Center Utrecht
Doctors in training are particularly susceptible to making errors. A majority of medical errors have been attributed to human factors such as communication habits and insufficient teamwork. Both objective and subjectively experienced stress contribute to these errors. Medical schools rarely offer training in skills to cope with stress.
Dr. Kato will talk about her most recent game, Air Medic Sky 1. AMS1 is a game designed to improve patient safety by both instructing about patient safety and practicing stress reduction exercises as a human factor. A non-invasive biofeedback device allows the user to control events in the game by managing their physiological stress response. Players learn about patient safety through mini-games, lectures from experts, and patient care simulations. The game is a novel training approach for medical interns and residents. Research on the game will also be discussed.
————————–
Fails: When Assessment Goes Bad…
Donald Clark, Free from the Tyranny of Employment
Never in our history have learners been subjected to so much assessment at so many levels. We have international league tables such as PISA and others, national league tables for schools and HE, then a battery of assessments for children and adults throughout their learning lives. Yet many of these assessments contain flaws that cause real concern, conceptual and statistical, that invalidate their results. Despite these flaws, readily admitted by the assessment bodies, politicians continue to exploit the results making claims well beyond the original scope of the assessments.
Donald will look at international, national and other assessments to show that, in many cases, the assessment tail is wagging the dog. He will look at PISA, HE, adult learning and training, the whole gamut of learning, to expose these serious problems and show that flawed pedagogy leads to flawed assessment. In addition, Donald has some strong views on e-portfolios (see his blog post Seven Reasons why I don’t want my life in a shoebox) and will not be pulling any punches in assessing the assessors, proposing some radical ideas for ‘regime change’ change in the world of assessment.
Workshops
The NXT Step - Assessing with Lego Robots
Kent Pledger, Mid Calder Primary School
A practical demonstration of how Lego NXT robots can be used in the classroom to challenge children’s thinking and to achieve the Experiences and Outcomes of Curriculum for Excellence through interdisciplinary learning. Delegates will experience how children can be motivated and engaged, how learning can be assessed, how children can be the lead learners and how teachers can become enablers.
The workshop will include a practical challenge with children fulfilling the role of mentors.
————————–
Tools to Enhance Assessment in Moodle
Matt Wingfield, TAG Developments
We are increasingly asking students to create digital evidence of their learning, however, marking such digital work is not always easy. Files can be very large, accessing them requires a range of software and mechanisms for leaving contextual feedback can be complex and can often involve altering the student’s work.
This workshop will provide hands-on use of a revolutionary assessment tool from TAG Developments called Red Pen Tool, which is now available for Moodle. The tool allows users to contextually annotate and mark any digital file – text, illustrations, presentations, photographs, and even video and sound – all through a web-browser.
The session will be lead by Matt Wingfield, MD of TAG Developments and Board member of the eAssessment Association, along with a teacher with direct experience in using Red Pen Tool, to illustrate how it can simplify digital assessment, engage students, provide rapid and contextual feedback and save money on printing.
————————–
Assessment Manager for Moodle
Jim Judges, ULCC
ULCC have been working in collaboration with a number of providers to develop an Assessment Manager module for Moodle that enables evidence tracking against course specific outcomes. Whilst initially aimed at those delivering BTEC and NVQ type qualifications this tool should appeal to any organisation using Moodle to deliver courses based on performance criteria. Following on from the popular ULCC eILP Moodle plugin, this new Moodle 2.0 based tool is due for community release via moodle.org in the autumn of 2011.
This practical workshop will provide an introduction to the Assessment Manger and give participants a hands-on opportunity to experience this tool from the student and teacher’s perspective.
————————–
App, App and Away: Using Google Forms for Assessment and More
Martin Hawksey, Ed. Tech Mash-up Artist
Within the academic community, there is growing interest in personalised learning environments at the same time we are seeing more staff create their own personalised tutoring environments. Using free tools, tutors are creating learning environments that are more aligned to their personal, and the students’, needs.
One such tool is the Google Docs suite of documents, spreadsheets, presentations and surveys. In this session, participants will learn how Google Forms can be used for self-grading quizzes, electronic submission with customs pro-formas and get a glimpse of how common tasks can be automated using Google Apps Script.
Read about how the workshop went on Martin’s MASHe blog.
————————–
MyShowcase - Controlling Your Digital Evidence
David Sowden, University of Hull
It can be a daunting and time-consuming task to seek out the right documents linking them to assessment criteria, as and when you need them. For example a tutor or mentor might want to have a look at your project work, and they might also want to have a look at all your media evidence too, linked to the assessment requirements. MyShowcase™ is an open source web application which offers personalised evidence gathering for e-portfolios.
Most importantly, the format your evidence doesn’t matter, MyShowCase is being developed to allow users to integrate learning evidence from a range of online sources to showcase for continuous professional development, career planning, and lifelong learning.
Participants will be given access to the MyShowcase toolset allowing them to:
Gather and aggregate
Users can bring together RSS feeds, Twitter accounts, Flickr sets, files, and weblinks and to create rich evidence streams of their online content (Note: participants should have access to these types of resource for this workshop).
Tag & Map
Tag and map evidence easily to competency frameworks, lists of learning objectives, CPD frameworks and more.
Showcase
Select, format and publish evidence for reviews, assessments, job applications, and ongoing CPD.
————————–
A Dummies Guide to Secure Digital eAssessment
Martin Gower, eCom Scotland
You have heard all about eAssessment so now it’s time to see it in action. In conjunction with the SQA, eCom Scotland has developed a new high tech, secure, online solution for the Security Industry Qualifications (SIA) qualifications. For a no-nonsense practical hands-on workshop, embark on a journey of discovery.
From creating your own digital forms to using the webcam and digital pen, eCom is all about making eAssessment easy. Experience first hand with eCom Scotland how our secure “tried and tested” solution is impacting eAssessment.
Seminars
Supporting Students’ Writing Skills Using Turnitin
Dr. Sharon Flynn, National University of Ireland, Galway
Over the last three years, the Centre for Excellence in Learning and Teaching (CELT) at NUI Galway has been investigating the use of Turnitin to support student writing skills, particularly in large enrolment classes. As part of a number of on-going and evolving case studies across the institution, the text matching service has been used successfully by academic staff in their teaching and assessment practices. Turnitin has been found to be a useful teaching tool when integrated into an assessment strategy that emphasises original work. When students are given access to their own originality reports, accompanied by support from tutors, student writing is enhanced.
This presentation will report on outcomes from some of the case studies and highlight the factors that contribute to a successful assessment strategy using Turnitin.
————————–
24-hour Papers: the Open-Book Alternative to Exams for Online Assessment
David Hopkins, Bournemouth University
Common unit specifications covering delivery of subject-identical units across different courses, often with different delivery methods, are increasingly being implemented. The inclusion of a ‘coursework’ element of assessment allows for flexibility. This is different when an ‘exam’ is required; with students on a fully-online course, unable to attend an exam centre, due to differences in time zones and/or locations, the concept of an open-book exam is used. The exam paper is released to students through our VLE (Blackboard) at a time that is agreed and broadcast to students in advance. Submission of their work is required within a 24-hour window via an upload of their files to the VLE (using either the standard submission tool or Turnitin).
This presentation will draw upon the Bournemouth University’s substantial experience of presenting time-constrained papers to students studying at a distance and will consider the issues surrounding this approach. Particular consideration will be given to the importance of question design to limit scope for academic dishonesty and the University’s plans to modify this approach in the forthcoming academic year.
————————–
Assessment in the ‘Digital Age’ and the Role of the Student
Professor Ian Pirie, Assistant Principal, Edinburgh College of Art (left)
Stewart Cordiner, Head of Visual Communication, Edinburgh College of Art (right)
During the past four years, Edinburgh College of Art has developed and implemented a constructively aligned, grade profile and assessment scheme. Since September 2010, assessment and feedback is now fully supported online within a purpose-designed Learning Management System (LMS) that enables students to self-evaluate, reflect, grade their own work and write their own feedback and for staff to design their projects, create learning resources, provide written feedback, indicate required actions and conduct assessment.
Assessment and feedback is being used proactively to support students in their learning and to help them understand their strengths, weaknesses and where particular attributes and competencies begin to be developed. The history of the student journey and progress made is available to the student and their tutors 24/7 - it now becomes quite difficult for students to maintain they ‘never get feedback’.
The presentation will cover the journey so far and explore the types of information and data that the College is now in a position to collate and analyse to evidence improvement and inform further enhancements.
————————–
Formative e-Assessment: Just Another Way to Use a Blog
Natalie Lafferty, University of Dundee
The Medical School at the University of Dundee has been piloting the use of the WordPress blogging platform to support their teaching. These pilots have been well received by staff and students and one of the features that has proved particularly popular has been the inclusion of formative e-Assessment opportunities as teachers have created cases of the week, x-rays of the week and quizzes into their blogs. Course feedback has frequently highlighted that students would like to see more formative assessment provided and students have asked for more of these formative elements to be included as the blogs are further developed and new ones are created.
This presentation will provide an overview of how using Wordpress is helping to increase the engagement of teaching staff in the use of educational technology and also stimulating students to explore their own ideas and work with academics and the e-learning team to develop formative e-Assessments to meet their learning needs.
————————–
Some Unexpected Benefits of Technology-supported Peer Collaboration which Aggressively Attack Traditional Methods of Assessment
Sarah Honeychurch, University of Glasgow
Steve Draper, University of Glasgow
The benefits of peer collaboration in terms of building a learning community are well known, but the reasons why this happens have not been fully understood. We began with an interest in collaborative work and investigated various models of implementing this by using Web 2.0 technologies, noting that these technologies allow the teacher to evaluate individual input and to ensure that proper credit is given to each student. We ended with a startling conclusion: students are very good at providing constructive feedback to their peers and, even when this is implemented in a way that appears to be informal, formative learning happens. This led us to challenge traditional models of assessment and to re-evaluate basic principles of learning and teaching.
There is obviously a place for assessment by the teacher, but the concept of constructive feedback needs to be reconsidered. In order to explain this, we begin with an overview of some models of peer collaboration and, by presenting short case studies, we show how subtle group effects can help individual students to improve their academic performance.
————————–
Harnessing the Potential of PeerMark to Enhance and Enrich the Student Experience of Assessment and Feedback
Dr Neil McPherson, University of the West of Scotland
The importance of providing students with an enriched assessment and feedback experience by way of empowering them as self-regulated learners is at the centre of contemporary discussion surrounding the role of assessment (see, e.g. Nicol & McFarlane-Dick 2006; Nicol 2007). Allied to this is the growing awareness of the need to develop effective learning environments that locate the student at the centre of the appraisal process (Nicol 2010; Sadler 2010).
Taking these positions as a point of departure, the presentation examines the potential of Turnitin’s PeerMark tool to enhance and enrich the student experience of assessment and feedback through the provision of a structured environment that facilitates and encourages the production of a developed matrix of peer and self-appraisal. It focuses on the use PeerMark in the construction and delivery of a formative/summative framework of assessment and argues that PeerMark can empower both students and tutors by closing the gap between students’ understanding of assessment and tutors’ expectations of student performance.
————————–
Tips for Creating and Delivering Effective Mobile Assessments
Ivan Forward, Questionmark
Handheld, Internet-connected devices such as Android smartphones, the Apple iPad and iPhone have become increasingly more common in today’s world. These devices provide teachers with exciting new options for delivering learning and assessments to students or employees on-the-go.
This presentation will provide tips and mobile delivery best practices, and also outline the potential applications of mobile assessments. The session will also demonstrate Questionmark’s assessment management system, which enables you to author an assessment once, and then participants can take them on their device of choice.
————————–
e-Assessment for Recruitment, Training and Testing in Private and Public Sectors: The Edumatic case at Airbus and Selor
Bert Wylin, Televic Education
Televic Education’s e-assessment platform, Edumatic, has many different use cases, ranging from recruitment and selection evaluations in HR-companies or departments (Airbus, FR) to real exams in universities and high school, or for the public sector in the national language exam at Selor, B.
The technology for this is both generic and very advanced. “Generic” meaning not content specific and very open to all kinds of content input, including many multimedia options, “advanced” meaning that very specific question types and correction algorithms will allow to create very authentic activities to train and evaluate candidates. Optional metadata allow even more detailed and flexible reporting. Recently, oral activities with automatic correction (ASR, automatic speech recognition) have been added.
————————–
Numbas: A Powerful Open Source Tool for e-Assessment
Bill Foster, Newcastle University (left)
Christian Perfect, Newcastle University (right)
Numbas is a web-based e-assessment system developed at Newcastle University. It consists of a set of tools which produce SCORM-compliant exam packages. Features include:
- Simple installation
- Web-based, so it can run on a wide range of computers and browsers
- Exams can be deployed in a variety of locations and platforms – VLE, DVD and stand-alone on the web
- Support for all standard question types including symbolic mathematical input
- Write questions using simple markup with any text editor
- Questions can be fully randomised
- Rich content such as videos and interactive graphs can be included easily
- SCORM 2004 standards compliant
- Support for themes to change the look and user interface of exams
- Support for extensions to add new features, such as new question types, mathematical and statistical libraries
Follow this link for an interactive demonstration of the Numbas platform.
————————–
Authentic Assessment in Law: SIMPLE
Karen Barton, University of Strathclyde Law School
The SIMulated Professional Learning Environment (SIMPLE), developed at Strathclyde University, enables students to collaborate on professional simulations across professions, disciplines, countries and cultures, and at any educational level. The open-source software enables powerful interactions between simulated roles and users, and the completion of professional tasks. Students learn by doing the professional tasks through innovative and authentic assessment, and by interpreting and reflecting upon their experiences. The result is situated experience that deepens learning, is hugely enjoyable for students, and fascinating for staff because it enables students to learn effectively using sophisticated technology and a constructivist approach to learning and assessment.
The concept of authentic assessment is central to the education paradigm of Transactional Learning on which the SIMPLE environment has been based and is an interesting example of how technology has enhanced the practice of assessment in this area.
————————–
Assessment Futures: the Role for e-Assessment?
Peter Hartley, University of Bradford
Over the last decade across UK HE, various national and institutional teaching initiatives have shown how assessment can be changed to improve student learning. This has been accompanied by developments in conceptual models focused on how assessment affects student learning and by new methodologies for evaluating the impact of changes to assessment (Gibbs 2005; 2007; 2009; Rust et al. 2008; Nicol and McFarlane-Dick, 2006). However, almost all this innovation and development has been at the level of individual course units/modules.
Module/unit innovation can fail because the programme-level environment which students experience is not congruous or aligned. This may explain some of the variance which institutions notice in student reactions to their assessment and feedback. For example, the Student Course Evaluation Questionnaire in Australia and the UK National Student Survey (NSS) are both used as part of national QA efforts and to inform student choice. Both assume that students respond to entire programmes in their learning, not just individual units/modules. In the UK, institutional NSS scores vary more for items concerning assessment than for any other aspects of students’ higher education experience, and so students’ responses to programme-level assessment environments contribute very significantly to national rankings of institutions. There is a growing awareness that assessment needs to change at programme level if students’ programme-level learning responses are to be affected positively.
What might these changes involve and what does this mean for e-assessment?
This session will discuss ‘assessment futures’ from two perspectives:
1. Developments in integrative and programme-based assessment
Studies have identified greater differences in students’ learning responses to assessment between institutions than between courses or programmes within institutions (Gibbs et al 2007, Gibbs et al, 2009), indicating that there are different institutional assessment environments. Building on interim findings and fundamental issues from two current multi-institutional projects (PASS and TESTA), I will examine and discuss the impact of different forms of assessment at programme level on students’ overall learning responses.
2. What could this mean for e-assessment?
The accessibility and power of new computing hardware and software gives us a range of opportunities to improve the ways we assess and provide feedback to students. The most promising of these include new methods for audio/video feedback, developments in computer-aided assessment, and approaches to curriculum design which integrate assessment and learning. I will suggest how these developments can best support more integrative approaches to assessment.
————————–
Getting Vocal About Feedback
Malcolm Mactavish, University of Abertay Dundee
We want students to be the best they can be. In pursuit of excellence, we provide written feedback; where they went wrong, could have done better, etc. Good feedback helps weaker students improve their chance of a successful reassessment. BUT how many actually read and understand what we write? AND being realistic, how much time are we prepared to devote to 80+ students?
Malcolm ran a successful trial using feedback by audio recording – quicker for staff, more comprehensive for weaker students and universally popular across the cohort. So what could possibly go wrong .. go wrong .. go wrong ..
————————–
Personalising the Assessment Experience: Closing the Gap Between Learning, Assessment, Feedback and Future Performance
Shane Sutherland, PebblePad
Drawing upon real examples of practice, delegates will be guided through an array of online assessment features designed specifically for deployment in the seemingly paradoxical context of personal learning spaces in an institutional context. Assessment should always be viewed as a catalyst for learning and the functions shared in this session demonstrate that the needs of the tutor and institution need not compromise the needs of the learner and their learning.
Specifically there will be an opportunity to see how tools such as the scaffolded feedback forms, the comment bank and the ‘feedback conversations’ are used. The inclusion of validation functions, foot-printing and commenting enables tutors, assessors and moderators to ‘converse’ and reflect on the assessment process in the tutor blog. Including students in the assessment process is easily facilitated through permissions that allow peer review to be anonymised. In more formal approaches to assessment work can be blind/double blind marked and snapshots of all work, feedback, grades and moderation can be made.
Adding to the list of assessment functions designed to support learning is the recently developed ‘quiz builder’ tool. PebbleQuizzes promote extensive feedback and provide the learner with a durable record of their performance which can be shared, reflected upon and treated like all other portfolio assets.
————————–
Mahara in Action
Gordon McLeod, Royal Scottish Academy of Music and Drama
In this talk, Gordon will give a very practical overview of some of the functionality, practical uses and features of Mahara which include:
- Learner-Led Portfolio-building and why personalised learning matters.
- Storing and accessing learner files and data
- Showcasing Learner/Tutor skills/knowledge
- Planning our own learning with goals and action plans
- How Communities of Interest (Groups) can enhance practical skill development
- Formal assessment within walled gardens - considerations
- Mahara 1.4 – initial thoughts
————————–
Building the e-Assessment Centre
Gavin Lang & Claire Fulton, Edinburgh’s Telford College
Building and setting up a designated e-Assessment centre can be a daunting and difficult project to manage. This short presentation will provide details on how we built our centre and how we have embedded the use of e-Assessment within our curriculum delivery, the presentation will cover the following areas;
- How did it all start?
- Building the centre
- Advantages of e-Assessment
- Setting up processes and procedures
- Types of assessments used
- Promotional activities
- Usage statistics
————————–
Can We Rely on Intuition Alone When Designing Assessments?
Margarida Amaral, University of Porto
The University of Porto has been developing its capacity to support Computer-Based Assessment (CBA), allowing teaching staff to use technology to deliver tests in a transparent, secure and reliable environment. The transfer to an online mode has been recognised as an opportunity to revisit the the basic principles and practice surrounding assessment, with new tools revealing more about the performance of students and individual question items.
The in-house New Technologies in Education (NTE) unit supports staff through the delivery of an online course which looks at the assembly of test items for online delivery and analysis of performance. This presentation will describe how the University has supported the shift to CBT, and detail the experiences encountered on the way.