Here is a quick (ish) summary of the CIT-eA project, its outputs and findings to help you decide if it is right for your needs. Although our focus is on the Scottish further education college sector, much of this guide will apply equally to other sectors such as higher education, community based learning and work-based learning etc.

Remit & Aims

The remit of our project included ‘explore and identify the barriers to adopting e-assessment and identify workable solutions’. This is what we hope you will find in this guide.

Assessment: the ‘sharp’ end of education

Our education systems are still in the process of moving from being paper based to becoming digital, institutions tend to move slowly in adapting to change – both technological and social.

Assessment is at the ‘sharp end’ of education. Colleges, Universities and private sector providers all largely exist thanks to being able to provide certified evidence of the level of learning achieved by their students, with teaching related income being their largest source of revenue by far. The certification of learning is based on assessment procedures, which in turn is monitored by regulating authorities such as the SQA, the City & Guilds, the QAA and others. Any changes to assessment procedures have the potential for disruption. This guide and website provides a grounding in how to approach these changes and encourages the adoption of a critical and analytical approach. In this respect, we depart from some of the more exuberant claims made for the transformative power of technology in education and are concerned with actually getting things to work in real educational settings.

We use the popular and adaptable ADDIE instructional design model to provide a coherent and effective structure for readers to follow. A terminological note here – in the UK (particularly in Higher Education) sometimes the term ‘Learning Design’ is used. Whatever the chosen terminology, the main thing to grasp here is that the concept of ‘designing’ teaching is an important success factor in the adoption of technology in education. This involves some different approaches compared to ‘traditional’ classroom education – as well as keeping the best of existing practices.

The style of the guidance materials we have produced (at the request of the lecturers we worked with) is informal and direct with an emphasis on providing quick access in the form of checklists and tips, backed up by longer discussions. References and links to further information are provided in the text for those who would like to explore topics further.

Systematic Approach

A central concept in this guide is the concept of viewing assessment activities as part of a larger connected system; we argue that without this kind of approach introducing e-assessment will be much less successful. Experience elsewhere has shown this is part of the cultural change needed to make better use of technology in any profession or workplace – it’s closely allied to engineering methods with their emphasis on problem solving. So, it is useful to view adopting e-assessment as really involving the ‘re-engineering’ of existing educational practices and processes, something that requires a holistic and detailed approach and can assist those in management positions to succeed.

This guide does not contain detailed training materials for how to operate particular technologies used in colleges such as Moodle, Mahara or Blackboard – there are already lots of existing resources that do that and are kept up to date.

Scottish FE Focus

This guide concentrates on the Scottish FE sector and largely deals with the SQA qualification system, where the learning outcomes, assessment criteria, evidence requirements and conditions for assessment are specified in the ‘unit descriptors’. Having this information specified in detail provides a good foundation for implementing e-assessment.

What We Found

Read the Specs! A close reading of the unit descriptors is always recommended when starting the process of redesigning existing assessment practice to incorporate greater use of technology. The rationale behind this, stems from the findings of the project – that one of the main systemic factors holding back greater use of technology in assessment in Scottish FE is staff perceptions of the external SQA quality control procedures used to monitor change; ‘External Verification’ (EV).

Engage with the SQA – As the forward from the SQA to our guide makes clear; they want to promote greater take up of e-assessment. So make use of their prior verification facilities if you have any doubts.

Student IT and Information Management skills – the idea that students can easily work with technology is largely a myth; they will need support – especially to use college systems.

College e-learning infrastructures – can suffer from usability and performance issues ranging from minor to substantial. Access to enough networked computers to undertake summative exams in an invigilated environment can be a particular challenge. College network policies can be problematical in terms of access to web sites and downloading files from the web.

Administrative systems – may not be fully integrated into the digital assessment lifecycle resulting in delay and duplication of multiple paper and digital records that need to be maintained and coordinated.

Staff IT and information management skills – while using standard ‘Office’ type tools and shared network drives is reasonably widespread. The use of web-based college tools like Virtual Learning Environments (VLEs) and E-portfolios is more problematic due to usability issues of those systems and shortcomings in integration with college administration systems, such as student records.

Teamwork and working practices – the move to a greater use of e-assessment (and e-learning in general) needs more of an emphasis on team teaching, the sharing of resources, and greater up-front analysis and design activities. A useful way of looking at this is that in order to gain the benefits we need to have an up-front investment of time and effort, these types of changes can be a challenge in any workplace.

Small things make a big difference – what may seem small or insignificant to one person in the ‘e-assessment chain’ can create big problems or improvements for others. It is vital to grasp the connected nature of this kind of work – which is why we stress that analysis and testing as being so important. In this category would be included file formats and network policies about permissions to download and open files.

Test, Test and Test again – it is essential to thoroughly test your e-assessments, both technically and by conducting ‘walkthroughs’ with your colleagues and students.

Things will take longer than you think – all our participants found developing e-assessment a lengthy process. The payback can be substantial but it does take up-front investment of time and effort.

Take personal responsibility – don’t leave thing to the last minute or assume a support worker will do it for you. Plan well ahead and work as a team, if things go wrong learn from it. Develop a ‘Plan B’.

A phased approach works best – although our overall target is the greater use of e-assessments for summative assessment. It makes a great deal of sense to start by concentrating on formative assessments in order to develop capacity and skills and, crucially, to identify and understand what the existing technical and institutional limitations may be.

Creativity is vital – it is essential in ‘ordinary teaching’ in order to adapt our teaching practices to the needs of different students in traditional classroom education. It is also essential to implement e-assessment and e-learning in order to overcome the limitations of both the technology and the institutional context. The key to this is in developing a thorough analysis and understanding of our own working environments in order to provide a sound foundation for designing creative solutions.