On Tuesday 25th May I was at Oxford Brookes University as part of a review event for the Programme Assessment Strategies (PASS) Project. Also at the event was Graham Gibbs. Graham spoke about his experiences of effecting change within programmes and institutions. His views mirrored our experiences on the ESCAPE project, namely that there is a "hidden" network of influence that you must be able to tap into, if you want to effect change within a school . A component of the "hidden" network are the informal meetings that take place in schools. For example; in the corridor, in social areas, over coffee and at lunchtimes. These informal venues are often where the hierarchy is relaxed and a more open discourse is possible. The discussions are often robust and play key part in forming opinions within schools.

Graham's views certainly mirrored our experiences.It was something that we had anticipated when considering our approach to working with our stakeholders. One of the early parts of the ESCAPE project involved mapping the influences of individuals and teams within schools. We looked at who were the key opinion formers within the schools and how we could get them "on side". Additionally we looked what were the formal and as important, informal channels of communication within schools.

We also looked at how we could encourage collateral effects - how the stakeholders we were working with could influence others. Both over the course of the project and beyond as part of our sustainability planning.

As part of the event I was invited by Peter Hartley - who chaired, to present an outline the ESCAPE project. It was extremely valuable to get the benefit of the teams experience experience in the discussion that followed.

Although the tenure of the meeting was one of looking a programme level interventions rather than at modular level ( - which is the ESCAPE perspective), there are common assessment themes that transcend both projects which started to emerge - such as ownership of the assessment and managing change within teams.

I was able to suggest some alternative approaches for the PASS project that involved taking a cross module approach to assessment that is designed to break down the barriers between modules as a "halfway house" to full programme level assessments. I spoke about the experiences of two of the programmes at the University of Hertfordshire ,that have worked towards implementing a more integrated approach to assessment that attempts to transcend the module based assessment model.

As part of the meeting Graham Gibbs discussed his guide "using assessment to support student learning" which is available at:

http://www.leedsmet.ac.uk/100317_36641_Formative_Assessment3Blue_WEB.pdf

Graham provided me with a copy of the guide during the meeting. It is an extremely useful resource for teams embarking on using assessment as a vehicle for learning. The ideas set out by Graham are illustrated and supported by case studies, which makes it very accessible and relevant to the reader.

On my train journey home I was able to reflect on what had been really useful and informative day!

Off to Scotland

Helen Barefoot and I were invited by the HEA to talk run a workshop on assessment and feedback at an all day Assessment event at the Robert Gordon University. What follows is a collection of (very quick) thoughts on the day …

Helen and I have run a few workshops together (inside UH) and so it was just great to take our work (much of which is guided by our ESCAPE activity) to colleagues outside UH. For various reasons the workshop did not run :-(. Despite our disappointment we did get to hear some great assessment related presentations.

Dai Hounsell presented a really grounded key-note and, in fact covered, much of what we were covering too. That assessment is not a new challenge, that good assessment is planned activity and that good assessment stimulates learning. We did not need the NSS to get us thinking that assessment is important. Some really useful slides from Dai that I will explore and come back to in a later post. Great start to the day.

A couple of student perspective presentations followed.
A student led campaign that successful introduced a turn-around-time policy for coursework and interestingly a policy to provide feedback on examination scripts. The learning gains to be had from providing feedback on examination scripts seems rather limited to me. I’m always banging-on about feedback creating consequences, and I’m just not convinced I know what consequences flow, or are able to flow, from feedback on end-of-process, high stakes assessment tasks. That’s a post for another day. But just to say, I’m off the fence on this one. I just don’t get it. Another delegate did note they had a similar policy at his institution and only 15% of the scripts (with feedback) were picked up. Surely we would be better placed putting our feedback on work that will be picked up and more importantly attended to, by the students. And relax!

Steve Draper, engaging as ever, had a couple of threads running through his presentation. First, was the interesting anomaly that overall a department was rated 5th against other departments (107 in total) for the NSS question overall, I am satisfied with the quality of the course and yet questions relating to feedback were ranked much lower. Feedback on my work has been prompt (ranked 54/107), Feedback on my work has helped me clarify things I did not understand (ranked 79/107) have received detailed comments on my work (ranked 101/107). Steve asked us to explore what might be going on. Was there a complex weighting algorithm for all the items on the NSS? Should the individual items of the NSS not sum to the overall score? If not, what was missing, what was the missing ingredient? Second, Steve separated declarative and procedural learning and pondered as to where our efforts on providing feedback might prove most effective. i.e. might we get more learning from less or (better targeted) feedback?

Module Coordinator Experiences

This week we held a lunch time seminar to give staff a chance to learn about the project and to hear from two of the module co-ordinators that we have been working with. James Johnstone is module coordinator for a second year module - Principles and Practice of Sports Science and Hazel Wagner is module coordinator for Advanced Corporate Reporting a third year module. Hazel and James spoke about the issues with the modules prior to the ESCAPE project and took us through the process of re engineering that they went through in order to increase the amount of engagement that the students had with their assessments. We had alluded to this at the Ulster conference ( see previous blog entry) but it was really exciting to hear the people responsible for managing and applying the changes speak with passion about their modules. The presentations gave an indication of what was possible if to quote James " .......we leave the Rolls Royce in the garage" This is a reference to the work done by Win Hornby, of Robert Gorden University on efficiency and effectiveness in assessment .

The assessments developed by Hazel and James were effectively multi component designed to engage students on a regular basis over the whole semester. They have multiple opportunities for feedback to students and for students to engage with the feedback,using it to inform their application to the next component of their assessment.

One of the themes of the assessments is that it gives the chance for the lecturers to understand where the students are at each part of the module - to really get a feel for the students grasp of topics or issues. This reminded me of the example of when I walk Holly, my Springer Spaniel, we are only together at the start and the end of the process - where she goes in the middle bit I am not too sure - we meet again at the end. This is all to often what happens with our students -we know what they are thinking at the start of the lecture and we know where they are at the end - but they probably have not been following us all the way through.

The seminar finished with question and answer session with the audience keen to quiz James and Hazel on their experience of applying the new assessment.

Collateral Effects

As the ESCAPE project moves forward through its various stages it is becoming increasingly apparent that thee are a great many collateral effects from the project. Many of the conversations that are taking place include references to stakeholders using the techniques , technologies and practices that they have been exposed to in modules other than the "official ESCAPE "ones. There are many incidences of the ESCAPE influences spilling over in to related module areas. Examples include:

• a new approach to supporting practical work in Life Sciences, using short videos of practical techniques made available on studynet.This will have a wider impact across the school on other modules that use practical work
• The increased use of flip cams as a teaching aid - with students being encouraged to produce their own videos
• the adoption and embedding of the WATS mathematical tutorial system across a number of on ESCAPE modules, rather than the initially envisaged single module
• approaches to group work and submission of draft course work - whilst not adopted in a particular level 3 ESCAPE module being considered relevant for a level 1 non ESCAPE module, that the module leader teaches on.
• ESCAPE module leaders starting to act as "agents for change " across other modules and with other module teams - this is possibly a consequence of the growing confidence felt by the ESCAPE participants

It will be interesting to see how these collateral effects develop, as they will be key to the project's sustainability over the long term - when the ESCAPE pebble is sitting at the bottom of the pond its ripples will still be felt!

Railway Journey

Yesterday I attended the "Engaging and Responding to Learners" workshop run by JISC 'in Birmingham. It was an extremely valuable day - both in terms of the workshop content and as a forum to network and engage with colleagues on other projects.

I took the train from Euston. The different types bright shiny new rolling stock that seemed to appear briefly and then whizz past the window provided an interesting distraction during the journey. It struck me that when there was just British Rail operating the railways, there was a one size fits all policy - such that often trains would depart having four, eight or eleven nearly empty carriages.

The situation on the railways has a parallel with education - we are investing heavily in technological solutions for the students educational journeys. There is much more willingness to look at bespoke solutions and to embrace new ways of thinking than stay with the one size fits all delivery mechanisms of the past. The result? - faster smoother journeys for all!

For those interested - on my journey the train was hauled by a EWS Class 90 and I returned to Euston on a Virgin Pendolino

What if ?................

It has been another busy couple of weeks.The reflective interviews are continuing with the module coordinators and module teams from the four Business School modules that we are working with. As I transcribe these interviews the passion that the lecturers have for their subject shines through. I suspect as long as the passion is there any problems can be overcome. I have had a positive reception from every one in the schools I have been working with and people are keen to share what they feel the issues are and how they (and we!) can work towards a improving the student engagement and attainment.

The next phase is to start to build up case studies using the interview material and the information gleaned from the base line audit of assessment practice that we have carried out. The case studies are quite comprehensive as the explore the module assessment landscape from the perspective of the people who set and assess the assignments . It will detail the role played by individual lecturers in assessment and teaching on a module and look to build a holistic picture of the subsequent assessment experience of a student taking the module. We are looking beneath the surface of the assessment practice to see exactly how and why the module works the way it does.

We are also trying to capture the student view via a student assessment questionnaire and are looking to conduct some focus groups. This information will be critical to our evaluation of the project to look at the before and after picture of student engagement and attainment on the modules.

We are now starting to thing about the next phase of the project - the most exciting - that of the re engineering of the module - this is when we dare to ask "What if ?"

Final Version of Project Plan to JISC

We have just submitted our " final version" as opposed to "draft version" of our project plan. the plan was revised in the light of comments received from our colleagues at JISC and from the steering group. The Plan is now quite a hefty document - including the work packages it runs to 48 pages. We have revised some of our time lines. In particular we have allowed much longer for the design and development of the assessments for learning phase. The modules we are working with are a mix of one and two semesters long. The single semester modules will run between October 2009 and February 2010 - which means in March we will have an idea of how well our redesign has worked.

This week we are starting to apply our appreciative inquiry methods to the school of Life Sciences. We are conducting interviews with members of the module teams. This will allow us to get to know each other and start the inquire phase of the Appreciative Inquiry method. We will be asking them about good learning experiences they have encountered. We will be exploring what was it that made it a good experience. with questions such as:

• what happened?
• who was involved?
• what was your part in the experience?
• what was it that made it possible?

Using these reflections we will to start to evaluate the assessment practices of the modules. We will be looking to establish a way of evaluating the assessment against efficiency and effectiveness.

ESCAPE coverage in the Universe

Hi there, my name is Elizabeth Terry and I am a Business Studies student currently on placement with the Blended Learning Unit/ Learning and Teaching Institute (BLU/LTI). Working in such a committed learning environment, I have come to realise just how much the BLU/LTI value the students' opinion. Before working here, I had no idea how much thought went into enhancing the student experience, particularly within the classroom. Because of this, I have decided to do all that I can to make the students fully aware of what activities we do and the benefits that they bring to not only the university, but also their studies.

Liaising closely with Dawn Hamlet (Vice President Academic Support and Campaigns), I have successfully been able to get numerous stories on various projects that the BLU/LTI are running into the Universe (the student’s paper that has on average 4,000 readers). This week, the Universe features a story on the ESCAPE project. The article highlights concerns that are attached to assessment and invites the students to give their opinion and views on assessment and feedback.