FInal hurrah?

So Dr Helen Barefoot and I presented the ESCAPE work and the ESCAPE (Assessment for Learning) toolkit at the recent International Soiety for the Scholarship of Teaching and Learning (ISSOTL) conference in Liverpool. There was something really nice and rounded presenting with Helen. She and I put the original bid together all those years ago. Really pleased that Helen had another chance to articulate some of her thinking on assessment and feedback.

We presented to a smallish (but packed) room. I hear they had to turn people away form our session due to the crowded conditions. Anyway, it was useful to see, what appeared as, genuine enthusiasm for our work and the toolkit. Prof. Chris Rust (long standing expert in assessment) asked for access to the resources - he wants to take them to an assessment workshop he is running at Kingston University.

Really pleasing that as the project finishes - so the legacy remains and the resources are there for others to use and engage with. Sustainability, and growing better assessment practice was at the heart of the ESCAPE project and it seems like we are doing exactly that.

Mark

On Tuesday 25th May I was at Oxford Brookes University as part of a review event for the Programme Assessment Strategies (PASS) Project. Also at the event was Graham Gibbs. Graham spoke about his experiences of effecting change within programmes and institutions. His views mirrored our experiences on the ESCAPE project, namely that there is a "hidden" network of influence that you must be able to tap into, if you want to effect change within a school . A component of the "hidden" network are the informal meetings that take place in schools. For example; in the corridor, in social areas, over coffee and at lunchtimes. These informal venues are often where the hierarchy is relaxed and a more open discourse is possible. The discussions are often robust and play key part in forming opinions within schools.

Graham's views certainly mirrored our experiences.It was something that we had anticipated when considering our approach to working with our stakeholders. One of the early parts of the ESCAPE project involved mapping the influences of individuals and teams within schools. We looked at who were the key opinion formers within the schools and how we could get them "on side". Additionally we looked what were the formal and as important, informal channels of communication within schools.

We also looked at how we could encourage collateral effects - how the stakeholders we were working with could influence others. Both over the course of the project and beyond as part of our sustainability planning.

As part of the event I was invited by Peter Hartley - who chaired, to present an outline the ESCAPE project. It was extremely valuable to get the benefit of the teams experience experience in the discussion that followed.

Although the tenure of the meeting was one of looking a programme level interventions rather than at modular level ( - which is the ESCAPE perspective), there are common assessment themes that transcend both projects which started to emerge - such as ownership of the assessment and managing change within teams.

I was able to suggest some alternative approaches for the PASS project that involved taking a cross module approach to assessment that is designed to break down the barriers between modules as a "halfway house" to full programme level assessments. I spoke about the experiences of two of the programmes at the University of Hertfordshire ,that have worked towards implementing a more integrated approach to assessment that attempts to transcend the module based assessment model.

As part of the meeting Graham Gibbs discussed his guide "using assessment to support student learning" which is available at:

http://www.leedsmet.ac.uk/100317_36641_Formative_Assessment3Blue_WEB.pdf

Graham provided me with a copy of the guide during the meeting. It is an extremely useful resource for teams embarking on using assessment as a vehicle for learning. The ideas set out by Graham are illustrated and supported by case studies, which makes it very accessible and relevant to the reader.

On my train journey home I was able to reflect on what had been really useful and informative day!

Collateral Effects

We have been approached by some of the teams working in the sports therapy area to see if we can work with them to look at their approach to assessment and learning on two of their modules. They had come along to one of the university wide presentations that we gave on the ESCAPE project .

We are exploring with these modules how the the use of student produced videos of therapeutic techniques can be used to support the students with their practical classes. One of the themes that a is emerging is motivating students to practice the techniques on a regular basis so that they can master them.

After speaking with one of the module coordinators this morning it striking how similar many of the assessment issues resonate with those that we are already working on. It is clear that the foundation that we have laid with the original module teams is providing a jump off point in applying the assessment approaches that have been adopted to these "secondary effect" modules . We had of course expected the to be secondary and collateral effects - viz: others we would work with or would influence, but what is surprising is how portable some of the techniques and approaches that our module teams have developed piloted may turn out to be.

Use of a Visualiser in Mathematics Support

One of the technologies that we are looking at to support students is the use of short videos. We are working with staff and students using flipcams to demonstrate laboratory and mathematical techniques.

IN the case of the mathematical techniques, we are using a visualiser to project the image onto a white board and focusing on the projection of the paper that the calculation is laid out on.

The calculation fills the screen and the lecturer explains the mathematical technique, carrying it out step by step. The lecturer's narration and explanation indicate the difficult parts of the calculation and guides the student through the technique. The videos will be made available on Studynet and students opinion on their usefulness in supporting the mathematics workshops will be sought. Their particular use will be demonstrating applied mathematical laboratory techniques when students which students traditionally have problems with.

Is a good example of how simple technology can support the appropriate pedagogy in delivering student centered learning. Furthermore we are able to gain the student view as to their usefulness through the discussion feature on Studynet.

Here comes the data (1) ...

We are now starting to see the fruits of our labour on the ESCAPE project. Some of the 'fruits' arise when we talk to staff about their assessment practice and resulting experience whereas other 'fruit' arises from observations of student performance and their engagement with their studies. I thought I would share some of what we are finding ...

A new module (not one of the original ESCAPE modules) wanted help with Peer Assessment. The member of staff was already engaging her students with peer assessment and hence were reaping numerous learning gains.

* Students were able to see how their peers responded to the same task
* Students were able to engage more with the marking criteria and standards
*

Previously, however, she was kept very busy after the peer assessment by dealing with more than expected students questioning their marks. Whilst it is highly appropriate that the students are exposed to a fair and reliable assessment, many of the efficiency gains made by the staff member were lost due to the need to deal with students on a one-to-one basis.

With the help of the ESCAPE project we were able to re-purpose a web-based data collection facility (developed to support computer based assessment). Using the web based data collection facility we posted to the students a series of questions (asking them to reflect on their own submission and the peer assessment process. This was an addition to the work previously done and hence created addiction learning gains. Importantly, we also included an opportunity for the students to 'comment on their mark' and note that if they were over or under marked to provide evidence where this was the case -with reference to the marking criteria used in the peer assessment process. The result was of which a vast reduction in 'additional' time required by the staff member to look at the concerns.

This addition was a real win-win. Students were now reflecting on the process (and sharing the reflections with staff) and the staff were reaping efficiency gains.

Slightly self-promoting but this typifies exactly the type of things this project is about. Using technology (led by pedagogy) to reap learning and efficiency gains.

I can't see this technology-supported intervention stopping when the project finishes.

Interim Reporting and Future Plans

We are just completing our second (and last) interim report. It deals with the project's progress over the last six months. A large part of the report deals with mapping the proposed project objectives and outcomes against what we have been able to achieve so far. One important theme that emerges from the initial draft of the report is the sustainability of the project, that is what will happen when the funding finishes in October. How will we ensure that the projects legacy will continue? The report provides a useful focus for articulating these plans in the light of our experiences over the last 18 months.

It was good to meet with the ESCAPE module teams on Wednesday - the half day event was a chance to catch up and swap experiences. The fact that we were able to take people away from the university for a day is a real bonus and gives people a chance to take stock and to reflect without distractions.

We were able to capture the reflections using the flipcams - now in their second generation so that they can record for two hours, although nobody took the machine to its limits. Our reflections to camera lasted about five minutes for each participant.

Along with the reflective theme that we were developing we are also looking at the collateral effects that the project is generating. some examples of these are:

• Plans for the WATS system to be used with the MSc Life Science students next academic year - building on the successes in using it with with a first year Life Science module


• Working with another three modules from the division of sports science over the coming months - facilitating and supporting a re - engineering approach to their assessments


• Working with one of the ESCAPE module coordinators on a masters module to support the use of video to assess group work


• Working with one of the compulsory first year modules in the Business School that has over 1600 students taking it in two cohorts.


• Being approached by a module coordinator to help to re-engineer the assessment for a large Health & Human module that involves students on placement - assessing them them through the use of group work and a piece of reflective writing.

We are looking at what is the best way to work with people who have approached us and we will probably work with individual teams or areas as and when we are asked to. This approach offers us quite a bit of flexibility in the approaches we use - rather than using a larger " come into the big tent and see the show" approach.

Belfast Bound!

Last week Mark and I presented some of our observations from the ESCAPE Project at the University of Ulster eighth annual eLearning Conference. I had never been to Northern Ireland before and was excited by what I saw - it certainly challenged my preconceptions . Belfast is a vibrant town, new buildings and infra- structure projects were springing up everywhere - and there was a real energy to the city. The venue for the conference was the School of Art and Design. The campus consists of two buildings, one a turn of the century the other a much more modern one. The buildings are connected by a new covered walkway. They were really excellent learning spaces, well designed and resourced.

Our presentation, which we gave twice, looked at the background to the ESCAPE project and the wider assessment landscape. Mark spoke about the challenges of the project and how the extensive literature on learning and assessment should be informing teachers practice. I spoke about working with module teams and discussed some of the changes that teams had put in place as a consequence of working with ESCAPE. We also discussed the role Appreciative Inquiry played in the project .

On the way back to the airport the taxi driver took us to see the Thompson Graving Dock. This is where the Titanic, which was built at Harland and Wolf in Belfast, was brought and fitted out after she was launched.

I contrasted this with the the keynote address at the conference, where professor Paul Moore spoke about the changing technologies that we use and how they have influenced us, using an in car record player from his father ford Anglia to illustrate his point. He spoke passionately about the iPhone and contrasted this with his blackberry which he referred to as a " wheelbarrow for emails". His perspective was an interesting one and his excitement about the impending launch of the iPad was infectious. Paul felt that this device would have a huge influence on learning over the coming years - interesting times!

For those of you that have not realised - the flag at the top of the blog is that of the White Star Line, the shipping line that operated the Titanic.

eBiolabs University of Bristol

We have been really impressed by the eBiolabs project at the University of Bristol - one of the three other project in our cluster. The project has many resonances with the work we are doing with one of our 9 modules -" Introduction to Biochemistry, Microbiology and Pharmacology". The module contains a large element of practical work and the eBiolabs approach of preparing students before they undertake the practicals by taking them through a series of online tasks. This element of engagement prior to the practical work itself and the corresponding post practical online tasks would be something that we would like to incorporate in the module.

We have spoken with Gus Cameron - the eBiolabs project manager and we are hoping to be able to use some of the stand-alone resources that his team have produced. This is an exciting opportunity for projects to start to disseminate the products that they have developed. We In turn anticipate that the case studies that we are developing along with our change management model - incorporating the AI approach will become widely available to others. The use of the JISC Design Studio is the ideal forum for such dissemination of project deliverables

Sustainability – thoughts from the Experts.

The JISC operate a Pedagogic Experts Group which I have been fortunate to have been a member for a little while. At a recent event a cluster from the Curriculum Design programme were invited to engage the experts as were a cluster from the Curriculum Delivery programme. We (ESCAPE) were part of the Curriculum Delivery cluster invited to talk to the Experts. I jumped at the chance and suggested that our session be about sustainability.

I am keen that our project does not wither on the vine when the funding runs out, nor do I want our leaning to stay confined to us at UH. And so I was keen that we didn’t just engage in a traditional show-and-tell routine but use the Experts Group to help shape and / or validate our thinking on sustaining our projects.


After a brief into from Malcolm Ryan – the critical friend of the cluster, Gunter Saunders outlined some of the work from Westminster’s project. I followed and briefly introduced ESCAPE. We then moved to ask the group …

* What would you do to ensure that the projects are not sustained ! (reversing the question is just great to get the ideas flowing ) and

* What one thing would you suggest the projects consider / do to ensure the sustainability of the projects.

Some great ideas flowed

Working with Dominic we I will go through the list and use the thoughts / ideas to challenge our own work.

I always like the buzz that seems to be present at the JISC Experts Group and I was particularly (selfishly?) pleased that some of the buzz and thinking will flow into our ESCAPE project.

Collateral Effects

As the ESCAPE project moves forward through its various stages it is becoming increasingly apparent that thee are a great many collateral effects from the project. Many of the conversations that are taking place include references to stakeholders using the techniques , technologies and practices that they have been exposed to in modules other than the "official ESCAPE "ones. There are many incidences of the ESCAPE influences spilling over in to related module areas. Examples include:

• a new approach to supporting practical work in Life Sciences, using short videos of practical techniques made available on studynet.This will have a wider impact across the school on other modules that use practical work
• The increased use of flip cams as a teaching aid - with students being encouraged to produce their own videos
• the adoption and embedding of the WATS mathematical tutorial system across a number of on ESCAPE modules, rather than the initially envisaged single module
• approaches to group work and submission of draft course work - whilst not adopted in a particular level 3 ESCAPE module being considered relevant for a level 1 non ESCAPE module, that the module leader teaches on.
• ESCAPE module leaders starting to act as "agents for change " across other modules and with other module teams - this is possibly a consequence of the growing confidence felt by the ESCAPE participants

It will be interesting to see how these collateral effects develop, as they will be key to the project's sustainability over the long term - when the ESCAPE pebble is sitting at the bottom of the pond its ripples will still be felt!

Sustaining ESCAPE - Learning from other projects

At the heart of the ESCAPE project is the need to sustain and embed our practice. We are after all about developing sustainable change. Using the standard JISC template for our Project and Evaluation Plans we were required to present some thoughts on Sustainability and Exit. I really like the idea of asking the project to think about these issues at the outset and not at its end. Thumbs up from me!

The University of Bradford are running another JISC funded project - Audio Supported Enhanced Learning (ASEL) - to which we (UH) are partners. The project is nearing completion and today (11 Feb 2009) they ran a UH / Bradford meeting to review how the institutions would sustain the ASEL activity and continue to reap the benefits suggested of audio supported learning.

I am really keen to see legacy activity from these type of projects and not just see all the good work fizzle out and dwindle just because the project and funding has formally closed. I was invited as part of the UH team to attend the day to which I seized the opportunity. I want our project, ESCAPE, to learn from ASEL activities and see how we might sustain ourselves long after we have finished. In particular I am interested in what things we might do now to help our project sustain itself and what questions might we ask at the start of the project that would help us at the project close.

Whilst I was happy to support ASEL, want to see UH benefit from a real legacy of the ASEL project, I also wanted, (more selfishly), to see what lessons I could take away from the meeting to benefit ESCAPE.

We had an interesting start to the day where as a group we explored how the project had met its objectives. I think this will be a useful exercise for ESCAPE too i.e. to keep reminding ourselves what we set out to achieve and constantly relate our activity to that.

Bob Rotheram from the Leeds Metropolitan University followed that session with a great presentation about the JISC funded Sounds Good Project - Sounds Good is about giving better feedback and using audio to support the feedback activity.

After Bob outlined his project and the benefits, I was keen to find out how we going to sustain the Sounds Good activity (i.e. in line with the purpose of the meeting and for my selfish needs!)

Bob spoke about the importance of enthusiasm and then I asked, in addition to enthusiasm what three things do you think are important to sustain Sounds Good.?

Bob spoke about the time savings it could bring staff and qualified this with some great examples.
It is quicker (audio feedback) than written feedback if …
• You give a lot of feedback
• You speak faster than you can write

Be a really useful exercise to quantify this and show staff some break information
Bob also said that technology needs to be simple and useable and not be a barrier to uptake. Some great thoughts.

Naturally Bob also\mentioned the positive benefits to students in terms of receiving audio feedback. I really liked the pragmatism about being both student and staff focused. Staff are very busy and they need convincing of a need to change practice and see the benefits for them and their students

Peter Chatterton asked a follow-up question …

If you had a pot of gold what would you use it for to sustain Sounds Good ?
Fortunately Bob said not much was needed, sounded like a ripple effect was kickimg in, but he did say …
• Give staff decent devices
• Run workshops
• Give a chance for staff to come together to share and discuss activity


It struck me also that just as e-learning was a Trojan Horse to pedagogy so too was the use of Audio. The morning session gave a few examples which suggested, for me, it was not developing pedagogy but giving what we already know about learning to be surfaced.


In the afternoon we broke into our Uni teams and discussed how we might sustain ASEL. Asking ourselves …
why bother?
How do we convince an academic not connected with the project to engage?

We collected many ideas - which will `be posted here when they are summarised, but some things for now …

Provide some training packages
Utilising existing support - not develop new systems
Resources might include how do you do it
What do you need to avoid
i.e. a collection of resources related to ‘how to’
as well as a collection of resources relating to ‘why to’

We also mentioned strengthening the student mentor scheme run at UH - and if new / different skills are needed for audio activity then we need to describe what these are and up-skill the mentors accordingly

* Linking it to podcasting campaign
* Tagging resources so that we don’t reinvent stuff
* Networking
* Sustaining communities
* Inter-faculty meetings/
* Cross University inter-faculty meetings
* Link with the Podcasting for Pedagogic Purposes group?
* Not invent what is already out there

A couple of things ASEL might do now
• Develop crisp selling messages
• Use the right language for different groups
• Tap the why do it into responding to challenges
• Sell to Heads of School
• Share an Executive summary


I was interested in asking the team to map out the audio landscape post ASEL and then establish what is needed to support a vision

i.e. where we will be in
1 week
4 weeks
2 months
6 months
1 year

I would like to do this for ESCAPE - map out what we want UH to look like after ESCAPE and establish what we need to make that vision-landscape a reality.