FInal hurrah?

So Dr Helen Barefoot and I presented the ESCAPE work and the ESCAPE (Assessment for Learning) toolkit at the recent International Soiety for the Scholarship of Teaching and Learning (ISSOTL) conference in Liverpool. There was something really nice and rounded presenting with Helen. She and I put the original bid together all those years ago. Really pleased that Helen had another chance to articulate some of her thinking on assessment and feedback.

We presented to a smallish (but packed) room. I hear they had to turn people away form our session due to the crowded conditions. Anyway, it was useful to see, what appeared as, genuine enthusiasm for our work and the toolkit. Prof. Chris Rust (long standing expert in assessment) asked for access to the resources - he wants to take them to an assessment workshop he is running at Kingston University.

Really pleasing that as the project finishes - so the legacy remains and the resources are there for others to use and engage with. Sustainability, and growing better assessment practice was at the heart of the ESCAPE project and it seems like we are doing exactly that.

Mark

Assessment patterns - not long now!

As this project draws to a close so we are keen to keep supporting UH. Indeed our University has funded a year long, UH-wide project related to assessment. Naturally, the ESCAPE project will feed beautifully into one of the strands of the project.

One of things I'm drawing together is a set of assessment patterns. The patterns are graphic images that show the likely consequences of different assessment regimes. The patterns show differently weighted assessments located at different places on a timescale. I've shown these evolving patterns to a couple of (internal) audiences and they seem to be well liked. Just tidying up the document now and I'll post it here for comment. Watch this space.

I'd be really interested to hear your thoughts on the usefulness (or other) of the graphic representations.

Mark

Final Cluster Meeting

We have just had our last cluster meeting, which was held at the University of Exeter. The theme of the meeting was to give each project a chance to present what had been achieved over the two years and to look to the future - beyond the official end of the projects at the end of October. It gave us a chance to per critique each other projects. We presented on the work of ESCAPE and Mark demonstrated aspects of the ESCAPE Toolkit. Themes that we covered included:

• the development of a set of ESCAPE Principles


• mapping the current assessment landscape to these principles


• considering efficiency verses effectiveness


• what does transformative change look like


• demonstration of some of the "themes in practice" videos

We talked about how the project has laid a foundation for staff engagement with a year long university wide assessment project that is running post ESCAPE.

The meeting was in the usual format of a two day timetable with meeting spread over lunchtime on the Thursday to lunchtime on the Friday. Helen Beetham joined us on Friday to facilitate a session on exploring what we have learnt as a cluster. We were looking to build upon the collaborative efforts of our joint cluster presentation at the University of Greenwich e- learning Conference. There were some interesting ideas for further collaboration including a collaboration with the University of Exeter (INTEGRATE Project) with a sharing of resources from our projects.

We agreed that beyond the final programme meting in October we would look to meet in 12 - 18 months to look at what impact our projects have had.

Video of our ESCAPE themes

I am in the process of capturing our thinking and work in the form of short Camtasia videos. What follows is a video on our ESCAPE themes.

Thoughts on the format and themes are, as always, most welcome

Mark

ESCAPE themes - so far so good!

Readers of the ESCAPE blog, and those of you that know our project, will be aware that we are trying to support staff develop both 'educationally effective' and 'resource efficient' assessment.

We believe that all staff want to do a good job but also acknowledge they experience various demands on their time. Research, consultancy activity, teaching, assessing students, providing pastoral care etc.

To reconcile the desire to help staff develop good assessments against the 'time-demanding' backdrop, part of our engagement is to provide ready access to the literature. Sure, staff might want quick 'hints and tips', and we are providing them, but we are keen too to make sure that the hints and tips etc. are located against a set of themes that describe what good assessment looks like.

For completeness, although stated elsewhere our ESCAPE themes are:

Good assessment for learning...

• Engages students with the assessment criteria
• Supports personalised learning
• Ensures feedback leads to improvement
• Focuses on student development
• Stimulates dialogue
• considers staff and student effort

These themes have evolved over the duration of the project and now follow us around as we talk about the project and work with our ESCAPE partners.


It's really helpful to know that as we share our themes in different arenas we are not seeing any adverse reaction. They seem to capture the thoughts of the practitioner too; indeed we wrote them with a view on being accessible and having face validity. They have been shown on numerous events, both inside and outside the University of Hertfordshire, and are seemingly doing what we set out to do.

In fact a participant at a recent event indicated he seen them before, Unlikely, (highly unlikely), but at least it shows the notion that the themes are not challenging 'appropriate' thinking about good assessment and that staff already indicate they like them.


Mark

ESCAPE @ IBLC 10

Dominic and I recently presented our work at the International Blended Learning Conference. The room was full – which was just great. We were joined too by Sarah knight and Marianne Sheppard from the JISC and so it was just great that they could pick up another update of our work and hear delegates thoughts of our work from delegates.

Essentially ..
1. We outlined the context for our project
2. We gave a run through of main features of the ESCAPE toolkit

• Themes and questions (Diagnostic)
• Features and consequences
• Ideas (including our three minute assessment Tips, Themes in Practice, see earlier blog post)

3. We demonstrated an example of change in action

It was really good to see some of the concrete things emerging from the work. It was particularly rewarding too to look back at some of our Partner modules and just see how far we have taken them; both in terms of their thinking about assessment and feedback practice and also in their use of technology. .

We took a couple of questions at the end of the session. Perhaps the most encouraging was a question asking if access to our ESCAPE toolkit was possible – It is! Such interest validates our thinking and also shows how projects run by one institution can also help and influence practice elsewhere too.
I was really pleased that Sarah and Marianne were able to join us and see the fruits of their programme being shared with others.

Making an existing assessment activity more efficient

One of the 'additional' modules that sought our advice was Human Physiology - School of Life Sciences. Within that module the students were already engaging in peer assessment.

The peer assessment activity brought many learning opportunities but some of the hoped for time savings were lost with students immediately questioning the marks given by their peers.
Staff were engaged in time consuming activity associated with moderating the marks.


ESCAPE involvement sought to support the existing work but use technology to drive more reflection, try to get the to students' understand their strengths and areas for improvement and also require the students to present a considered and evidenced-based argument where they believed they were over or under marked.

The use of a web based data collection facility (offered by the ESCAPE project) provided some great data, encouraged the learning and also saved staff time.

The story was presented as a poster at the Fifth International Blended Learning Conference and represents a really good example of ESCAPE in practice.

Poster can be found here

Here comes the data (1) ...

We are now starting to see the fruits of our labour on the ESCAPE project. Some of the 'fruits' arise when we talk to staff about their assessment practice and resulting experience whereas other 'fruit' arises from observations of student performance and their engagement with their studies. I thought I would share some of what we are finding ...

A new module (not one of the original ESCAPE modules) wanted help with Peer Assessment. The member of staff was already engaging her students with peer assessment and hence were reaping numerous learning gains.

* Students were able to see how their peers responded to the same task
* Students were able to engage more with the marking criteria and standards
*

Previously, however, she was kept very busy after the peer assessment by dealing with more than expected students questioning their marks. Whilst it is highly appropriate that the students are exposed to a fair and reliable assessment, many of the efficiency gains made by the staff member were lost due to the need to deal with students on a one-to-one basis.

With the help of the ESCAPE project we were able to re-purpose a web-based data collection facility (developed to support computer based assessment). Using the web based data collection facility we posted to the students a series of questions (asking them to reflect on their own submission and the peer assessment process. This was an addition to the work previously done and hence created addiction learning gains. Importantly, we also included an opportunity for the students to 'comment on their mark' and note that if they were over or under marked to provide evidence where this was the case -with reference to the marking criteria used in the peer assessment process. The result was of which a vast reduction in 'additional' time required by the staff member to look at the concerns.

This addition was a real win-win. Students were now reflecting on the process (and sharing the reflections with staff) and the staff were reaping efficiency gains.

Slightly self-promoting but this typifies exactly the type of things this project is about. Using technology (led by pedagogy) to reap learning and efficiency gains.

I can't see this technology-supported intervention stopping when the project finishes.

Cluster Meeting 25th & 26th March 2010

We have just hosted the third meeting of our cluster (universities of Bristol, Exeter, Hertfordshire and Westminster). The venue was the Comfort Hotel in St.Albans. The hotel was at one time the home of Samuel Ryder of Ryder Cup fame. The meeting ran as is usual from lunchtime on the 25th to lunchtime on the 26th. This gives everyone a chance to arrive in good time to start the meeting and for a social event in the evening - usually a group meal at a local restaurant. This time is valuable as much of the meeting is about sharing experiences of our projects and seeking advice from our fellow cluster project teams and our critical friend Malcolm Ryan.

The first day kicked of with a welcome and introduction by Malcolm and a review of the ground rules for the two days. The Chatham House rules mean that we can speak candidly about issues and events arising in the projects. We then moved on to looking how we had used the Appreciative Inquiry approach with our project and a facilitated session run by Rachel Harris of Inspire Research. We then moved on to evaluation and looked at how we are approaching this- thinking about "what a successful project looks like " and discussing the value in understanding why something did not work out as planned. The day was rounded of by a very pleasant meal in which the topics discussed during the day were further explored.

The second day looked at the plans for the final cluster meeting at Exeter - a draft agenda was proposed and celebrations planned! We then moved on to plans for a joint conference submission by the cluster at the Geenwich e learning conference in the summer. The day finished with a session on using Fishbone Analysis to explore the sustainability agenda of the projects.

All in all a really useful two days - we have similar experiences and issues and it was an excellent opportunity to raise them and to identify ways to move forward.

JISC Experts Group

On Wednesday 17th March I presented a workshop on the ESCAPE Project at a meeting of the JISC experts group in Birmingham. The workshop gave an overview of the Appreciative Inquiry(AI) approach that we took with module teams. We then explored the emerging findings from two of the modules that we are working with. In particular we looked at how educational technologies (online submission, wikis, and online groups for example) provided a medium for achieving the pedagogical aims of the project (for example: a more consistent engagement with the curriculum, increased opportunities for students to engage with feedback and the facilitation of more personalised leaning opportunities). The day provided a good opportunity for getting feedback on the project and there were many questions on the use of AI.

It is indicative of the stage that we are at in the project that we are now starting to present the emerging results of the project on a national stage .This was always was part of our dissemination plan and its nice to be in a position to start to see all the hard work done by the module teams starting to bear fruit.

Sharing what we do

As a project team we are really keen to share with the sector  our work.

This manifests as presentations both internally and externally to the University of Hertfordshire. You will see that we have received a few invites too to present our work.

The University of Hertfordshire has recently re-launched its in house journal relating to teaching and learning. Or more specifically, Blended Learning. The journal has a practitioner focus and uses a variety of formats to engage the readers. The title of the Journal is aptly named Blended Learning in Practice (BLiP)

Dominic and I recently published in the journal. The article can be found here.

Supporting Peer Assessment

During the recent half day ESCAPE review meeting we took along our Flipcams and 'invited' modules leaders (we are working with) to record an introduction to their developments. These short videos will provide a really useful resource-bank and will be useful for colleagues both inside and outside UH to see what we have been doing and more importantly consider the adoption of some of our approaches to their own practice. Thanks to all that contributed so positively :-)

Helen Barefoot and I took a Flipcam outside (for a walk) to hear about the recent ESCAPE activity supporting Peer Assessment.

Module Coordinator Experiences

This week we held a lunch time seminar to give staff a chance to learn about the project and to hear from two of the module co-ordinators that we have been working with. James Johnstone is module coordinator for a second year module - Principles and Practice of Sports Science and Hazel Wagner is module coordinator for Advanced Corporate Reporting a third year module. Hazel and James spoke about the issues with the modules prior to the ESCAPE project and took us through the process of re engineering that they went through in order to increase the amount of engagement that the students had with their assessments. We had alluded to this at the Ulster conference ( see previous blog entry) but it was really exciting to hear the people responsible for managing and applying the changes speak with passion about their modules. The presentations gave an indication of what was possible if to quote James " .......we leave the Rolls Royce in the garage" This is a reference to the work done by Win Hornby, of Robert Gorden University on efficiency and effectiveness in assessment .

The assessments developed by Hazel and James were effectively multi component designed to engage students on a regular basis over the whole semester. They have multiple opportunities for feedback to students and for students to engage with the feedback,using it to inform their application to the next component of their assessment.

One of the themes of the assessments is that it gives the chance for the lecturers to understand where the students are at each part of the module - to really get a feel for the students grasp of topics or issues. This reminded me of the example of when I walk Holly, my Springer Spaniel, we are only together at the start and the end of the process - where she goes in the middle bit I am not too sure - we meet again at the end. This is all to often what happens with our students -we know what they are thinking at the start of the lecture and we know where they are at the end - but they probably have not been following us all the way through.

The seminar finished with question and answer session with the audience keen to quiz James and Hazel on their experience of applying the new assessment.

Belfast Bound!

Last week Mark and I presented some of our observations from the ESCAPE Project at the University of Ulster eighth annual eLearning Conference. I had never been to Northern Ireland before and was excited by what I saw - it certainly challenged my preconceptions . Belfast is a vibrant town, new buildings and infra- structure projects were springing up everywhere - and there was a real energy to the city. The venue for the conference was the School of Art and Design. The campus consists of two buildings, one a turn of the century the other a much more modern one. The buildings are connected by a new covered walkway. They were really excellent learning spaces, well designed and resourced.

Our presentation, which we gave twice, looked at the background to the ESCAPE project and the wider assessment landscape. Mark spoke about the challenges of the project and how the extensive literature on learning and assessment should be informing teachers practice. I spoke about working with module teams and discussed some of the changes that teams had put in place as a consequence of working with ESCAPE. We also discussed the role Appreciative Inquiry played in the project .

On the way back to the airport the taxi driver took us to see the Thompson Graving Dock. This is where the Titanic, which was built at Harland and Wolf in Belfast, was brought and fitted out after she was launched.

I contrasted this with the the keynote address at the conference, where professor Paul Moore spoke about the changing technologies that we use and how they have influenced us, using an in car record player from his father ford Anglia to illustrate his point. He spoke passionately about the iPhone and contrasted this with his blackberry which he referred to as a " wheelbarrow for emails". His perspective was an interesting one and his excitement about the impending launch of the iPad was infectious. Paul felt that this device would have a huge influence on learning over the coming years - interesting times!

For those of you that have not realised - the flag at the top of the blog is that of the White Star Line, the shipping line that operated the Titanic.

eBiolabs University of Bristol

We have been really impressed by the eBiolabs project at the University of Bristol - one of the three other project in our cluster. The project has many resonances with the work we are doing with one of our 9 modules -" Introduction to Biochemistry, Microbiology and Pharmacology". The module contains a large element of practical work and the eBiolabs approach of preparing students before they undertake the practicals by taking them through a series of online tasks. This element of engagement prior to the practical work itself and the corresponding post practical online tasks would be something that we would like to incorporate in the module.

We have spoken with Gus Cameron - the eBiolabs project manager and we are hoping to be able to use some of the stand-alone resources that his team have produced. This is an exciting opportunity for projects to start to disseminate the products that they have developed. We In turn anticipate that the case studies that we are developing along with our change management model - incorporating the AI approach will become widely available to others. The use of the JISC Design Studio is the ideal forum for such dissemination of project deliverables

Poster and Video

This last week we have been involved in the production poster and video in preparation for a joint meeting of the curriculum design and the curriculum development projects in Manchester on the 13th and 14th of October. The poster is a visual representation of the stages and mechanisms of the project. It captures the journey through the project by the stake holders and illustrates the approaches taken at the various stages of the project by the ESCAPE team. It was influenced heavily by the change management development activities that we were involved in at our last CAMEL meeting. One of the aspects of change management was looking at a four stage model of change management, viz:

• Awareness
• Interest
• Trying
• Adopting

It was interesting mapping our project activities to these four stages and realising the the stages were not atomic- that is to say that they could overlap - and that it was possible for stakeholders to drop back a stage( or two!).

The poster does not really capture the complexity of what we are doing - for example we are working with 9 different module teams, with some module attempting quite complex and innovative changes in assessment practice, but is a good start as a "top layer" map of the project.

I am now working on a three minute flip cam video that will introduce the ESCAPE project and include a stakeholder voice. This along with videos introducing all the other projects, can be found at www.youyube.com/jisccdd - a JISC dedicated channel on YouTube.

Correspondence with Steve Draper

Following Steve Draper’s presentation at the HEA Assessment and Feedback day we have been passing emails back to one another. Steve is a great thinker and a really energising presenter. Go listen to him whenever you get the chance.

Two things came up in the correspondence.

1. The difference in the use of ppt between Steve and I. Steve amplified and told wonderfully engaging stories about the words he had on his slides. (I tend to shy away from .ppt slides that are littered with just text) and my talking over pictures. We also presented our ESCAPE activity and we wanted to help capture the attention of the delegates and thought a visual stimulus might help. We overlaid animations on graphs since we wanted delegates to really understand what we were talking about - I think the animations helped us do that (I hope) – it was just after lunch too – and I am aware I can be a tad soporific!

We discussed the usefulness of .ppt slides after such an event. With my approach it would be particularly difficult for anyone that wasn’t there to ‘get’ what I was showing. I am aware of this but also wondered how many people actually revisit the slides after such an event. Steve has a simple solution to this and brings a ready-made, single-sided hand out. The hand out comprises useful refs and links.

2. The second point we batted about, and of more relevance here, is the approach you take to engage staff in re-engineering activity. I am talking here about re-engineering curricula and in relation to ESCAPE re engineering and rethinking the assessment activities.

Although slightly extreme we discussed the merits of small steps of change versus big-bang change. It will be of little surprise that we both converged on the notion that both have something to offer.

With small steps of change …

It is more likely to keep staff engaged and on board i.e. in a way we are showing that we value what staff do but help to nudge things along (to be more educational effective and resource efficient) in a way that is both manageable and supportive.

For some staff a big-bang approach might wrench from under their feet everything they thought was good about their work. I wonder too if a step-wise approach is a great conversation starter and less threatening. Small-steps might prepare staff for some fun big-bang stuff and also develop their scholarship in a risk managed way.

The problem with small steps of change is that often you might never get to the heart of the problem. You tinker, fiddle on the edges, and whilst you might be adding incremental enhancements to a system / assessment (curriculum design approach), the approach /system / curricula you are tinkering with might actually need dropping!

Big-bang gives a great chance to ‘really’ explore the fundamental issue, establish a vision and help set out a curricular that responds to that vision.

We are doing both with ESCAPE and will look at how personalities as well as features of the modules/assessment arrangements best suit the alternatives.

Watch this space!

ESCAPE coverage in the Universe

Hi there, my name is Elizabeth Terry and I am a Business Studies student currently on placement with the Blended Learning Unit/ Learning and Teaching Institute (BLU/LTI). Working in such a committed learning environment, I have come to realise just how much the BLU/LTI value the students' opinion. Before working here, I had no idea how much thought went into enhancing the student experience, particularly within the classroom. Because of this, I have decided to do all that I can to make the students fully aware of what activities we do and the benefits that they bring to not only the university, but also their studies.

Liaising closely with Dawn Hamlet (Vice President Academic Support and Campaigns), I have successfully been able to get numerous stories on various projects that the BLU/LTI are running into the Universe (the student’s paper that has on average 4,000 readers). This week, the Universe features a story on the ESCAPE project. The article highlights concerns that are attached to assessment and invites the students to give their opinion and views on assessment and feedback.

Scottish Journey

Mark and I travelled up to Glasgow last week to meet with Rachel Harris from Inspire Research. We were looking at how using appreciative enquiry techniques could help us in our work with the module teams in the schools. AI looks at growing what works in a organisation to foster change rather than to think of things in terms of something needing fixing.

The day was very productive and we were able to flesh out exactly how we we going to approach things on a module level. In addition we have started to put a detailed timeline together for work we are doing with the schools. We had a chance to knock around quite a few ideas and we were able to develop some ways of looking at things that should really help the sustainability of the project's influence after the project itself finishes. In particular we looked at how many modules you would need to influence in order to bring about an institutional change . Mark compared it to how many people were needed to start a mexican wave in a stadium. Was there a critical mass of modules that you needed? and if so what were the factors that influenced that number. We also looked at things from person centred approach and looked at how we could measure a persons influence within a school as an agent for change. We wanted to encapsulate how many modules a particular person could influence and see if we could express that as a kind of index or quotient. It was good to have Rachel's input and experience to keep us both grounded on what was possible within our time frame.

At the end of last week we also got the feedback on the project plan from JISC - so this week will be concerned with updating the plan in the light of these and the steering groups comments. In addition we will be and arranging meetings with the module teams within the schools to get them to examine their current practice within an AI framework.

First Meeting of the Steering Group

Last Thursday we had our first meeting of the steering group, we had an excellent turnout. It was good to see Malcolm Ryan again - our critical friend from JISC and fellow VW owner and to meet Prof. Margaret Price - from Oxford Brookes University. In addition Prof. David Nicol from the University of Strathclyde joined us via elluminate along with Prof. Peter Bullen head of BLU at UH.

We used a very large plasma screen to display the elluminate interface and Mark managed to solve some technical hitches prior to the start of the meeting so the sound was excellent.

The steering group were very supportive of what we had achieved so far and were in agreement with our approach as detailed in the draft project plan - which was approved. They were able to offer advice on areas where they felt their experience was relevant. It is good to know that we are backed up in our endeavours by such a depth and breadth of experience. The steering group’s guidance and advice will be invaluable to helping make the project a success.

Some things that I got from the meeting:

• look at the student perspective of assessment & compare with the lecturer view
  
• think about how teaching & learning staff development for new (and existing) staff can support the sustainability of the project
  
• create hard data for analysis
  
• what are the staff and student drivers here
  
• students will always want more feedback - we need to encourage and facilitate self regulated learners
  
• look at the national survey of student engagement
  
• think about the context of the change we are trying to achieve - make sure that senior champions are on side
  
• the challenge of using general principles of good assessment practice - of which there are a number of schools of thought - and turning them into actual an assessment regime for a particular subject module.
  

One of the main themes that I picked up on is how much work is being done in similar areas to our own and how we need to be aware of it. One of the discussions that Mark and I have had is his feeling that we need to have a good idea of what is out there - perhaps by carrying out a survey exercise.