The ripples are still visible

I presented on some of the aspects of the work emerging from the ESCAPE project at a development day -"Using Technology to Enhance Feedback" run by the HEA Subject Centre for BioScience and hosted by the University of Roehampton. I talked about the work that James Johnstone had been doing with his second year sports studies students. How the wiki had been used as a forum for collaborative development. I spoke about the way that the students had received formative feedback and how their work was monitored each week with James compiling a blog for general feedback.

I also spoke at length about the barriers for adopting technologies such as wikis and blogs - both from a teachers and also a students perspective,
The session was well received with lots of interest being generated. There were three workshops in the afternoon., where we had a chance to explore the use of wikis in a bit more depth. It was a very appropriate way to finish the project- showing what we had been able to achieve in partnership with the modules we worked with.

Assessment Patterns

Regular readers will note that I have previously posted about the assessment patterns.
The assessment patterns show graphically alternative assessment strategies and highlights the possible consequences of the different patterns.

The assessment patterns have been further developed and separated to allow ready access to three areas. Of course the assessment patterns are complimentary but each set relates to a particular area.

The areas relate to

* Moving away from high-stakes end of process assessment
* Making more of feedback
* Programme view of assessment

Links to the assessment patterns (docs) and a wiki and video page can be found here


https://jiscdesignstudio.pbworks.com/ESCAPE+-+Assessment+timelines

Comments welcome

Mark

FInal hurrah?

So Dr Helen Barefoot and I presented the ESCAPE work and the ESCAPE (Assessment for Learning) toolkit at the recent International Soiety for the Scholarship of Teaching and Learning (ISSOTL) conference in Liverpool. There was something really nice and rounded presenting with Helen. She and I put the original bid together all those years ago. Really pleased that Helen had another chance to articulate some of her thinking on assessment and feedback.

We presented to a smallish (but packed) room. I hear they had to turn people away form our session due to the crowded conditions. Anyway, it was useful to see, what appeared as, genuine enthusiasm for our work and the toolkit. Prof. Chris Rust (long standing expert in assessment) asked for access to the resources - he wants to take them to an assessment workshop he is running at Kingston University.

Really pleasing that as the project finishes - so the legacy remains and the resources are there for others to use and engage with. Sustainability, and growing better assessment practice was at the heart of the ESCAPE project and it seems like we are doing exactly that.

Mark

Assessment patterns - a video

Thought I would produce a video introducing the concept of the assessment timeline (assessment patterns) drawings. The timing of the assessment as well as its design and alignment with the module learning outcomes is a major consideration in getting the most from the assessment activity.


Thoughts / ideas most welcome


Introductory video follows ...

Assessment patterns - not long now!

As this project draws to a close so we are keen to keep supporting UH. Indeed our University has funded a year long, UH-wide project related to assessment. Naturally, the ESCAPE project will feed beautifully into one of the strands of the project.

One of things I'm drawing together is a set of assessment patterns. The patterns are graphic images that show the likely consequences of different assessment regimes. The patterns show differently weighted assessments located at different places on a timescale. I've shown these evolving patterns to a couple of (internal) audiences and they seem to be well liked. Just tidying up the document now and I'll post it here for comment. Watch this space.

I'd be really interested to hear your thoughts on the usefulness (or other) of the graphic representations.

Mark

ESCAPE themes - student focus

The ESCAPE (Assessment for Learning) themes have been synthesised from the literature. Indeed purposely draw on the literature to create the questions that prompt teachers to think about their assessment design in light of the six ESCAPE themes.

But the design is only one facet of good assessment. The role of the students also needs to be considered. Sure, the design should stimulate appropriate student behaviors etc. but we wanted to go further.

What we are working on now, is a set of 'accessing questions' for our ESCAPE themes that are explicitly written for students. We have the same themes (of course) but the accessing questions are different. Although, we have not tested the ideas out yet, I wanted to share some of our current thinking. Comments, questions, thoughts, most welcome.


Good Practice in Assessment-For-Learning:
Engages students with the assessment criteria
Assessment is an important aspect of student learning and should be used to help reinforce the expected standards. Our interactions with students, through assessment and feedback, should help students engage with the assessment criteria.

Q1.1 I seek out opportunities to help me understand the academic standards expected of Higher Education.
Q1.2 I take advantage of the resources available (across UH) to ensure my work meets the academic standards expected of me.
Q1.3 When presented with an assessment task I read and ensure I correctly understand the assessment criteria
Q1.4 When I receive feedback on my work I look at my feedback and link it back to the assessment criteria to support my future learning
Q1.5 I ensure my assessment submission responds to all the assessment criteria / learning outcomes described in the assessment briefing documents

In what ways do you engage with the assessment criteria?


Good Practice in Assessment-For-Learning:
Supports personalised learning
Students have their own motivations and interests. As individuals, students also have differing needs to support their learning. Whilst individual assessment tasks are likely to be an impractical proposition it is helpful to consider how assessment can support the personalisation of learning.

Q2.1 Where appropriate, I use my own personal experiences to support my assessment submissions.
Q2.2 I take opportunities to let my lecturer know about the areas I would like feedback on (strengths and weaknesses).
Q2.3 I take advantage of any assessment choices presented to me to suit my learning preferences (topic/weighting/timing/criteria).
To finish!
In what ways do seek out opportunities for personalised learning?


Good Practice in Assessment-For-Learning:
Ensures feedback leads to improvement
Feedback is an essential aspect of assessment activity. Feedback will be more effective if it is prompt and makes sense to the students. Moreover, good feedback provides a commentary on the students’ submissions, offers advice on how the work could be developed and provides opportunities for students to demonstrably engage with the feedback.

Q3.1 I recognise the many ways that feedback is presented to me about my work and my learning
Q3.2 I know when and where feedback on my assessment is available and pick up my feedback as soon as it is released.
Q3.3 I take opportunities to discuss feedback with my lecturers and my peers.
Q3.4 I take the time to identify (by myself and / or with my peers) the strengths and weaknesses of my own assessment.
Q3.5 I use feedback from previous assessment tasks to help me improve my understanding and my next assessment task

In what ways do you ensure that the feedback leads to your improvement?



Good Practice in Assessment-For-Learning:
Focuses on student development
Assessment has a significant influence on student motivation and the ways in which students approach their learning. Good assessment develops the students’ interests, motivations and encourages appropriate study behaviours. Ultimately good assessment motivates good learning.

Q4.1 When constructing my assessment submission I focus my effort on learning (i.e. linking concepts together) rather than just remembering information.
Q4.2 When I receive feedback on my assessment I look carefully at the comments, advice and encouragement and do not just concentrate on the marks I received
Q4.3 I take the time to review my own assessment (self assessment) before and after I submit my work
Q4.4 I make sure I identify the positive aspects of my own work as well as areas for improvement

In what ways do you ensure your activity focuses on the development of your learning?


Good Practice in Assessment-For-Learning:
Stimulates dialogue
A good learning environment considers the individual student whilst also recognising the importance of a learning community. Further, learning is enhanced if students are able to share their conceptions and misconceptions. Good assessments support the development of a learning community and provide opportunities for students to engage in a dialogue about their learning.

Teachers too should have an opportunity to engage in a dialogue. A dialogue that helps them shape their teaching and engage in staff, module and programme development activity.

Q5.1. I take every opportunity to contribute to group and class discussions relating to assessment and learning.
Q5.2. I look for opportunities to discuss my assessment with my peers and my teachers
Q5.3 I look for other sources of help to support my assessment. This might include reading lists, learning groups, central support systems etc.
Q5.4. I use the assessment tasks (and subsequent feedback) to help me develop my understanding of the standards expected of me
Q5.5. I always read the feedback I receive and use it to help me shape my learning

In what ways do you engage in dialogue about your learning and assessment activity


Good Practice in Assessment-For-Learning:
Considers student and staff effort
Good assessments create a good educational experience. Good assessment set out high expectations, foster appropriate study behaviours and stimulate students’ inquisitiveness, motivation and interest. Good assessment should distribute the students’ effort across the study-period and topic areas. Good assessments will demand an appropriate amount of student effort. Good assessments will not, however overload students nor their teachers. Good assessments ensure there is adequate time for teachers to create and deliver feedback in ways that supports student learning.

Q6.1. I put all my assessment deadlines in a diary/online calendar so that I am aware of what is expected of me
Q6.2 I plan my work so that I am able to work on assessment tasks that have overlapping deadlines
Q6.3. I avoid cramming and spread out my time on assessment tasks
Q6.4 I follow the advice given (on the assignment briefing document) regarding how much time I should typically spend on my assessment activity
Q6.5 For each assessment task I carefully plan each stage of the work (e.g. reviewing previous feedback, reading/research for new assignment, creating a first draft, reviewing and amending, proof reading, self evaluation, submission)

In what ways do you plan and manage your effort to enhance your learning?


Again, comments / thought / questions most welcome.

Mark

Final Cluster Meeting

We have just had our last cluster meeting, which was held at the University of Exeter. The theme of the meeting was to give each project a chance to present what had been achieved over the two years and to look to the future - beyond the official end of the projects at the end of October. It gave us a chance to per critique each other projects. We presented on the work of ESCAPE and Mark demonstrated aspects of the ESCAPE Toolkit. Themes that we covered included:

• the development of a set of ESCAPE Principles


• mapping the current assessment landscape to these principles


• considering efficiency verses effectiveness


• what does transformative change look like


• demonstration of some of the "themes in practice" videos

We talked about how the project has laid a foundation for staff engagement with a year long university wide assessment project that is running post ESCAPE.

The meeting was in the usual format of a two day timetable with meeting spread over lunchtime on the Thursday to lunchtime on the Friday. Helen Beetham joined us on Friday to facilitate a session on exploring what we have learnt as a cluster. We were looking to build upon the collaborative efforts of our joint cluster presentation at the University of Greenwich e- learning Conference. There were some interesting ideas for further collaboration including a collaboration with the University of Exeter (INTEGRATE Project) with a sharing of resources from our projects.

We agreed that beyond the final programme meting in October we would look to meet in 12 - 18 months to look at what impact our projects have had.

Themes In Practice (2)

The following video outlines another of the technology-rich approaches to assessment used in the ESCAPE project. The Weekly Assessed Tutorial Sheet (WATS) approach to assessment has been used elsewhere at UH (I developed the approach and the technologies to support my engineering undergraduates) but it's just great to see it being used here too.


The video is another of the assessment TIP's (Themes In Practice) and has been produced to show how our ESCAPE themes relate to assessment practice.

Mapping the assessment landscape

Initial activity of the ESCAPE project related to us 'mapping the assessment landscape'.

The following demonstrates our activity and offers the idea as a concept to help you in your explorations

Mapping educational effectiveness and resource efficiency

One of the instruments we used to engage our ESCAPE partners was a simple two-dimensional matrix. The matrix has axis for educational effectiveness and resource efficiency.

we found this to be a valuable tool to work with our partners in developing better educative assessments that are not overly demanding on their time. The following video sets out the matrix.

Video of our ESCAPE themes

I am in the process of capturing our thinking and work in the form of short Camtasia videos. What follows is a video on our ESCAPE themes.

Thoughts on the format and themes are, as always, most welcome

Mark

Good assessment - the right of all our students

On my travels I have come across Kate Wicklow (Feedback Officer) of the NUS. I'm always been impressed at Kate's energy and enthusiasm for the work she does. I can see the passion for the work she does. anyway, Kate, via Sarah Knight, invited me to write a piece for the NUS journal HE Focus. Naturally I responded positively.

The 'draft' piece follows. Lets see what happens and see if it get included


Good assessment - the right of all students!


The ESCAPE project
Good assessment is the right of all our students - fact! Good assessment should not be the experience of a lucky few taught by academics that understand the significant influence of assessment. An influence that, inter alia, shapes students’ study behaviours, stimulates an appropriate ‘approach’ to learning, and arouses the students’ inquisitiveness in learning and their subject discipline. Such aspirations, although not impossible, are becoming every more difficult given the increase in student numbers and associated reduction in resource.
Our JISC funded project, Effecting Sustainable Change in Assessment Practice and Experience (ESCAPE) seeks to bring about enhancements to assessment in a resource efficient way. Working with two Academic Schools at the University of Hertfordshire, (The Business School and The School of Life Sciences), we set out to explore current assessment practice, surface conceptions of good assessment and subsequently support our partners as they review and develop their assessment practice.

What is good?
Review the assessment literature and you will unearth numerous sets of principles of good assessment and feedback practice. See for instance (Gibbs & Simpson, 2004; McDowell et al., 2006; Nicol, 2007; Nicol & Macfarlane-Dick, 2006; Weston-Manor-Group, 2007), and of course the principles espoused by the National Union of Students (separately on assessment and feedback). To help us engage with busy academics, and being mindful of the so-called magic number seven, we have drawn together the existing principles and produced a set of six overarching ESCAPE themes. These are:
Good assessment for learning…
• Engages students with the assessment criteria
• Supports personalised learning
• Ensures feedback leads to improvement
• Focuses on student development
• Stimulates dialogue
• Considers student and staff effort

Although we recognise the importance of assessment as an instrument to measure learning our primary interest is using assessment to encourage and stimulate, and not measure, learning.

Working with our ESCAPE partner schools we are using the themes to establish a better assessment and hence educational experience for their students. Using the themes along with a purposely positive exploration of current practice we are working with staff to develop assessment that is both educative and resource efficient. By working at the conceptual level of good assessment, rather than offering quick fixes, we are providing a framework for our partners to take their developing assessment expertise to other modules. Indeed we are already seeing assessment developments being translated on other non-ESCAPE modules.

An example in practice
One of our ESCAPE modules previously required students to provide an individual laboratory report. The laboratory is traditionally a formal document describing the aims of the study, apparatus, method, results and discussion. The nature of the setting however, around 90 students and an individual report, meant that students were not provided with feedback immediately after their submission and the students’ thinking and developing conceptions could not readily be seen by their lecturer. The lecturer could only see the students’ conceptions when the final submission was made.

Following the module teams engagement with the ESCAPE project, the students now work in groups and co-construct their laboratory report on a wiki. Importantly, the wiki ensures that individual contributions are seen and the evolving laboratory report is visible to the lecturer. As such the lecturer now engages with the students’ work at regular intervals and provides on-going feedback on the students work, thinking and analysis. As such this feedback can now be used by the students to shape their thinking and their work.

At the end of the group process, and to alleviate concerns over working in a group that has different levels of student contributions, the students also provide a mark along with an justification statement (for the mark) for each of their group members. These student marks help individualise the mark provided by the lecturer.
The lecturer, new to wiki’s at the start of the process, believes his students now have a better educational experience and he has saved time; win-win at its best.


Numerous examples of effective and resources efficient practice are available and naturally we want to spread our ways of working more widely across the University. Good assessment is, after all, the right of all our students!

For more information about the ESCAPE project take a look at http://escape-uh-jisc.blogspot.com/


Gibbs, G., & Simpson, C. (2004). Conditions under which assessment supports students' learning. Learning and Teaching in Higher Education,, 1, 8.
McDowell, L., Sambell, K., Bazin, V., Penlington, R. W., D, Wickes, H., & Smailes, J. (2006). Assessment for learning: Current practice exemplars from the Centre for Excellence in Teaching and Learning in Assessment for Learning o. Document Number)
Nicol, D. (2007). Principles of good assessment and feedback: Theory and practice. Paper presented at the Assessment design for learner responsibility, Reap On-Line.
Nicol, D., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218.
Weston-Manor-Group. (2007). Assessment Standards : A manifesto for change. Retrieved 20 October 2008, from http://www.business.brookes.ac.uk/learningandteaching/aske/Manifesto%20final.pdf

ESCAPE themes - so far so good!

Readers of the ESCAPE blog, and those of you that know our project, will be aware that we are trying to support staff develop both 'educationally effective' and 'resource efficient' assessment.

We believe that all staff want to do a good job but also acknowledge they experience various demands on their time. Research, consultancy activity, teaching, assessing students, providing pastoral care etc.

To reconcile the desire to help staff develop good assessments against the 'time-demanding' backdrop, part of our engagement is to provide ready access to the literature. Sure, staff might want quick 'hints and tips', and we are providing them, but we are keen too to make sure that the hints and tips etc. are located against a set of themes that describe what good assessment looks like.

For completeness, although stated elsewhere our ESCAPE themes are:

Good assessment for learning...

• Engages students with the assessment criteria
• Supports personalised learning
• Ensures feedback leads to improvement
• Focuses on student development
• Stimulates dialogue
• considers staff and student effort

These themes have evolved over the duration of the project and now follow us around as we talk about the project and work with our ESCAPE partners.


It's really helpful to know that as we share our themes in different arenas we are not seeing any adverse reaction. They seem to capture the thoughts of the practitioner too; indeed we wrote them with a view on being accessible and having face validity. They have been shown on numerous events, both inside and outside the University of Hertfordshire, and are seemingly doing what we set out to do.

In fact a participant at a recent event indicated he seen them before, Unlikely, (highly unlikely), but at least it shows the notion that the themes are not challenging 'appropriate' thinking about good assessment and that staff already indicate they like them.


Mark

ESCAPE @ IBLC 10

Dominic and I recently presented our work at the International Blended Learning Conference. The room was full – which was just great. We were joined too by Sarah knight and Marianne Sheppard from the JISC and so it was just great that they could pick up another update of our work and hear delegates thoughts of our work from delegates.

Essentially ..
1. We outlined the context for our project
2. We gave a run through of main features of the ESCAPE toolkit

• Themes and questions (Diagnostic)
• Features and consequences
• Ideas (including our three minute assessment Tips, Themes in Practice, see earlier blog post)

3. We demonstrated an example of change in action

It was really good to see some of the concrete things emerging from the work. It was particularly rewarding too to look back at some of our Partner modules and just see how far we have taken them; both in terms of their thinking about assessment and feedback practice and also in their use of technology. .

We took a couple of questions at the end of the session. Perhaps the most encouraging was a question asking if access to our ESCAPE toolkit was possible – It is! Such interest validates our thinking and also shows how projects run by one institution can also help and influence practice elsewhere too.
I was really pleased that Sarah and Marianne were able to join us and see the fruits of their programme being shared with others.

Making an existing assessment activity more efficient

One of the 'additional' modules that sought our advice was Human Physiology - School of Life Sciences. Within that module the students were already engaging in peer assessment.

The peer assessment activity brought many learning opportunities but some of the hoped for time savings were lost with students immediately questioning the marks given by their peers.
Staff were engaged in time consuming activity associated with moderating the marks.


ESCAPE involvement sought to support the existing work but use technology to drive more reflection, try to get the to students' understand their strengths and areas for improvement and also require the students to present a considered and evidenced-based argument where they believed they were over or under marked.

The use of a web based data collection facility (offered by the ESCAPE project) provided some great data, encouraged the learning and also saved staff time.

The story was presented as a poster at the Fifth International Blended Learning Conference and represents a really good example of ESCAPE in practice.

Poster can be found here

Assessment TIPs are on their way

You will be aware that we are using Themes (relating to good assessment and feedback practice) to help drive our work forward.

We are keen that the Themes are not abstract concepts. In fact we have written them so that the language is accessible and they have face-validity.

I am now in the process of preparing a collection of short videos showing the Themes in Practice (TIPs).

I'm going for 'three minute TIPs' since I want to be able to get the TIPs seen by busy academics. The first of our TIPs is attached.

On Tuesday 25th May I was at Oxford Brookes University as part of a review event for the Programme Assessment Strategies (PASS) Project. Also at the event was Graham Gibbs. Graham spoke about his experiences of effecting change within programmes and institutions. His views mirrored our experiences on the ESCAPE project, namely that there is a "hidden" network of influence that you must be able to tap into, if you want to effect change within a school . A component of the "hidden" network are the informal meetings that take place in schools. For example; in the corridor, in social areas, over coffee and at lunchtimes. These informal venues are often where the hierarchy is relaxed and a more open discourse is possible. The discussions are often robust and play key part in forming opinions within schools.

Graham's views certainly mirrored our experiences.It was something that we had anticipated when considering our approach to working with our stakeholders. One of the early parts of the ESCAPE project involved mapping the influences of individuals and teams within schools. We looked at who were the key opinion formers within the schools and how we could get them "on side". Additionally we looked what were the formal and as important, informal channels of communication within schools.

We also looked at how we could encourage collateral effects - how the stakeholders we were working with could influence others. Both over the course of the project and beyond as part of our sustainability planning.

As part of the event I was invited by Peter Hartley - who chaired, to present an outline the ESCAPE project. It was extremely valuable to get the benefit of the teams experience experience in the discussion that followed.

Although the tenure of the meeting was one of looking a programme level interventions rather than at modular level ( - which is the ESCAPE perspective), there are common assessment themes that transcend both projects which started to emerge - such as ownership of the assessment and managing change within teams.

I was able to suggest some alternative approaches for the PASS project that involved taking a cross module approach to assessment that is designed to break down the barriers between modules as a "halfway house" to full programme level assessments. I spoke about the experiences of two of the programmes at the University of Hertfordshire ,that have worked towards implementing a more integrated approach to assessment that attempts to transcend the module based assessment model.

As part of the meeting Graham Gibbs discussed his guide "using assessment to support student learning" which is available at:

http://www.leedsmet.ac.uk/100317_36641_Formative_Assessment3Blue_WEB.pdf

Graham provided me with a copy of the guide during the meeting. It is an extremely useful resource for teams embarking on using assessment as a vehicle for learning. The ideas set out by Graham are illustrated and supported by case studies, which makes it very accessible and relevant to the reader.

On my train journey home I was able to reflect on what had been really useful and informative day!

Collateral Effects

We have been approached by some of the teams working in the sports therapy area to see if we can work with them to look at their approach to assessment and learning on two of their modules. They had come along to one of the university wide presentations that we gave on the ESCAPE project .

We are exploring with these modules how the the use of student produced videos of therapeutic techniques can be used to support the students with their practical classes. One of the themes that a is emerging is motivating students to practice the techniques on a regular basis so that they can master them.

After speaking with one of the module coordinators this morning it striking how similar many of the assessment issues resonate with those that we are already working on. It is clear that the foundation that we have laid with the original module teams is providing a jump off point in applying the assessment approaches that have been adopted to these "secondary effect" modules . We had of course expected the to be secondary and collateral effects - viz: others we would work with or would influence, but what is surprising is how portable some of the techniques and approaches that our module teams have developed piloted may turn out to be.

Use of a Visualiser in Mathematics Support

One of the technologies that we are looking at to support students is the use of short videos. We are working with staff and students using flipcams to demonstrate laboratory and mathematical techniques.

IN the case of the mathematical techniques, we are using a visualiser to project the image onto a white board and focusing on the projection of the paper that the calculation is laid out on.

The calculation fills the screen and the lecturer explains the mathematical technique, carrying it out step by step. The lecturer's narration and explanation indicate the difficult parts of the calculation and guides the student through the technique. The videos will be made available on Studynet and students opinion on their usefulness in supporting the mathematics workshops will be sought. Their particular use will be demonstrating applied mathematical laboratory techniques when students which students traditionally have problems with.

Is a good example of how simple technology can support the appropriate pedagogy in delivering student centered learning. Furthermore we are able to gain the student view as to their usefulness through the discussion feature on Studynet.

WATS in BioSciences

One of the modules that the ESCAPE Project has been working with has been the Introduction to Biochemistry, Microbiology and Pharmacology module. This is a level one introductory module for 200 life science students that is taken by students on a number of pathways. Students have traditionally struggled on this module in applying mathematical techniques. particular areas that are flagged up through diagnostic testing are: calculation of dilutions, use of logs and ratios and percentages . These areas all present students with problems( see slide at top of blog entry). In order to address this skills gap the mathematical techniques required for the module were taught using a series of five maths workshops in semester A.

This year Ella Bryson and Jackie Willis have integrated WATS - the Weekly Assessed Tutorial Sheets into their teaching. The WATS system developed by Mark Russell is an automated assessment system that allows teachers to set questions ,which are the same for each student - however the data within the question is different - so that no two sets of answers are the same. The students log on to WATS download a set of questions, do the calculations and submit their answers electronically. The Answers are marked automatically and students are emailed their results.

Ella and Jackie harvested typical questions from the colleagues who deliver the teaching and the practical work on the module, and Incorporated these into a set of 15 WATS problem sets. The WATS system was used to build upon the material covered in a series of workshops.

Students were tasked with answering a set of questions each week for 15 weeks.

Students were given an anonymity number (a three letter code) after each set of questions were submitted,a league table of results produced so that students could see their (anonymous) ranking. After submission students were emailed the answers with feedback messages to support their learning. Additionally the WATS was integrated into the personal tutorial system.

Ella Bryson has commented that "WATS has allowed us to teach maths to students in a way that actively encourages them, it promotes independent thinking by making it impossible to copy answers from others. It gives immediate feedback on how each student did, where they went wrong and how they compare with the rest of the class , stimulating competition."

Jackie Willis felt that " BioScience students don't always appreciate that maths is a fundamental part of their course." and " Students are so used to compartmentalising information once the ( Maths) workshops are completed they..fail to apply what they have learnt in their practicals. WATS engaged the students on a regular basis and we were able to contextualise the maths so that it complemented and supported what was happening in their practical programme"

Thanks to Ella and Jackie's hard work they have really been able to support their students using this innovative and effective approach. Ella and Jackie are now exploring ways to roll this out to support second year students.

JISC Conference 13th April 2010

Last week Mark Russell and I presented at the JISC Annual Conference, which this year was held at the Queen Elizabeth II Conference Centre in London. The opening keynote speaker was Martin Bean, Vice Chancellor of the Open University. One of his Keynote themes was that students are often well able to find information - there certainly is enough of it out there, but are less adept at assessing the veracity of the information they do find. He spoke about teaching students to not only discover, but also to teach them to "triangulate" the information. He then went on to challenge the audience - always a good thing(" I'm an Australian - you can't offend me!" was one of Martin's opening gambits). He spoke of the different perspectives between British and Australian textbooks in dealing with the Galliploi campaign of the first world war. This is still clearly an emotive topic after over 90 years and brings into sharp focus the concept of triangulating knowledge - and the need for students to develop their own understanding and perspective.

All in all a really challenging and informative keynote address

It's Quality Time!

One of the things that has struck me this week is the how often subtle changes to a modules delivery can have quite wide ranging changes to the learning interactions that take place. The example I am thinking of is the Corporate Social Responsibility and Sustainable Marketing Module. The module coordinator - using his experience from previous years, has posted a set of FAQs on his module Studynet site. The questions address points and issues that were raised by students last year and were generated from the experiences of the whole module team.

As a result, module team's interactions with students - both face to face and online, been able to focus more on developing the students understanding of the material covered in the module . Students can be pointed in the direction of where to find specific information. This leaves the team time to develop the "high quality " dialogue and interaction with students that can really support and personalise the students learning. This simple strategy(one of a number adopted by the team) of using a well primed bank of FAQS has acted as a filter so that the more complex and interesting questions that students generate can form the basis of meaningful discussion with the module team.

Here comes the data (2)

A previous post (Here comes the data 1) outlined the use of technology to collect feedback from students engaged in a peer assessment activity. In the previous post I suggested that was a real win-win. Better learning and time savings.

Here are some of the findings. Click the image to read the responses.

Here comes the data (1) ...

We are now starting to see the fruits of our labour on the ESCAPE project. Some of the 'fruits' arise when we talk to staff about their assessment practice and resulting experience whereas other 'fruit' arises from observations of student performance and their engagement with their studies. I thought I would share some of what we are finding ...

A new module (not one of the original ESCAPE modules) wanted help with Peer Assessment. The member of staff was already engaging her students with peer assessment and hence were reaping numerous learning gains.

* Students were able to see how their peers responded to the same task
* Students were able to engage more with the marking criteria and standards
*

Previously, however, she was kept very busy after the peer assessment by dealing with more than expected students questioning their marks. Whilst it is highly appropriate that the students are exposed to a fair and reliable assessment, many of the efficiency gains made by the staff member were lost due to the need to deal with students on a one-to-one basis.

With the help of the ESCAPE project we were able to re-purpose a web-based data collection facility (developed to support computer based assessment). Using the web based data collection facility we posted to the students a series of questions (asking them to reflect on their own submission and the peer assessment process. This was an addition to the work previously done and hence created addiction learning gains. Importantly, we also included an opportunity for the students to 'comment on their mark' and note that if they were over or under marked to provide evidence where this was the case -with reference to the marking criteria used in the peer assessment process. The result was of which a vast reduction in 'additional' time required by the staff member to look at the concerns.

This addition was a real win-win. Students were now reflecting on the process (and sharing the reflections with staff) and the staff were reaping efficiency gains.

Slightly self-promoting but this typifies exactly the type of things this project is about. Using technology (led by pedagogy) to reap learning and efficiency gains.

I can't see this technology-supported intervention stopping when the project finishes.

Feedback

We have sought numerous opportunities to collect feedback throughout our project and not just at its end. In fact, we are awash with data and feedback. It will be a real challenge to use the data and feed it into our final evaluation. We have so many threads of activity and so many things we would want to share.

But I do want to post here one of the observations made by one of our ESCAPE partners. In many ways the quote captures everything we're trying to do. Support our partners, help them with their practice and allow them to take the work we are doing with them into other modules. i..e Spreading the learning and benefit.

The quote came from a conversation between our ESCAPE partners and was not solicited in any way. You can imagine my scrabble for a pen and paper!

The ESCAPE project has made me think about the way I give feedback and it has changed my practice in all modules.

She went on to say that

I now organise myself such that I am able to give feedback straight away. I now plan my diary around large coursework to free up 2-3 and re-arrange teaching sessions so that I can get a chance to give feedback.

Arguably, the above quotes demonstrate the transformative nature of the project and shows that the learning and benefit is not constrained to the modules we are working with.


Great stuff!

Learning from the slopes!

I'm not a skier. I have never been skiing, until recently that is.

But as I was learning to ski (or rather learning how to pick myself up from the white stuff) lots of my time was spent thinking about my learning (on the slopes) and the things that supported my ability to learn and get going.

I'm really interested in notions of personalised learning.
Personalised learning requires personal activity and learning activities that are adaptive to the demonstrable needs of the learner. This is true in the classroom and on the slopes. Personalised learning does not throw the learner in too far at the deep end but develops the learners confidence and hence their inquisitiveness and willingness to progress. Personalised learning does not swamp the learner with feedback nor provide fedback that is inaccessible to the learner either.

All of these things were so beautifully reinforced to me as I tried to ski. Putting ourselfs in situations that make us the novice (skiing in my case) was great to see some of the things about learning relate to me.

Being a novice and thinking about learning was just great.

Interim Reporting and Future Plans

We are just completing our second (and last) interim report. It deals with the project's progress over the last six months. A large part of the report deals with mapping the proposed project objectives and outcomes against what we have been able to achieve so far. One important theme that emerges from the initial draft of the report is the sustainability of the project, that is what will happen when the funding finishes in October. How will we ensure that the projects legacy will continue? The report provides a useful focus for articulating these plans in the light of our experiences over the last 18 months.

Cluster Meeting 25th & 26th March 2010

We have just hosted the third meeting of our cluster (universities of Bristol, Exeter, Hertfordshire and Westminster). The venue was the Comfort Hotel in St.Albans. The hotel was at one time the home of Samuel Ryder of Ryder Cup fame. The meeting ran as is usual from lunchtime on the 25th to lunchtime on the 26th. This gives everyone a chance to arrive in good time to start the meeting and for a social event in the evening - usually a group meal at a local restaurant. This time is valuable as much of the meeting is about sharing experiences of our projects and seeking advice from our fellow cluster project teams and our critical friend Malcolm Ryan.

The first day kicked of with a welcome and introduction by Malcolm and a review of the ground rules for the two days. The Chatham House rules mean that we can speak candidly about issues and events arising in the projects. We then moved on to looking how we had used the Appreciative Inquiry approach with our project and a facilitated session run by Rachel Harris of Inspire Research. We then moved on to evaluation and looked at how we are approaching this- thinking about "what a successful project looks like " and discussing the value in understanding why something did not work out as planned. The day was rounded of by a very pleasant meal in which the topics discussed during the day were further explored.

The second day looked at the plans for the final cluster meeting at Exeter - a draft agenda was proposed and celebrations planned! We then moved on to plans for a joint conference submission by the cluster at the Geenwich e learning conference in the summer. The day finished with a session on using Fishbone Analysis to explore the sustainability agenda of the projects.

All in all a really useful two days - we have similar experiences and issues and it was an excellent opportunity to raise them and to identify ways to move forward.

JISC Experts Group

On Wednesday 17th March I presented a workshop on the ESCAPE Project at a meeting of the JISC experts group in Birmingham. The workshop gave an overview of the Appreciative Inquiry(AI) approach that we took with module teams. We then explored the emerging findings from two of the modules that we are working with. In particular we looked at how educational technologies (online submission, wikis, and online groups for example) provided a medium for achieving the pedagogical aims of the project (for example: a more consistent engagement with the curriculum, increased opportunities for students to engage with feedback and the facilitation of more personalised leaning opportunities). The day provided a good opportunity for getting feedback on the project and there were many questions on the use of AI.

It is indicative of the stage that we are at in the project that we are now starting to present the emerging results of the project on a national stage .This was always was part of our dissemination plan and its nice to be in a position to start to see all the hard work done by the module teams starting to bear fruit.

Sharing what we do

As a project team we are really keen to share with the sector  our work.

This manifests as presentations both internally and externally to the University of Hertfordshire. You will see that we have received a few invites too to present our work.

The University of Hertfordshire has recently re-launched its in house journal relating to teaching and learning. Or more specifically, Blended Learning. The journal has a practitioner focus and uses a variety of formats to engage the readers. The title of the Journal is aptly named Blended Learning in Practice (BLiP)

Dominic and I recently published in the journal. The article can be found here.

Supporting Peer Assessment

During the recent half day ESCAPE review meeting we took along our Flipcams and 'invited' modules leaders (we are working with) to record an introduction to their developments. These short videos will provide a really useful resource-bank and will be useful for colleagues both inside and outside UH to see what we have been doing and more importantly consider the adoption of some of our approaches to their own practice. Thanks to all that contributed so positively :-)

Helen Barefoot and I took a Flipcam outside (for a walk) to hear about the recent ESCAPE activity supporting Peer Assessment.

It was good to meet with the ESCAPE module teams on Wednesday - the half day event was a chance to catch up and swap experiences. The fact that we were able to take people away from the university for a day is a real bonus and gives people a chance to take stock and to reflect without distractions.

We were able to capture the reflections using the flipcams - now in their second generation so that they can record for two hours, although nobody took the machine to its limits. Our reflections to camera lasted about five minutes for each participant.

Along with the reflective theme that we were developing we are also looking at the collateral effects that the project is generating. some examples of these are:

• Plans for the WATS system to be used with the MSc Life Science students next academic year - building on the successes in using it with with a first year Life Science module


• Working with another three modules from the division of sports science over the coming months - facilitating and supporting a re - engineering approach to their assessments


• Working with one of the ESCAPE module coordinators on a masters module to support the use of video to assess group work


• Working with one of the compulsory first year modules in the Business School that has over 1600 students taking it in two cohorts.


• Being approached by a module coordinator to help to re-engineer the assessment for a large Health & Human module that involves students on placement - assessing them them through the use of group work and a piece of reflective writing.

We are looking at what is the best way to work with people who have approached us and we will probably work with individual teams or areas as and when we are asked to. This approach offers us quite a bit of flexibility in the approaches we use - rather than using a larger " come into the big tent and see the show" approach.

Tomorrow we are running a half day event for some of the modules that we have been working with. Four of our modules have now completed their piloting phase , four are still running and one has just started . Now at the end of semester A is a good time to get people together to reflect on the changes made over the last semester. We have once again become aware of the time pressures that colleagues are under and the fact that it is extremely difficult to find a time when all who need to can attend. Any work that comes out of this project will need to reflect on how this is overcome - we have achieved it by running separate sessions to cater for staff availability.

The event tomorrow will have three themes:

• Reflecting on the changes made to the module in the light of ESCAPE.


• Mapping these changes to agreed principles of assessment for learning.


• Planning for sustainability - ensuring that the changes that are put in place in the piloting phase are built upon.

We are aiming to capture the reflections with the use of the ubiquitous flipcam. We will be exploring:

1. What went well - share and celebrate your successes


2. What needed more development


3. what was the change in the student experience


4. what was the student response to the changes

Incidentally quite a few of the module coordinators have flipcams and I will blog about what they have been used for soon.

I am looking forward to a really good afternoon - it will be the first chance that many of the lecturers have had to meet with their peers since before Christmas. And hopefully it will provide good opportunities for sharing of experiences and ideas.

Off to Scotland

Helen Barefoot and I were invited by the HEA to talk run a workshop on assessment and feedback at an all day Assessment event at the Robert Gordon University. What follows is a collection of (very quick) thoughts on the day …

Helen and I have run a few workshops together (inside UH) and so it was just great to take our work (much of which is guided by our ESCAPE activity) to colleagues outside UH. For various reasons the workshop did not run :-(. Despite our disappointment we did get to hear some great assessment related presentations.

Dai Hounsell presented a really grounded key-note and, in fact covered, much of what we were covering too. That assessment is not a new challenge, that good assessment is planned activity and that good assessment stimulates learning. We did not need the NSS to get us thinking that assessment is important. Some really useful slides from Dai that I will explore and come back to in a later post. Great start to the day.

A couple of student perspective presentations followed.
A student led campaign that successful introduced a turn-around-time policy for coursework and interestingly a policy to provide feedback on examination scripts. The learning gains to be had from providing feedback on examination scripts seems rather limited to me. I’m always banging-on about feedback creating consequences, and I’m just not convinced I know what consequences flow, or are able to flow, from feedback on end-of-process, high stakes assessment tasks. That’s a post for another day. But just to say, I’m off the fence on this one. I just don’t get it. Another delegate did note they had a similar policy at his institution and only 15% of the scripts (with feedback) were picked up. Surely we would be better placed putting our feedback on work that will be picked up and more importantly attended to, by the students. And relax!

Steve Draper, engaging as ever, had a couple of threads running through his presentation. First, was the interesting anomaly that overall a department was rated 5th against other departments (107 in total) for the NSS question overall, I am satisfied with the quality of the course and yet questions relating to feedback were ranked much lower. Feedback on my work has been prompt (ranked 54/107), Feedback on my work has helped me clarify things I did not understand (ranked 79/107) have received detailed comments on my work (ranked 101/107). Steve asked us to explore what might be going on. Was there a complex weighting algorithm for all the items on the NSS? Should the individual items of the NSS not sum to the overall score? If not, what was missing, what was the missing ingredient? Second, Steve separated declarative and procedural learning and pondered as to where our efforts on providing feedback might prove most effective. i.e. might we get more learning from less or (better targeted) feedback?

Off to Wales

I was invited to present some work on Blended Learning (which included assessment activity) to Glamorgan University. Rather than just present ‘stuff’ I tried to engage those that gave up their lunchtime by asking them to consider their own challenges and also ask what they thought supports learning. Clearly, technology enhanced learning is about learning, not technology, and so I wanted to help surface their own values and pedagogic perspectives. I presented the UH context and gave some examples of Blended Learning in practice.

Following the session I was whisked away to share some of our ESCAPE activity. Glamorgan have a Change Academy project which focuses on assessment . I tried to outline some of the ways in which we (the ESCAPE team)( were working and also highlight some of things worth considering.

I tried to emphasise

* That whilst assessment was the focus of our attention, our project was about change management too.

* the importance of, and benefits to be gained, by considering an Appreciative Inquiry approach to evaluation

* The importance of trying to understand the culture and constraints and of the Schools we were working with

* The importance of working with a principled approach to curriculum design and that we (the project lead) need to bring the principles to the Schools.

* That innovation and change is stimulated by many things and that we need to be alert to the different stimuli and create situations where they can flourish

* That developing educationally effective assessments alone might not stimulate all colleagues. Many staff are experiencing numerous time pressures and so consideration of resource efficiency along with educational effectiveness is important too.

It’s a real shame that I didn’t get more / any time to hear about their work but having established contact I will follow things up to see how they are working and how we might benefit from their insights too.

Module Coordinator Experiences

This week we held a lunch time seminar to give staff a chance to learn about the project and to hear from two of the module co-ordinators that we have been working with. James Johnstone is module coordinator for a second year module - Principles and Practice of Sports Science and Hazel Wagner is module coordinator for Advanced Corporate Reporting a third year module. Hazel and James spoke about the issues with the modules prior to the ESCAPE project and took us through the process of re engineering that they went through in order to increase the amount of engagement that the students had with their assessments. We had alluded to this at the Ulster conference ( see previous blog entry) but it was really exciting to hear the people responsible for managing and applying the changes speak with passion about their modules. The presentations gave an indication of what was possible if to quote James " .......we leave the Rolls Royce in the garage" This is a reference to the work done by Win Hornby, of Robert Gorden University on efficiency and effectiveness in assessment .

The assessments developed by Hazel and James were effectively multi component designed to engage students on a regular basis over the whole semester. They have multiple opportunities for feedback to students and for students to engage with the feedback,using it to inform their application to the next component of their assessment.

One of the themes of the assessments is that it gives the chance for the lecturers to understand where the students are at each part of the module - to really get a feel for the students grasp of topics or issues. This reminded me of the example of when I walk Holly, my Springer Spaniel, we are only together at the start and the end of the process - where she goes in the middle bit I am not too sure - we meet again at the end. This is all to often what happens with our students -we know what they are thinking at the start of the lecture and we know where they are at the end - but they probably have not been following us all the way through.

The seminar finished with question and answer session with the audience keen to quiz James and Hazel on their experience of applying the new assessment.

Belfast Bound!

Last week Mark and I presented some of our observations from the ESCAPE Project at the University of Ulster eighth annual eLearning Conference. I had never been to Northern Ireland before and was excited by what I saw - it certainly challenged my preconceptions . Belfast is a vibrant town, new buildings and infra- structure projects were springing up everywhere - and there was a real energy to the city. The venue for the conference was the School of Art and Design. The campus consists of two buildings, one a turn of the century the other a much more modern one. The buildings are connected by a new covered walkway. They were really excellent learning spaces, well designed and resourced.

Our presentation, which we gave twice, looked at the background to the ESCAPE project and the wider assessment landscape. Mark spoke about the challenges of the project and how the extensive literature on learning and assessment should be informing teachers practice. I spoke about working with module teams and discussed some of the changes that teams had put in place as a consequence of working with ESCAPE. We also discussed the role Appreciative Inquiry played in the project .

On the way back to the airport the taxi driver took us to see the Thompson Graving Dock. This is where the Titanic, which was built at Harland and Wolf in Belfast, was brought and fitted out after she was launched.

I contrasted this with the the keynote address at the conference, where professor Paul Moore spoke about the changing technologies that we use and how they have influenced us, using an in car record player from his father ford Anglia to illustrate his point. He spoke passionately about the iPhone and contrasted this with his blackberry which he referred to as a " wheelbarrow for emails". His perspective was an interesting one and his excitement about the impending launch of the iPad was infectious. Paul felt that this device would have a huge influence on learning over the coming years - interesting times!

For those of you that have not realised - the flag at the top of the blog is that of the White Star Line, the shipping line that operated the Titanic.

A Brand New Year

It's been quite a snowy start to the New Year here at the University of Hertfordshire with the University being effectively closed for two days. The snow has started to clear and things are now getting back to normal. Christmas seems to be a fast fading memory.

There are four modules that the ESCAPE team are working with, that finish in semester A - which is in two weeks time. We will be capturing the module team experiences through interviews and running the student online assessment survey again to capture the student view.

Conditioning and Exercise in Sport is a level 3 module due to start in semester B. We are working with them - looking to use students to produce a series of conditioning and exercise techniques as part of the formative assessment process. It will be interesting to see how the students get on .We will be issuing students with flipcams to record their sessions.