Blogging from the 'mud-movers'

The ripples are still visible

2010-12-16T06:53:00.000-08:00

I presented on some of the aspects of the work emerging from the ESCAPE project at a development day -"Using Technology to Enhance Feedback" run by the HEA Subject Centre for BioScience and hosted by the University of Roehampton. I talked about the work that James Johnstone had been doing with his second year sports studies students. How the wiki had been used as a forum for collaborative development. I spoke about the way that the students had received formative feedback and how their work was monitored each week with James compiling a blog for general feedback.

I also spoke at length about the barriers for adopting technologies such as wikis and blogs - both from a teachers and also a students perspective,
The session was well received with lots of interest being generated. There were three workshops in the afternoon., where we had a chance to explore the use of wikis in a bit more depth. It was a very appropriate way to finish the project- showing what we had been able to achieve in partnership with the modules we worked with.

Assessment Patterns

2010-11-04T12:50:00.000-07:00

Regular readers will note that I have previously posted about the assessment patterns.
The assessment patterns show graphically alternative assessment strategies and highlights the possible consequences of the different patterns.

The assessment patterns have been further developed and separated to allow ready access to three areas. Of course the assessment patterns are complimentary but each set relates to a particular area.

The areas relate to

* Moving away from high-stakes end of process assessment
* Making more of feedback
* Programme view of assessment

Links to the assessment patterns (docs) and a wiki and video page can be found here

https://jiscdesignstudio.pbworks.com/ESCAPE+-+Assessment+timelines

Comments welcome

Mark

FInal hurrah?

2010-10-27T05:54:00.000-07:00

So Dr Helen Barefoot and I presented the ESCAPE work and the ESCAPE (Assessment for Learning) toolkit at the recent International Soiety for the Scholarship of Teaching and Learning (ISSOTL) conference in Liverpool. There was something really nice and rounded presenting with Helen. She and I put the original bid together all those years ago. Really pleased that Helen had another chance to articulate some of her thinking on assessment and feedback.

We presented to a smallish (but packed) room. I hear they had to turn people away form our session due to the crowded conditions. Anyway, it was useful to see, what appeared as, genuine enthusiasm for our work and the toolkit. Prof. Chris Rust (long standing expert in assessment) asked for access to the resources - he wants to take them to an assessment workshop he is running at Kingston University.

Really pleasing that as the project finishes - so the legacy remains and the resources are there for others to use and engage with. Sustainability, and growing better assessment practice was at the heart of the ESCAPE project and it seems like we are doing exactly that.

Mark

Assessment patterns - a video

2010-10-01T09:55:00.000-07:00

Thought I would produce a video introducing the concept of the assessment timeline (assessment patterns) drawings. The timing of the assessment as well as its design and alignment with the module learning outcomes is a major consideration in getting the most from the assessment activity.

Thoughts / ideas most welcome

Introductory video follows ...

Assessment patterns - not long now!

2010-09-29T12:20:00.000-07:00

As this project draws to a close so we are keen to keep supporting UH. Indeed our University has funded a year long, UH-wide project related to assessment. Naturally, the ESCAPE project will feed beautifully into one of the strands of the project.

One of things I'm drawing together is a set of assessment patterns. The patterns are graphic images that show the likely consequences of different assessment regimes. The patterns show differently weighted assessments located at different places on a timescale. I've shown these evolving patterns to a couple of (internal) audiences and they seem to be well liked. Just tidying up the document now and I'll post it here for comment. Watch this space.

I'd be really interested to hear your thoughts on the usefulness (or other) of the graphic representations.

Mark

ESCAPE themes - student focus

2010-09-09T04:14:00.001-07:00

The ESCAPE (Assessment for Learning) themes have been synthesised from the literature. Indeed purposely draw on the literature to create the questions that prompt teachers to think about their assessment design in light of the six ESCAPE themes.

But the design is only one facet of good assessment. The role of the students also needs to be considered. Sure, the design should stimulate appropriate student behaviors etc. but we wanted to go further.

What we are working on now, is a set of 'accessing questions' for our ESCAPE themes that are explicitly written for students. We have the same themes (of course) but the accessing questions are different. Although, we have not tested the ideas out yet, I wanted to share some of our current thinking. Comments, questions, thoughts, most welcome.

Good Practice in Assessment-For-Learning:
Engages students with the assessment criteria
Assessment is an important aspect of student learning and should be used to help reinforce the expected standards. Our interactions with students, through assessment and feedback, should help students engage with the assessment criteria.

Q1.1 I seek out opportunities to help me understand the academic standards expected of Higher Education.
Q1.2 I take advantage of the resources available (across UH) to ensure my work meets the academic standards expected of me.
Q1.3 When presented with an assessment task I read and ensure I correctly understand the assessment criteria
Q1.4 When I receive feedback on my work I look at my feedback and link it back to the assessment criteria to support my future learning
Q1.5 I ensure my assessment submission responds to all the assessment criteria / learning outcomes described in the assessment briefing documents

In what ways do you engage with the assessment criteria?

Good Practice in Assessment-For-Learning:
Supports personalised learning
Students have their own motivations and interests. As individuals, students also have differing needs to support their learning. Whilst individual assessment tasks are likely to be an impractical proposition it is helpful to consider how assessment can support the personalisation of learning.

Q2.1 Where appropriate, I use my own personal experiences to support my assessment submissions.
Q2.2 I take opportunities to let my lecturer know about the areas I would like feedback on (strengths and weaknesses).
Q2.3 I take advantage of any assessment choices presented to me to suit my learning preferences (topic/weighting/timing/criteria).
To finish!
In what ways do seek out opportunities for personalised learning?

Good Practice in Assessment-For-Learning:
Ensures feedback leads to improvement
Feedback is an essential aspect of assessment activity. Feedback will be more effective if it is prompt and makes sense to the students. Moreover, good feedback provides a commentary on the students’ submissions, offers advice on how the work could be developed and provides opportunities for students to demonstrably engage with the feedback.

Q3.1 I recognise the many ways that feedback is presented to me about my work and my learning
Q3.2 I know when and where feedback on my assessment is available and pick up my feedback as soon as it is released.
Q3.3 I take opportunities to discuss feedback with my lecturers and my peers.
Q3.4 I take the time to identify (by myself and / or with my peers) the strengths and weaknesses of my own assessment.
Q3.5 I use feedback from previous assessment tasks to help me improve my understanding and my next assessment task

In what ways do you ensure that the feedback leads to your improvement?

Good Practice in Assessment-For-Learning:
Focuses on student development
Assessment has a significant influence on student motivation and the ways in which students approach their learning. Good assessment develops the students’ interests, motivations and encourages appropriate study behaviours. Ultimately good assessment motivates good learning.

Q4.1 When constructing my assessment submission I focus my effort on learning (i.e. linking concepts together) rather than just remembering information.
Q4.2 When I receive feedback on my assessment I look carefully at the comments, advice and encouragement and do not just concentrate on the marks I received
Q4.3 I take the time to review my own assessment (self assessment) before and after I submit my work
Q4.4 I make sure I identify the positive aspects of my own work as well as areas for improvement

In what ways do you ensure your activity focuses on the development of your learning?

Good Practice in Assessment-For-Learning:
Stimulates dialogue
A good learning environment considers the individual student whilst also recognising the importance of a learning community. Further, learning is enhanced if students are able to share their conceptions and misconceptions. Good assessments support the development of a learning community and provide opportunities for students to engage in a dialogue about their learning.

Teachers too should have an opportunity to engage in a dialogue. A dialogue that helps them shape their teaching and engage in staff, module and programme development activity.

Q5.1. I take every opportunity to contribute to group and class discussions relating to assessment and learning.
Q5.2. I look for opportunities to discuss my assessment with my peers and my teachers
Q5.3 I look for other sources of help to support my assessment. This might include reading lists, learning groups, central support systems etc.
Q5.4. I use the assessment tasks (and subsequent feedback) to help me develop my understanding of the standards expected of me
Q5.5. I always read the feedback I receive and use it to help me shape my learning

In what ways do you engage in dialogue about your learning and assessment activity

Good Practice in Assessment-For-Learning:
Considers student and staff effort
Good assessments create a good educational experience. Good assessment set out high expectations, foster appropriate study behaviours and stimulate students’ inquisitiveness, motivation and interest. Good assessment should distribute the students’ effort across the study-period and topic areas. Good assessments will demand an appropriate amount of student effort. Good assessments will not, however overload students nor their teachers. Good assessments ensure there is adequate time for teachers to create and deliver feedback in ways that supports student learning.

Q6.1. I put all my assessment deadlines in a diary/online calendar so that I am aware of what is expected of me
Q6.2 I plan my work so that I am able to work on assessment tasks that have overlapping deadlines
Q6.3. I avoid cramming and spread out my time on assessment tasks
Q6.4 I follow the advice given (on the assignment briefing document) regarding how much time I should typically spend on my assessment activity
Q6.5 For each assessment task I carefully plan each stage of the work (e.g. reviewing previous feedback, reading/research for new assignment, creating a first draft, reviewing and amending, proof reading, self evaluation, submission)

In what ways do you plan and manage your effort to enhance your learning?

Again, comments / thought / questions most welcome.

Mark

Final Cluster Meeting

2010-09-07T06:30:00.000-07:00

We have just had our last cluster meeting, which was held at the University of Exeter. The theme of the meeting was to give each project a chance to present what had been achieved over the two years and to look to the future - beyond the official end of the projects at the end of October. It gave us a chance to per critique each other projects. We presented on the work of ESCAPE and Mark demonstrated aspects of the ESCAPE Toolkit. Themes that we covered included:

the development of a set of ESCAPE Principles

mapping the current assessment landscape to these principles

considering efficiency verses effectiveness

what does transformative change look like

demonstration of some of the "themes in practice" videos

We talked about how the project has laid a foundation for staff engagement with a year long university wide assessment project that is running post ESCAPE.

The meeting was in the usual format of a two day timetable with meeting spread over lunchtime on the Thursday to lunchtime on the Friday. Helen Beetham joined us on Friday to facilitate a session on exploring what we have learnt as a cluster. We were looking to build upon the collaborative efforts of our joint cluster presentation at the University of Greenwich e- learning Conference. There were some interesting ideas for further collaboration including a collaboration with the University of Exeter (INTEGRATE Project) with a sharing of resources from our projects.

We agreed that beyond the final programme meting in October we would look to meet in 12 - 18 months to look at what impact our projects have had.

Themes In Practice (2)

2010-07-29T08:48:00.000-07:00

The following video outlines another of the technology-rich approaches to assessment used in the ESCAPE project. The Weekly Assessed Tutorial Sheet (WATS) approach to assessment has been used elsewhere at UH (I developed the approach and the technologies to support my engineering undergraduates) but it's just great to see it being used here too.

The video is another of the assessment TIP's (Themes In Practice) and has been produced to show how our ESCAPE themes relate to assessment practice.

Mapping the assessment landscape

2010-07-10T01:10:00.000-07:00

Initial activity of the ESCAPE project related to us 'mapping the assessment landscape'.

The following demonstrates our activity and offers the idea as a concept to help you in your explorations

Mapping educational effectiveness and resource efficiency

2010-07-03T07:03:00.000-07:00

One of the instruments we used to engage our ESCAPE partners was a simple two-dimensional matrix. The matrix has axis for educational effectiveness and resource efficiency.

we found this to be a valuable tool to work with our partners in developing better educative assessments that are not overly demanding on their time. The following video sets out the matrix.

Video of our ESCAPE themes

2010-06-29T03:12:00.001-07:00

I am in the process of capturing our thinking and work in the form of short Camtasia videos. What follows is a video on our ESCAPE themes.

Thoughts on the format and themes are, as always, most welcome

Mark

Good assessment - the right of all our students

2010-06-29T02:41:00.001-07:00

On my travels I have come across Kate Wicklow (Feedback Officer) of the NUS. I'm always been impressed at Kate's energy and enthusiasm for the work she does. I can see the passion for the work she does. anyway, Kate, via Sarah Knight, invited me to write a piece for the NUS journal HE Focus. Naturally I responded positively.

The 'draft' piece follows. Lets see what happens and see if it get included

Good assessment - the right of all students!

The ESCAPE project
Good assessment is the right of all our students - fact! Good assessment should not be the experience of a lucky few taught by academics that understand the significant influence of assessment. An influence that, inter alia, shapes students’ study behaviours, stimulates an appropriate ‘approach’ to learning, and arouses the students’ inquisitiveness in learning and their subject discipline. Such aspirations, although not impossible, are becoming every more difficult given the increase in student numbers and associated reduction in resource.
Our JISC funded project, Effecting Sustainable Change in Assessment Practice and Experience (ESCAPE) seeks to bring about enhancements to assessment in a resource efficient way. Working with two Academic Schools at the University of Hertfordshire, (The Business School and The School of Life Sciences), we set out to explore current assessment practice, surface conceptions of good assessment and subsequently support our partners as they review and develop their assessment practice.

What is good?
Review the assessment literature and you will unearth numerous sets of principles of good assessment and feedback practice. See for instance (Gibbs & Simpson, 2004; McDowell et al., 2006; Nicol, 2007; Nicol & Macfarlane-Dick, 2006; Weston-Manor-Group, 2007), and of course the principles espoused by the National Union of Students (separately on assessment and feedback). To help us engage with busy academics, and being mindful of the so-called magic number seven, we have drawn together the existing principles and produced a set of six overarching ESCAPE themes. These are:
Good assessment for learning…
• Engages students with the assessment criteria
• Supports personalised learning
• Ensures feedback leads to improvement
• Focuses on student development
• Stimulates dialogue
• Considers student and staff effort

Although we recognise the importance of assessment as an instrument to measure learning our primary interest is using assessment to encourage and stimulate, and not measure, learning.

Working with our ESCAPE partner schools we are using the themes to establish a better assessment and hence educational experience for their students. Using the themes along with a purposely positive exploration of current practice we are working with staff to develop assessment that is both educative and resource efficient. By working at the conceptual level of good assessment, rather than offering quick fixes, we are providing a framework for our partners to take their developing assessment expertise to other modules. Indeed we are already seeing assessment developments being translated on other non-ESCAPE modules.

An example in practice
One of our ESCAPE modules previously required students to provide an individual laboratory report. The laboratory is traditionally a formal document describing the aims of the study, apparatus, method, results and discussion. The nature of the setting however, around 90 students and an individual report, meant that students were not provided with feedback immediately after their submission and the students’ thinking and developing conceptions could not readily be seen by their lecturer. The lecturer could only see the students’ conceptions when the final submission was made.

Following the module teams engagement with the ESCAPE project, the students now work in groups and co-construct their laboratory report on a wiki. Importantly, the wiki ensures that individual contributions are seen and the evolving laboratory report is visible to the lecturer. As such the lecturer now engages with the students’ work at regular intervals and provides on-going feedback on the students work, thinking and analysis. As such this feedback can now be used by the students to shape their thinking and their work.

At the end of the group process, and to alleviate concerns over working in a group that has different levels of student contributions, the students also provide a mark along with an justification statement (for the mark) for each of their group members. These student marks help individualise the mark provided by the lecturer.
The lecturer, new to wiki’s at the start of the process, believes his students now have a better educational experience and he has saved time; win-win at its best.

Numerous examples of effective and resources efficient practice are available and naturally we want to spread our ways of working more widely across the University. Good assessment is, after all, the right of all our students!

For more information about the ESCAPE project take a look at http://escape-uh-jisc.blogspot.com/

Gibbs, G., & Simpson, C. (2004). Conditions under which assessment supports students' learning. Learning and Teaching in Higher Education,, 1, 8.
McDowell, L., Sambell, K., Bazin, V., Penlington, R. W., D, Wickes, H., & Smailes, J. (2006). Assessment for learning: Current practice exemplars from the Centre for Excellence in Teaching and Learning in Assessment for Learning o. Document Number)
Nicol, D. (2007). Principles of good assessment and feedback: Theory and practice. Paper presented at the Assessment design for learner responsibility, Reap On-Line.
Nicol, D., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218.
Weston-Manor-Group. (2007). Assessment Standards : A manifesto for change. Retrieved 20 October 2008, from http://www.business.brookes.ac.uk/learningandteaching/aske/Manifesto%20final.pdf

ESCAPE themes - so far so good!

2010-06-28T04:44:00.000-07:00

Readers of the ESCAPE blog, and those of you that know our project, will be aware that we are trying to support staff develop both 'educationally effective' and 'resource efficient' assessment.

We believe that all staff want to do a good job but also acknowledge they experience various demands on their time. Research, consultancy activity, teaching, assessing students, providing pastoral care etc.

To reconcile the desire to help staff develop good assessments against the 'time-demanding' backdrop, part of our engagement is to provide ready access to the literature. Sure, staff might want quick 'hints and tips', and we are providing them, but we are keen too to make sure that the hints and tips etc. are located against a set of themes that describe what good assessment looks like.

For completeness, although stated elsewhere our ESCAPE themes are:

Good assessment for learning...

Engages students with the assessment criteria
Supports personalised learning
Ensures feedback leads to improvement
Focuses on student development
Stimulates dialogue
considers staff and student effort

These themes have evolved over the duration of the project and now follow us around as we talk about the project and work with our ESCAPE partners.

It's really helpful to know that as we share our themes in different arenas we are not seeing any adverse reaction. They seem to capture the thoughts of the practitioner too; indeed we wrote them with a view on being accessible and having face validity. They have been shown on numerous events, both inside and outside the University of Hertfordshire, and are seemingly doing what we set out to do.

In fact a participant at a recent event indicated he seen them before, Unlikely, (highly unlikely), but at least it shows the notion that the themes are not challenging 'appropriate' thinking about good assessment and that staff already indicate they like them.

Mark

ESCAPE @ IBLC 10

2010-06-27T02:03:00.000-07:00

Dominic and I recently presented our work at the International Blended Learning Conference. The room was full – which was just great. We were joined too by Sarah knight and Marianne Sheppard from the JISC and so it was just great that they could pick up another update of our work and hear delegates thoughts of our work from delegates.

Essentially ..
1. We outlined the context for our project
2. We gave a run through of main features of the ESCAPE toolkit

Themes and questions (Diagnostic)
Features and consequences
Ideas (including our three minute assessment Tips, Themes in Practice, see earlier blog post)

3. We demonstrated an example of change in action

It was really good to see some of the concrete things emerging from the work. It was particularly rewarding too to look back at some of our Partner modules and just see how far we have taken them; both in terms of their thinking about assessment and feedback practice and also in their use of technology. .

We took a couple of questions at the end of the session. Perhaps the most encouraging was a question asking if access to our ESCAPE toolkit was possible – It is! Such interest validates our thinking and also shows how projects run by one institution can also help and influence practice elsewhere too.
I was really pleased that Sarah and Marianne were able to join us and see the fruits of their programme being shared with others.

Making an existing assessment activity more efficient

2010-06-21T01:47:00.000-07:00

One of the 'additional' modules that sought our advice was Human Physiology - School of Life Sciences. Within that module the students were already engaging in peer assessment.

The peer assessment activity brought many learning opportunities but some of the hoped for time savings were lost with students immediately questioning the marks given by their peers.
Staff were engaged in time consuming activity associated with moderating the marks.

ESCAPE involvement sought to support the existing work but use technology to drive more reflection, try to get the to students' understand their strengths and areas for improvement and also require the students to present a considered and evidenced-based argument where they believed they were over or under marked.

The use of a web based data collection facility (offered by the ESCAPE project) provided some great data, encouraged the learning and also saved staff time.

The story was presented as a poster at the Fifth International Blended Learning Conference and represents a really good example of ESCAPE in practice.

Poster can be found here

Assessment TIPs are on their way

2010-06-03T12:44:00.000-07:00

You will be aware that we are using Themes (relating to good assessment and feedback practice) to help drive our work forward.

We are keen that the Themes are not abstract concepts. In fact we have written them so that the language is accessible and they have face-validity.

I am now in the process of preparing a collection of short videos showing the Themes in Practice (TIPs).

I'm going for 'three minute TIPs' since I want to be able to get the TIPs seen by busy academics. The first of our TIPs is attached.

2010-05-27T01:20:00.000-07:00

On Tuesday 25th May I was at Oxford Brookes University as part of a review event for the Programme Assessment Strategies (PASS) Project. Also at the event was Graham Gibbs. Graham spoke about his experiences of effecting change within programmes and institutions. His views mirrored our experiences on the ESCAPE project, namely that there is a "hidden" network of influence that you must be able to tap into, if you want to effect change within a school . A component of the "hidden" network are the informal meetings that take place in schools. For example; in the corridor, in social areas, over coffee and at lunchtimes. These informal venues are often where the hierarchy is relaxed and a more open discourse is possible. The discussions are often robust and play key part in forming opinions within schools.

Graham's views certainly mirrored our experiences.It was something that we had anticipated when considering our approach to working with our stakeholders. One of the early parts of the ESCAPE project involved mapping the influences of individuals and teams within schools. We looked at who were the key opinion formers within the schools and how we could get them "on side". Additionally we looked what were the formal and as important, informal channels of communication within schools.

We also looked at how we could encourage collateral effects - how the stakeholders we were working with could influence others. Both over the course of the project and beyond as part of our sustainability planning.

As part of the event I was invited by Peter Hartley - who chaired, to present an outline the ESCAPE project. It was extremely valuable to get the benefit of the teams experience experience in the discussion that followed.

Although the tenure of the meeting was one of looking a programme level interventions rather than at modular level ( - which is the ESCAPE perspective), there are common assessment themes that transcend both projects which started to emerge - such as ownership of the assessment and managing change within teams.

I was able to suggest some alternative approaches for the PASS project that involved taking a cross module approach to assessment that is designed to break down the barriers between modules as a "halfway house" to full programme level assessments. I spoke about the experiences of two of the programmes at the University of Hertfordshire ,that have worked towards implementing a more integrated approach to assessment that attempts to transcend the module based assessment model.

As part of the meeting Graham Gibbs discussed his guide "using assessment to support student learning" which is available at:

http://www.leedsmet.ac.uk/100317_36641_Formative_Assessment3Blue_WEB.pdf

Graham provided me with a copy of the guide during the meeting. It is an extremely useful resource for teams embarking on using assessment as a vehicle for learning. The ideas set out by Graham are illustrated and supported by case studies, which makes it very accessible and relevant to the reader.

On my train journey home I was able to reflect on what had been really useful and informative day!

Collateral Effects

2010-05-19T04:21:00.000-07:00

We have been approached by some of the teams working in the sports therapy area to see if we can work with them to look at their approach to assessment and learning on two of their modules. They had come along to one of the university wide presentations that we gave on the ESCAPE project .

We are exploring with these modules how the the use of student produced videos of therapeutic techniques can be used to support the students with their practical classes. One of the themes that a is emerging is motivating students to practice the techniques on a regular basis so that they can master them.

After speaking with one of the module coordinators this morning it striking how similar many of the assessment issues resonate with those that we are already working on. It is clear that the foundation that we have laid with the original module teams is providing a jump off point in applying the assessment approaches that have been adopted to these "secondary effect" modules . We had of course expected the to be secondary and collateral effects - viz: others we would work with or would influence, but what is surprising is how portable some of the techniques and approaches that our module teams have developed piloted may turn out to be.

Use of a Visualiser in Mathematics Support

2010-05-05T06:25:00.000-07:00

One of the technologies that we are looking at to support students is the use of short videos. We are working with staff and students using flipcams to demonstrate laboratory and mathematical techniques.

IN the case of the mathematical techniques, we are using a visualiser to project the image onto a white board and focusing on the projection of the paper that the calculation is laid out on.

The calculation fills the screen and the lecturer explains the mathematical technique, carrying it out step by step. The lecturer's narration and explanation indicate the difficult parts of the calculation and guides the student through the technique. The videos will be made available on Studynet and students opinion on their usefulness in supporting the mathematics workshops will be sought. Their particular use will be demonstrating applied mathematical laboratory techniques when students which students traditionally have problems with.

Is a good example of how simple technology can support the appropriate pedagogy in delivering student centered learning. Furthermore we are able to gain the student view as to their usefulness through the discussion feature on Studynet.

WATS in BioSciences

2010-05-04T01:48:00.000-07:00

One of the modules that the ESCAPE Project has been working with has been the Introduction to Biochemistry, Microbiology and Pharmacology module. This is a level one introductory module for 200 life science students that is taken by students on a number of pathways. Students have traditionally struggled on this module in applying mathematical techniques. particular areas that are flagged up through diagnostic testing are: calculation of dilutions, use of logs and ratios and percentages . These areas all present students with problems( see slide at top of blog entry). In order to address this skills gap the mathematical techniques required for the module were taught using a series of five maths workshops in semester A.

This year Ella Bryson and Jackie Willis have integrated WATS - the Weekly Assessed Tutorial Sheets into their teaching. The WATS system developed by Mark Russell is an automated assessment system that allows teachers to set questions ,which are the same for each student - however the data within the question is different - so that no two sets of answers are the same. The students log on to WATS download a set of questions, do the calculations and submit their answers electronically. The Answers are marked automatically and students are emailed their results.

Ella and Jackie harvested typical questions from the colleagues who deliver the teaching and the practical work on the module, and Incorporated these into a set of 15 WATS problem sets. The WATS system was used to build upon the material covered in a series of workshops.

Students were tasked with answering a set of questions each week for 15 weeks.

Students were given an anonymity number (a three letter code) after each set of questions were submitted,a league table of results produced so that students could see their (anonymous) ranking. After submission students were emailed the answers with feedback messages to support their learning. Additionally the WATS was integrated into the personal tutorial system.

Ella Bryson has commented that "WATS has allowed us to teach maths to students in a way that actively encourages them, it promotes independent thinking by making it impossible to copy answers from others. It gives immediate feedback on how each student did, where they went wrong and how they compare with the rest of the class , stimulating competition."

Jackie Willis felt that " BioScience students don't always appreciate that maths is a fundamental part of their course." and " Students are so used to compartmentalising information once the ( Maths) workshops are completed they..fail to apply what they have learnt in their practicals. WATS engaged the students on a regular basis and we were able to contextualise the maths so that it complemented and supported what was happening in their practical programme"

Thanks to Ella and Jackie's hard work they have really been able to support their students using this innovative and effective approach. Ella and Jackie are now exploring ways to roll this out to support second year students.

JISC Conference 13th April 2010

2010-04-23T03:22:00.000-07:00

Last week Mark Russell and I presented at the JISC Annual Conference, which this year was held at the Queen Elizabeth II Conference Centre in London. The opening keynote speaker was Martin Bean, Vice Chancellor of the Open University. One of his Keynote themes was that students are often well able to find information - there certainly is enough of it out there, but are less adept at assessing the veracity of the information they do find. He spoke about teaching students to not only discover, but also to teach them to "triangulate" the information. He then went on to challenge the audience - always a good thing(" I'm an Australian - you can't offend me!" was one of Martin's opening gambits). He spoke of the different perspectives between British and Australian textbooks in dealing with the Galliploi campaign of the first world war. This is still clearly an emotive topic after over 90 years and brings into sharp focus the concept of triangulating knowledge - and the need for students to develop their own understanding and perspective.

All in all a really challenging and informative keynote address

It's Quality Time!

2010-04-23T02:47:00.000-07:00

One of the things that has struck me this week is the how often subtle changes to a modules delivery can have quite wide ranging changes to the learning interactions that take place. The example I am thinking of is the Corporate Social Responsibility and Sustainable Marketing Module. The module coordinator - using his experience from previous years, has posted a set of FAQs on his module Studynet site. The questions address points and issues that were raised by students last year and were generated from the experiences of the whole module team.

As a result, module team's interactions with students - both face to face and online, been able to focus more on developing the students understanding of the material covered in the module . Students can be pointed in the direction of where to find specific information. This leaves the team time to develop the "high quality " dialogue and interaction with students that can really support and personalise the students learning. This simple strategy(one of a number adopted by the team) of using a well primed bank of FAQS has acted as a filter so that the more complex and interesting questions that students generate can form the basis of meaningful discussion with the module team.

Here comes the data (2)

2010-04-23T01:39:00.000-07:00

A previous post (Here comes the data 1) outlined the use of technology to collect feedback from students engaged in a peer assessment activity. In the previous post I suggested that was a real win-win. Better learning and time savings.

Here are some of the findings. Click the image to read the responses.

Here comes the data (1) ...

2010-04-23T00:58:00.000-07:00

We are now starting to see the fruits of our labour on the ESCAPE project. Some of the 'fruits' arise when we talk to staff about their assessment practice and resulting experience whereas other 'fruit' arises from observations of student performance and their engagement with their studies. I thought I would share some of what we are finding ...

A new module (not one of the original ESCAPE modules) wanted help with Peer Assessment. The member of staff was already engaging her students with peer assessment and hence were reaping numerous learning gains.

* Students were able to see how their peers responded to the same task
* Students were able to engage more with the marking criteria and standards
*

Previously, however, she was kept very busy after the peer assessment by dealing with more than expected students questioning their marks. Whilst it is highly appropriate that the students are exposed to a fair and reliable assessment, many of the efficiency gains made by the staff member were lost due to the need to deal with students on a one-to-one basis.

With the help of the ESCAPE project we were able to re-purpose a web-based data collection facility (developed to support computer based assessment). Using the web based data collection facility we posted to the students a series of questions (asking them to reflect on their own submission and the peer assessment process. This was an addition to the work previously done and hence created addiction learning gains. Importantly, we also included an opportunity for the students to 'comment on their mark' and note that if they were over or under marked to provide evidence where this was the case -with reference to the marking criteria used in the peer assessment process. The result was of which a vast reduction in 'additional' time required by the staff member to look at the concerns.

This addition was a real win-win. Students were now reflecting on the process (and sharing the reflections with staff) and the staff were reaping efficiency gains.

Slightly self-promoting but this typifies exactly the type of things this project is about. Using technology (led by pedagogy) to reap learning and efficiency gains.

I can't see this technology-supported intervention stopping when the project finishes.

Feedback

2010-04-02T04:19:00.000-07:00

We have sought numerous opportunities to collect feedback throughout our project and not just at its end. In fact, we are awash with data and feedback. It will be a real challenge to use the data and feed it into our final evaluation. We have so many threads of activity and so many things we would want to share.

But I do want to post here one of the observations made by one of our ESCAPE partners. In many ways the quote captures everything we're trying to do. Support our partners, help them with their practice and allow them to take the work we are doing with them into other modules. i..e Spreading the learning and benefit.

The quote came from a conversation between our ESCAPE partners and was not solicited in any way. You can imagine my scrabble for a pen and paper!

The ESCAPE project has made me think about the way I give feedback and it has changed my practice in all modules.

She went on to say that

I now organise myself such that I am able to give feedback straight away. I now plan my diary around large coursework to free up 2-3 and re-arrange teaching sessions so that I can get a chance to give feedback.

Arguably, the above quotes demonstrate the transformative nature of the project and shows that the learning and benefit is not constrained to the modules we are working with.

Great stuff!

Learning from the slopes!

2010-04-02T03:57:00.000-07:00

I'm not a skier. I have never been skiing, until recently that is.

But as I was learning to ski (or rather learning how to pick myself up from the white stuff) lots of my time was spent thinking about my learning (on the slopes) and the things that supported my ability to learn and get going.

I'm really interested in notions of personalised learning.
Personalised learning requires personal activity and learning activities that are adaptive to the demonstrable needs of the learner. This is true in the classroom and on the slopes. Personalised learning does not throw the learner in too far at the deep end but develops the learners confidence and hence their inquisitiveness and willingness to progress. Personalised learning does not swamp the learner with feedback nor provide fedback that is inaccessible to the learner either.

All of these things were so beautifully reinforced to me as I tried to ski. Putting ourselfs in situations that make us the novice (skiing in my case) was great to see some of the things about learning relate to me.

Being a novice and thinking about learning was just great.

Interim Reporting and Future Plans

2010-03-31T05:53:00.000-07:00

We are just completing our second (and last) interim report. It deals with the project's progress over the last six months. A large part of the report deals with mapping the proposed project objectives and outcomes against what we have been able to achieve so far. One important theme that emerges from the initial draft of the report is the sustainability of the project, that is what will happen when the funding finishes in October. How will we ensure that the projects legacy will continue? The report provides a useful focus for articulating these plans in the light of our experiences over the last 18 months.

Cluster Meeting 25th & 26th March 2010

2010-03-31T03:32:00.000-07:00

We have just hosted the third meeting of our cluster (universities of Bristol, Exeter, Hertfordshire and Westminster). The venue was the Comfort Hotel in St.Albans. The hotel was at one time the home of Samuel Ryder of Ryder Cup fame. The meeting ran as is usual from lunchtime on the 25th to lunchtime on the 26th. This gives everyone a chance to arrive in good time to start the meeting and for a social event in the evening - usually a group meal at a local restaurant. This time is valuable as much of the meeting is about sharing experiences of our projects and seeking advice from our fellow cluster project teams and our critical friend Malcolm Ryan.

The first day kicked of with a welcome and introduction by Malcolm and a review of the ground rules for the two days. The Chatham House rules mean that we can speak candidly about issues and events arising in the projects. We then moved on to looking how we had used the Appreciative Inquiry approach with our project and a facilitated session run by Rachel Harris of Inspire Research. We then moved on to evaluation and looked at how we are approaching this- thinking about "what a successful project looks like " and discussing the value in understanding why something did not work out as planned. The day was rounded of by a very pleasant meal in which the topics discussed during the day were further explored.

The second day looked at the plans for the final cluster meeting at Exeter - a draft agenda was proposed and celebrations planned! We then moved on to plans for a joint conference submission by the cluster at the Geenwich e learning conference in the summer. The day finished with a session on using Fishbone Analysis to explore the sustainability agenda of the projects.

All in all a really useful two days - we have similar experiences and issues and it was an excellent opportunity to raise them and to identify ways to move forward.

JISC Experts Group

2010-03-31T02:56:00.000-07:00

On Wednesday 17th March I presented a workshop on the ESCAPE Project at a meeting of the JISC experts group in Birmingham. The workshop gave an overview of the Appreciative Inquiry(AI) approach that we took with module teams. We then explored the emerging findings from two of the modules that we are working with. In particular we looked at how educational technologies (online submission, wikis, and online groups for example) provided a medium for achieving the pedagogical aims of the project (for example: a more consistent engagement with the curriculum, increased opportunities for students to engage with feedback and the facilitation of more personalised leaning opportunities). The day provided a good opportunity for getting feedback on the project and there were many questions on the use of AI.

It is indicative of the stage that we are at in the project that we are now starting to present the emerging results of the project on a national stage .This was always was part of our dissemination plan and its nice to be in a position to start to see all the hard work done by the module teams starting to bear fruit.

Sharing what we do

2010-03-10T06:53:00.000-08:00

As a project team we are really keen to share with the sector our work.

This manifests as presentations both internally and externally to the University of Hertfordshire. You will see that we have received a few invites too to present our work.

The University of Hertfordshire has recently re-launched its in house journal relating to teaching and learning. Or more specifically, Blended Learning. The journal has a practitioner focus and uses a variety of formats to engage the readers. The title of the Journal is aptly named Blended Learning in Practice (BLiP)

Dominic and I recently published in the journal. The article can be found here.

Supporting Peer Assessment

2010-02-26T08:32:00.000-08:00

During the recent half day ESCAPE review meeting we took along our Flipcams and 'invited' modules leaders (we are working with) to record an introduction to their developments. These short videos will provide a really useful resource-bank and will be useful for colleagues both inside and outside UH to see what we have been doing and more importantly consider the adoption of some of our approaches to their own practice. Thanks to all that contributed so positively :-)

Helen Barefoot and I took a Flipcam outside (for a walk) to hear about the recent ESCAPE activity supporting Peer Assessment.

2010-02-26T06:32:00.001-08:00

It was good to meet with the ESCAPE module teams on Wednesday - the half day event was a chance to catch up and swap experiences. The fact that we were able to take people away from the university for a day is a real bonus and gives people a chance to take stock and to reflect without distractions.

We were able to capture the reflections using the flipcams - now in their second generation so that they can record for two hours, although nobody took the machine to its limits. Our reflections to camera lasted about five minutes for each participant.

Along with the reflective theme that we were developing we are also looking at the collateral effects that the project is generating. some examples of these are:

Plans for the WATS system to be used with the MSc Life Science students next academic year - building on the successes in using it with with a first year Life Science module

Working with another three modules from the division of sports science over the coming months - facilitating and supporting a re - engineering approach to their assessments

Working with one of the ESCAPE module coordinators on a masters module to support the use of video to assess group work

Working with one of the compulsory first year modules in the Business School that has over 1600 students taking it in two cohorts.

Being approached by a module coordinator to help to re-engineer the assessment for a large Health & Human module that involves students on placement - assessing them them through the use of group work and a piece of reflective writing.

We are looking at what is the best way to work with people who have approached us and we will probably work with individual teams or areas as and when we are asked to. This approach offers us quite a bit of flexibility in the approaches we use - rather than using a larger " come into the big tent and see the show" approach.

2010-02-23T07:53:00.001-08:00

Tomorrow we are running a half day event for some of the modules that we have been working with. Four of our modules have now completed their piloting phase , four are still running and one has just started . Now at the end of semester A is a good time to get people together to reflect on the changes made over the last semester. We have once again become aware of the time pressures that colleagues are under and the fact that it is extremely difficult to find a time when all who need to can attend. Any work that comes out of this project will need to reflect on how this is overcome - we have achieved it by running separate sessions to cater for staff availability.

The event tomorrow will have three themes:

Reflecting on the changes made to the module in the light of ESCAPE.

Mapping these changes to agreed principles of assessment for learning.

Planning for sustainability - ensuring that the changes that are put in place in the piloting phase are built upon.

We are aiming to capture the reflections with the use of the ubiquitous flipcam. We will be exploring:

What went well - share and celebrate your successes

What needed more development

what was the change in the student experience

what was the student response to the changes

Incidentally quite a few of the module coordinators have flipcams and I will blog about what they have been used for soon.

I am looking forward to a really good afternoon - it will be the first chance that many of the lecturers have had to meet with their peers since before Christmas. And hopefully it will provide good opportunities for sharing of experiences and ideas.

Off to Scotland

2010-02-04T03:59:00.000-08:00

Helen Barefoot and I were invited by the HEA to talk run a workshop on assessment and feedback at an all day Assessment event at the Robert Gordon University. What follows is a collection of (very quick) thoughts on the day …

Helen and I have run a few workshops together (inside UH) and so it was just great to take our work (much of which is guided by our ESCAPE activity) to colleagues outside UH. For various reasons the workshop did not run :-(. Despite our disappointment we did get to hear some great assessment related presentations.

Dai Hounsell presented a really grounded key-note and, in fact covered, much of what we were covering too. That assessment is not a new challenge, that good assessment is planned activity and that good assessment stimulates learning. We did not need the NSS to get us thinking that assessment is important. Some really useful slides from Dai that I will explore and come back to in a later post. Great start to the day.

A couple of student perspective presentations followed.
A student led campaign that successful introduced a turn-around-time policy for coursework and interestingly a policy to provide feedback on examination scripts. The learning gains to be had from providing feedback on examination scripts seems rather limited to me. I’m always banging-on about feedback creating consequences, and I’m just not convinced I know what consequences flow, or are able to flow, from feedback on end-of-process, high stakes assessment tasks. That’s a post for another day. But just to say, I’m off the fence on this one. I just don’t get it. Another delegate did note they had a similar policy at his institution and only 15% of the scripts (with feedback) were picked up. Surely we would be better placed putting our feedback on work that will be picked up and more importantly attended to, by the students. And relax!

Steve Draper, engaging as ever, had a couple of threads running through his presentation. First, was the interesting anomaly that overall a department was rated 5th against other departments (107 in total) for the NSS question overall, I am satisfied with the quality of the course and yet questions relating to feedback were ranked much lower. Feedback on my work has been prompt (ranked 54/107), Feedback on my work has helped me clarify things I did not understand (ranked 79/107) have received detailed comments on my work (ranked 101/107). Steve asked us to explore what might be going on. Was there a complex weighting algorithm for all the items on the NSS? Should the individual items of the NSS not sum to the overall score? If not, what was missing, what was the missing ingredient? Second, Steve separated declarative and procedural learning and pondered as to where our efforts on providing feedback might prove most effective. i.e. might we get more learning from less or (better targeted) feedback?

Off to Wales

2010-02-04T03:00:00.000-08:00

I was invited to present some work on Blended Learning (which included assessment activity) to Glamorgan University. Rather than just present ‘stuff’ I tried to engage those that gave up their lunchtime by asking them to consider their own challenges and also ask what they thought supports learning. Clearly, technology enhanced learning is about learning, not technology, and so I wanted to help surface their own values and pedagogic perspectives. I presented the UH context and gave some examples of Blended Learning in practice.

Following the session I was whisked away to share some of our ESCAPE activity. Glamorgan have a Change Academy project which focuses on assessment . I tried to outline some of the ways in which we (the ESCAPE team)( were working and also highlight some of things worth considering.

I tried to emphasise

* That whilst assessment was the focus of our attention, our project was about change management too.

* the importance of, and benefits to be gained, by considering an Appreciative Inquiry approach to evaluation

* The importance of trying to understand the culture and constraints and of the Schools we were working with

* The importance of working with a principled approach to curriculum design and that we (the project lead) need to bring the principles to the Schools.

* That innovation and change is stimulated by many things and that we need to be alert to the different stimuli and create situations where they can flourish

* That developing educationally effective assessments alone might not stimulate all colleagues. Many staff are experiencing numerous time pressures and so consideration of resource efficiency along with educational effectiveness is important too.

It’s a real shame that I didn’t get more / any time to hear about their work but having established contact I will follow things up to see how they are working and how we might benefit from their insights too.

Module Coordinator Experiences

2010-01-29T03:08:00.001-08:00

This week we held a lunch time seminar to give staff a chance to learn about the project and to hear from two of the module co-ordinators that we have been working with. James Johnstone is module coordinator for a second year module - Principles and Practice of Sports Science and Hazel Wagner is module coordinator for Advanced Corporate Reporting a third year module. Hazel and James spoke about the issues with the modules prior to the ESCAPE project and took us through the process of re engineering that they went through in order to increase the amount of engagement that the students had with their assessments. We had alluded to this at the Ulster conference ( see previous blog entry) but it was really exciting to hear the people responsible for managing and applying the changes speak with passion about their modules. The presentations gave an indication of what was possible if to quote James " .......we leave the Rolls Royce in the garage" This is a reference to the work done by Win Hornby, of Robert Gorden University on efficiency and effectiveness in assessment .

The assessments developed by Hazel and James were effectively multi component designed to engage students on a regular basis over the whole semester. They have multiple opportunities for feedback to students and for students to engage with the feedback,using it to inform their application to the next component of their assessment.

One of the themes of the assessments is that it gives the chance for the lecturers to understand where the students are at each part of the module - to really get a feel for the students grasp of topics or issues. This reminded me of the example of when I walk Holly, my Springer Spaniel, we are only together at the start and the end of the process - where she goes in the middle bit I am not too sure - we meet again at the end. This is all to often what happens with our students -we know what they are thinking at the start of the lecture and we know where they are at the end - but they probably have not been following us all the way through.

The seminar finished with question and answer session with the audience keen to quiz James and Hazel on their experience of applying the new assessment.

Belfast Bound!

2010-01-29T01:33:00.000-08:00

Last week Mark and I presented some of our observations from the ESCAPE Project at the University of Ulster eighth annual eLearning Conference. I had never been to Northern Ireland before and was excited by what I saw - it certainly challenged my preconceptions . Belfast is a vibrant town, new buildings and infra- structure projects were springing up everywhere - and there was a real energy to the city. The venue for the conference was the School of Art and Design. The campus consists of two buildings, one a turn of the century the other a much more modern one. The buildings are connected by a new covered walkway. They were really excellent learning spaces, well designed and resourced.

Our presentation, which we gave twice, looked at the background to the ESCAPE project and the wider assessment landscape. Mark spoke about the challenges of the project and how the extensive literature on learning and assessment should be informing teachers practice. I spoke about working with module teams and discussed some of the changes that teams had put in place as a consequence of working with ESCAPE. We also discussed the role Appreciative Inquiry played in the project .

On the way back to the airport the taxi driver took us to see the Thompson Graving Dock. This is where the Titanic, which was built at Harland and Wolf in Belfast, was brought and fitted out after she was launched.

I contrasted this with the the keynote address at the conference, where professor Paul Moore spoke about the changing technologies that we use and how they have influenced us, using an in car record player from his father ford Anglia to illustrate his point. He spoke passionately about the iPhone and contrasted this with his blackberry which he referred to as a " wheelbarrow for emails". His perspective was an interesting one and his excitement about the impending launch of the iPad was infectious. Paul felt that this device would have a huge influence on learning over the coming years - interesting times!

For those of you that have not realised - the flag at the top of the blog is that of the White Star Line, the shipping line that operated the Titanic.

A Brand New Year

2010-01-11T03:53:00.000-08:00

It's been quite a snowy start to the New Year here at the University of Hertfordshire with the University being effectively closed for two days. The snow has started to clear and things are now getting back to normal. Christmas seems to be a fast fading memory.

There are four modules that the ESCAPE team are working with, that finish in semester A - which is in two weeks time. We will be capturing the module team experiences through interviews and running the student online assessment survey again to capture the student view.

Conditioning and Exercise in Sport is a level 3 module due to start in semester B. We are working with them - looking to use students to produce a series of conditioning and exercise techniques as part of the formative assessment process. It will be interesting to see how the students get on .We will be issuing students with flipcams to record their sessions.

eBiolabs University of Bristol

2009-11-13T03:43:00.000-08:00

We have been really impressed by the eBiolabs project at the University of Bristol - one of the three other project in our cluster. The project has many resonances with the work we are doing with one of our 9 modules -" Introduction to Biochemistry, Microbiology and Pharmacology". The module contains a large element of practical work and the eBiolabs approach of preparing students before they undertake the practicals by taking them through a series of online tasks. This element of engagement prior to the practical work itself and the corresponding post practical online tasks would be something that we would like to incorporate in the module.

We have spoken with Gus Cameron - the eBiolabs project manager and we are hoping to be able to use some of the stand-alone resources that his team have produced. This is an exciting opportunity for projects to start to disseminate the products that they have developed. We In turn anticipate that the case studies that we are developing along with our change management model - incorporating the AI approach will become widely available to others. The use of the JISC Design Studio is the ideal forum for such dissemination of project deliverables

Collateral Effects

2009-11-13T03:07:00.000-08:00

The ESCAPE project is at its most critical and interesting stage - we have launched the new assessments with 8 of the 9 modules we are working with ( one is starting in semester b) and we are closely monitoring the results. Each of the modules has a different approach with some changing things drastically other less so. It is interesting to see the role that technology is playing with the use of virtual groups, wikis and flipcams in some modules - while others rely on a total realignment of the pedagogy and eschewing a technological aspect to the re-enginerring and concentrating on the pedagogical delivery in a traditional way - albeit constructively realigned.

We have been trying to capture the collateral effects that the project has had on teaching and learning across the schools. Examples include people trying approaches new to assessment that have spilled over from the project into other modules that they teach on. We are seeing things such as a sports coaching module using wikis as a collaborative vehicle with groups of students. A second year accounting module using virtual group work. The most interesting and unexpected secondary effect was a colleague taking a ESCAPE flipcam to the arctic with master's students and recording a not been recorded before ice flow phenomenon. It is this kind of unplanned but hoped for secondary effect that underpins our hopes for the sustainability of the project.

We are looking to post the video of the phenomenon on our University of Hertfordshire iTunesU channel.

Continuing with the video theme, we are looking a producing a series of quick instructional videos with Bio Science students for use in their laboratory sessions - demonstrating practical techniques. looking for inspiration and to show the team what was already available I looked on YouTube and was amazed at the number of instructional videos that are already available online. Clearly this is a popular subject and one where there are already off the shelve resources available.

Sustainability – thoughts from the Experts.

2009-10-28T14:16:00.000-07:00

The JISC operate a Pedagogic Experts Group which I have been fortunate to have been a member for a little while. At a recent event a cluster from the Curriculum Design programme were invited to engage the experts as were a cluster from the Curriculum Delivery programme. We (ESCAPE) were part of the Curriculum Delivery cluster invited to talk to the Experts. I jumped at the chance and suggested that our session be about sustainability.

I am keen that our project does not wither on the vine when the funding runs out, nor do I want our leaning to stay confined to us at UH. And so I was keen that we didn’t just engage in a traditional show-and-tell routine but use the Experts Group to help shape and / or validate our thinking on sustaining our projects.

After a brief into from Malcolm Ryan – the critical friend of the cluster, Gunter Saunders outlined some of the work from Westminster’s project. I followed and briefly introduced ESCAPE. We then moved to ask the group …

* What would you do to ensure that the projects are not sustained ! (reversing the question is just great to get the ideas flowing ) and

* What one thing would you suggest the projects consider / do to ensure the sustainability of the projects.

Some great ideas flowed

Working with Dominic we I will go through the list and use the thoughts / ideas to challenge our own work.

I always like the buzz that seems to be present at the JISC Experts Group and I was particularly (selfishly?) pleased that some of the buzz and thinking will flow into our ESCAPE project.

ESCAPE – surfacing and using existing technologies.

2009-10-28T01:02:00.001-07:00

A few years back I developed a technology rich assessment programme. The approach to assessment uses technology to create, collect, mark and provide feedback on student-unique, Weekly Assessed Tutorial Sheets (WATS). The WATS approach to assessment was developed to support my own engineering undergraduates but has also been used and influenced practice elsewhere - both inside and outside the university.

The ESCAPE project provided another opportunity to showcase the WATS approach to assessment and debate how it might respond to the challenges of the ESCAPE partner schools.

Due to the ESCAPE project, the School of Life Sciences are now engaging with WATS. In addition to using the WATS approach to assessment in ways which we already know supports students‘ learning,, the School of Life Sciences have put a new twist on its use. The School of Life Sciences are using WATS to set tasks in IBMAP – a core module taken by many students. The students tackle weekly tasks and are presented with prompt and personalised feedback. Additionally, the students’ performance data is fed to the students’ personal tutors. Now the students study profiles, and performance form a useful part of the pastoral care conversations. Although not specifically developed to support pastoral care, the nature of the data that flows form the WATS approach to assessment suggests it can readily be used, not only to support learning and shape teaching, but provide early warning signs of struggling students.

I will keep you posted of how the WATS approach to assessment shapes up in the IBMAP.

Collateral Effects

2009-10-07T05:07:00.000-07:00

As the ESCAPE project moves forward through its various stages it is becoming increasingly apparent that thee are a great many collateral effects from the project. Many of the conversations that are taking place include references to stakeholders using the techniques , technologies and practices that they have been exposed to in modules other than the "official ESCAPE "ones. There are many incidences of the ESCAPE influences spilling over in to related module areas. Examples include:

a new approach to supporting practical work in Life Sciences, using short videos of practical techniques made available on studynet.This will have a wider impact across the school on other modules that use practical work
The increased use of flip cams as a teaching aid - with students being encouraged to produce their own videos
the adoption and embedding of the WATS mathematical tutorial system across a number of on ESCAPE modules, rather than the initially envisaged single module
approaches to group work and submission of draft course work - whilst not adopted in a particular level 3 ESCAPE module being considered relevant for a level 1 non ESCAPE module, that the module leader teaches on.
ESCAPE module leaders starting to act as "agents for change " across other modules and with other module teams - this is possibly a consequence of the growing confidence felt by the ESCAPE participants

It will be interesting to see how these collateral effects develop, as they will be key to the project's sustainability over the long term - when the ESCAPE pebble is sitting at the bottom of the pond its ripples will still be felt!

Poster and Video

2009-10-07T04:21:00.001-07:00

This last week we have been involved in the production poster and video in preparation for a joint meeting of the curriculum design and the curriculum development projects in Manchester on the 13th and 14th of October. The poster is a visual representation of the stages and mechanisms of the project. It captures the journey through the project by the stake holders and illustrates the approaches taken at the various stages of the project by the ESCAPE team. It was influenced heavily by the change management development activities that we were involved in at our last CAMEL meeting. One of the aspects of change management was looking at a four stage model of change management, viz:

Awareness
Interest
Trying
Adopting

It was interesting mapping our project activities to these four stages and realising the the stages were not atomic- that is to say that they could overlap - and that it was possible for stakeholders to drop back a stage( or two!).

The poster does not really capture the complexity of what we are doing - for example we are working with 9 different module teams, with some module attempting quite complex and innovative changes in assessment practice, but is a good start as a "top layer" map of the project.

I am now working on a three minute flip cam video that will introduce the ESCAPE project and include a stakeholder voice. This along with videos introducing all the other projects, can be found at www.youyube.com/jisccdd - a JISC dedicated channel on YouTube.

Cluster Meeting

2009-09-24T06:44:00.000-07:00

We have just returned from our latest CAMEL meeting with our three cluster partners - The Universities of Bristol, Exeter and Westminster. The meeting was hosted by the team from Westminster at their New Cavendish Street site, and as usual was held over two days. One of the most important aspects of these meetings is the chance to meet colleagues and to share our experiences of how our projects are going.

The first day consisted of a couple of really informative session on change management followed by a computer simulation exercise which involved guiding a change management through to completion. Needles to say it started to get quite competitive with teams eager to claim victory. The simulation was in many respects quite realistic and mirrored some of the issues that arise in the ESCAPE project.

The evening was spent catching up with the experiences of the other teams and mapping out the coming year. Plans for the joint JISC Curriculum D&D Programme Meeting in October were discussed over a very pleasant evening meal with the other teams.

On the second day we were asked to give a short presentation on the progress we have made - specifically looking at pedagogical tools that we had developed.The University of Bristol team demonstrated their eBiolabs project . They have developed an interactive online laboratory manual that is designed to prepare and support students through their practical work. The resource being developed is an excellent one and builds on previous work done by the chemists at Bristol, who had a similar project.The link to eBiolabs is http://www.bristol.ac.uk/ebiolabs/ and is well worth a look.

We were able to talk about our experiences at Hertfordshire and the kinds of approaches that we have taken. We outlined the progress made and mapped out how we saw the next six months as the new assessment landscape is bedded in.

Due to building work being carried out at Exeter we will be hosting the next CAMEL meeting which is scheduled for around April 2010.

Now the hard work really starts!

2009-09-14T05:44:00.000-07:00

At present I am in the process of meeting with the module co-coordinators to flesh out their plans for re-engineering their assessment regime. This picks up from the work done in July in the assessment re-engineering workshops that we ran. This was a chance for the module teams to reflect on their current practice and to start to apply some of the Appreciative Inquiry techniques that we have been looking at. This includes using reflective interviews to tease out what works well on the module and use this as a basis for the review of the assessments.We are also using "Fish bone Analysis" as a tool to get the teams to see what needs to occur for the re-engineering to happen.

Rachel Harris from Inspire Research has worked with the project team over the last year on the AI approach - acting in a consultancy role. Rachel lead a couple of the sessions at our two day event and writes about her experience in her blog http://www.inspire-research.co.uk/blog.html

I have just finished the draft interim report for the project. This is a good chance for us to reflect on the progress made over the last year with the project. It is quite sobering to think that we are halfway through the life of the project - now we are at the really exciting stage where we can really start to influence the assessment practice in the schools.

Mud moving in Africa

2009-08-19T13:57:00.001-07:00

A few weeks a go we ran a 2 day workshop for ESCAPE module leaders and some of their team members. It was really useful to consider the modules in details. The module leaders used a fishbone analysis to identify areas for improvement and the actions required. The activities were then transfered to a time line. With the support of the ESCAPE team it will be really exciting to see the changes occurring.

Much of the discussions involved consideration of the use of technologies to improve effectiveness whilst ensuring efficiency. We talked about the use of video with students, using Camtasia and Cam studio, podcasts, an assessment managment/marking tool, WATS, online submission, tablet PCs ... all sorts of options that we are lucky enough to have access to.

I have just returned from visiting friends who are doing voluntary work in Ghana. It was a very humbling trip. I took a tour around a technical college (16-19yr) (Cape Tech- see video) and Cape Coast University. The lack of faciltites shocked me and trying to teach practical subjects e.g. buidling techniques, biology, optometry and physics, with little, if any, equipment must be incredibly difficult.

My friends, Vicky and Steve Ager are teaching at HE and FE, respectively. These are their first teaching roles so it was fun to sit with them and to think about how then can encorporate active learning and collaborative activities within their sessions and how to improvise practical classes with basic and cheap alternatives to the equipment we are accustomed too (e.g. using drinking straws to simulate scaffolding). The basis for our discussions were Chickering and Gamson's (1987) Seven Principles of Good Practice in Undergraduate Education. My friends soon started identifying where they could meet the priniples within different activitites.

'Encouraging contact between students and lecturers' and 'Developing reciprocity and cooperation among students' will be achieved when students gain valuable work experience within community projects e.g working in the local eye clinic. The students from the technical college will literally do some mud moving when they help with the construction of a mother's union building.

Visiting Ghana reminded me that we don't always need the latest technologies or equipment to ensure a valuable learning experience. Our intention is always to focus on the learning primarily and then to consider how we can use technologies which are fit for purpose and enhance efficiency. I am incredibly grateful that we live in a country where we have access to the equipment and resources we need.

A picture is worth a thousand words

2009-07-15T09:26:00.000-07:00

last week I attended the Greenwich e-Learning conference. It was really good to see so many colleges there from the He and FE sector. It was also a chance to catch up with Malcolm Ryan one of the co - organisers and our projects critical Friend. The Conference is always informative and enjoyable - this year was no exception.

Interestingly there were also presentations from secondary schools - who it seems are really starting to use e-learning in innovative ways. A particularly interesting example was that of a school using reflective blogs with pupils, to capture their developmental journey over a year. Looking at the honesty and sophistication of the some of the later postings the pupils made and comparing them with the initial ones really showed how the students had matured in their use of reflection as a tool in personal development.

The Keynote speaker was Professor Jonathan Drori a former BBC producer. His personal insights into what made a good learning experience really opened the minds of the audience. He spoke passionately about how learning is a personal thing and was able to illustrate some of the many misconceptions people have about what constitutes a good learning experience. It was interesting to hear him relate his experiences of life as a broadcaster and the opportunities he has had to challenge peoples ideas on education. The pictures at the top of the post are from his presentation. It is of an art gallery in York that invited people to write on a wall what the pictures meant to them. The idea of being able to interact in this way - individually but also collaboratively had resonances with what we were trying to achieve.

It was good to be able to take so many of the Cable3.0 team with me to the conference. We have been working very hard for a year on developing virtual groups of students to work across the four FE colleges in Hertfordshire. And it was an excellent opportunity for them to get an idea what others were doing in the sector.

We are now in full preparation mode for the major re-engineering event planned for the 21st and 22nd July. This will involve 8 of the 9 modules we are working with. The first day will be spent reflecting on the current practice and looking at some possible solutions. This is followed by a day of re-engineering, working with the module coordinators to start to put into place the assessment for learning tasks that should add value to the student experience.

As an interesting aside we were able to travel to Greenwich on a mix of travel modes: Car, train , TUBE ,DLR and by boat along the Thames - a truly blended approach to travel !

Railway Journey

2009-06-26T05:55:00.001-07:00

Yesterday I attended the "Engaging and Responding to Learners" workshop run by JISC 'in Birmingham. It was an extremely valuable day - both in terms of the workshop content and as a forum to network and engage with colleagues on other projects.

I took the train from Euston. The different types bright shiny new rolling stock that seemed to appear briefly and then whizz past the window provided an interesting distraction during the journey. It struck me that when there was just British Rail operating the railways, there was a one size fits all policy - such that often trains would depart having four, eight or eleven nearly empty carriages.

The situation on the railways has a parallel with education - we are investing heavily in technological solutions for the students educational journeys. There is much more willingness to look at bespoke solutions and to embrace new ways of thinking than stay with the one size fits all delivery mechanisms of the past. The result? - faster smoother journeys for all!

For those interested - on my journey the train was hauled by a EWS Class 90 and I returned to Euston on a Virgin Pendolino

Casting Bread On The Water.................

2009-06-04T03:16:00.000-07:00

Yesterday afternoon we went live with the student assessment survey. We emailed all students registered on the nine modules that we are working with, to invite them to fill in the survey. The university exam period only ended last Friday . We were unsure when students would log on again to pick up their university emails after their post exam celebrations and the consequent recuperation period ! When I checked this morning there were 51 responses already! many students have given quite detailed feedback on their assessment experience , filling out the optional parts of the survey with their thoughts and suggestions. Lets hope that this level of response continues.This data is really important - along with the module case studies, it will provide us with the information to ensure the assessment benchmarking exercise is as comprehensive as it can be.

Regarding our two day event - one of our module teams is having difficulty making the dates - 21 and 22 July, so we are looking to have a separate event . This module is the most complex in terms of the delivery and assessment modes and has the largest module team. It is a Bio Science double credit module that encompasses three different areas of science.

It's a Two Day Event

2009-05-28T01:34:00.000-07:00

Things are starting to pick up speed with the project. We are planning a two day off site event at the university conference venue - the Fielder Centre. The likely date is towards the end of July - once all the exam boards are out of the way. The aim of the event is to give the nine module teams that we are working with the time and space to critically examine their assessment and feedback practices. And also then to start the reengineering process. We are planing ( at this early stage)the two days to be a mixture of presentations , facilitated workshops and module team meetings. The module teams will be able to call on the resources of BLU and the LTI to examine the particular pedagogical and technological aspects of possible assessment scenarios. In particular we are calling on our experts in curriculum design and change management.

By the end of this week all the module coordinators will have had appreciative enquiry reflective interviews in order to build up case studies of how the module is delivered and assessed . These case studies along with data from the student assessment surveys that we are carrying out, will give us a really clear picture of the issues surrounding each particular module. They will become the jumpimg off point for the reengineering process .

What if ?................

2009-05-15T02:08:00.000-07:00

It has been another busy couple of weeks.The reflective interviews are continuing with the module coordinators and module teams from the four Business School modules that we are working with. As I transcribe these interviews the passion that the lecturers have for their subject shines through. I suspect as long as the passion is there any problems can be overcome. I have had a positive reception from every one in the schools I have been working with and people are keen to share what they feel the issues are and how they (and we!) can work towards a improving the student engagement and attainment.

The next phase is to start to build up case studies using the interview material and the information gleaned from the base line audit of assessment practice that we have carried out. The case studies are quite comprehensive as the explore the module assessment landscape from the perspective of the people who set and assess the assignments . It will detail the role played by individual lecturers in assessment and teaching on a module and look to build a holistic picture of the subsequent assessment experience of a student taking the module. We are looking beneath the surface of the assessment practice to see exactly how and why the module works the way it does.

We are also trying to capture the student view via a student assessment questionnaire and are looking to conduct some focus groups. This information will be critical to our evaluation of the project to look at the before and after picture of student engagement and attainment on the modules.

We are now starting to thing about the next phase of the project - the most exciting - that of the re engineering of the module - this is when we dare to ask "What if ?"

Twitter - a tool for education 2!

2009-05-14T09:20:00.000-07:00

Had a really stimulating comment to my earlier post re Twitter and education - Thank you Anne Marie - It's just great that people read this stuff :-)

Okay, I know I should post this as a comment but I did not want to lose my thoughts in a comment - I also cheekily thought that other comments might flow to this post with some other ideas.

Blogging, by the way, is just great for getting you thinking and comments on the blog take that thinking to the next level. I like blogging me and I like learning too :-) Keep them comments coming. Rambling now. hmm, not so good after all. To business ...

The comment related to the use of Twitter as a tool for educational purposes and what might it *really* add in terms of communicating with students. I have to agree that was aligned with my original thoughts too. i.e. What seriously can we do with 140 chars? But after my introductory tour from Lindsay Jordan, and observations that I should immerse myself and have a go, I have become converted.

... and so I can honestly see some uses of Twitter for genuine educational gains. Sure, it's not the answer to everything, but little is. An essay does not explore students' numerical understanding nor their practical problem solving ability. In fact you might argue that a 3000 word essay does little to promote crispness of thought. Equally, Twitter has a certain functionality that might limit it's use but its functionality might be useful in specific situations.

I have dumped a few thoughts together with a minor preamble ...

I'm keen on Just-In-Time-Teaching (JiTT) (Novak et al) and also the Classroom Assessment Techniques (CAT) wonderfully described, collated and themed by Angelo and Cross.

Hence how about using Twitter as a tool to collect responses to a One Minute Paper?

* What did you learn today?
* what was the stickiest / muddiest point?
* What questions remian uppermost in your mind?
* What two things do you want me to cover next week?
* What question do you think your peers should be able to answer based on this weeks lecture.

I'm sure you get the picture and can add to my quick question list. Using a #hashtag will allow you to keep all the response together for each weeks questions. Not only do you get them thinking but you get a large resource for them to review.

I'm keen too that we see our students as valuable resources and so from one post each (in a class of 100) they get to see 100 posts - not a bad return.

I have written elsewhere that some of these questions they might just copy - and indeed that might be okay for the muddiest point etc, but other questions might explicitly require an individual response.

i.e. drawing on today's lecture relate the content / concept / idea to an experience that you have had. i.e. get them to relate it to them. That way they can't copy.

And then you write if your idea is already on the list then you have to write another. Not only does that get reading what others have posted, which in itself is good, but it gets them doing stuff quickly - i.e. reduces cramming - helps develop reflection and inculcates an appropriate study pattern

* What about asking students to post think-alouds to Twitter outside the class as they are thinking about the module? Guess these should really be called tweet-alouds (Russell, 2009).

* You might go further and ask if you had 140 characters per post and you could 5 posts two days after the lecture what would you posts be? Make those posts now.

* Or, as you open up the coursework what things are you thinking?
What theories will you draw on?
How will you show that ...

I think all of these have some genuine potential and could be used both inside the class (CAT) and also outside the class Non-CAT! and to support adaptive teaching, student centered teaching or JiTT.

Thoughts?

(ps am I rambling still?)

Mark

Twitter - a tool for education?

2009-05-14T00:31:00.000-07:00

A couple of weeks back I posted a fairly edgy item to a discussion forum entitled ‘twitter – tool or toy?’ http://blendedlearningconference.ning.com/forum/topics/twitter-tool-or-toy

My motivation was to see how it was being used and gain some insights into what we can really do in a teaching, learning and assessment context with 140 characters.

Twitter is limited to 140 character input and also was founded on the simple premise of the staus update found in Facebook etc. i.e. what are you doing now.

The forum entry took a couple of quick responses (notably from Twitter enthusiasts that were gentle enough not to berate my ignorance but to say, ‘have a go’. I wanted to make a reasoned judgement on its usefulness and so ‘have a go I did’ Or rather am!

I was fortunate enough to get a bit of a guided tour and given advice as to what it might / might not do. Twitter, I was told, does not do everything and rather than think of it a tool to engage in meaningful dialogue, why not use it as a seed to establish the dialogue and then take it elsewhere?

Well, I have to say that (at the moment) I’m sold. I like the connections I am forming and I like the so-called micro-blogging. I have used it to engage in back channel conversations at conferences and also at JISC meetings. I tweeted whilst at the recent Curriculum Delivery Programme meeting relating to Change and enjoyed reading the tweets coming out of sister Curriculum Design Programme meeting relating to Change.

Sure, I tell my followers what I’m up to – which includes my gym habits but I also use it for throwing out some thoughts relating to teaching, learning and assessment.

I really think Twitter-type tools have a place in education. 140 characters is much less daunting to a student than a blank blog page when responding to the questions such as ‘what did you get from today’s lecture?’

I’m going to take twitter to our ESCAPE teams and see what they make of it.

Oh, want to follow me and my gym exploits? I’m MarkRussell

Electonic Voting Systems - a workshop to the University of Northampton

2009-05-14T00:05:00.001-07:00

Out of the blue I received an invite to present some of my Electronic Voting Systems (EVS) activity to the University of Northampton - which was nice :-) It’s always good to get invited out, show your stuff and gain insights from others too.

After a few false starts we finally managed to fix a date.

I am always slightly weary of being asked to talk about technology. I’m much more interested in talking about teaching, learning and assessment and then, and only then, move to discussing the potential for technology. I took the same slant at my two hour session in Northampton. Numbers were much lower than I had anticipated, and hoped for, but nevertheless those that turned up were really engaged and contributed lots to the session. Thank you!

I opened by asking them to describe what they thought supported learning in a non-technology rich space. Lindsay Jordan and I did the same exercise at a workshop at the Plymouth e-learning Conference.

Lot’s of useful things came out including the value of

* Feedback
* Safety of the environment
* Being able to show what you know and what you don’t
* Dialogue
* Collaboration
* Enthusiasm
…

I collected these ideas and located them against what we already know about good learning. Anyone that knows me will be bored with my affinity with the Seven Principles … (Chickering and Gamson), the How People Learn framework (Bransford et al) and also the ideas behind Learning that Lasts (Mentkwoski and associates).

The idea was to show that good learning is good learning. We then moved to see how EVS might respond to some of the principles. I showed some of my work which also included the linking of questions to show the irony of some of the students’ answers, answer guessing and genuine knowledge of concepts.

I think the participants liked the idea. It makes a change from the more obvious use of EVS. I also showed the students views on the use of EVS in the module.

I closed the session with a rather non-conventional slide for an EVS session. I was really saying that my approach was not about EVS but rather about responding to principles (relating to learning) and that might these principles be served by technologies – and as for my last slide, by other technologies too.

Why do I need to buy expensive handsets when many students have mobile phones? What about digital ink on student own pc’s? Further, given that most disciplines, like life, is not multiple choice, what about technologies such as Twitter to engage students in class?

I enjoyed the session and got some great feedback afterwards – people buzzing apparently!

I am aware that Northampton is about to ramp-up its EVS activity and I wish them well with their endeavours

Thanks for the invite and for giving me a chance to hear about your work and present some of mine.

Both the Schools engaged in the ESCAPE project (Life Sciences & the Business School) use EVS but I would want to run a session with them to see how systematically they are using it to support student learning - I would also like to show them the last slide too.

i.e. what else might they do to support their students?

The session made me think – which was just great

Passion and Reflection

2009-04-29T08:19:00.000-07:00

I have spent the last few days having individual interviews with the Life Science module coordinators and their team members. The interviews involve reflecting on their involvement with the module and the assessment practices . The object of the exercise is to build up case studies of the current assessment practice ,to see what the particular assessment drivers are and to give people a chance to critically reflect on their current practice.

The appreciative enquiry approach that we are taking is really paying dividends - it is allowing the passion that these practitioners have for their subject to really shine through. It is interesting to watch people during the interviews start realise that they can say " wait - what if...........". Simple questions such as "tell me what you enjoy most about being involved with this module" are acting as a catalyst for open and honest reflection about the assessment practices.

This honest reflection along with being able to harness the passion of the staff are the key elements to the success of the project. They will be the drivers that will allow the re-engineering from within the module teams.

e-portfpolios

2009-04-24T07:57:00.001-07:00

Having attended a HEA Physical Science subject centre event on Wednesday about e-portfolios, I have been thinking about the benefits and downsides to reporting/reflecting mechanisms such as these. Not being a 'diary' user, my reflection tends to occur whilst riding home from work on the bus or at the weekend. My action on this tends to be random e-mails sent to Mark, Peter Bullen (Director of the Blended Learning Unit) or Jon Alltree (Director of Learning & Teaching) with my ideas and thoughts about what we should do. I am sure they get a little fed up with my e-mails but this system seems to work for me as I am pretty much an activist so like to get things done, rather than pontificating for too long. Most presenters at the e-portfolio conference had experience of using Pebblepad but in most cases staff and students were engaging with the e-portfolio when it was part of an assessment or a requirement. Is this because for most of us, we have so much going on that we only use new mediums when it is a requirement or is it because there are so many ways of reflecting and recording that it is really for individuals to find the medium which suits them?

I hadn't 'blogged' before writing on the mud mover blog, but I have found it to be an enjoyable process and I now occasionally blog after teaching activities. I also opened a twitter account yesterday so have been enjoying 'tweeting' today, but whether this is enjoyable because it is a novelty or a useful means of recording and communicating my thoughts, is yet to be seen.

I did wonder about suggesting to Mark and Dominic about the thought of staff and students on the ESCAPE project keeping e-portfolios of their activities. This could be a great way to share and record the experiences, but with FLIP cameras, questionnaires, evaluation reports, attendance at events and numerous meetings I just wonder if the time burden on busy staff and students is too great?

At the e-portfolio event I was chatting with Lisa Gray, who mentioned that she had thought about asking project teams to keep e-portfolios. I can see the opportunities as Lisa and other members of the JISC and steering groups/cluster teams etc. could access the portfolios and keep up to date with activities. This could reduce the burden in formal report mechanisms, however, I'm not sure that the JISC are ready to change their reporting mechanisms and again, I am mindful to the fact that project teams will have found their own ways to update colleagues on progress (e.g. via Blogs, workshop events, social networking sites, postings on the CIRCLE site etc).

I haven't really come to any conclusions about the using e-portfolios and I probably won't until I start one myself. I plan to do this in September when our 'in house' VLE activates its e-portfolio system, but I am increasingly aware that people like to reflect and record in different ways. Whether people collate their 'tweets' or maintain e-portfolios, it is the process of reflection not the medium, that is important and we must encourage staff to reflect in their own way and act on their thoughts to enhance activities for students and staff.

Correspondence with Steve Draper

2009-04-17T02:39:00.000-07:00

Following Steve Draper’s presentation at the HEA Assessment and Feedback day we have been passing emails back to one another. Steve is a great thinker and a really energising presenter. Go listen to him whenever you get the chance.

Two things came up in the correspondence.

1. The difference in the use of ppt between Steve and I. Steve amplified and told wonderfully engaging stories about the words he had on his slides. (I tend to shy away from .ppt slides that are littered with just text) and my talking over pictures. We also presented our ESCAPE activity and we wanted to help capture the attention of the delegates and thought a visual stimulus might help. We overlaid animations on graphs since we wanted delegates to really understand what we were talking about - I think the animations helped us do that (I hope) – it was just after lunch too – and I am aware I can be a tad soporific!

We discussed the usefulness of .ppt slides after such an event. With my approach it would be particularly difficult for anyone that wasn’t there to ‘get’ what I was showing. I am aware of this but also wondered how many people actually revisit the slides after such an event. Steve has a simple solution to this and brings a ready-made, single-sided hand out. The hand out comprises useful refs and links.

2. The second point we batted about, and of more relevance here, is the approach you take to engage staff in re-engineering activity. I am talking here about re-engineering curricula and in relation to ESCAPE re engineering and rethinking the assessment activities.

Although slightly extreme we discussed the merits of small steps of change versus big-bang change. It will be of little surprise that we both converged on the notion that both have something to offer.

With small steps of change …

It is more likely to keep staff engaged and on board i.e. in a way we are showing that we value what staff do but help to nudge things along (to be more educational effective and resource efficient) in a way that is both manageable and supportive.

For some staff a big-bang approach might wrench from under their feet everything they thought was good about their work. I wonder too if a step-wise approach is a great conversation starter and less threatening. Small-steps might prepare staff for some fun big-bang stuff and also develop their scholarship in a risk managed way.

The problem with small steps of change is that often you might never get to the heart of the problem. You tinker, fiddle on the edges, and whilst you might be adding incremental enhancements to a system / assessment (curriculum design approach), the approach /system / curricula you are tinkering with might actually need dropping!

Big-bang gives a great chance to ‘really’ explore the fundamental issue, establish a vision and help set out a curricular that responds to that vision.

We are doing both with ESCAPE and will look at how personalities as well as features of the modules/assessment arrangements best suit the alternatives.

Watch this space!

Curriculum Design Toolkit

2009-04-10T08:44:00.000-07:00

Although I am the ESCAPE Project Director I only (officially) get to work on the project for 0.2 of my time The other 0.8 is dedicated to my role as the Deputy of the Blended Learning Unit (BLU). In the 0.8 capacity part of my remit is to lead a team of seconded staff in the area of Curriculum Design and Innovation (CDI)

Some of the CDI team are currently working on developing a Curriculum Design Toolkit. The toolkit has three components and is intend to help individuals, module teams and programme teams reflect on, and where appropriate re-engineer, their curricula. The primary components of the toolkit are

A Diagnostic tool – used to establish where the problems in the curricula might lay
A Features and Consequences map – used to show what the ‘features and consequences’ of the diagnosed curricula are likely to be.
A Suggestions for Improvement resource bank – used to provide staff with different forms of advice should they wish to tackle any diagnosed problems

Wednesday and Thursday of this week I drew the team together to review progress and accelerate the development. The team are only seconded the BLU on a fractional basis and so they too have many varied demands placed on them.

The two days was full of energy and enthusiasm for the development.
After I outlined the vision for the toolkit and fed back initial thoughts from users of a pilot Diagnostic tool day one was set aside for the team to show where they are at.

Marija Cubric outlined the Technology-to-Principles Matrix. Given that UH is wedded to the notion of Blended Learning there is a need for us to help staff make appropriate decisions about the use of the varied technologies available. For us it is never ‘the answer is a wiki, now what was the question?’

Following Marija’s presentation / discussion we spent the rest of the day exploring the progress of the various CD strands.

Currently the strands under development are ‘Curriculum Design Toolkits for…

* Core values (Here we use the Principles of Good Practice in Undergraduate Education offered by Chickering and Gamson)

* Employability – this development is being led by Amanda (Mandi) Relph and Sarah Flynn

* Research Informed Teaching – this development is being led by Phil Porter

* Assessment for Learning – this development is being led by Maria Banks and Myself

Other strands to be developed include

* Internationalisation
* Entrepreneurship

The Diagnostic tool offers a set of research informed principles to the user. The Principles are different for each strand.

Each of the Principles are accessed separately by five questions. The Diagnostic tool rates the responses to the questions and shows, graphically, if there is good, or other, alignment with each of the Principles. We spent the whole of Wednesday looking at the Principles and the ‘accessing questions’.

Wednesday was really productive and we moved on our developments and thinking significantly. Wednesday also included a trip out to a local Indian Restaurant. It was great to continue the discussions as well spend some time socialising with colleagues.

Thursday was due to be a testing day. We were hoping (somewhat ambitiously) to get other members of the BLU / LTI to test the toolkits under different scenarios. Alas we were just too ambitious and so we stood the ‘testers’ down.

We continued with our work from Wednesday and also spent part of the day engaging with students. This was to incorporate their voice into the Features and Consequences Map. We thought getting the students involved would be a valuable contribution. I’m not sure any of us knew just how valuable that activity was going to be. We took input from Lauren Anderson and Elizabeth Terry (placement students in the BLU), Yoeri Goosens – BLU Student Consultant, Ruth Hyde – BLU research assistant recently engaged in the JISC funded STROLL project and Dawn Hamlet. Dawn is our link with the UH student Union. We have worked with this group before and they are always dedicated and committed. Thank you!

Each of the strand leaders met separately with the students to get the student view. It was really interesting to see how the strand leaders engaged the students differently. I suspect the different facilitation techniques kept the students alert too. That is they were not overwhelmed by the same technique.

Gaining the voice of the student was followed by a super-efficient and dialogic facilitation technique (Marija Cubric) to get us to vote on the likely ability of the various technologies to meet the needs of the various Principles.

We came a long way in those two days and although we are left with some on-going activity I doubt we would have so far if we had stayed in our offices and worked on the toolkits on an ad-hoc basis.

Oh, why am I writing this here? Mainly because of the Assessment-for-Learning strand that is being developed. It will be just great to get this and the Technology-to-Principles matrix fed into ESCAPE.

Watch this space…

Great job team!

ESCAPE: The first outing: HEA Assessment and Feedback Event

2009-04-02T00:47:00.000-07:00

The first outing of ESCAPE came at the HEA event on Assessment and Feedback. This event was created by the HEA to support the JISC Curriculum Design and Delivery programme. There were not too many other C-Delivery/ C-Design teams there.

I presented to a quiet room of just about 50 participants. It was after lunch and so maybe that’s why they were quiet! Hmm, maybe it was just me and my soporific presentation style!

I was given 10 minutes – I used it all. No time for q’s either :-( I did not spend any time outlining the case for assessment. It is already well made ! and was reinforced again by Brenda Smith in the morning session. I showed some of our approach in the hope that it might raise questions about what we are doing. I wanted to show a practitioner focus and evidenced based activity.

I showed our ‘mapping of the landscape’ and noted that this gave a great opportunity to ask the Schools if the map reflected what they wanted. i.e. did their conception of learning and their view of what their graduates attributes would be was supported by the landscape map?

I showed a modified landscape that I would have more allegiance with. i..e less reliance on final examinations and more opportunities on assessment for learning..

I spoke about capturing the secondary effects and showed that although we are only working with five modules by tracking staff and logging secondary and tertiary effects we are able to see how other modules have benefitted. We are really keen that a good assessment experience is the right of all of our students and not just those that happen to be taught by a member of staff that understands about assessment, alignment and the importance of feedback.

I then showed a technology-to-assessment-principles grid. This grid is intended to help staff make purposeful decisions about what technologies to use and how the different technologies’ relate to principles of good practice. The grid also shows how the technologies are being considered in terms of their likely
* Effectiveness
* Risk
* Cost

I.e. accepting that practitioners are likely to have different skill sets and resources. These skill sets and resources, as well as the ability of the technology to meet the Principles, are likely to be considered when selecting technologies.

Finally I presented the location of assessments on an efficiency versus efficiency matrix. I then tried to show how can make incremental change to separately improve the effectiveness and efficiency of the assessment.

I had fun animating all of this on the train!

No time for questions – let’s see if we get contacted after the event.

Putting assessment at the Heart of Learning : HEA Assessment and Feedback Event

2009-04-02T00:44:00.000-07:00

Norah Jones and Alice Lau (Glamorgan)presented a really piece on their work about engaging faculties and putting assessment at the heart of learning

Some notes I scribbled down as I was listening …

Different approaches are possible Plan vs Experiment
Experimenting allows people to make mistakes. The mistakes give potential to improve. See mistakes as a positive potential.

Students’ viewed were considered really important too – it seems as though they have done some great work here. Not only do they do the more tradional stuff (focus groups and surveys etc) but they used Facebook too to elicit students views – Interesting the Facebook group was initiated by the SU – They seemed to have lots of engagement and collected data that did not come through form their other methods.

They spoke about them collecting and sharing good practice too. And this was collected from the Institution – which, for me, helps contextualise the setting. i.e. stops at source, this cant be done here retort.

They spoke too of the creation of assessment diaries – these show bunching of assessments etc. The team suggested this initiative, the study diaries, had improved their NSS results. I wondered if it might be some secondary effects too – i.e. the engagement with th students in creating the diaries and seeing th challenges that the students face – Trojan horse rearing its head again!

Norah mentioned that assessment is a process not just a method and that to help them on their way the Change Academy activity was really helpful for. There is lots of information captured on web site which we will look at.
They have a journal too – will look to see if there is an opportunity to get ESCAPE published through this journal.

Really great presentation and a great job at Glamorgan – thumbs up!

A few thoughts from Prof. Brenda Smith. HEA Assessment and Feedback Event

2009-04-02T00:41:00.000-07:00

Brenda, as always, presents a really coherent argument and draws on a wealth of experience and expertise. Brenda’s position allows her to talk with numerous institutions which subsequently feed back into her knowledge. She uses these examples really well to allow her to provide some concrete examples of practice in the sector.

A few things I wrote down from her presentation …

* Engage students in the planning and curriculum planning – students can be great change agents.
* Assessments ‘can’ be set at week 3
* Get the Heads of Academic Schools at Teaching and Learning sessions – get T&L valued.
* Policies of professorship for T&L not valued or limited take up?
* A student friendly academic School gives a higher student feedback return
* The University of Lincoln, provides a booklet relating to students’ feedback. The booklet includes ‘this is what you asked for, this is what you said, and this is what we did. Helen and I are trying to get a You Said – We Did campaign up and running at UH. This resonated with us.
* Staff taking students to lunch and other simple schemes that attempt to engage students with staff. This again reminds me of Chickering and Gamson’s first principle ‘ Good practice im undergraduate education encourages staff student contact.
* Too much feedback might not be helpful
* Don’t just respond to students’ needs – Brenda gave the example of groups work and students not wanting it– Don’t forget to tell them why you are using it and what the purposes are. It’s not a secret!
* Stratchlyde – podcast lectures and use lecture time more productively – this frees up time for better feedback – This reminds me of what we do with the Blended Learning - use the technology o enhance and extend the traditional teaching sessions.
* Use of voice – audio supported feedback
* What do students do with feedback?
* Useful ways of presenting feedback – quickly!

Lots and lots more really useful ideas - I'm sure I've only touched the surface and dine an injustice to Brenda's session.

Hurrah for Herts. HEA Assessment and Feedback Event

2009-04-02T00:39:00.000-07:00

The HEA event on Assessment and Feedback (1 April 2009) provided a great opportunity to hear from some A&F experts I was flattered too to get an invite to present the work of the ESCAPE project. Thanks for the invite Eddie Gulc :-) The invite provided an opportunity to raise the profile of the ESCAPE project as well as raise the profile of UH. It was just great too to hear other speakers also refer to the good work of UH. Andy Gillett and Angela Hammond’s map of the assessment profile was raised by Prof. Brenda Smith and so too was our CABLE activity.

Thanks for the helping us fly-the-UH-flag..

Always Learning - HEA Assessment and Feedback Event

2009-04-02T00:37:00.000-07:00

Helen and I have this notion of Always Learning. It’s said with a slight tease but it reinforces the idea that everywhere you look you are presented with opportunities to learn new things.

Much of the traditional forms of learning are undertaken in lecture rooms, seminar spaces and laboratories, etc. but the informal settings provide fruitful settings for learning too. On our taxi Journey to the Higher Education Academy for an Assessment and Feedback one day Event (one of the three events organised by the Academy to support the JISC Curriculum Delivery / Curriculum Design programme, we were taxied by Gordon.

After a few passing pleasantries about weather etc., Gordon rattled off fact-after-fact about York Minster. Gordon had said that he was not interested in History in School but was just passionate about the space. Gordon was not an undergraduate, Gordon was not a postgraduate student neither he was just a guy that reminded me that learning is a lifelong endeavour and learning opportunities occur everywhere.

Final Version of Project Plan to JISC

2009-03-30T01:27:00.000-07:00

We have just submitted our " final version" as opposed to "draft version" of our project plan. the plan was revised in the light of comments received from our colleagues at JISC and from the steering group. The Plan is now quite a hefty document - including the work packages it runs to 48 pages. We have revised some of our time lines. In particular we have allowed much longer for the design and development of the assessments for learning phase. The modules we are working with are a mix of one and two semesters long. The single semester modules will run between October 2009 and February 2010 - which means in March we will have an idea of how well our redesign has worked.

This week we are starting to apply our appreciative inquiry methods to the school of Life Sciences. We are conducting interviews with members of the module teams. This will allow us to get to know each other and start the inquire phase of the Appreciative Inquiry method. We will be asking them about good learning experiences they have encountered. We will be exploring what was it that made it a good experience. with questions such as:

what happened?
who was involved?
what was your part in the experience?
what was it that made it possible?

Using these reflections we will to start to evaluate the assessment practices of the modules. We will be looking to establish a way of evaluating the assessment against efficiency and effectiveness.

ESCAPE coverage in the Universe

2009-03-25T08:44:00.000-07:00

Hi there, my name is Elizabeth Terry and I am a Business Studies student currently on placement with the Blended Learning Unit/ Learning and Teaching Institute (BLU/LTI). Working in such a committed learning environment, I have come to realise just how much the BLU/LTI value the students' opinion. Before working here, I had no idea how much thought went into enhancing the student experience, particularly within the classroom. Because of this, I have decided to do all that I can to make the students fully aware of what activities we do and the benefits that they bring to not only the university, but also their studies.

Liaising closely with Dawn Hamlet (Vice President Academic Support and Campaigns), I have successfully been able to get numerous stories on various projects that the BLU/LTI are running into the Universe (the student’s paper that has on average 4,000 readers). This week, the Universe features a story on the ESCAPE project. The article highlights concerns that are attached to assessment and invites the students to give their opinion and views on assessment and feedback.

Meeting Life Sciences Module Teams - Scene Setting

2009-03-20T09:31:00.000-07:00

So, after much behind the scenes activity we recently met with the module teams from the School of Life Sciences. We are engaging in Appreciative Inquiry (as an Evaluation Framework) and before we enter the Inquire Phase (with them) we wanted to set the scene and help get them on board with our work. They, like other Schools, are very busy and I have no desire to draw on their time unnecessarily. We set aside 2.5 hours for a meeting where we could outline the project and get them interacting. The time flew by - for me anyway. Dominic did a great job introducing the project - I simply made up numbers and responded to some of their concerns.

Naturally (given it was a 2.5 hour session), we built in stuff for them to do and feedback - I thought (and I did say that I hoped I had not misread the mood of the room) that they were very positive.

Helen Barefoot came along too since this is her School and she spans across the Learning and Teaching Institute and also the School. Helen was a great advocate and could help relate our ideas to their context.

We asked for their immediate thoughts - which are now being collated and will be fed back here.

At the end of the session we also gave them Flip Cameras and asked them to introduce themselves and talk very briefly about a positive assessment experience. The room was lively and vibrant and although it was late and at the end of the 2.5 hours we got some great clips. This will now start to set a very positive picture when we start the Inquire Phase.

Thanks LIfe Sciences - Great start!

All roads lead to ESCAPE

2009-03-17T12:40:00.000-07:00

Peter Bullen and I have been invited to act as guest editors for a special edition of Engineering Education. Engineering Education is the Journal of the Higher Education Engineering Subject Centre. We were approached (I suspect) since we are engineers, are passionate about engineering Education and because we are Director and Deputy Director of the Blended Learning Unit (respectively).

The focus of the special edition is technology supported engineering
education. See the fit?

We were overwhelmed with the response, it far exceeded our expectations and I suspect it exceeded the expectations of the Subject Centre :-) Naturally we then had the pleasure of reading the proposals. This 'pleasure' IS genuine since I'm always interested to hear and read about colleagues activity. Although we are guest-editing the regular editorial team are still in place to support us. I had a lengthy telephone conversation today regarding the proposals and how we move forward from here.

Some things that came from the conversation included ...

* What to do with a great proposal that exceeded the word limit?
* How it was comforting when the 5 independent reviewers agreed on the outcome of many of the proposals
* How it was frustrating when we didn't.
* How we can be encouraging yet maintain the standards of the Journal

Sure, much of this relates to the Journal but it really struck a chord with me since we were essentially assessing the proposals for suitability for inclusion in the journal. And many of the issues really relate to assessment per se.

* Congruence of mark / feedback - i.e. reliability of assessment
* Engagement with the rules
* Sharing and maintaining standards

Seems like wherever I look nowadays I see assessment and activity related to ESCAPE.

Steering Group - More thoughts

2009-03-17T02:13:00.001-07:00

Colleagues at our inaugural Steering Group meeting offered support, encouragement and (quite rightly) probing - I thought I would post some of the observations and thoughts here too.

A few 'things' that resonated with me from the discussions -

1 the NSEE vs. NSS
2 taking my (our) own agenda to the Schools
3 Making due consideration to all items of the plan
4 Ambition – making sure that we are not being overly ambitious.
5 Drawing on projects that have already engaged in similar-ish activity

1. I have flirted with the NSSE / or seen pointers to it (the NSSE) on my USA conference travels and it something that I would like to explore further. Clearly we will have to balance the institutional requirements of 'success' in the NSS and so this will need to stay on our radar too. I would like us to focus on learning and seeing how that aligns with the NSS.
i.e. not wanting to be NSS driven.

2. David Nicol asked what agenda we will be taking to the Schools - i.e. in addition to working with the agendas of the Schools. Great question! I have thought long-and-hard about this after the meeting - I am aware of David' self regulation agenda for instance. Most of my work has been about trying to develop a personalised experience for the students. That is not to trample over the benefits of collaboration, community centeredness or seeing that socialisation is important but I am keen to see students as individuals in the current mass HE system. In some sense this is in part response to one of the 14 grand challenges raised by the National Academy of Engineering. I think Personlised Learning / seing students as individuals will resonate with us here since we can quite rightly draw on David's work, promote the benefits of community centeredness but also remind ourselves that we are dealing with individuals. We are a large University and losing contact with our students (as individuals) will be of concern to many of us here.

3. Our project plan was details in some areas but less so in others - we need to balance our endeavors and show commitment to all aspects of the project and its stages. Lisa Gray raised similar observations when she asked for more specificity.

4. We write in our project plan about transformative and sustained change. I don't wish to retreat from these aims but the Steering encouraged a close look at the size of the 'ambition'.

5. I am aware of the FAST and SENLEF projects and also the Engaging Students with Assessment Feedback project. Actually now I come to write about those and other A&F related projects (SPRINTA etc) that Chris Rust and I were involved with at the University of Essex, its kind of bonkers that there are still so many issues with some of the basics of assessment and feedback practice.
I will make sure that we revisit those projects and use what the others have already found. I will also draw on the experiences and expertise of Malcolm Ryan and the Learner Experience projects. Learner Experience is also part of this project

Hmm. Lots to do - but it was just great to get the input from colleagues. I have every confidence that our project will benefit from the experience and expertise of our Steering Group - Great input - thank you!

Mark

Scottish Journey

2009-03-09T04:01:00.000-07:00

Mark and I travelled up to Glasgow last week to meet with Rachel Harris from Inspire Research. We were looking at how using appreciative enquiry techniques could help us in our work with the module teams in the schools. AI looks at growing what works in a organisation to foster change rather than to think of things in terms of something needing fixing.

The day was very productive and we were able to flesh out exactly how we we going to approach things on a module level. In addition we have started to put a detailed timeline together for work we are doing with the schools. We had a chance to knock around quite a few ideas and we were able to develop some ways of looking at things that should really help the sustainability of the project's influence after the project itself finishes. In particular we looked at how many modules you would need to influence in order to bring about an institutional change . Mark compared it to how many people were needed to start a mexican wave in a stadium. Was there a critical mass of modules that you needed? and if so what were the factors that influenced that number. We also looked at things from person centred approach and looked at how we could measure a persons influence within a school as an agent for change. We wanted to encapsulate how many modules a particular person could influence and see if we could express that as a kind of index or quotient. It was good to have Rachel's input and experience to keep us both grounded on what was possible within our time frame.

At the end of last week we also got the feedback on the project plan from JISC - so this week will be concerned with updating the plan in the light of these and the steering groups comments. In addition we will be and arranging meetings with the module teams within the schools to get them to examine their current practice within an AI framework.

2009-03-04T09:24:00.000-08:00

Yesterday Mark and I ran a workshop on 'assessment and feedback' open to any member of staff at the university. We had 24 participants who contributed to some lively debate and discussion regarding the challenges of setting effective assignments and providing useful feedback. As you will be aware from previous blog entries, we are very keen at modelling behaviour, so we set the participants some pre-workshop activity, and then an activity within the session. We asked participants to bring along a case study of an assessment task that they felt needed improving. We discussed principles of good assessment and feedback practice, considered student and staff opinions regarding assessment and feedback and then asked the participants to consider where their assessment case study was currently located on the above grid of effectiveness vs efficiency. With support from colleagues, they then discussed how they could move their assessment up/along the grid to improve the effectiveness and efficiency of the assessment. In some cases this meant improving marking turn around times to provide prompt feedback by considering for example, sharing out the marking, online submission and group work. Other cases considered improving the effectiveness of the assessment by considering activities such as peer and self assessment and involving students in a dialogue around assessment policy (i.e. enabling some choice within the assessment task, discussion of assessment criteria and the activities involved in the marking process ). For post workshop activity, we have asked the participants to identify changes that they will make as individuals to their assessments, changes that they will make within module teams, plus changes that they hope to influence at School level. We will follow up their activities and hopefully gain more examples and suggestions that will benefit the ESCAPE project. It was great to see staff engaged in this agenda and there were 24 enthusiastic people in the room. My hope is that these individuals plus the academic staff involved in ESCAPE will make small steps of change that will influence their colleagues and we will see an improvement in assessment and feedback practice across the whole institution - there's nothing wrong with being optimistic right?

Model Relationship

2009-03-04T07:20:00.000-08:00

This week I have been concerned with trying to model the relationships between the modules that we are looking at in the School of Life Sciences and the other modules within the school. We are looking at the module profiles, the assessment mix and the key individuals that are involved with the modules to see what links and patterns emerge. We are looking at ways of using our results to model and predict how the project's influences will spread thought the school. It is helping us to identify the key "secondary targets" that will be able to be influenced by the second phase of the project. We have been helped this week by Ruth Hyde - welcome aboard Ruth. Ruth has been doing some work on building the model and looking at the variants that are emerging.

In addition I have had a meeting with the Business School to see how we can get the data that we need to be able to carryout our baseline audit of achievement for our "before" part of the "before and after" comparison of results. Lots of enthusiasm for the project in the Business School which is manifesting itself in a plethora of offers of support and encouragement. They are very excited by the project and can see how it will address some of the assessment and acheivement issues that they have with certain programmes.

It’s off to Glasgow Thursday and Friday for the team - we are having some training in appreciative enquiry provided for us by inspire research. We want to see how we can use the techniques of appreciative enquiry to manage the change we want to encourage in the assessment practices in the schools we are working with.

Innovations week at the University of Bath

2009-02-26T11:26:00.001-08:00

I have been invited to a few universities to present at their one or two day annual T&l conferences - and I am always impressed at the enthusiasm for the event and also the effort required to plan and manage such an event. The University of Bath seem a little discontent with one / two days events. They are now in the middle of an Innovations Week – wow. A really ambitious plan to put on a week of activities related to teaching and learning – or more specifically Feedback to Students and Engagement with Students . One of the days was dedicated to showcasing / brokering relations with the HEA Subject Centres and the CETL’s. We were invited as the Blended Learning Unit (BLU) – my day job is the BLU Deputy Director.

I have a few contacts in Bath and so took the opportunity to take BLU to the event and also catch up with some of my Bath contacts.

I had mixed views about how many people might turn up and how many Subject Centres and CETL’s the University would get. The event far exceeded my expectations.

I addition to showing some BLU activity I took the opportunity to plan a workshop - Clearly I remained in eye contact with the BLU space / poster. Hence managed to support Bath and also get some work done.

Really great job Bath and the Learning and Teaching Enhancement Office for coordinating the week - I took the idea of a week long event back to Hertfordshire and it will be just great if we could mimic some of the energy created at Bath.

Full Programme

First Meeting of the Steering Group

2009-02-25T02:07:00.000-08:00

Last Thursday we had our first meeting of the steering group, we had an excellent turnout. It was good to see Malcolm Ryan again - our critical friend from JISC and fellow VW owner and to meet Prof. Margaret Price - from Oxford Brookes University. In addition Prof. David Nicol from the University of Strathclyde joined us via elluminate along with Prof. Peter Bullen head of BLU at UH.

We used a very large plasma screen to display the elluminate interface and Mark managed to solve some technical hitches prior to the start of the meeting so the sound was excellent.

The steering group were very supportive of what we had achieved so far and were in agreement with our approach as detailed in the draft project plan - which was approved. They were able to offer advice on areas where they felt their experience was relevant. It is good to know that we are backed up in our endeavours by such a depth and breadth of experience. The steering group’s guidance and advice will be invaluable to helping make the project a success.

Some things that I got from the meeting:

look at the student perspective of assessment & compare with the lecturer view
think about how teaching & learning staff development for new (and existing) staff can support the sustainability of the project
create hard data for analysis
what are the staff and student drivers here
students will always want more feedback - we need to encourage and facilitate self regulated learners
look at the national survey of student engagement
think about the context of the change we are trying to achieve - make sure that senior champions are on side
the challenge of using general principles of good assessment practice - of which there are a number of schools of thought - and turning them into actual an assessment regime for a particular subject module.

One of the main themes that I picked up on is how much work is being done in similar areas to our own and how we need to be aware of it. One of the discussions that Mark and I have had is his feeling that we need to have a good idea of what is out there - perhaps by carrying out a survey exercise.

From alpha to omega

2009-02-12T07:31:00.001-08:00

We had a very valuable meeting with Lisa Gray and her colleagues from JISC. It was a chance for the JISC team to find out what we had been doing over the last month or so and for us to get some feedback on our draft project plan. They also had a chance to meet our stakeholders in the Business School and Life Sciences. The first part of the meeting was with Mark and myself and concentrated on the work we had done to date and looking at some of the issues that the plan had raised.

The round table that followed teased out some really interesting issues within the schools about the need for this kind of project and the kinds of solutions that were being developed - both bu early adopters and having to us technology as the only way to cope with a situation - for example the impracticalities of 650 students all handing in hard copies of coursework at 3.00 on a Friday afternoon. There was a rich conversation on how schools at UH are using of the electronic voting systems and the hidden benefits that emerge once you start to use this kind of technology. Studynet was also discussed in detail and some of us spoke with passion about what we and our have been able to achieve with it.

There was a valuable discussion on the level of detail needed in the work packages - which we took on board. The work packages will need to be more comprehensive when detailing our baseline audit of what is happening in the schools at present - a fair comment. This detail will come in the next couple of weeks as we start to work much more closely with the module teams to dissect the modules and the staff and student experience.

We also discussed the resources and support that will be made available to us over the course of the project. The meeting was held in the Learning Resources Centre where the JISC team were able to see a cross section of the way our students work from informal small study groups that form due to almost chance meetings to more formal student led sessions in dedicated seminars rooms. As I looked around it was clear that technology in the form of laptops, wireless Internet and the web 2.0 technologies was very central to the students learning - the place was a hive of student activity.

All in all a very useful morning, it is good to know that we are heading in the right direction and that we have the support and resources to deliver the next part of the project. I am pleased the way things have gone so far and liken the project to picture of my daughter trapezing - its a delicate balancing act that we are attempting and it will not be easy to get the embedding that we are after - but it is possible and when you get it right it is exhilarating!

Sustaining ESCAPE - Learning from other projects

2009-02-11T09:52:00.000-08:00

At the heart of the ESCAPE project is the need to sustain and embed our practice. We are after all about developing sustainable change. Using the standard JISC template for our Project and Evaluation Plans we were required to present some thoughts on Sustainability and Exit. I really like the idea of asking the project to think about these issues at the outset and not at its end. Thumbs up from me!

The University of Bradford are running another JISC funded project - Audio Supported Enhanced Learning (ASEL) - to which we (UH) are partners. The project is nearing completion and today (11 Feb 2009) they ran a UH / Bradford meeting to review how the institutions would sustain the ASEL activity and continue to reap the benefits suggested of audio supported learning.

I am really keen to see legacy activity from these type of projects and not just see all the good work fizzle out and dwindle just because the project and funding has formally closed. I was invited as part of the UH team to attend the day to which I seized the opportunity. I want our project, ESCAPE, to learn from ASEL activities and see how we might sustain ourselves long after we have finished. In particular I am interested in what things we might do now to help our project sustain itself and what questions might we ask at the start of the project that would help us at the project close.

Whilst I was happy to support ASEL, want to see UH benefit from a real legacy of the ASEL project, I also wanted, (more selfishly), to see what lessons I could take away from the meeting to benefit ESCAPE.

We had an interesting start to the day where as a group we explored how the project had met its objectives. I think this will be a useful exercise for ESCAPE too i.e. to keep reminding ourselves what we set out to achieve and constantly relate our activity to that.

Bob Rotheram from the Leeds Metropolitan University followed that session with a great presentation about the JISC funded Sounds Good Project - Sounds Good is about giving better feedback and using audio to support the feedback activity.

After Bob outlined his project and the benefits, I was keen to find out how we going to sustain the Sounds Good activity (i.e. in line with the purpose of the meeting and for my selfish needs!)

Bob spoke about the importance of enthusiasm and then I asked, in addition to enthusiasm what three things do you think are important to sustain Sounds Good.?

Bob spoke about the time savings it could bring staff and qualified this with some great examples.
It is quicker (audio feedback) than written feedback if …
• You give a lot of feedback
• You speak faster than you can write

Be a really useful exercise to quantify this and show staff some break information
Bob also said that technology needs to be simple and useable and not be a barrier to uptake. Some great thoughts.

Naturally Bob also\mentioned the positive benefits to students in terms of receiving audio feedback. I really liked the pragmatism about being both student and staff focused. Staff are very busy and they need convincing of a need to change practice and see the benefits for them and their students

Peter Chatterton asked a follow-up question …

If you had a pot of gold what would you use it for to sustain Sounds Good ?
Fortunately Bob said not much was needed, sounded like a ripple effect was kickimg in, but he did say …
• Give staff decent devices
• Run workshops
• Give a chance for staff to come together to share and discuss activity

It struck me also that just as e-learning was a Trojan Horse to pedagogy so too was the use of Audio. The morning session gave a few examples which suggested, for me, it was not developing pedagogy but giving what we already know about learning to be surfaced.

In the afternoon we broke into our Uni teams and discussed how we might sustain ASEL. Asking ourselves …
why bother?
How do we convince an academic not connected with the project to engage?

We collected many ideas - which will `be posted here when they are summarised, but some things for now …

Provide some training packages
Utilising existing support - not develop new systems
Resources might include how do you do it
What do you need to avoid
i.e. a collection of resources related to ‘how to’
as well as a collection of resources relating to ‘why to’

We also mentioned strengthening the student mentor scheme run at UH - and if new / different skills are needed for audio activity then we need to describe what these are and up-skill the mentors accordingly

* Linking it to podcasting campaign
* Tagging resources so that we don’t reinvent stuff
* Networking
* Sustaining communities
* Inter-faculty meetings/
* Cross University inter-faculty meetings
* Link with the Podcasting for Pedagogic Purposes group?
* Not invent what is already out there

A couple of things ASEL might do now
• Develop crisp selling messages
• Use the right language for different groups
• Tap the why do it into responding to challenges
• Sell to Heads of School
• Share an Executive summary

I was interested in asking the team to map out the audio landscape post ASEL and then establish what is needed to support a vision

i.e. where we will be in
1 week
4 weeks
2 months
6 months
1 year

I would like to do this for ESCAPE - map out what we want UH to look like after ESCAPE and establish what we need to make that vision-landscape a reality.

Per ardua ad alta

2009-02-11T05:20:00.000-08:00

We are welcoming Lisa Gray of the JISC e-Learning Programme team, Marianne Sheppard from the Support and Synthesis team, and Sheila MacNeill, a representative from JISC CETIS to the university tomorrow . They are coming to meet the ESCAPE team and to have a round table meeting with us and our key stakeholders in the schools. It prom ices be a really useful day - we are keen to show JISC what we have achieved to date and to get feedback on our draft project plan. It will also be a chance to talk about the arrangements for the management of the project - both strategic with the steering group and advisory group and from an operational perspective. We will also be talking about the communication strategy and reporting mechanisms along with the Project evaluation plan and baseline data gathering exercises that we are nearly ready to start. I will be interested to see what support the meeting feels the project needs.
We are on the launch pad, we have done the preparation with our partner schools and we are ready (almost) to light the blue touch paper and launch the baseline assessment of the current assessment practice.

Winter's discontent made glorious

2009-02-11T02:46:00.000-08:00

I have now been working on the ESCAPE project full time for nearly two weeks. And what a two weeks! Hertfordshire has seen the worst snow in many years with the result that the univesrsity effectively closed for three days last week. In spite of this we have managed to have extremely fruitful meetings with senior colleagues and stakeholders in the School of Life Sciences and the Business School. In the School of Life Sciences We have had meetings with Peter Stanbury, Associate Head of Department and Areles Molleman one of the LTI teachers who has a unique cross school perspective - they are both extremely supportive of the project and we are really looking forward to working with them. There are some changes to the mix of modules we are going to be looking at with one module being swapped as it is not running again for a similar one that has exhibited that has some novel and interesting assessment practices. Excitingly such is the enthusiasm for the project within the school - we have been approached to see if we can also include fifth module in from a master degree programme run by a keen supporter of the project, who has been involved in some novel work on assessment. This is I hope an indication of the building momentum that the project is generating.The ground work has been laid for the next stage of the project which is engaging with the module leaders in the school to look at a baseline audit of the assessment practice in their modules.

We have also been busy with colleges from the Business School - we have had meetings with Professor Mick Broadbent Head of Department, Karen Robins Associate Head of Department and Jan Filosof the programme manager of the programme we are looking at. We are in the process of narrowing down the choice of modules that we will be looking at through speaking to the module leaders, looking at the historical statistical data, AMERs and programmereview documentation etc.. We want suitable modules which would lend any developments to portability and sustainability within the school. There are a number of possibilities at the moment.

We submitted our draft project plan to JISC on Monday, with a deadline of the end of the month for the final version. The work package timelines have changed from the original bid document in that we have moved the time that we engage with the Business School forward so that we are working with the schools in tandem. This will mean that synergies can be encouraged and developed between the module teams and managers in the two schools schools. In addition the workpackages have been deconstructed slightly to enable more transparency in the plan.

15 (shh 45 !) minutes with the School of Life Sciences

2009-02-05T09:35:00.000-08:00

We , (the ESCAPE team), managed to get an invite to talk more widely with one of our Partner Schools - The School of Life Sciences. We were invited to give a project background to the School’s Academic Quality and Enhancement Committee. Clearly we are not working with the entire School nor all their modules and so it was a good chance to alert others as to what we are doing and what we are trying to achieve.

My 15 minute slot ran into 45 minutes. Not convinced it was all my fault – I took many questions and the meeting ‘seemed’ happy for the discussion to spill over. Many questions were upbeat and some, perhaps rightly, felt a little more challenging. Why should they engage with us? What were we bringing to the table to a team that is already looking at its assessment diet? (and hopefully alignment with the Module Learning Outcomes) Why also the focus on the National Student Survey?

All good questions and naturally we need to provide a convincing argument about why we are doing what we are doing. Academics are very busy right now and want to know why they might need to revisit their curricula. If it aint broke …

I took a really positive email the following day from someone at the meeting. They wanted to offer their support for the project and the ways in the project was aligned with their interests some of what they had been trying to achieve. I was really grateful for the mail and for them letting me know that the project had rekindled an interest in this important area :-)

The meeting was a useful outlet to get ESCAPE out there but also a timely reminder that we need to marshal our evidence as we seek to engage and suggest a need for change.

The Best Laid Plans of Mice and Men

2009-02-02T11:56:00.000-08:00

I am really starting to get a feel for what the ESCAPE project can achieve now that I have had a chance to meet people in other schools and to get their input on what the project will mean for them. I meet last week with Peter Stanbury - a senior member of the School of Life Sciences in a similar way that Mark & I met with representatives of the Business School. In each case they could see a real need for the project and anticipated its benefits. In each case we were welcomed with offers of all the help and access that we would need to help make the project a success. In many cases they were able to show how it dovetailed with work that had already being going on in their schools to address issues that they were aware of. It will be interesting to see how we can help and influence each other.

At the moment I am involved in writing up part of the project plan. Working on the workpackages aspect of the plan made me think of how in practice we will work with the stakeholders in the schools. Whatever model of working we adopt I am confident - having met with the schools -that we will get all the support, encouragement and cooperation we need. This is one of the key foundations for successthat the project needs - we are building on these initial meetings with a further series of meetings - snow permitting- this week.

Appreciating what works ...

2009-02-02T07:35:00.000-08:00

As we develop and map out our evaluation plans at the forefront of our thinking is a desire to ...

* evaluate 'as we go' and learn lessons quickly
* ensure the evaluation activity does not upset or interfere with our collaborative endeavors with our partner schools
* seek out what already works
* get as much stakeholder input into the evaluation activity as possible.

Appreciative Inquiry (AI) seems to offer an evaluation framework that meets many of the characteristics we are looking for. We will frame our questions in the spirit of AI and go looking for what works and how we can use more of it.

Visioning a brighter future and not taking a deficit approach is something we are looking forward to engage with.

Will keep you informed of the progress and, because the approach to the ESCAPE team, will reflect back things that challenge and excite us

Online marking

2009-02-02T06:57:00.000-08:00

Having read Mark's post about meeting with the Business School, I wanted to comment on the benefits of online marking. I am a strong advocate of online submission. As well as reducing queueing times for students and as Mark rightly mentioned, burdens in marking work submitted in plastic wallets etc. the reduction in administrative time is really beneficial and it means as a lecturer, I can start marking assignments as soon as they come in. I do not have to wait for the paper copy to be logged, sorted and passed on to me. It also saves me time, as I don't have to physically pass the copy onto the second marker/moderator and back to the administrator. I think we save atleast 3 days by having work submitted online, plus it reduces the risk of paper copies getting lost and limits the need for storage space. I appreciate that some colleagues do not like reading work on the screen, but I know colleagues who print out the work to read it, but annotate the online version. I use my Tablet PC to mark which enables me to handwrite comments on the screen in exactly the same way i would do if i was annotating a paper script, however, those without a Tablet PC can easily use 'track changes' to annotate scripts. The other benefit to online marking, is that the students are more likley to read the feedback, and hopefully use the feedback. Student marks are released through the virtual learning environment (VLE)and the mark is shown at the bottom of the page below the feedback comments. If you haven't yet tried it, give online marking a go.

A challenge from the Vice Chancellor

2009-02-01T07:13:00.000-08:00

Our Vice Chancellor (Professor Tim Wilson) lives in a different world from me. He seems to have firmly gone metric and now has 25 hours in a day and 10 days a week! Part of his busy schedule includes him keeping a blog where regular entries communicate thoughts, successes and challenges.

He had clearly read the Times Higher Education article on assessment, which was pleasing giving its relation to this project and UH's inclusion, and made many positive comments about the piece. At the end of his blog entry he asks the question about the format of the traditional examination. Why shouldn’t we allow computers and technology into the examination room? A great question and, for sure, there might be an appropriate reason why we don’t but for some cases I suspect there might not.

I might stretch the challenge further and not only ask why not make greater use of technology in the examination room but why the examination room itself?

What I really want from the ESCAPE project is to stimulate similar questioning with our partner academic schools and by the use of the project, appropriate change management techniques and visioning a future, explore why we do what we do.

Because we do, as a response would be interesting to unpack and see what lies behind keeping the norm.

Great to see that our VC is up for a challenge and wants us to explore current practice.

Times Higher Education

2009-01-29T05:46:00.001-08:00

Some time ago I was passed an email from the Times Higher Education (THE) looking for examples of innovative practice in assessment. I responded positively on behalf of the University - Clearly I don't know all the good work that is going on at UH, but given my position in the Blended Learning Unit, I do get a scent of good things outside of engineering too!

Little happened for a while and then a couple of weeks back I engaged in a telephone interview with Rebecca Attwood. The THE journalist writing the piece. Really pleased to see that 'some' of what I said found its way to the article and I am pleased too to help put Hertfordshire on the map. We have many things to be proud of here - just as other HEI's do too. The article can be found here

I really enjoyed the article - great Job Rebecca, but it was a tad disappointing to note that the contributions came from the same 'names'. Surely there are more of us out there with a passion for assessment?

I would love to read a piece with great contributions from academics / staff developers whose names I have never heard of / or are not so intimately connected with assessment but nevertheless are as passionate about the students' experience and the impact that assessment has. Just a thought :-)

Meeting with Management Team of the Business School

2009-01-28T08:14:00.000-08:00

The ESCAPE project will work with the Business School and also the School of Life Sciences. Naturally we are presently on many paper trails looking at current practice. This will inform our project and evaluation plans. To support and enhance our ‘paper-trailing’ Dominic and I sat with Mike Broadbent Head of Department: Accounting, Finance and Economics (Chair of Faculty Learning and Teaching Group) and Karen Robins (Associate Head of Department (Management Systems)), from the Business School yesterday. We had a really productive meeting. They seem to have as much energy and enthusiasm for the ESCAPE project as we have. We discussed some of their general Teaching, Learning and Assessment challenges as well as the ESCAPE project.

Predominantly their challenges lie in the area of large groups and direct entry students. Large classes present many different challenges which we need to unpack and provide appropriate responses to. I am aware of the excellent work done by David Kraithman (and others) where they spent some time and effort providing really useful and accessible electronic resources to the students. The motivation being to move the lectures from being didactic information giving affairs to being more seminar based interactions. It will be of little surprise that class performance improved. Great job David! Not only does this show how technology can help the teaching and learning nexus but also how the out of class engagement can, and indeed should, inform the in class engagement.

I was also musing also at some of the practicalities of assessing large classes too. My largest class probably peaked at around 200. For some modules in the Business School student numbers are higher. I heard that a self service coursework hand in-box was withdrawn from use due to the potential Health and safety issues as queues of students formed to submit their work just before the submission deadline. I then got thinking about a class size of say 360 students and considered the case where each student submits their work in a plastic wallet - a reasonably common phenomenon Assuming it takes around 5 seconds to pick up and take out the work from the wallet and then another 5 seconds to put it back after marking (i.e.10 seconds per student), this trivial activity alone adds another hour to the assessment burden. Okay, this might not seem like much but as class sizes grow so too does the time and distraction for engaging in trivial non learning oriented activity.

Clearly there is much potential in electronic submission, which UH has, but making those systems useable and also configurable to suit the various usage patterns is no trivial activity.

We spoke of the Business School’s desire to meet the University’s target to reduce its time to return coursework and conversations moved to marinating the standard of assessment. Again, I have views here, which essential centre on the notion of ‘what’s the use of an integrated, challenging, well thought through piece of coursework if the attendant marking demand suggests its takes 6 weeks to return the feedback?’. Cleary I am not suggesting moving away from establishing authentic and challenging assessment tasks but there might be some work here in looking at the assessment experience in its entirety.

A really good meeting and one which established a positive tone for engagement but quite rightly set down some challenges from the chalk (pc) face for us to consider.

A presentation to the universities Academic Quality and Enhancement Committee (AQEC)

2009-01-28T07:05:00.000-08:00

I was invited to present some examples of effective assessment and feedback practice to the Universities Academic Quality and Enhancement Committee (Tuesday 26th Jan 2009). Although the presentation was not specifically related to our ESCAPE activity it was a committee that the ESCAPE project needs to touch. If we are to be successful in making our activity mainstream and embedded then such committees need to be shown the value of our endeavours and we need them to help us respond to our challenges. Working with enthusiasts and champions is not enough. By definition, ESCAPE has an explicit intent to be sustained long after the project funding has finished.

I started by outlining the importance of assessment and feedback - both in terms of the challenges raised by the results of the National Student Survey (NSS) and also (and ultimately) learning! Any meaningful and aligned curricula has assessment at its core.

I then moved to show some work that Helen Barefoot and I have been doing in terms of collecting case studies from the LTI and more importantly overlaying (on the case studies) principles of good assessment and feedback practice. The case studies are tagged to help staff find resources relating to their challenge.

I think (hope) the committee liked the presentation and also the pragmatism of recognising that academic colleagues are busy and so not only do we need to collect resources but help staff access them in ways that meets their needs.

The perpetual problem was raised about engaging students with feedback and the frustrations felt by staff in producing feedback that is not even collected let alone used by the students. I have my own views on why this is the case - but in many cases I think the problem resides with us. Just as assessment creates activity so too should feedback. Feedback also should create consequences.

Evaluation Meeting- Birmingham 15th January 2009

2009-01-16T08:51:00.000-08:00

This meeting was my first chance to really immerse myself in the project and I was looking forward to meeting our critical friend Malcolm Ryan and the others from the JISC support network. I was very interested in having a chance to meet with people from our sister projects - particularly those in our cluster. I am hoping that as our projects progress our links will grow stronger and we will develop ways of supporting each other.

The venue in Birmingham was an excellent one as was flexible and spacious enough to accommodate and encourage all the meetings - scheduled and unscheduled, formal and informal that took place. It really brought home to me how a space can really influence the productivity and the modality of the events that occur within it.

Reflecting on the evaluation of chocolate chip cookies ( see Mark's previous entry) , which was one of the exercises the we performed to stimulate a discussion on evaluation, made me think about how evaluation should be right there at the design process. Chocolate chip cookies do not appear naturally in nature (at least not in Hertfordshire) therefore there will have been (- I would hope) an evaluation policy before the biscuits were put into general production. This would not have been the case with say a banana - which does occur naturally in nature. We can evaluate it - but it is an artificial evaluation one that we will impose on it from our perspective. We want to eat it so we want it to taste nice, be nutritious and not poison us. This occurs by chance ......not design.

This simple exercise has really made me think about evaluation an how it is so important for it to be there right at the planning and design stage.

Evaluation Mtg - part 2

2009-01-15T10:42:00.000-08:00

Evaluating a chocolate-chip cookie

During the evaluation meeting (14 Jan 2008) we were asked to evaluate a chocolate chip cookie. We did this in our cluster.

We were presented with four bowls filled with a whole cookie and bite size pieces of the same cookie. We were asked to devise evaluation criteria that we could use to choose the best cookie. Sure the exercise might not be overly related to our project but (putting that aside and chomping away) the exercise drew up lots of interesting observations.

We spent a long time drawing up a very long and extensive list.

Naturally taste was on the list but so too was colour, distribution of choc-chips, density of choc-chips, look of cookie, crunchiness, chewiness, buttery-ness, smell, size, cost, packaging, etc. The list goes on...

Having identified the list we then tasted them. It was interesting that after producing the list someone immediately said something about a cookie that was not on the list!

This asks the question can all useful evaluation criterion be identified before the evaluation takes place ?- I wonder if this resonates with the notion of the unexpected consequences?

I struggled to offer any sensible thoughts and did not have a sophisticated enough palette to sense/taste and hence talk about buttery-ness. It was also suggested that just because we might be able to identify criteria they might not be relevant. This was certainly the case with me on the buttery-ness criterion.

Another group mentioned they just went for ‘taste’ And this reminded me of notions of connoisseurship. Can we really set out what we looking to measure or is there a sense of connoisseurship that has a part to part to play here too?.

The obvious things arose from the wider group about

* just because you can measure it, it doesn’t mean it's important and
* nothing ever improved/grew/enhanced by being measured alone.
Hmm, I’m mindful here of the Hawthorne Effect.

Nevertheless, these were some useful reminders as we now engage in developing our evaluation plan. There were certainly some useful lessons to learn from this exercise.

MBR

Evaluation Mtg - part 1

2009-01-15T10:21:00.000-08:00

Evaluation meeting 15 Jan - part 1

Dominic Bygate (our new ESCAPE project Manager) and I attended the JISC-organised-Inspire Research-Facilitated Evaluation Event in Maple House, Birmingham. The event presented many opportunities - not only a great chance to catch up with our cluster (Universities of … Westminster, Exeter and Bristol) but also our Critical Friend - Malcolm Ryan. Malcolm had suggested we catch up the night before the event so that we can meet on a more informal and social basis. It was great to meet Inspire Research (Aka Rachel and Alison) who also joined us, and also good too to discuss our work outside the framework of the agenda items on Thursday - great job Malcolm - thank you!

What follows is a collection of thoughts that came out of the Evaluation Event …

We started the day by trying to sketch out what we saw as evaluation. Many metaphors were produced from our table

They included …
* A running track that had a start and finish line but was littered with hurdles to encounter and pathways that accelerate the journey.
* Shark infested waters, firing squads and comfy chairs
* Scales of judgement and also cycles and cycles
* Mirrors that gave the evaluator a chance to reflect on the whole and not just the individual pieces of the project - I liken this to suggesting it’s a great tie and a fab shirt - but together they look just lousy!
* Boulders of various sizes suggesting that evaluation has many levels of granularity

Other things I scribbled from this session was an Ice Cream Sundae that showed much pleasure hidden below the surface, but potentially also some hidden fruits that might be sour to the palette. Notions of known’s and unknowns also were presented. I wondered too about unknown-unknowns as we engaged with our project.

What the metaphors were describing were journeys, multiple pathways, some good times ahead and also some difficult things to confront. The importance of stakeholder engagement was raised and a great point was offered in terms of helping students develop their skills so that they might better articulate and describe their needs and experiences. I liked that.

After this visioning exercise we were presented with some useful definitions of evaluation. Notions of paradigms and where you are located suggested that some definitions better fit the different projects. It is conceivable too that a hybrid definition might better reflect what at is needed for some projects.

Using some of the information from the ‘exploring the tangible benefits of e-learning' document we were asked to reflect as a cluster why we might engage in evaluation. We were presented with a list that included
* Cost savings / resources
* Improved retention & recruitment
* Enhancing skills and employability
* Increased student achievement
* Improved inclusion

My immediate thoughts were that whilst the items on the list are extremely laudable our project wants to get to the items by working with staff - and so notions of staff development and changing thinking were vital. Closer inspection of the exploring the tangible benefits of e-learning document suggest that the initial list given to us was only a subset and supporting staff was indeed noted.

Our cluster also spoke about
* the tension in evaluation between ‘numbers and stories’ - the quantitative and qualitative debate. So ypou can go watch a 5 star rated film - But what does 5 stars mean?
* the potential conflict between different stakeholders as to what evidence might be needed
* notions of developing transformative change and also culture change
(Such notions are at the heart of ESCAPE project)
* the need for alignment of project with institutional direction for sustainable change or possible the alignment / engagement with external communities to support sustainability - discipline / areas of interest etc

Other groups fed back

* All the initial items on the list related to output (as we had indicted)
* Various stakeholders and key outputs
* Sustainability, culture, embedding, lessons learned for the sector
* Proving improvement
* Need to develop systems and measures
* Benefits to the individual learner
* And also supporting a community
(resonates with sections of the How People Learn framework)
* Approaches and evaluation methods were presented and also the importance of base lining new and existing data.
* Getting a decent response which was not just the number of responses but levels of sophistications that will help the project develop - This idea was nicely aligned with the previous idea of helping students articulate their experiences and n9ot just expecting them to be able to.
* Quality of conversations and how to measure and evidence the conversation.
* Does the process of capturing get in the way of / does it create its own evidence?
* How to measure behavioural change
* The idea of the reverse novelty effect was described which raised issues on
* When to evaluate and how to evaluate

It was really interesting to see how the groups tackled the same task
One group focused ‘more’ on methods, one more on ‘stakeholders’ and two (us included) probably focused more on wider issues.

MBR

Project manager

2009-01-10T00:39:00.000-08:00

Our new project manager, Dominic, started work on the ESCAPE project this week. I am sure he will be a great asset to the project and will drive the work forward. Dominic has experience of working with staff on change management projects and I am sure his expertise in this area will be invaluable when he works with colleagues in the Business School and Life Sciences.

I am looking forward to being a part of the ESCAPE steering group and am feeling better about taking a back seat. I currently have a number of other projects which I am working on and keeping all the plates spinning in the air is certainly proving challenging.

I am looking forward to following the progress of ESCAPE via the blog and hope that Mark and Dominic keep us up to date with their thoughts.

Letting go - part two

2008-12-05T02:27:00.000-08:00

It is with much disappointment that Helen cannot stay too intimately connected with the project. Not only did she have significant part to play in developing and shaping the bid, (and hence is really attached to the project), but she is just great to work with. We had fun preparing for the two day event and she challenges me regularly to think about what I do.

But Helen has a role across UH and has a major piece of work to do in taking UH’s Learning and Teaching Institute forward. And I know she and the team will do a great job.

And so earlier this week we interviewed for our project co-ordinator. The project coordinator will support me but involve themselves in more of the day-to-day activities. Ideally we are looking for someone with a good understanding of project management, assessment for learning and change management. Hmm, quite a tall order.

Interviewing candidates is a tough process -some candidates you know well others you don’t. It is stressful for all involved. BUT without exception each candidate brought useful qualities to the interview

And so now we have the arduous task of identifying the individual that is likely to be able to best respond to all the demands of the project.

Whilst this is a tough decision it is an exciting that we will bring someone new to the project.

Will keep you informed of our progressed

MBR

Letting go...

2008-11-21T04:43:00.000-08:00

We are currently reviewing the applications for the post of project manager who will work with Mark, the project director. Once the manager has been appointed I will no longer be directly involved with the project. I will be a member of the steering committee but will be taking a back seat regarding the day-to-day activities. I have to confess that I am feeling a little sad about handing over the project. Mark and I, plus support from other colleagues, wrote the bid in the summer and I have been involved with the project from the beginning. I am passionate about ESCAPE and having met with Academic staff from the relevant Schools, am keen to share their endeavours. It feels very difficult to give something up that I have been so involved with and it’s also sad to walk away after having such a good time at the start up meeting last week. However, other work commitments will be keeping me busy and I shall be watching with interest about how the project develops.

200 yards with the Business School

2008-11-20T14:22:00.000-08:00

The ESCAPE project will work with two Schools - the School of Life Sciences and the Business School. On Monday 17th November I ran a workshop on assessment and feedback for the Business School. Although not specifically tied to the project it was a great opportunity to start/feed into the dialogue on assessment / feedback in the school and in particular help the School to respond to the challenges raised by the National Student Survey (NSS).

I had a full-ish room - (around 40 - 50 people - I think!) I mixed the session up with some presentational material - which mainly focused around principles of good assessment and feedback practice, with the chance for the School to reflect on their own assessment practice and identify where and how their current assessment practice aligns with the principles. We had a really engaging discussion, which naturally included observations suggesting students don’t really understand what feedback is, through to trying to identify opportunities for each person to think about how they can make a small commitment to change.

Although good assessment is an embedded and aligned activity (and not as John Heywood writes ‘too often afterthought of higher education’) assessment practice can be enhanced by constant reflection and a commitment to make on-going small changes.

I am minded of the wonderful notion that I cannot map out how to get to china. But I know how to walk 200 yards - and in 200 yards time I will ask someone else the direction towards china. Join my 200 yards together and soon enough I will be in China.

I challenged the Business School to work with me and help them be the most improved School in the University relating to Assessment and Feedback - NSS wise. Let’s see if they rise to my challenge.

Thanks Business School - great session

Objects on the table

2008-11-19T14:31:00.000-08:00

It was really good to hear about the other JISC funded projects at the start-up meeting. In addition to the two-minute pitch - see previous blog entry - each project team was invited to bring to the meeting an artefact that described the project. These were anonymised and put on a table for us all to tie the artefact with the project. There was naturally a vast array of objects on the table - The justification for the chosen objects was as varied as the objects - we had great fun trying to establish which object related to which project.

Reflecting from the trenches

2008-11-18T07:53:00.001-08:00

The brief set by JISC for 2 minute pitches was a great idea and allowed us a snapshot of each of the projects. It was also really interesting to see how other project teams approached the pitch and the creative nature of teams when required to present without PowerPoint.

Interestingly, although used to presenting, we both felt very nervous as we watched the other pitches and waited for our turn. I’m not sure if this was because we had scripted our pitch, which neither of us is used to, or if we just felt out of our comfort zones as a certain degree of acting was required. Either way it was a really useful reminder to us of how some of our students feel when they are challenged to do a presentation.

We both felt that our pitch went well and we had some very nice feedback from other participants - thanks to those who took the time to speak to us.
Following the pitches we were put into cluster groups based on our project interests. It was good to discuss our projects in more detail and share some ideas and offers of support.

Our Pitch

2008-11-18T07:45:00.000-08:00

Our two minute pitch was practised and timed at just over three minutes! Sure this is 50% over the allotted time, but surely the JISC would let us run on a bit - they will wont they? We decided to see how others got on, see how ruthless with the clock the JISC would be, and if needed, do some last minute editing. Arriving at the event we had a quick scan of the agenda. This said we would be CUT OFF after two minutes. Hmm. Now what? Oh, and this was to be the first session too - so no time to decide exactly what we would cut.

We sat and listened to the first pitch - The College of West Anglia gave a succinct and well planned presentation of their project with a bit of role play which set an immediate high standard. They timed it perfectly and after the two minute timer had run, down the National Anthem automatically played. Given our Great Escape analogy, we wondered - we just wondered…

Our turn arrived - given the really tight timing we played the Great Escape theme tune (through an mda) as we walked up to get ready - this we hoped would set the scene. With the theme tune playing we launched into our pitch and as the time ran out, the National Anthem started just as I was just making the point about us fighting the enemies of the bad assessment, in the classrooms, in the VLE’s and in the staffrooms. The National Anthem (for us) was a great addition - shame we did not think of it - thank you JISC for not cutting us off! We enjoyed that session.

Modelling appropriate behaviours

2008-11-17T11:44:00.000-08:00

Having established our pitch idea - we went through a series of ways in which to reinforce the analogy - we really wanted to use pictures of the characters from the film backed with pictures of our real life ESCAPE team. We also wanted to rouse the listeners with a background play of ‘the great escape’ theme tune. We were on a roll.

It then dawned on us - copyright :-( neither of us are copyright experts - but we are sensitive to the issue and wanted to be sure we did not infringe copyright law. We sought no personal gain or profit from the use but were simply unsure of the rules of use. In the Learning and Teaching Institute/Blended Learning Unit at the University Hertfordshire - one of our values is modelling appropriate behaviours - clearly we could not knowingly ‘sneak-in’ copyright photo’s or music and at the same time believe we were modelling. We took advice from within UH and it seems as though we could not use the pictures.

Naturally in no way did we wish to gain profit from the use of others work but we felt really deflated at how the law impacted on what we wanted to do. Although our limited creativity was stifled, the sock-idea remained in the laptop case.

Oh, the music - that’s another story - Helen’s magical brass ensemble (‘shakin’ that brass) played the theme tune to purchased music. Go Brass Shakers! A dodgy recording later and we’re back on track. I think they will be able to tell that it is not the original sound track!

Starting out: project kick-off meeting.

2008-11-17T11:36:00.000-08:00

We were invited to the two day JISC workshop (getting going and getting to know each other). The invitation e-mail set the tone of the day’s activities which were described as being interactive, lively and relaxed. Jeans and tee shirts were the order of the day. Naturally not only would we hear a little more about the project strand from the JISC but we would also have an opportunity to share to the group a little more about our project.

To help us to share the JISC challenged the teams to make a pitch to JISC tv-
The rules of engagement were …
In your 2 minute pitch tell us:

The Plot (the challenges you are facing and how you intend to address them)
The Characters (an introduction to the project team and key stakeholders)
The Target Audience (who will benefit from the success of your project?)
Why we should watch this project (why is it of interest to the wider sector?)

Hmm, neither of us (Helen Barefoot & Mark Russell) felt particularly creative - but we knew we did not want to just ‘stand-and-deliver’

After a sock was found in Mark’s laptop case we ventured the idea of sock puppetry. It didn’t take too long to realise this was a no-goer.

Our project title ESCAPE represents our core ideals - to create and maintain Effective Sustainable Change in Assessment Practices and Experience. Hmmm ESCAPE, plot, characters - Steve McQueen came to mind - Or Helen’s at least!

And so our pitch centred on drawing an analogy between ‘The Great Escape’ and our work’.
We identified the central characters - the Squadron Leader, the Tunnel King, the Cooler King, the Forger, the Scrounger, Mud-Movers and the Kommandant. And related them to our project team. Although there is some fun in the pitch - we believe the analogy works and in no way are we trivialising the issue.

Let’s see how others receive the pitch!

What's this blog about?

2008-11-17T11:00:00.000-08:00

This blog describes the University of Hertfordshire's endeavours on the ESCAPE project. The ESCAPE project (Effective Sustainable Change in Assessment Experiences and Practices) is funded by the JISC under their Curriculum Delivery Call.

Our full project proposal can be found here

We are really excited about the opportunities the project will bring to enhance the student assessment and hence learning experience.

The project start date was the November 2008
The project duration is 24 months
The project is being directed and managed by the Blended Learning Unit /Learning and Teaching Institute

We will post regular updates and welcome comments and thoughts.

MBR