It's Quality Time!

One of the things that has struck me this week is the how often subtle changes to a modules delivery can have quite wide ranging changes to the learning interactions that take place. The example I am thinking of is the Corporate Social Responsibility and Sustainable Marketing Module. The module coordinator - using his experience from previous years, has posted a set of FAQs on his module Studynet site. The questions address points and issues that were raised by students last year and were generated from the experiences of the whole module team.

As a result, module team's interactions with students - both face to face and online, been able to focus more on developing the students understanding of the material covered in the module . Students can be pointed in the direction of where to find specific information. This leaves the team time to develop the "high quality " dialogue and interaction with students that can really support and personalise the students learning. This simple strategy(one of a number adopted by the team) of using a well primed bank of FAQS has acted as a filter so that the more complex and interesting questions that students generate can form the basis of meaningful discussion with the module team.

Here comes the data (1) ...

We are now starting to see the fruits of our labour on the ESCAPE project. Some of the 'fruits' arise when we talk to staff about their assessment practice and resulting experience whereas other 'fruit' arises from observations of student performance and their engagement with their studies. I thought I would share some of what we are finding ...

A new module (not one of the original ESCAPE modules) wanted help with Peer Assessment. The member of staff was already engaging her students with peer assessment and hence were reaping numerous learning gains.

* Students were able to see how their peers responded to the same task
* Students were able to engage more with the marking criteria and standards
*

Previously, however, she was kept very busy after the peer assessment by dealing with more than expected students questioning their marks. Whilst it is highly appropriate that the students are exposed to a fair and reliable assessment, many of the efficiency gains made by the staff member were lost due to the need to deal with students on a one-to-one basis.

With the help of the ESCAPE project we were able to re-purpose a web-based data collection facility (developed to support computer based assessment). Using the web based data collection facility we posted to the students a series of questions (asking them to reflect on their own submission and the peer assessment process. This was an addition to the work previously done and hence created addiction learning gains. Importantly, we also included an opportunity for the students to 'comment on their mark' and note that if they were over or under marked to provide evidence where this was the case -with reference to the marking criteria used in the peer assessment process. The result was of which a vast reduction in 'additional' time required by the staff member to look at the concerns.

This addition was a real win-win. Students were now reflecting on the process (and sharing the reflections with staff) and the staff were reaping efficiency gains.

Slightly self-promoting but this typifies exactly the type of things this project is about. Using technology (led by pedagogy) to reap learning and efficiency gains.

I can't see this technology-supported intervention stopping when the project finishes.

Feedback

We have sought numerous opportunities to collect feedback throughout our project and not just at its end. In fact, we are awash with data and feedback. It will be a real challenge to use the data and feed it into our final evaluation. We have so many threads of activity and so many things we would want to share.

But I do want to post here one of the observations made by one of our ESCAPE partners. In many ways the quote captures everything we're trying to do. Support our partners, help them with their practice and allow them to take the work we are doing with them into other modules. i..e Spreading the learning and benefit.

The quote came from a conversation between our ESCAPE partners and was not solicited in any way. You can imagine my scrabble for a pen and paper!

The ESCAPE project has made me think about the way I give feedback and it has changed my practice in all modules.

She went on to say that

I now organise myself such that I am able to give feedback straight away. I now plan my diary around large coursework to free up 2-3 and re-arrange teaching sessions so that I can get a chance to give feedback.

Arguably, the above quotes demonstrate the transformative nature of the project and shows that the learning and benefit is not constrained to the modules we are working with.


Great stuff!

Learning from the slopes!

I'm not a skier. I have never been skiing, until recently that is.

But as I was learning to ski (or rather learning how to pick myself up from the white stuff) lots of my time was spent thinking about my learning (on the slopes) and the things that supported my ability to learn and get going.

I'm really interested in notions of personalised learning.
Personalised learning requires personal activity and learning activities that are adaptive to the demonstrable needs of the learner. This is true in the classroom and on the slopes. Personalised learning does not throw the learner in too far at the deep end but develops the learners confidence and hence their inquisitiveness and willingness to progress. Personalised learning does not swamp the learner with feedback nor provide fedback that is inaccessible to the learner either.

All of these things were so beautifully reinforced to me as I tried to ski. Putting ourselfs in situations that make us the novice (skiing in my case) was great to see some of the things about learning relate to me.

Being a novice and thinking about learning was just great.

Off to Scotland

Helen Barefoot and I were invited by the HEA to talk run a workshop on assessment and feedback at an all day Assessment event at the Robert Gordon University. What follows is a collection of (very quick) thoughts on the day …

Helen and I have run a few workshops together (inside UH) and so it was just great to take our work (much of which is guided by our ESCAPE activity) to colleagues outside UH. For various reasons the workshop did not run :-(. Despite our disappointment we did get to hear some great assessment related presentations.

Dai Hounsell presented a really grounded key-note and, in fact covered, much of what we were covering too. That assessment is not a new challenge, that good assessment is planned activity and that good assessment stimulates learning. We did not need the NSS to get us thinking that assessment is important. Some really useful slides from Dai that I will explore and come back to in a later post. Great start to the day.

A couple of student perspective presentations followed.
A student led campaign that successful introduced a turn-around-time policy for coursework and interestingly a policy to provide feedback on examination scripts. The learning gains to be had from providing feedback on examination scripts seems rather limited to me. I’m always banging-on about feedback creating consequences, and I’m just not convinced I know what consequences flow, or are able to flow, from feedback on end-of-process, high stakes assessment tasks. That’s a post for another day. But just to say, I’m off the fence on this one. I just don’t get it. Another delegate did note they had a similar policy at his institution and only 15% of the scripts (with feedback) were picked up. Surely we would be better placed putting our feedback on work that will be picked up and more importantly attended to, by the students. And relax!

Steve Draper, engaging as ever, had a couple of threads running through his presentation. First, was the interesting anomaly that overall a department was rated 5th against other departments (107 in total) for the NSS question overall, I am satisfied with the quality of the course and yet questions relating to feedback were ranked much lower. Feedback on my work has been prompt (ranked 54/107), Feedback on my work has helped me clarify things I did not understand (ranked 79/107) have received detailed comments on my work (ranked 101/107). Steve asked us to explore what might be going on. Was there a complex weighting algorithm for all the items on the NSS? Should the individual items of the NSS not sum to the overall score? If not, what was missing, what was the missing ingredient? Second, Steve separated declarative and procedural learning and pondered as to where our efforts on providing feedback might prove most effective. i.e. might we get more learning from less or (better targeted) feedback?

It's a Two Day Event

Things are starting to pick up speed with the project. We are planning a two day off site event at the university conference venue - the Fielder Centre. The likely date is towards the end of July - once all the exam boards are out of the way. The aim of the event is to give the nine module teams that we are working with the time and space to critically examine their assessment and feedback practices. And also then to start the reengineering process. We are planing ( at this early stage)the two days to be a mixture of presentations , facilitated workshops and module team meetings. The module teams will be able to call on the resources of BLU and the LTI to examine the particular pedagogical and technological aspects of possible assessment scenarios. In particular we are calling on our experts in curriculum design and change management.

By the end of this week all the module coordinators will have had appreciative enquiry reflective interviews in order to build up case studies of how the module is delivered and assessed . These case studies along with data from the student assessment surveys that we are carrying out, will give us a really clear picture of the issues surrounding each particular module. They will become the jumpimg off point for the reengineering process .

A few thoughts from Prof. Brenda Smith. HEA Assessment and Feedback Event

Brenda, as always, presents a really coherent argument and draws on a wealth of experience and expertise. Brenda’s position allows her to talk with numerous institutions which subsequently feed back into her knowledge. She uses these examples really well to allow her to provide some concrete examples of practice in the sector.

A few things I wrote down from her presentation …

* Engage students in the planning and curriculum planning – students can be great change agents.
* Assessments ‘can’ be set at week 3
* Get the Heads of Academic Schools at Teaching and Learning sessions – get T&L valued.
* Policies of professorship for T&L not valued or limited take up?
* A student friendly academic School gives a higher student feedback return
* The University of Lincoln, provides a booklet relating to students’ feedback. The booklet includes ‘this is what you asked for, this is what you said, and this is what we did. Helen and I are trying to get a You Said – We Did campaign up and running at UH. This resonated with us.
* Staff taking students to lunch and other simple schemes that attempt to engage students with staff. This again reminds me of Chickering and Gamson’s first principle ‘ Good practice im undergraduate education encourages staff student contact.
* Too much feedback might not be helpful
* Don’t just respond to students’ needs – Brenda gave the example of groups work and students not wanting it– Don’t forget to tell them why you are using it and what the purposes are. It’s not a secret!
* Stratchlyde – podcast lectures and use lecture time more productively – this frees up time for better feedback – This reminds me of what we do with the Blended Learning - use the technology o enhance and extend the traditional teaching sessions.
* Use of voice – audio supported feedback
* What do students do with feedback?
* Useful ways of presenting feedback – quickly!

Lots and lots more really useful ideas - I'm sure I've only touched the surface and dine an injustice to Brenda's session.