Final Version of Project Plan to JISC

We have just submitted our " final version" as opposed to "draft version" of our project plan. the plan was revised in the light of comments received from our colleagues at JISC and from the steering group. The Plan is now quite a hefty document - including the work packages it runs to 48 pages. We have revised some of our time lines. In particular we have allowed much longer for the design and development of the assessments for learning phase. The modules we are working with are a mix of one and two semesters long. The single semester modules will run between October 2009 and February 2010 - which means in March we will have an idea of how well our redesign has worked.

This week we are starting to apply our appreciative inquiry methods to the school of Life Sciences. We are conducting interviews with members of the module teams. This will allow us to get to know each other and start the inquire phase of the Appreciative Inquiry method. We will be asking them about good learning experiences they have encountered. We will be exploring what was it that made it a good experience. with questions such as:

• what happened?
• who was involved?
• what was your part in the experience?
• what was it that made it possible?

Using these reflections we will to start to evaluate the assessment practices of the modules. We will be looking to establish a way of evaluating the assessment against efficiency and effectiveness.

ESCAPE coverage in the Universe

Hi there, my name is Elizabeth Terry and I am a Business Studies student currently on placement with the Blended Learning Unit/ Learning and Teaching Institute (BLU/LTI). Working in such a committed learning environment, I have come to realise just how much the BLU/LTI value the students' opinion. Before working here, I had no idea how much thought went into enhancing the student experience, particularly within the classroom. Because of this, I have decided to do all that I can to make the students fully aware of what activities we do and the benefits that they bring to not only the university, but also their studies.

Liaising closely with Dawn Hamlet (Vice President Academic Support and Campaigns), I have successfully been able to get numerous stories on various projects that the BLU/LTI are running into the Universe (the student’s paper that has on average 4,000 readers). This week, the Universe features a story on the ESCAPE project. The article highlights concerns that are attached to assessment and invites the students to give their opinion and views on assessment and feedback.

Meeting Life Sciences Module Teams - Scene Setting

So, after much behind the scenes activity we recently met with the module teams from the School of Life Sciences. We are engaging in Appreciative Inquiry (as an Evaluation Framework) and before we enter the Inquire Phase (with them) we wanted to set the scene and help get them on board with our work. They, like other Schools, are very busy and I have no desire to draw on their time unnecessarily. We set aside 2.5 hours for a meeting where we could outline the project and get them interacting. The time flew by - for me anyway. Dominic did a great job introducing the project - I simply made up numbers and responded to some of their concerns.

Naturally (given it was a 2.5 hour session), we built in stuff for them to do and feedback - I thought (and I did say that I hoped I had not misread the mood of the room) that they were very positive.

Helen Barefoot came along too since this is her School and she spans across the Learning and Teaching Institute and also the School. Helen was a great advocate and could help relate our ideas to their context.

We asked for their immediate thoughts - which are now being collated and will be fed back here.

At the end of the session we also gave them Flip Cameras and asked them to introduce themselves and talk very briefly about a positive assessment experience. The room was lively and vibrant and although it was late and at the end of the 2.5 hours we got some great clips. This will now start to set a very positive picture when we start the Inquire Phase.

Thanks LIfe Sciences - Great start!

All roads lead to ESCAPE

Peter Bullen and I have been invited to act as guest editors for a special edition of Engineering Education. Engineering Education is the Journal of the Higher Education Engineering Subject Centre. We were approached (I suspect) since we are engineers, are passionate about engineering Education and because we are Director and Deputy Director of the Blended Learning Unit (respectively).

The focus of the special edition is technology supported engineering
education. See the fit?

We were overwhelmed with the response, it far exceeded our expectations and I suspect it exceeded the expectations of the Subject Centre :-) Naturally we then had the pleasure of reading the proposals. This 'pleasure' IS genuine since I'm always interested to hear and read about colleagues activity. Although we are guest-editing the regular editorial team are still in place to support us. I had a lengthy telephone conversation today regarding the proposals and how we move forward from here.

Some things that came from the conversation included ...

* What to do with a great proposal that exceeded the word limit?
* How it was comforting when the 5 independent reviewers agreed on the outcome of many of the proposals
* How it was frustrating when we didn't.
* How we can be encouraging yet maintain the standards of the Journal

Sure, much of this relates to the Journal but it really struck a chord with me since we were essentially assessing the proposals for suitability for inclusion in the journal. And many of the issues really relate to assessment per se.

* Congruence of mark / feedback - i.e. reliability of assessment
* Engagement with the rules
* Sharing and maintaining standards


Seems like wherever I look nowadays I see assessment and activity related to ESCAPE.

Steering Group - More thoughts

Colleagues at our inaugural Steering Group meeting offered support, encouragement and (quite rightly) probing - I thought I would post some of the observations and thoughts here too.

A few 'things' that resonated with me from the discussions -

1 the NSEE vs. NSS
2 taking my (our) own agenda to the Schools
3 Making due consideration to all items of the plan
4 Ambition – making sure that we are not being overly ambitious.
5 Drawing on projects that have already engaged in similar-ish activity


1. I have flirted with the NSSE / or seen pointers to it (the NSSE) on my USA conference travels and it something that I would like to explore further. Clearly we will have to balance the institutional requirements of 'success' in the NSS and so this will need to stay on our radar too. I would like us to focus on learning and seeing how that aligns with the NSS.
i.e. not wanting to be NSS driven.

2. David Nicol asked what agenda we will be taking to the Schools - i.e. in addition to working with the agendas of the Schools. Great question! I have thought long-and-hard about this after the meeting - I am aware of David' self regulation agenda for instance. Most of my work has been about trying to develop a personalised experience for the students. That is not to trample over the benefits of collaboration, community centeredness or seeing that socialisation is important but I am keen to see students as individuals in the current mass HE system. In some sense this is in part response to one of the 14 grand challenges raised by the National Academy of Engineering. I think Personlised Learning / seing students as individuals will resonate with us here since we can quite rightly draw on David's work, promote the benefits of community centeredness but also remind ourselves that we are dealing with individuals. We are a large University and losing contact with our students (as individuals) will be of concern to many of us here.

3. Our project plan was details in some areas but less so in others - we need to balance our endeavors and show commitment to all aspects of the project and its stages. Lisa Gray raised similar observations when she asked for more specificity.

4. We write in our project plan about transformative and sustained change. I don't wish to retreat from these aims but the Steering encouraged a close look at the size of the 'ambition'.

5. I am aware of the FAST and SENLEF projects and also the Engaging Students with Assessment Feedback project. Actually now I come to write about those and other A&F related projects (SPRINTA etc) that Chris Rust and I were involved with at the University of Essex, its kind of bonkers that there are still so many issues with some of the basics of assessment and feedback practice.
I will make sure that we revisit those projects and use what the others have already found. I will also draw on the experiences and expertise of Malcolm Ryan and the Learner Experience projects. Learner Experience is also part of this project

Hmm. Lots to do - but it was just great to get the input from colleagues. I have every confidence that our project will benefit from the experience and expertise of our Steering Group - Great input - thank you!

Mark

Scottish Journey

Mark and I travelled up to Glasgow last week to meet with Rachel Harris from Inspire Research. We were looking at how using appreciative enquiry techniques could help us in our work with the module teams in the schools. AI looks at growing what works in a organisation to foster change rather than to think of things in terms of something needing fixing.

The day was very productive and we were able to flesh out exactly how we we going to approach things on a module level. In addition we have started to put a detailed timeline together for work we are doing with the schools. We had a chance to knock around quite a few ideas and we were able to develop some ways of looking at things that should really help the sustainability of the project's influence after the project itself finishes. In particular we looked at how many modules you would need to influence in order to bring about an institutional change . Mark compared it to how many people were needed to start a mexican wave in a stadium. Was there a critical mass of modules that you needed? and if so what were the factors that influenced that number. We also looked at things from person centred approach and looked at how we could measure a persons influence within a school as an agent for change. We wanted to encapsulate how many modules a particular person could influence and see if we could express that as a kind of index or quotient. It was good to have Rachel's input and experience to keep us both grounded on what was possible within our time frame.

At the end of last week we also got the feedback on the project plan from JISC - so this week will be concerned with updating the plan in the light of these and the steering groups comments. In addition we will be and arranging meetings with the module teams within the schools to get them to examine their current practice within an AI framework.

Yesterday Mark and I ran a workshop on 'assessment and feedback' open to any member of staff at the university. We had 24 participants who contributed to some lively debate and discussion regarding the challenges of setting effective assignments and providing useful feedback. As you will be aware from previous blog entries, we are very keen at modelling behaviour, so we set the participants some pre-workshop activity, and then an activity within the session. We asked participants to bring along a case study of an assessment task that they felt needed improving. We discussed principles of good assessment and feedback practice, considered student and staff opinions regarding assessment and feedback and then asked the participants to consider where their assessment case study was currently located on the above grid of effectiveness vs efficiency. With support from colleagues, they then discussed how they could move their assessment up/along the grid to improve the effectiveness and efficiency of the assessment. In some cases this meant improving marking turn around times to provide prompt feedback by considering for example, sharing out the marking, online submission and group work. Other cases considered improving the effectiveness of the assessment by considering activities such as peer and self assessment and involving students in a dialogue around assessment policy (i.e. enabling some choice within the assessment task, discussion of assessment criteria and the activities involved in the marking process ). For post workshop activity, we have asked the participants to identify changes that they will make as individuals to their assessments, changes that they will make within module teams, plus changes that they hope to influence at School level. We will follow up their activities and hopefully gain more examples and suggestions that will benefit the ESCAPE project. It was great to see staff engaged in this agenda and there were 24 enthusiastic people in the room. My hope is that these individuals plus the academic staff involved in ESCAPE will make small steps of change that will influence their colleagues and we will see an improvement in assessment and feedback practice across the whole institution - there's nothing wrong with being optimistic right?

Model Relationship

This week I have been concerned with trying to model the relationships between the modules that we are looking at in the School of Life Sciences and the other modules within the school. We are looking at the module profiles, the assessment mix and the key individuals that are involved with the modules to see what links and patterns emerge. We are looking at ways of using our results to model and predict how the project's influences will spread thought the school. It is helping us to identify the key "secondary targets" that will be able to be influenced by the second phase of the project. We have been helped this week by Ruth Hyde - welcome aboard Ruth. Ruth has been doing some work on building the model and looking at the variants that are emerging.

In addition I have had a meeting with the Business School to see how we can get the data that we need to be able to carryout our baseline audit of achievement for our "before" part of the "before and after" comparison of results. Lots of enthusiasm for the project in the Business School which is manifesting itself in a plethora of offers of support and encouragement. They are very excited by the project and can see how it will address some of the assessment and acheivement issues that they have with certain programmes.

It’s off to Glasgow Thursday and Friday for the team - we are having some training in appreciative enquiry provided for us by inspire research. We want to see how we can use the techniques of appreciative enquiry to manage the change we want to encourage in the assessment practices in the schools we are working with.