12 June 2012

On "assessment for learning"; the petrifaction of a process.

I've been reading with interest a detailed blog on ESOL by Sam Shepherd, and in particular a point from a (fairly) recent post:
My personal experiment here was around learning outcomes, the value of “sharing” them I have always wondered about. And I have to be honest and say that for this lesson  I might as well have just whistled the learning outcomes at the beginning, for all it meant to the learners. [...] But the whole learning outcomes thing, really, not impressed. For this group it just didn’t make sense. [...] But I want to emphasise that it was just one lesson: and in the spirit of research and experimentation, I am going to persist and see how we get on.
This is an issue which crops up regularly in my own classes with people already teaching in post-compulsory education taking a part-time teaching qualification, and I usually come in for some stick from "students" who tell me they are obliged by college policy to announce the learning outcomes of every session at the start, and one very upset man told me that the only reason Ofsted had not deemed his class "outstanding" was because he had been too nervous to recite them. My usual response is to say that the only decent reason I can see for doing it is that it's a bit of ritual which you can use to communicate, "this is where the class really starts", but there are lots of ways of doing that. (I used to illustrate this by reciting at least one of the module outcomes in Anglican chant--I gave up on that when it became clear that it meant nothing to most students, and I don't have the voice for it... Pity.) 

So what is my objection? It is certainly not about wanting to confuse the students (although sometimes that is legitimate and effective; surprise can be an effective teaching tool). I've no objection to outlining what we are going to be doing, or "looking at" in this session, or giving an idea of about how much time we are likely to be spending on something; especially when it is a two-hour session, they like to know when the comfort-break is likely to come up. 

Students don't understand them--why should they?

My first objection is precisely what Shepherd identifies. Learning Outcomes (LOs) are written in teacher-speak gobbledygook. Even my students frequently don't understand them--and at the beginning of the module we do actually study not only what the outcomes say but also how they are written. They were difficult enough in the days when my colleagues and I wrote them, but since they have been laid down centrally by the now-defunct and unlamented "Lifelong Learning UK" they are well-nigh impossible. Granted that these are at the module level rather than the session level, here are two examples from the 2007 LLUK specification:
Understand the application of theories and principles of learning and communication to inclusive practice.
Understand how to apply theories and principles of learning and communication in planning and enabling inclusive learning.
(Shepherd's outcomes are SMART; one thing anyone formulating LOs learns from the outset is that "understand" is not an acceptable verb for an LO. See here and here for my heretical take on that. But the guardians of the Gradgrindian fog at LLUK used it all the time.) It's not clear what they mean. It's not clear how you show their LOs have been met. And there are two of them, and it's not clear what the difference is between them! They add nothing. Time spent on introducing them could be spent more profitably on doing practically anything else.
    Such as using an Advance Organiser. OK, the evidence for its effectiveness as a teaching device is not very high, but it's a respectable tool to have in the box, and it takes very little effort to use it. AOs (let's carry on with this silly game) refer to the students' experience, so they engage them, and set them up for the session. LOs on the other hand, distance them, and assure them that teaching is about to be done to them, so that they will emerge at the end of the session having been reliably processed through the sausage-machine.

Where's the evidence? Substitution of the sign for the signified.

My second objection is that there is no evidence that this is good practice. (This is a more convoluted and longer argument.)

A few months ago a correspondent got in touch about my Heterodoxy pages;
Very interested in your article re reflective practice, much of which can, in my opinion, be attributed to today's idea which cannot be questioned - 'Assessment for Learning'.

It seems to me that the impact of this seemingly effective form of practice has not born fruit [...] and that opportunities for practicing a particular skill or trying something more difficult have been replaced by pointless navel gazing. 

I'd like to see you take a swipe at AfL.
(Link above inserted). I'm not "taking a swipe" just because CJ requested it (but apologies to him/her for taking nine months to get into this), but because this kind of issue is a classic case of the ritual ossification of what he/she describes as "this seemingly effective form of practice"--concentrating on a relatively arbitrary sign rather than on the substance towards which it is supposed to point. And to explore it is also to explore the fate of many other educational innovations...

So I've done a lot of reading and talked to a lot of people--no, I didn't "do a literature review" and "interview respondents"--I live in the real world nowadays! And I'll spare you (most of) the references.

I am a fan of assessment for learning. The real McCoy, though, not the institutionalised tick-box version being peddled by Ofsted and their intimidated toadies. But how did we get the version we have now?

How did the principle of maximising feedback to learners, and thereby getting feedback from them, become this sterile ritual observance of reciting LOs, testing in every class, recapitulating the pre-determined learning points, and then writing it all up as if it constituted evidence of something?

There are several complementary perspectives on the process, and they all tend to push in the same direction...

The (proximate) origins of AfL


To cut through a lot of the origins of the idea, we can say that while it was enshrined in a Department of Education and Science report in 1988, it was championed by Black and Wiliam (1998), although they of course acknowledged that it was what good teachers had been doing for decades or even centuries--certainly at least long enough for them to amass 250 studies for a meta-analysis. They note:
"Typical effect sizes of the formative assessment experiments were between 0.4 and 0.7. These
effect sizes are larger than most of those found for educational interventions." (1998 p.3)
And Hattie's much larger meta-analysis, published 2010, confirms the figures (my take on it is here, including a note on what "effect-sizes" are).

But were they talking about what is commonly understood today as "Assessment for Learning"? It's clear from re-reading Black and Wiliam that they envisage a long-term cultural shift in classrooms. They repeat several times that it will be a slow process. It will be characterised by introducing measures which enable teachers to get a clearer picture of where their students are, in relation to particular topics and skills within a subject area, drawing on, inter alia, student self-assessment. Open communication is the key.

But Black and Wiliam are too experienced to believe that teachers can re-adjust simply on the basis of principles--they need concrete practices to follow:
"Teachers will not take up ideas that sound attractive, no matter how extensive the research base, if the ideas are presented as general principles that leave the task of translating them into everyday practice entirely up to the teachers. Their classroom lives are too busy and too fragile for all but an outstanding few to undertake such work. What teachers need is a variety of living examples of implementation, as practiced by teachers with whom they can identify and from whom they can derive the confidence that they can do better. They need to see examples of what doing better means in practice." (1998 p.10)
That is entirely understandable. I don't want to over-simplify the convoluted processes of change, particularly in education, but I suspect that nevertheless one unintended consequence of such thinking, exemplified in professional development programmes at one end of the scale, and in Ofsted inspection frameworks at the other, is to concentrate on the outward and visible signs, rather than the more elusive and protean processes of cultural shift. (We also need to see this process in a political and historical context, of obsessional micro-management of public services under the Labour government...)

We are also now looking at a second or even third generation of teachers since the desirability of AfL became the conventional wisdom--and quite possibly because of the way these teachers have themselves been taught--they have simply accepted all these signs and rituals as given, with no realisation of the rationale or principles underlying them. Reflective practice (from ten years earlier) has similarly become a matter of going through the motions without knowing why.

The managerial argument has been: If we are looking for a more open flow of information about understanding and achievement within the classroom, how do we know that it is happening? We need it to be reported. We need ILPs (Individual Learning Plans) to be set up and recorded on forms, and in order to ensure that the information is hard enough, that needs to be based on testing.

But! Given the time it takes to set up and implement and record such tests, and their inherent bias towards summative assessment, and Black and Wiliam's point above that "[Teachers'] classroom lives are too busy and too fragile for all but an outstanding few to undertake such work." --it's not surprising that their point that:
For assessment to function formatively, the results have to be used to adjust teaching and learning; thus a significant aspect of any program will be the ways in which teachers make these adjustments.  (1998 p.4, emphasis added.)
has in many cases not been met. It's just too much effort to complete the circle, and as with any cyclical model, there is nothing more disappointing that trying again and again to start something which continually peters out. (Think of trying to start an engine with a pull-cord--or even a starting-handle--when it won't "catch".)

The social construction of learning and its institutions.


Nevertheless, there is a venerable descriptive cyclical model in play here--see Berger and Luckmann (1967). (It is consistent with Wenger (1998): the cycle can start at any point.)
Put very crudely, it represents how we put ideas out into the world (externalise them). We do that by talking, writing, making objects. Ideas become social "things" (reified), such as institutions, laws, languages, art, etc. but of course changed in the process; and we in turn internalise them, which changes us and the next ideas we try to externalise. It is the reification of AfL which has lost the point (or if you like, the spirit) of the whole thing, and this cycle carries on independently of the will of the teachers. (There are fairly clear parallels with this model.)

What happens in the process of reification is that the idea, for want of a better word, has to accommodate to all the other reified ideas out there if it is to survive, and it is this adapted version which we (and the next generation) internalise. Thus, any social institution (and any approach to teaching is a social "institution" in a broad sense) gets knocked into shape by the dominant political, economic, technological and other powerful cultural factors of the day.

This is not merely an "academic" digression. What counts as "learning"--especially in the sense of "what is supposed to go on in education institutions"--is socially constructed. And so, therefore, is its assessment. At the level of universities, Stephan Collini writes (2012 ch.5) on  the substitution of metrics and superficial proxies for valid evaluative tools, because the rhetoric of "a public good" is no longer recognised as a reason for doing anything unless it can also be justified in primarily economic or other instrumental terms. (He writes as an apologist for the humanities.)

And he notes that inspection and evaluation of processes of delivery have taken over from any engagement with disciplines and content themselves, because of the value-questions which are inevitably encountered through such an engagement.

So, much as Ofsted and other quality assurance institutions would wish us to believe that their work is value-neutral, and that an "outstanding" class is better than a merely "good" one on any terms, that is not the case. (I grant there are some practices clearly to be avoided in teaching, so there is likely to be a more substantial consensus on distinguishing between "outstanding" and "failing".)

In other words, what kind of learning is "assessment for learning" about? The choice of what kind of assessment to focus on is critical in determining what kind of learning is promoted. In assessing a piece of written work, what counts most? Is it spelling and grammar? Elegance of expression? Originality of ideas? Arguments based on evidence? Underlying research? All of them, of course, but emphasising some beyond others depending on the age and stage of the learner, the subject matter of the essay, and so on. Choosing the "correct" criteria, choosing what to pay attention to and what to ignore are all based on those socially constructed values.

(The issues are of course generally much more straightforward in STEM* subjects.)

Accept only substitutes!


Frankly, all that is too hard to measure, so the system contents itself with these relatively trivial, perhaps harmless (other than their cost in time, effort and morale, and distortion of priorities) proxies which at one time might have pointed towards the quality of classroom communication, but have long since lost the connection. And perhaps that is all one can do in and inspection of just a few days. But it's not surprising if institutions focus on compliance with the letter rather than the spirit of the approach. (I'm giving inspectors, principals and other managers the benefit of the doubt here; my suspicion is that they no longer realise that there is a problem.

Just to round off, though, there is of course a quis custodiet question about these guardians of teaching standards: here's an interesting take on that, and more generally here is one on all those who presume to diagnose, predict and treat within ill-defined systems, exhibiting "deluded self-limiting prescriptivism".  Thanks to David Stone for the link.

But see here for a post on an LSE blog which rather grudgingly concedes some validity to Ofsted's judgements.

Update 27 July: Dylan Wiliam himself is now pointing out how misunderstood the AfL is, and how little implemented (TES, 13 July)


*Science, Technology, Engineering and Mathematics

Berger P L and Luckmann T (1967) The Social Construction of Reality London; Penguin (And Peter Berger is still blogging--largely about the social practice of religion--here.)

Collini S (2012) What Are Universities For? London; Penguin (Review here)
Posted by at No comments:

09 June 2012

Items to share: 9 June

Education Focus
Other Business
  • Leonardo of the Deep - "Gas pods" (small bumps on the roof of a car) are claimed to increase fuel efficiency by mimicking the effects of similar bumps occurring naturally on whales and lobsters.
Posted by at No comments:

03 June 2012

Items to share: 2 June

Posted by at No comments:

26 May 2012

Items to share: 26 May


Education Focus Other Business Asides
Posted by at No comments:

21 May 2012

On handing on...

(19 May 12) The Olympic torch has arrived, and the relay begins...

Chelsea (football/soccer team) have won the European champions' league, in what some commentators have claimed to be the last gasp of an ageing team. I have no idea about that in detail, but the handover to the next generation is a periodic make or break challenge for any long-term organisation.

When Star Trek returned to TV in the 1987, the series was identified as "the Next Generation", and indeed it represented a shift in values, perhaps from the free-wheeling '60s to the more formal and task-oriented '80s.

And at a minor level, the coincidence of minor personnel "churn" and institutional re-organisation in our network of teaching centres means that at unpredictable ("stick and slip") intervals we are faced with the challenge of passing on the torch--or even the "brand"--of the programme to a new generation.

All of which brings to mind the challenge of passing on--or adapting--underlying values, rather than formal structures.

In the case of  the PCE programme, the quality assurance procedures have been mind-boggling. For various reasons (none to do with the programme itself) it has been inspected by Ofsted three times in the past five years as well as reviewed by its validating body (now abolished); it has recently emerged from a routine quinquennial review by the university, and last year a couple of typos in the documentation prompted an intrusive (and, it transpires, unauthorised) review apparently motivated by the personal animosity of a senior figure in the Faculty, who is now no longer with the university...

Stefan Collini (2012:107ff.) points out that the procedures and metrics of "quality assurance" are good at ensuring and indeed enhancing the rigour and comprehensiveness of everything which does not really matter in universities.

But how does one pass on the vision and the distinctiveness of the programme, which cannot be contained by the details of second-marking samples and moderation and module specifications (syllabi)?
    It could of course be argued that possibly they should not be passed on. Perhaps they were misguided to begin with. Perhaps they have been overtaken by social, political, economic--even heaven help us, technological changes. After all, this programme was first validated in 1996.  It's true that it has dropped increasingly out of step with relentless policy initiatives in the sector, but even the hostile unauthorised review of 2011 did not try to make a case on that basis.)
The community of practice which is the teaching team on the programme (around forty people at any one time in ten colleges, and fifteen or so who regularly attend network meetings) is the bearer of the (sorry! Hate to use the term, but it is  the most powerful) of the "brand". Its ultimate origins may be found in a range of philosophers and educational theorists and practitioners but its proximate base rests on the two people who have been with it from the beginning, and both of us have now retired.

For the rest of the community, the characteristics of the programme can seem quirky, because they have only encountered them in their "mature" form, and were never involved in their development. The programme leader who has recently retired--and who has been by far the most significant figure in the development of the course--has frequently commented over the past few years how much time in network meetings has been spent in explaining over again the original rationale behind, for example, the policy of negotiating submissions through learning contracts rather than setting assignments--and occasionally discovering that the policy has been sacrificed to administrative convenience by some (generally peripheral) tutors.

So do we now have to recognise that it is futile to cling to the past out of some misguided delusion that the course is our own creation? Indeed, may we have become so obsessed with its quirks that we are no longer able to evaluate them dispassionately? May they be merely trivial decorative flourishes which mean nothing or may--as some students and probably tutors believe--actually obstruct learning?

Quite possibly. But then it may be the sense of working on something different and distinctive which enthuses and energises colleagues teaching the course, and gives them a sense of belonging to a team... even if that which is distinctive does not instrumentally matter that much. That certainly seems to be the experience of colleagues at one of the colleges in the network, which has changed their validating university, for irrelevant reasons; they have reported a sense of loss. One neglects such an agenda at one's peril.

But spare a thought for those who are taking up the baton, and their challenge of making the course their own--putting their stamp on it--while engaging with issues of loyalty to a prior generation.

And read the Saber-Tooth Curriculum. (There's a note on it, here.)
Collini S (2012) What are Universities For? London; Penguin
Posted by at No comments:

19 May 2012

On reasonable demands

This is one of those confluence posts--when several sources flow together... (started a couple of months ago, but re-animated by further prompts--5 onwards below).
  1. We have just had a Study Day at which Kathryn Ecclestone raised pertinent questions about current approaches to assessment--particularly in the further and vocational education sector, but also at more advanced levels. The students discussed those issues in discipline/setting specific groups and articulated their particular concerns and solutions on posters.
  2. A few months ago I was an external member of an exam board when I had to express concern about the very detailed brief--almost a recipe--for the production of an assessed piece of work at Master's level. (Since fixed)
  3. But the leader of the dissertation module on one of our Master's programmes has recently approached the tutorial team, requesting suggestions for reading as the basis of the literature review for the dissertation
  4. ...and in the past few hours I've had two, much appreciated, thank-you emails from readers of my pages on the structure of a dissertation, and writing at Master's level. #1 is the proximate cause of this post, but it is fuelled by #2 and #3, and #4 poses the question whether I am complicit in this game--if that is what it is.
  5. A couple of former colleagues* and I are working on a conference paper concerning the inability of current educational systems to accommodate liminal states on the part of students. Liminality occurs in conditions of qualitative change (I've elsewhere used the term disorientation for the experience of liminality, with a slightly different emphasis). A precursor of the paper can be found here.
    Karl Marx (Capital vol 1.) argues that "Merely quantitative differences beyond a certain point pass into qualitative changes." We can turn that on its head and suggest that because that is the case, and qualitative change generates liminality, therefore quantitative changes need to be constrained below that trigger point.
  6. Stefan Collini (What are Universities For? Penguin, 2012) in a well-argued (but padded, repetitious and under-evidenced enlightenment-style "pamphlet") contends that the managerialist rhetoric of current neo-liberal politics in which everything has to be accountable and costed, has forced those who would run universities (and indeed other educational institutions) to embrace spurious metrics as distorted proxies for fuzzy contestable aims such as "education" and "scholarship" which are no longer accepted as goods in themselves. In particular he refers to fatuous notions of "continuous improvement" "beyond excellence"--that can only work if the standards do not change. Discontinuities--gear changes--are inimical to the model...
All this is not to fall into a jeremiad about the state of further and higher education; there's plenty of evidence of its achievements, and I continually come across people producing great work, for whom and for which I have great admiration. But they're having to do it despite the structures and the culture of the institutions.

And many of them are not even aware that it could be different. Or that by becoming complicit in merely implementing a cravenly incremental model of curriculum and learning, they are becoming part of a problem rather than that of a solution.
* Peter Hadfield and Peter Wolstencroft
Posted by at No comments:

Items to share: 19 May

Education Focus:
  • The Learning Spy - Who inspects Ofsted? General verdict--the model is not fit for purpose. (I'm working on a post on what happens to good ideas in education when they become institutionalised.)
Other Business

    Posted by at No comments:

    18 May 2012

    On hoarding

    There have been a number of recent TV programmes in the UK on compulsive hoarders. The programme template is simple; show us the grotesque extent of the subject's hoarding compulsion (including revealing pest infestations and their by-products...); dramatise their struggles to throw out their stuff, with the alleged help of therapists [discounting the potentially much more potent and prolonged presence of a TV crew with a clear story to tell, backed up of course with a legal contract...]

    Eventually, the former hoarder is shown in their now uncluttered and cleaned up house, protesting their relief at having room to move, and being able once again to access their bathroom and kitchen.

    The formula raises a host of issues. Let's leave aside the question of the status of compulsive hoarding as a psychiatric disorder (a very live issue since we are currently in the consultation period for an [or even the] authoritative manual on this), important though it is.

    Instead, let's return to the former hoarder, consider how long she or he remains free of the problem and a rather more venerable way of describing what happens:
    Luke 11:24-26:
    24 When the unclean spirit is gone out of a man, he walketh through dry places, seeking rest; and finding none, he saith, I will return unto my house whence I came out.
    25 And when he cometh, he findeth it swept and garnished.
    26 Then goeth he, and taketh to him seven other spirits more wicked than himself; and they enter in, and dwell there: and the last state of that man is worse than the first.
    Unsurprisingly, treatment is very difficult and uncertain. Without necessarily adopting any kind of religious discourse, it's all about taking something which served some psychological purpose--however bizarrely and counter-productively--and often not offering anything in exchange. One of the few frameworks which addresses this explicitly is drama therapy, in particular the Magic Shop exercise.

    Its principles are applicable not only in cases of pathology, but also in teaching, wherever learning or change is experienced as loss.
    Posted by at No comments:

    13 May 2012

    Items to share: 12 May (belated)

    Education focus:
    • Do We Need the Next Generation Science Standards?: OK, the question presupposes "No", of course. It's US-centric, so the point of the link is not the detail--it is the downright incomprehensibility of the whole thing, and losing sight of the wood for the trees.
    • Prezi - The Zooming Presentation Editor Prezi continues to evolve and become more flexible, but I confess I haven't found it as useful as I expected. Its vertiginous zooms require great restraint, and are often more distracting than illuminating (same goes for the dancers above), and there are ways of getting the same effects (for teaching purposes, at least) with more conventional means.
    Other business:
    • Saturday 8 May 1669 (Pepys' Diary): Why mention this now? Because this "real-time" replay of the world's most famous diary--with scholarly annotations--concludes at the end of this month, and although the archive will continue, there's something about the quotidien revelations which relies on their sequential publication, especially because much of Pepys' life was as boring as ours! I hope there will be a replay. 
    Posted by at No comments:

    05 May 2012

    Items to share: 5 May

    Education Focus
    • The Learning Paradox (Farnam Street) On "productive failure" (another version is here). This works for students who will not be de-motivated--an appropriate "mindset" is critical. (See also Tim Harford's latest: "Adapt; why success always starts with failure" [2011])
    General
    • Against Chairs Silly opening, but overall an interesting account of the history of the chair. Don't just sit there!
    • Pseud's corner-- a film about Slavoj Zizek I'm getting all post-modern... I'm drawing attention to this only in order to encourage you to ignore it. Although since the linked article takes a similar stance, perhaps two negatives make a positive? A few years ago, I did spend/waste almost a working week trying to suspend disbelief and evaluate this guy, from his own writings; so Zizek (or his gullible academic acolytes) robbed me of time which I could have spent more profitably washing the car or cleaning the cooker or doing practically anything. They deserved (perhaps) seconds, and I gave them hours.
    Posted by at No comments:

    28 April 2012

    Items to share: 28 April

    General

    • BPS Origins An excellent interactive timeline (from 1900) of the history of psychology--I hope it continues to grow in detail.
    • "Rupert Murdoch--a portrait of Satan" Adam Curtis reprises an earlier feature on relations between the BBC and Murdoch; not as conspiracy-focused as usual, but compelling nonetheless.
    • A Point of View: In defence of obscure words Will Self--inimitable (fortunately)--but I have to agree with him this time. A few months ago I was instructed to remove all Latin phrases from a course handbook (syllabus), particularly the notoriously difficult "et al." in case the students didn't understand them. I had naively thought we were in the business of expanding their understanding, not restricting ours... I somehow never got round to complying, and as expected no-one ever checked, and the person making the demands has quietly left the university.
    Education focus 
    • Great Lives: George Lyward (BBC Radio 4) I only met GL twice, shortly before he died and some maintain he was then "past it", but even so he was simply one of the most extraordinary people I have ever encountered. This is a great attempt at a tribute by Tom Robinson, but sadly it was bound to fall short... 
    Posted by at No comments:

    21 April 2012

    Items to share: 21 April

    Education focus:
    Other Items:
    Posted by at 1 comment:

    20 April 2012

    On the counter-productiveness of compulsion.

    The Higher Education Academy is advertising for accreditors. They will check applications for HEA fellowships, and programmes leading to eligibility for them. The work is moderately well-paid and flexible, and it sounds just up my street, if I behave myself...

    If I decide to apply, and get through the filtering despite being flaky on "Wide experience of working with the UK Professional Standards Framework"*, I can imagine that a sensible question at interview would be;
    "Tell me, Dr Atherton, what do you think are the major challenges facing the PFS programme in its implementation in HEIs?"
    I would have to answer "resentment" (followed by credibility). I can't trace the source of the adage that "managing academics is like herding cats"**,but it is very apposite.

    I've not done any formal research on this, but I am sure every colleague involved in "teacher training" in the HE sector will recognise the issues...
    • From a discussion at a conference--about learning and teaching in HE and therefore attended by enthusiasts--in Sheffield in the late '90s, when the new Institute for Learning and Teaching has announced its membership scheme: "Why would I be stupid enough to join the ILT, and announce my interest in teaching? I'll just get lumbered with the first-year teaching and lose time for research!"
    --and now it's a "requirement"--
    • "I've been in education for almost twenty years as a student--I think I should know something about teaching."
    • "It's bad enough trying to get your head around preparing lectures in your first year, and all the other rubbish we get dumped on us--that no-one ever told us about in advance--and then you want to take away our Wednesday afternoons for a compulsory teacher-training course. Dual professionalism my backside."
    • "This is all ideologically driven--principally to provide jobs for the girls (mainly), because they wouldn't be employable anywhere else in the university."
    • "It's an insult to have all this patronising jargon-ridden gobbledygook which passes for "research" in educational circles--mainly half-baked and methodologically very dubious progeny of long-out-dated fads from psychology and marxist theory--shoved down the throats of people who do actually know some stuff in depth."
    • "...and who in the Faculty is actually going to sack me for not getting a PGCert. in three years? And has anyone ever failed one, anyway?"
    I first started teaching on a PGCert in Learning and Teaching in HE in 1996, and of course it was then voluntary. (It was the first stage on the MA in Learning and Teaching.)  The requirement to undertake the course has been introduced piecemeal by different universities--I think it was introduced where I worked in 2003.

    So it was a joy to teach for seven years, with keen, motivated, and of course very bright colleagues as one's "students". We tended to start the three-hour session with one of the group reviewing something from the reading list for a nominal fifteen minutes. It was not unknown for them to fail to reach the end of their prepared remarks by the refreshment break 90 minutes later, such was the discussion. And I don't think I ever finished my prepared material. That's where I formulated my mayonnaise model of teaching.

    Of course they were critical, and of course there were some who were very irritating. Motivations were mixed. In the early days we had some older colleagues on the course who had been at the institution since before it became a polytechnic. Now it was a university and they were acutely aware of how under-qualified they were--and this course seemed the easiest way up. There were others changing tracks because their research career was stalling. And some were "encouraged" to attend because even their Heads of Department recognised that they did not even reach the very low and informal bar which represented the expected standard of teaching.

    But they were receptive and participative.

    Eventually, it became a requirement for all new members of staff without an existing teaching qualification to undertake the course, and that had knock-on effects. The numbers grew, of course, and that limited the opportunities for such highly participatory approaches to teaching. Heads of Department made representations to the Centre which ran the programme (it was not based in the School of Education) to make it more convenient, and more "practical". The Centre recruited more staff, with teaching qualifications but without the academic credibility which came from being based in a faculty (even in a School of Education, which is traditionally low in the academic pecking order). The course was revised and re-validated, responding to the explicit requests for more practicality and application--so the approach became more didactic. And structured.

    Dissident voices became more apparent. More time was spent justifying the very existence of the course--and more was also spent on inducting the participants into the QA procedures and paperwork of the institution. The Centre was absorbed into the ever-expanding Human Resources Department, and lost even more credibility. The course was no longer seen as a resource and a stimulus, but more of a penance, with lower and lower expectations of how it might help improve teaching...

    That story is perhaps a little over-stated, and it is not the whole story, but I know from speaking to former colleagues across the country that it is repeated with variations from Russell Group universities to post-1992 institutions.

    So how do you get colleagues to believe that the content of the course will actually help them, and make teaching better, easier and more enjoyable?
      *   How much more is "wide" than "some"? Does "working with" include "complaining about" and "failing to understand"? [I shan't mention this qualification in the application, but if it comes up I shall at least know someone at HEA reads the blog!]  

      **  Although my own contribution to the source hunt is that I first heard it used by Diana Laurillard at an Open University conference in Milton Keynes in 1997; although whether it appears in the transcript of her paper I don't know; it must have been fairly fresh because is got a genuine laugh. ( Laurillard D (1997) "A systems model of individual learning applied to organisational learning and the HE system" Paper at "Applying Systems Thinking to Higher Education" conference, Open University 12 July 97. [Read more: The Politics of RBL http://www.doceo.co.uk/rbl/polrbl.htm#ixzz1sVqmzXBE Under Creative Commons License: Attribution Non-Commercial No Derivatives] )
      Posted by at 3 comments:

      15 April 2012

      On sources for courses, so to speak

      I'm planning a book. The opportunities afforded by e-books for almost direct sales (not to mention the prospect of 70% royalties if one plays one's cards right, given that the external funding for my sites and blog dried up three years ago...) are very tempting.

      I have been wondering how to "pitch" it. Not in the sense of how to sell it, but how to find the right "voice" for it. And as I took a walk this afternoon, I realised I had already written it! At least, I wrote 40,000 words of it about fifteen years ago, and then life got in the way (and the publisher didn't like it) and it came to nothing. But it is still there on a backup drive, so this afternoon I sat down to read it.

      One: the publisher was right.

      Two; the content is OK, although of course way out-of-date (both academically, and in terms of my views), but I no longer have any idea of why anyone would want to read it, presented as it is. I'm pleased to find some of the grunt work has been done, of course, although I suspect that everything will have to be re-written from scratch. But it's unreadable; it's clear and well-structured but utterly lifeless. And...

      Three; that is in no small measure because I used the author/date referencing system (a.k.a. "Harvard"). It's not merely a technical device--it transforms writing. Not in a good way.

      In my retired leisure I have been reading a lot of interesting stuff, for pleasure. Several years ago I suffered Acute Fiction Failure. For some inexplicable reason I find myself incapable of engaging with novels any more, even old friends; the last one I finished was Ishiguro's Never let me go. Perhaps that was just too haunting?

      However! I've been devouring dozens of "popular" science/economics/social science/op-ed books and admiring their balance between doing justice to the content and being accessible to the reader. And clearly one of the ways in which they manage that balance is by using end-notes.

      So that set me thinking about academic writing styles.

      Sadly, the academic style I have become used to is primarily defensive. It assumes a hostile reader--a marker (grader) in the case of the student, or a peer reviewer in the case of a researcher. Hence the obsessional APA  referencing--never mind that the in-text citation continually interrupts and distracts from whatever "flow" the argument might have. And the adoption of the passive voice as default, as if to claim any personal involvement in the research is to give a hostage to fortune--although I am pleased to note that does seem to be in retreat, a little.

      It is certainly not about communicating any enthusiasm or even interest in the subject.

      Clearly the document of record, the peer-reviewed article reporting serious new findings in a serious discipline, needs to be scrupulous; but frankly looking at what passes for scholarship and even research in the dubious world of education, most of the published bumf is not in that league. It's glorified op-ed, covering itself in the fig-leaves of scholarship to conceal its nakedness.

      In many cases, the imponderables and immeasurables of the topics under discussion mean that is all we can aspire to. (And it's not helped by the pressure on academics in these soft fields to produce and publish ever more papers, just for the sake of it, without ever asking whether the world will in any sense be a better place for all the futile effort they devote to papers which will eventually aspire to a citation index of zero.)

      I am returning of course to topics I have touched on here and here.

      But the point this time is a little different; the authors and publishers of the books I have been reading, however serious (Kahneman, for one, is a Nobel laureate) are able to wear their scholarship more lightly and to concentrate on communicating their ideas, presumably because they are on trial in the market-place rather than in the more vicious arena of their peers.
      "The reason that academic politics is so vicious is that the stakes are so small." (often attributed to Henry Kissinger, probably wrongly--but the fact that I checked it out is itself a sort of recursive demonstration of the idea...)

      They use end-notes. That is not in itself a big deal, but it does mean that the author can write so as to give priority to the argument, but also allow the reader to get at the supporting evidence if she wants to.

      Ouch! I did actually start to write; "..it betokens a rhetoric which privileges the argument, while facilitating access to the scholarly evidence..." Sorry!!

      ...and yet... The two versions do not actually say quite the same thing. Enough!

      Meanwhile, back at the ranch... The different styles suit different contexts, of course. But having been corralled into writing academically for years, I am obsessed with buttressing the most banal assertion with scholarly scaffolding, and of course in-line author/date citation is the most in-your-face way of doing that.

      It's a con, of course. How often do you check even the source of the reference, never mind the content? So the concern of critics of this dangerously slack end-note style, that without at least numbered in-line references the reader cannot be sure that a statement has a legitimating provenance, is merely academic posturing.

      (And any self-respecting pedant will note that the preceding sweeping assertion is not supported by any references. That's because I can't be bothered, and it is purely unsubstantiated opinion, in any case.)

      Even more to the point... I realise I don't know how to write in this accessible style.
      • Carol Dweck wrote Self-theories; their role in motivation, personality and development in 1999. It's a solid and accessibly-written, if rather repetitious, academic book (APA referenced). Someone must have persuaded her that she could do something more popular, based on it. The chimerical outcome is Mindset: how you can fulfil your potential (2006) --the bit after the colon is the giveaway. Decades of (arguable, but) properly rigorous collaborative, peer-reviewed research, gets presented as '70s self-indulgent, self-promotional, psychobabbled pabulum. Sorry, but however well it sold, she didn't bring it off.

      • Jonah Lehrer is the primus inter pares of people who can bring it off:
        Lehrer didn’t invent his area of work. He is of a kind with Malcolm Gladwell and the Freakonomics authors, Steven Levitt and Stephen Dubner. [...] Like those authors, Lehrer is making a career of a tried and true structure: find a compelling anecdote to support counter-intuitive research from science and academia, and sum it all up with a pithy conclusion.
        Yet Lehrer and his ilk are popular for a reason. They make the hard-to-decipher work of scientists comprehensible to everyday readers and, at times, they give readers a belief that they too might be capable of extraordinary achievements.
      My proposal is not as formulaic as that quote would suggest, but it does put its finger on the art which conceals art, and which I am coming to realise, may well take a long time to acquire.

      Fascinating what can follow from a "simple" change in referencing conventions....


      Posted by at 1 comment:

      14 April 2012

      Items to share: 14 April

      Education focus:

      You Don't Remember Anything You Learn at School. Do You? - Lincoln Allison remembers lessons at Lancaster Royal Grammar School and University College, Oxford
      Lucky guy (as was I). But it's not the same now, is it?

      Felipe Fernández-Armesto recalls a golden age when learning was treasured (Times Higher Education)
      I thought I was heterodox, sometimes! (http://www.doceo.co.uk/heterodoxy/education.htm)

      The Stanford Education Experiment Could Change Higher Learning Forever (Wired.com) On the free open-access courses on Artificial Intelligence.

      Should Effort Count? Students Certainly Think So (Faculty Focus) --in terms of marks.

      To flip or not to flip? The pros and cons of flipped classes: new material presented on-line, class time spent on exercises and practice.

      Reading the dictionary (Joi Ito's Web) 'Shouldn't we be looking at the Internet as an amazing network enabling "The Power of Pull" and be empowering kids to learn through building things together rather than assessing their ability to complete courses and produce the right "answers"? (Comments also interesting.)

      More generally:

      The Knight’s Song, or What is a [scientific] theory? (Evolving Thoughts) Not as hard going as it first appears!

      (Mark) Twain on inference about the past (Evolving Thoughts)

      Age and wisdom: Older and wiser? (The Economist) Americans get wiser with age. Japanese are wise from the start

      The human advantage: middle age (The Washington Post) A cheering take on our middle years!

      What the 'limits of DNA' story reveals about the challenges of science journalism in the 'big data' age (The Last Word On Nothing)

      John Cleese on the 5 Factors to Make Your Life More Creative (Brain Pickings)

      Are humans naturally religious? (First Things)

      The Chocolate-and-Radish Experiment That Birthed the Modern Conception of Willpower (The Atlantic)
      Posted by at No comments:

      10 April 2012

      On "reflective practice" as a threshold concept

      This is in danger of becoming a little arcane! Is "reflection" (in the Schon/Kolb/Boud/Moon and co. sense) a threshold concept? And what difference does it make to anything if it is?

      The story so far--starting a couple of years ago, I've been doing occasional presentations, in many cases in association with Peter Hadfield, on re-evaluating the notion of "reflective practice" and its usefulness a quarter-century after it was promoted in a rather different professional culture. The other day I was invited to do a session in the Open Lecture series on learning and teaching at the University of Greenwich, and Peter was able to come too. So it was that we went through the lecture on the train, on my laptop, and he posed the question whether reflective practice is a threshold concept.

      Yes of course! ...Well, no. ...But it ought to be! It matters because this idea tests some of the supposed criteria for threshold concepts (although it has to be granted that those for TCs are tighter than those for RP--excuse the initials but they should be obvious by now). And certainly, the origins of TCs were found in the "Ways of Thinking and Practising" (WTP) which Entwistle identified. In many disciplines, RP is regarded as central to their WTP, and it is certainly intended/assumed to transform practitioners' approaches to their practice. It is considered not as an additional item of knowledge, but as a transformational perspective or frame of reference.

      In practice it doesn't work like that. And it so happened that the other day I did a teaching observation of a student actually teaching a module on "reflective practice" on the final year of an undergrad programme in... social work/ nursing/youth work/ community development/ informal education (It was one or more of those--I'm being coy for ethical reasons, but it doesn't affect the argument: and this has nothing to do with the performance of the student--we had interesting issues to discuss which concerned context and background and style, but her core performance was fine*).

      RP is regarded as a theme (perspective) running through the entire programme, but the existence of a discrete module accords it the status of (merely) additional knowledge. I can see how it happens, of course. As I cited in the lecture:
      Whilst many authors [...] have expressed doubts regarding the wisdom of formally assessing reflection, the currently favoured 'alignment' model of curriculum development ([...] makes it extremely difficult to make the development of reflective skills a core element of the curriculum design without it being overtly assessed. Despite my own reservations regarding the ethics
      of assessing student reflections [...] even I would struggle to envision a curriculum where reflection was central to the learning strategy but absent in the design of assessment.
      (Hargreaves 2010:91)

      Two aspects in particular struck me:

      Making it into a discrete module had transformed it; it was no longer the same activity. As Reflection (I'll capitalise it in its RP sense so I can use the more general term uncapitalised) was originally recommended by Boud and Schon et al, it was a characteristic of the accomplished practitioner, and effectively something which he or she could not help doing. In that form it is a reactive process, not a proactive one. It is something which takes place because one is that kind of practitioner, not in order to become one.

      For the Greenwich lecture, I looked up the phrase "reflective practice" on Ngram: I noted that when I published my first book, on Professional Supervision in Group Care in 1986 (written of course in 84-85), the term "reflect / reflection" appeared only once in the index, and then in a specialised context. The ngram graph seems to show that the idea came out of nowhere in the 'eighties.
      So if we are to believe one proponent of RP, practice prior to its identification could only be, "at best
      uninformed, and at worst ineffective, prejudiced and constraining" (Hillier 2002: p.xi) which is self-evident nonsense. It's clearly a case of  Jourdain's syndrome--the Moliere character who discovered he had been speaking prose all his life without knowing it. Identification and labelling is the first step in the process of systematisation, and that is what the Schon and co. started to do. Indeed, Schon was clear about his intentions. Nevertheless, that inevitably started a process of change.

      As I explored very briefly in discussing Bruner's cone of experience, in order to talk about an "enactive" experience, one has to step back and enter the "iconic" or even the "symbolic" zone--and such a step is an act of abstraction. So the act of reflection is in itself an abstraction and to talk about reflection is even more of an abstraction. And after talking about it for 25 years, and codifying it and evaluating it and representing it within Learning Outcomes and Module Descriptors--it has become something different.

      In particular it has become (at least on this course), a label for an approach to action-planning and target-setting, where despite the rhetoric of self-directed learning, the action and targets need to fit within the pre-determined framework of the curriculum. It's putting it too strongly to suggest any element of "indoctrination", but as Hargreaves points out above, when you start to assess Reflection, you have to go down that route.

      Perhaps that is not too bad an idea, really. There's nothing wrong with becoming proficient in planning and meeting targets. In current working systems and cultures, indeed, those skills may be more use than Reflection, but I did not get the impression that this position has been arrived at consciously and deliberately (let alone Reflectively).

      The students had learned it. Only problem, I (and my student) were no longer very sure what "it" was. And I did not know whether the students believed in it, or were just going through the motions, because they (at the end of three years, after all) knew very well how to play the game. It's nothing new: I've just found this, which I wrote in 1998:
      An external examiner on the social work course commented at the assessment board that although he was satisfied by the way in which anti-racist and anti-discriminatory practice was being taught on the programme, he saw little evidence from the work he had read, that students were actually using any of these principles, despite the fact that they were explicitly marked on the extent to which they were incorporated in all assignments. Discussion with some of the students after the final assessment revealed that for at least some of them:

      a)  The teaching represented a form of political correctness to which they knew they were supposed to pay lip-service in order to pass the course, but they did not actually believe that it was any use as a principle of practice.

      b)  More than that, several regarded it as inimical to their effective practice, because of the self-consciousness it engendered.

      Reflection is no different.

      But now it doesn't sound much like a concept, "akin to a portal, opening up a new and previously inaccessible way of thinking about something. It represents a transformed way of understanding, or interpreting, or viewing something without which the learner cannot progress." (Meyer and Land, 2006:3)

      And yet it started off that way. It's just that in its journey from being a necessary feature of a professional's manner of reacting to her practice, to this compliant choreographed process of planning to address external targets, it seems to have had a run-in with the Dementors and lost all its energy. It continues to exist as ritual knowledge, which has a “routine and rather meaningless character” (Perkins, 1999, p. 8**). I know it, but I no longer know why I know it, or what it is really for.

      Instead of opening doors, it is closing them.

      I am coming to realise, rather belatedly, that the threshold quality of a threshold concept, and the transformative potency of reflection, do not lie in their content as such. It lies in the manner in which learners are prepared to let them permeate and act on other areas of knowledge. And clearly caging these ideas up in discrete modular boxes and then treating them as mere additional gobbets of knowledge to be assimilated is guaranteed to emasculate them; we need to take the risks inherent in letting students engage with accommodating to them.

      * "Fine" does not do her justice, but we do not grade observations, and so I'm not going to use a term in Ofsted's vocabulary. Sorry if that is meaningless to anyone, but you're not missing anything!

      ** Many thanks to David Stone for finding this accessible source!

      References

      Hargreaves J (2010) "Voices from the past: professional discourse and reflective practice" in Bradbury H, Frost N, Kilminster S and Zukas M (eds) (2010) Beyond Reflective Practice; new approaches to professional lifelong learning Abingdon; Routledge

      Moliere (Poquelin, J-B) (1670) Le Bourgeois Gentilhomme (play)

      Posted by at No comments:

      09 April 2012

      Items to share: 7 April 12 (late)

       Education focus
      Other items
       And an Easter present:
      • Dropbox just got even better! Disclosure, for everyone who signs up on my recommendation I get more free space--but then so can you when you recommend it in turn. And I don't know if it works from a blog. Who cares? This is the easiest conceivable FREE "cloud backup solution". Yuck. Did I write that? (As long as you are not a paranoid security nut.)
      You can also browse this "stack" here.
      Posted by at No comments:
      Subscribe to: Posts (Atom)