On the Italian restaurant problem

Went out for a very pleasant evening yesterday with the finishing students. They clearly don't read the website--I drop heavy hints about my preferences for Greek food, although to be fair there's not much about in this town, currently.

But I was stupid enough to be surprised that when I suggested at the outset that we work out a policy for dealing with the bill, I wasn't exactly shouted down, but met with a wave of denial. "We'll just divide it up equally! It'll be OK!" I did suggest that wasn't necessarily the case and that this was a classic problem in game theory, but they were all sober and enjoying the oxytocin flush of relaxing with friends.

I  made my excuses and left quite early (for good reason, because I still had a paper to finish for a deadline today), just as the debate was beginning about who was planning to have dessert and who had drunk what... (I just put down cash worth more than I had eaten and drunk, and left it on the table).

But it did remind me of the eurozone crisis...

On Bloomsday

Today is the centenary of the publication of James Joyce's Ulysses. Or at least the 108th anniversary of the day (16 June 1904 a.k.a. "Bloomsday") which it supposedly chronicles.

Readings have interrupted regular scheduling on Radio 4 (which is tantamount to lese-majeste)

[Full disclosure: I am not, and have no relationship with, the James S Atherton, Joycean scholar and author of The Books at the Wake. But in my first days at university, in 1963, I did look up my own name in the card index--and found that reference. I found it on the shelf, and glanced in it for at least two minutes, before putting it back and beginning to wonder about the point of literary scholarship--beyond, of course, of its intrinsic delight, like that of solving a cryptic crossword puzzle.]

But I have read Joyce. My copy of the 1969 Penguin edition of Ulysses is beside me as I write, with pencil marks up to page 687. I confess that unlike my namesake, I never tackled the Wake.

So? Skip to the conclusion. Joyce invented literary bling.

I wanted to finish this post on Bloomsday, but I've missed that deadline (at least in BST terms). So, in short;

Ulysses is an  amazing tour de force. But the harder it tries the more contrived it gets.

Items to share: 16 June

Education focus

• Inklings: Please Rate the Overall Likability of the Instructor On evaluation instruments

• A Don's Life: What poems did you learn at school?

• A Master List of 500 Free Courses From Great Universities (Open Culture) It's just a list, but useful nonetheless

Other business

• How mapping neurons could reveal how experiences affect mental wiring (Wired UK) On the connectome and how it relates to the genome and the physical structure of the brain.

• Is War Inevitable? (Discover Magazine)  E O Wilson reckons it's part of human nature--just to cheer us all up! 

• Smart people are especially prone to stupid mistakes - Boing Boing  Jonah Lehrer on Daniel Kahneman 

• Separated by a common language: introducing yourself  In Britain and in the US--when do you mention your name?

• Deirdre N. McCloskey: Happyism (The New Republic) But how do you measure it?

• Dilbert makes up some numbers

• (Language Log) Young dialect mimic with a future? (Video)

• 'Sexual depravity' of penguins that Antarctic scientist dared not reveal (The Observer)  "March of the Penguins" was never like this... 

• First Contact (Colossal) Child meets manatee

• “Are you experienced?” (More or Less Bunk)  Guest post on what students actually learn on-line--and reflections on hunting for a first teaching job (US).

On "assessment for learning"; the petrifaction of a process.

I've been reading with interest a detailed blog on ESOL by Sam Shepherd, and in particular a point from a (fairly) recent post:

My personal experiment here was around learning outcomes, the value of “sharing” them I have always wondered about. And I have to be honest and say that for this lesson  I might as well have just whistled the learning outcomes at the beginning, for all it meant to the learners. [...] But the whole learning outcomes thing, really, not impressed. For this group it just didn’t make sense. [...] But I want to emphasise that it was just one lesson: and in the spirit of research and experimentation, I am going to persist and see how we get on.

from Playing it Straight « Sam Shepherd's Blog 

This is an issue which crops up regularly in my own classes with people already teaching in post-compulsory education taking a part-time teaching qualification, and I usually come in for some stick from "students" who tell me they are obliged by college policy to announce the learning outcomes of every session at the start, and one very upset man told me that the only reason Ofsted had not deemed his class "outstanding" was because he had been too nervous to recite them. My usual response is to say that the only decent reason I can see for doing it is that it's a bit of ritual which you can use to communicate, "this is where the class really starts", but there are lots of ways of doing that. (I used to illustrate this by reciting at least one of the module outcomes in Anglican chant--I gave up on that when it became clear that it meant nothing to most students, and I don't have the voice for it... Pity.) 

So what is my objection? It is certainly not about wanting to confuse the students (although sometimes that is legitimate and effective; surprise can be an effective teaching tool). I've no objection to outlining what we are going to be doing, or "looking at" in this session, or giving an idea of about how much time we are likely to be spending on something; especially when it is a two-hour session, they like to know when the comfort-break is likely to come up. 

Students don't understand them--why should they?

My first objection is precisely what Shepherd identifies. Learning Outcomes (LOs) are written in teacher-speak gobbledygook. Even my students frequently don't understand them--and at the beginning of the module we do actually study not only what the outcomes say but also how they are written. They were difficult enough in the days when my colleagues and I wrote them, but since they have been laid down centrally by the now-defunct and unlamented "Lifelong Learning UK" they are well-nigh impossible. Granted that these are at the module level rather than the session level, here are two examples from the 2007 LLUK specification:

Understand the application of theories and principles of learning and communication to inclusive practice.
Understand how to apply theories and principles of learning and communication in planning and enabling inclusive learning.

(Shepherd's outcomes are SMART; one thing anyone formulating LOs learns from the outset is that "understand" is not an acceptable verb for an LO. See here and here for my heretical take on that. But the guardians of the Gradgrindian fog at LLUK used it all the time.) It's not clear what they mean. It's not clear how you show their LOs have been met. And there are two of them, and it's not clear what the difference is between them! They add nothing. Time spent on introducing them could be spent more profitably on doing practically anything else.

Such as using an Advance Organiser. OK, the evidence for its effectiveness as a teaching device is not very high, but it's a respectable tool to have in the box, and it takes very little effort to use it. AOs (let's carry on with this silly game) refer to the students' experience, so they engage them, and set them up for the session. LOs on the other hand, distance them, and assure them that teaching is about to be done to them, so that they will emerge at the end of the session having been reliably processed through the sausage-machine.

Where's the evidence? Substitution of the sign for the signified.

My second objection is that there is no evidence that this is good practice. (This is a more convoluted and longer argument.)

A few months ago a correspondent got in touch about my Heterodoxy pages;

Very interested in your article re reflective practice, much of which can, in my opinion, be attributed to today's idea which cannot be questioned - 'Assessment for Learning'.

It seems to me that the impact of this seemingly effective form of practice has not born fruit [...] and that opportunities for practicing a particular skill or trying something more difficult have been replaced by pointless navel gazing. 

I'd like to see you take a swipe at AfL.

(Link above inserted). I'm not "taking a swipe" just because CJ requested it (but apologies to him/her for taking nine months to get into this), but because this kind of issue is a classic case of the ritual ossification of what he/she describes as "this seemingly effective form of practice"--concentrating on a relatively arbitrary sign rather than on the substance towards which it is supposed to point. And to explore it is also to explore the fate of many other educational innovations...

So I've done a lot of reading and talked to a lot of people--no, I didn't "do a literature review" and "interview respondents"--I live in the real world nowadays! And I'll spare you (most of) the references.

I am a fan of assessment for learning. The real McCoy, though, not the institutionalised tick-box version being peddled by Ofsted and their intimidated toadies. But how did we get the version we have now?

How did the principle of maximising feedback to learners, and thereby getting feedback from them, become this sterile ritual observance of reciting LOs, testing in every class, recapitulating the pre-determined learning points, and then writing it all up as if it constituted evidence of something?

There are several complementary perspectives on the process, and they all tend to push in the same direction...

The (proximate) origins of AfL

To cut through a lot of the origins of the idea, we can say that while it was enshrined in a Department of Education and Science report in 1988, it was championed by Black and Wiliam (1998), although they of course acknowledged that it was what good teachers had been doing for decades or even centuries--certainly at least long enough for them to amass 250 studies for a meta-analysis. They note:

"Typical effect sizes of the formative assessment experiments were between 0.4 and 0.7. These
effect sizes are larger than most of those found for educational interventions." (1998 p.3)

And Hattie's much larger meta-analysis, published 2010, confirms the figures (my take on it is here, including a note on what "effect-sizes" are).

But were they talking about what is commonly understood today as "Assessment for Learning"? It's clear from re-reading Black and Wiliam that they envisage a long-term cultural shift in classrooms. They repeat several times that it will be a slow process. It will be characterised by introducing measures which enable teachers to get a clearer picture of where their students are, in relation to particular topics and skills within a subject area, drawing on, inter alia, student self-assessment. Open communication is the key.

But Black and Wiliam are too experienced to believe that teachers can re-adjust simply on the basis of principles--they need concrete practices to follow:

"Teachers will not take up ideas that sound attractive, no matter how extensive the research base, if the ideas are presented as general principles that leave the task of translating them into everyday practice entirely up to the teachers. Their classroom lives are too busy and too fragile for all but an outstanding few to undertake such work. What teachers need is a variety of living examples of implementation, as practiced by teachers with whom they can identify and from whom they can derive the confidence that they can do better. They need to see examples of what doing better means in practice." (1998 p.10)

That is entirely understandable. I don't want to over-simplify the convoluted processes of change, particularly in education, but I suspect that nevertheless one unintended consequence of such thinking, exemplified in professional development programmes at one end of the scale, and in Ofsted inspection frameworks at the other, is to concentrate on the outward and visible signs, rather than the more elusive and protean processes of cultural shift. (We also need to see this process in a political and historical context, of obsessional micro-management of public services under the Labour government...)

We are also now looking at a second or even third generation of teachers since the desirability of AfL became the conventional wisdom--and quite possibly because of the way these teachers have themselves been taught--they have simply accepted all these signs and rituals as given, with no realisation of the rationale or principles underlying them. Reflective practice (from ten years earlier) has similarly become a matter of going through the motions without knowing why.

The managerial argument has been: If we are looking for a more open flow of information about understanding and achievement within the classroom, how do we know that it is happening? We need it to be reported. We need ILPs (Individual Learning Plans) to be set up and recorded on forms, and in order to ensure that the information is hard enough, that needs to be based on testing.

But! Given the time it takes to set up and implement and record such tests, and their inherent bias towards summative assessment, and Black and Wiliam's point above that "[Teachers'] classroom lives are too busy and too fragile for all but an outstanding few to undertake such work." --it's not surprising that their point that:

For assessment to function formatively, the results have to be used to adjust teaching and learning; thus a significant aspect of any program will be the ways in which teachers make these adjustments.  (1998 p.4, emphasis added.)

has in many cases not been met. It's just too much effort to complete the circle, and as with any cyclical model, there is nothing more disappointing that trying again and again to start something which continually peters out. (Think of trying to start an engine with a pull-cord--or even a starting-handle--when it won't "catch".)

The social construction of learning and its institutions.

Nevertheless, there is a venerable descriptive cyclical model in play here--see Berger and Luckmann (1967). (It is consistent with Wenger (1998): the cycle can start at any point.)

Put very crudely, it represents how we put ideas out into the world (externalise them). We do that by talking, writing, making objects. Ideas become social "things" (reified), such as institutions, laws, languages, art, etc. but of course changed in the process; and we in turn internalise them, which changes us and the next ideas we try to externalise. It is the reification of AfL which has lost the point (or if you like, the spirit) of the whole thing, and this cycle carries on independently of the will of the teachers. (There are fairly clear parallels with this model.)

What happens in the process of reification is that the idea, for want of a better word, has to accommodate to all the other reified ideas out there if it is to survive, and it is this adapted version which we (and the next generation) internalise. Thus, any social institution (and any approach to teaching is a social "institution" in a broad sense) gets knocked into shape by the dominant political, economic, technological and other powerful cultural factors of the day.

This is not merely an "academic" digression. What counts as "learning"--especially in the sense of "what is supposed to go on in education institutions"--is socially constructed. And so, therefore, is its assessment. At the level of universities, Stephan Collini writes (2012 ch.5) on  the substitution of metrics and superficial proxies for valid evaluative tools, because the rhetoric of "a public good" is no longer recognised as a reason for doing anything unless it can also be justified in primarily economic or other instrumental terms. (He writes as an apologist for the humanities.)

And he notes that inspection and evaluation of processes of delivery have taken over from any engagement with disciplines and content themselves, because of the value-questions which are inevitably encountered through such an engagement.

So, much as Ofsted and other quality assurance institutions would wish us to believe that their work is value-neutral, and that an "outstanding" class is better than a merely "good" one on any terms, that is not the case. (I grant there are some practices clearly to be avoided in teaching, so there is likely to be a more substantial consensus on distinguishing between "outstanding" and "failing".)

In other words, what kind of learning is "assessment for learning" about? The choice of what kind of assessment to focus on is critical in determining what kind of learning is promoted. In assessing a piece of written work, what counts most? Is it spelling and grammar? Elegance of expression? Originality of ideas? Arguments based on evidence? Underlying research? All of them, of course, but emphasising some beyond others depending on the age and stage of the learner, the subject matter of the essay, and so on. Choosing the "correct" criteria, choosing what to pay attention to and what to ignore are all based on those socially constructed values.

(The issues are of course generally much more straightforward in STEM* subjects.)

Accept only substitutes!

Frankly, all that is too hard to measure, so the system contents itself with these relatively trivial, perhaps harmless (other than their cost in time, effort and morale, and distortion of priorities) proxies which at one time might have pointed towards the quality of classroom communication, but have long since lost the connection. And perhaps that is all one can do in and inspection of just a few days. But it's not surprising if institutions focus on compliance with the letter rather than the spirit of the approach. (I'm giving inspectors, principals and other managers the benefit of the doubt here; my suspicion is that they no longer realise that there is a problem.

Just to round off, though, there is of course a quis custodiet question about these guardians of teaching standards: here's an interesting take on that, and more generally here is one on all those who presume to diagnose, predict and treat within ill-defined systems, exhibiting "deluded self-limiting prescriptivism".  Thanks to David Stone for the link.

But see here for a post on an LSE blog which rather grudgingly concedes some validity to Ofsted's judgements.

Update 27 July: Dylan Wiliam himself is now pointing out how misunderstood the AfL is, and how little implemented (TES, 13 July)

---

*Science, Technology, Engineering and Mathematics

Berger P L and Luckmann T (1967) The Social Construction of Reality London; Penguin (And Peter Berger is still blogging--largely about the social practice of religion--here.)

Collini S (2012) What Are Universities For? London; Penguin (Review here)

Items to share: 9 June

Education Focus

• How should mathematics be taught to non-mathematicians? (Gowers's Weblog)  A long and engaging post on a sixth-form "Use of Mathematics" curriculum--but in a good way.

• The Neurocritic: G r e a t e r / l e t t e r / s p a c i n g / helps reading in dyslexia

• The Benefits of Making It Harder to Learn  (The Chronicle of Higher Education)
  Not only a matter of "cognitive disfluency" but also of a degree of cognitive dissonance, perhaps. The comments are worth reading, too.

• A Technological Cloud Hangs Over Higher Education (The Chronicle of Higher Education) A bit of a jeremiad, but principally about how technology is mediating the encounter with subject-matter (particularly in the sciences) and hence distancing students from it. Some interesting links from the comments on the PowerPoint trope.

• Will Your Citations Pass or Fail?  (The Chronicle of Higher Education) Since it's the season for examiners' ritual complaints about the quality of referencing, here's a glimpse of what's involved at a professional/obsessional level.

Other Business

• The history of the paper clip: It was invented in 1899. It hasn't been improved upon since. (Slate Magazine)

• Haslam (2012) Toilet psychology  Redressing a history of neglect...

• Queen Elizabeth's Diamond Jubilee (The Atlantic) Picture gallery (the first one is brilliant).

• Mrs Windsor, anyone? (FT.com) on the conspicuous failure of republicanism to make any progress in the UK.

• Urbanscreen Light: Sydney Opera House (YouTube) The projection onto Buck House at the Jubilee concert had nothing on this.

• Leonardo of the Deep - "Gas pods" (small bumps on the roof of a car) are claimed to increase fuel efficiency by mimicking the effects of similar bumps occurring naturally on whales and lobsters.

• Atul Gawande: Failure and Rescue (The New Yorker) In medicine, it's not the ability to forestall failure which counts in survival rates, it's the capacity to respond and recover when it's happened.

Items to share: 2 June

• Sixty years of progress? (Simon Jenkins: Prospect) A token recognition of the Jubilee 

• Queueing theory: What people hate most about waiting in line. (Slate Magazine)

• Richard Dare: The Awfulness of Classical Music Explained Or at least the pretentiousness of the culture of devotees (comments worth reading, too). 

• Steven Pinker on the false fronts in the language wars. - Slate Magazine Prescriptivists vs. descriptivists.

• Derek Thompson: The New Economics of Happiness (The Atlantic)  And the sequel The 10 Things Economics Can Tell Us About Happiness.

• Monday 31 May 1669 (Pepys' Diary) The final entry. 

• As our languages make clear, context is a human specialty (LA Times)  "Research shows that humans are remarkably good at divining one another's intentions using context — a skill that computers have not yet mastered." Just a taster.

• Daniel Kahneman: Thinking That We Know (3quarksdaily) Almost as good as reading his book.

• The Perfected Self - The Atlantic  The resurgence of behaviour modification in the era of the app. (particularly for weight loss). 

• Noise and Signal — Nassim Taleb (Farnam Street) Why it is a mistake to obsess about indicators

• The Moral Molecule  (WSJ.com) The role of oxytocin in fostering reciprocity and trust (book review).