On surrogates and assessment

I'm starting off with two observations about quite different things.

The first is this one from Ben Goldacre's brilliant "Bad Science" blog at the Guardian. His topic concerns a press release about a potential new treatment for Duchenne's Muscular Dystrophy.

"...this story is also a reminder that we should always be cautious with "surrogate" outcomes. The biological change measured was important, and good grounds for optimism, because it shows the treatment is doing what it should in the body. But things that work in theory do not always work in practice, and while a measurable biological indicator is a hint something is working, such outcomes can often be misleading."

And later...

"...improvements on surrogate biological outcomes that can be measured in the body are a strong hint that something works – and I hope this new DMD treatment does turn out to be effective – but even in the most well-established surrogate measures, and drugs, these endpoints can turn out to be misleading."

A fairly basic point, of course, but it did set me thinking about how the extent to which we have become obsessed with measurement in many fields, including teaching and learning, had led to increasing reliance on fairly dubious surrogates.

And then I came across this commentary on Standard and Poor's revision of the the US credit rating:

"[Ratings agencies] ..are human enterprises, fallible institutions—and like other institutions, they have procedures, interests, and histories. Their records deserve inspection. In the scientific spirit, in the spirit of show me, they deserve scrutiny."

A credit rating is a complex construct (I presume). Since it is supposed to have predictive value (other than merely being part of a self-fulfilling prophecy), it must be put together from a raft of surrogate measures, presumably of directly observable factors which co-vary with an institutions credit-worthiness. But it is only as good as the choice of those surrogates*.

Which led to some general thoughts in relation to education.

Today is the day when GCSE results are published (the exams taken at age 16 by practically all pupils in the UK). The press stories are predictable, suggesting grade inflation and the exams being dumbed-down. (Or of course if the pass-rates were not an improvement on last year, there would be jeremiads about further decline in educational standards...) The press discussion will not be sophisticated, but it will at least acknowledge what the politicians and the educational establishment will deny, namely that the examinations are not realistic proxies for educational achievement.

This is leaving aside the issue of the tail wagging the dog, of "teaching to the test" without ever asking whether the test is valid or reliable. Beyond that, the logistics and practicalities of mass assessment distort the process, and it has ever been thus. When Liam Hudson (1967) discussed convergent and divergent thinking styles, he noted that convergent thinking was privileged in school at least in part because its testing could be standardised.

But these artificial surrogate assessments are increasingly separating the formal educational system from the "real world", particularly that of communities of practice. This is not an original observation; while I'm on "golden oldies", I'll refer to Becker's wonderful 1972 paper School is a Lousy Place to Learn Anything In, which is based inter alia on a similar argument.

It is out of an awareness of the intrinsic limitations of such surrogacy that a course on teaching with which I have long been involved has attempted to develop a more authentic assessment strategy. Of course, teaching courses have always routinely involved direct observation of teaching, but not everything is amenable to direct observation. The traditional solution on most** other courses has been set assignments; our course moved away from that to negotiated submissions based on a learning contract. Learning outcomes are specified and students decide, in consultation with a tutor, of course, what evidence they will submit to demonstrate that the outcomes have been met. This is a step closer to reality, but of course only insofar as the specified learning outcomes correspond to the real world.

The course has just been internally reviewed for routine reasons, and it is apparent that the bureaucrats hate the assessment scheme. Work is not graded, for example. The scheme is not suited to anonymous submission, because the students are talking about their own practice and work setting (it is an in-service course). Not all work is suited to electronic submission via Turnitin.... the list of complaints goes on.

The real problem is that validity, reliability and fairness--the traditional requirements of an assessment scheme--are now subordinated to standardisation, administrative convenience, and security***.

These are considerations for the legitimation of surrogates and proxies--the same kind of consideration as applies to the regulation of second or third-order derived financial instruments which no longer bear any relation to buying and selling stuff which is any actual use.

---

* I am not relying entirely on a single blog-post here! See also Dan Gardner's excellent and accessible Future Babble: Why expert predictions fail and why we believe them anyway. (London; Virgin Books, 2011) It's a great corrective to all the doom and gloom surrounding us. Incidentally, he draws a lot on the work of Philip Tetlock, the subject of this interview by Jonah Lehrer in Wired.

** Most but not all, our approach owes much to work at the University of Huddersfield, particularly in the early '90s.

*** Security in the sense of not being vulnerable to plagiarism, although the emphasis on discussion of one's own practice and production of examples and resources means that the approach is fairly protected in any case.

Becker H (1972) “School is a Lousy Place to Learn Anything In” American Behavioral Scientist (1972):85-105, reproduced in R G Burgess (ed.) (1995) Howard Becker on Education Buckingham: OU Press

(Update later today: Many thanks to David Stone, who writes; "I was happy to discover that my institutional subscription gave me access to the original Becker article. Just in case others should be as lucky, here is the DOI link:
http://dx.doi.org/10.1177/000276427201600109 ")

Hudson L (1967) Contrary Imaginations; a psychological study of the English Schoolboy Harmondsworth: Penguin

On teaching objects, tools, and frames

This post is prompted by interesting points made by Bruno Setola in a substantial post on his blog Gamification.nu, which is well worth reading (and I'm not just saying that because he makes some kind remarks about my sites). The relevant piece is headed "Levelling Up".

His post is packed with ideas and efforts to synthesise them into an approach to teaching Cross-Media Communication, amongst which he finds Threshold Concepts to be a very useful tool. When I first read his thoughts, though, I thought he hadn't really got the idea; he was emphasising the acquisition of a frame of reference rather than the actual content of the concepts.

However, in the course of the discussion he refers to a keynote at the Third Biennial Threshold Concepts Symposium in Sydney last year, given by David Perkins. Unfortunately I couldn't get to that meeting, but that made me even keener to watch the video, below. (Note that it is almost an hour long, but well worth the time. You might find it helpful to have the .pdf file of the full set of slides open so that you can switch to them, because the camera does not dwell on the screen.)



In essence, Perkins is now talking about threshold experiences rather than concepts, and explores the epistemic shifts which take place as they develop from object to tool to frame. (Hence the shift of emphasis in Bruno's account.)

Selectively, because there's a lot in the address, my attention was drawn to Perkins' thoughts about what is involved in managing these shifts and teaching material to serve as a tool rather than an object. (For more detail on the content of the tables, do watch the video; these notes are only about the gist of some parts which strike me on the basis of current interests.)

I was reminded of a couple of pretty poor classes I've observed this year, commented on here and here. In both cases the problem was not really with the actual teaching, but with the syllabus, and the way in which it treats each item of learning as a gobbet of what Perkins elsewhere calls "inert knowledge". Each item was to be stored in the students' brains, to be taken out and shown when called for, but there was no sense of doing anything with it. The academic level Perkins was talking about was higher than the classes I had observed, but he discussed how using the approaches in the left hand column of the table below tend to promote learning of material as a set of concepts, rather than tools to work with.

Object role	Tool role
Key features, 'toy' applications	Fully developed applications
Rival academic concepts	Rival tacit operative concepts
Comparison and critique	Select among several and apply

(Do not be tempted, incidentally, to see "Object" as merely equivalent to the lower levels of Bloom in the cognitive domain, and "Tool" as signifying applying the material. It is possible to teach at a very advanced level, still working with objects--and indeed as Perkins notes, that is often entirely appropriate, when the material is a "destination" rather than a "route"*, an end in itself or object of scholarship rather than something which earns its keep by serving as a tool.)

Tool role	Frame role
Several concepts	One concept
Somewhat closed problems	Somewhat open problems
Abundant time	Low-stress real time
Solo or large group	Small group, rapid turns

Tools have specific tasks, and need to be selected appropriately, and although they may become "extensions of the body" in practical tasks, they are nevertheless also objects which can be studied and refined (Setola discusses the "extensions" point in his post).

The third way in which ideas/knowledge/concepts etc. may be used is as a frame. A frame is an idea through which one sees stuff; a tool is an idea with which one works; an object is an idea one knows about. The critical difference is that by default a frame is part of oneself. It is not experienced as something other; indeed it may be very difficult to step outside one's habitual way of seeing things and take "my habitual way of seeing things" as an object of study.

Frames are what reveal the "inner game" of topics of study, for better or for worse, as Perkins (2009: ch.5) discusses. It needs to be emphasised that frames are not "superior" forms of knowledge (or skill, or values) to tools or objects. As Perkins' use of the term "role" suggests, it is a matter of what job you want this knowledge to do, and so how you teach it.

Bruno's concern is principally about how these transitions might be managed and "taught". Scaffolding, for example, with its implications of incremental development, no longer works when one reaches a discontinuity, such as this kind of epistemic shift between object, tool, and frame.

In short, I'm not sure it can reliably be managed. That is the nature of a threshold experience--the liminality, uncertainty, and indeed risk (although I don't want to over-dramatise) of how experience is re-organised by a new idea.

On the other hand, does it need to be managed? Does trying to manage it make it more likely to happen? Or is it wasted effort? But that's a question which might actually succumb to ingenious empirical research...

I'm reminded of Gestalt shifts in perception. But also of Ramsey (1967). I remember, almost 45 years ago, listening to Ian Ramsey delivering a guest lecture at Sussex on religious language--he must have been speaking about work in progress, because this was before 1967. He spoke about parallelisms in the psalms (I'm not going to digress that far) and the analogy of the polygon and the circle. Start with the simplest regular polygon--an equilateral triangle. Add a side = a square. Go on and on and the figure gets more and more circular, until at some point it is indistinguishable from a circle, and so it is a circle***.

And I hazily remember some basic physics from even longer ago! I seem to remember that phase transitions (such as ice melting, or water boiling) require an energy premium (not the correct phrase, I know)... A catalyst may help, chemically, but the basic transition is the product of "more of the same". It's just that in teaching, the "more of the same" needs to be about the epistemic status one is aiming at, not that which one is emerging from.

These properties are emergent...

This kind of thinking underpins Perkins (2009), where he is concerned about developing appropriate approaches to teaching to promote learning for understanding. (It's a term he is quite comfortable with, and discusses at some length on p.48 ff.)

The book is to a certain extent a reflection on his experience of learning to play baseball as a child; he found it easy, he argues, because he was exposed to the whole game. He practiced the components, of course, but he knew where they all fitted in and he saw them in context.

In formal education, on the other hand, there is in many cases no overall introduction to the whole game of a subject or discipline. Instead, each element of the knowledge base and skill set is likely to be introduced separately, and in isolation. Clearly this inhibits understanding of how it fits together; he calls this unfortunate curriculum strategy "elementitis".

And even if the whole is introduced, it is often discussed at a distance. In baseball (or other sport, or music, or language learning) newcomers get to play, from very early on. In education, the subject is described rather than participated in; he calls this aboutitis. (Perkins does address the question how the "whole game" can be introduced when it is enormous--such as mathematics, or science. He argues that just as baseball is introduced through a simplified form--simpler even than Little League--it is possible to develop an appropriate "junior" form of the game which students, of whatever age, can grasp.)

Back to practice. The sessions I observed were--inappropriately--focused on learning objects rather than tools, still less frames. But that was what the syllabus required. The mechanistic fragmentation of the whole into learning units and outcomes and assessment criteria effectively precluded any other approach. Moreover, the "whole game" was almost inconceivable. As the Wolf report suggests--although one could have wished for more detail--the arbitrary assemblage of  "competences" into courses, does not make for coherent and teachable programmes.

I may be critical of my students' application and implementation of their learning, but seen through this frame (or "lens" as Brookfield puts it) it is not clear how they can get better. Bottom line: if you are forced to teach a whole which does not make sense, the parts can't make sense either.

So that is what I did on my holidays.

7/10. You need to get out more.
Teacher

Notes/Asides

• I agreed with practically all of Perkins' book. I also found it highly readable, in part because does not let his references interrupt his flow--the evidence is there, but it is in the very accessible notes at the end.
  
• Indeed, I recognise much of his approach in mine, although he is more rigorous than me on "working on the hard parts" (ch.3), which is my failing. I would promise to do better next time, but at my age, there may not be a next time!
  
• His chapter on the "inner game" is a classic (ch.5), particularly on the hidden curriculum embodied in the physical and logistical elements of the classroom**.
  
• I'm being presumptuous here, but he does divide the basic idea, of concentrating on the whole, into seven principles, each of which has several aspects, each of which can in turn generate several strategies or exercises... Of course, if you approach the material as a tool-kit or even a frame, that's good. But, although I say it myself, I'm very good at that. I try to employ it all the time, but I did find I could not sustain the necessary frame all the way through the book. Perodically, I did lapse into thinking, "Do I have to learn all these particular techniques?" (Object orientation)
  
• P. writes in a US context. Syllabi in the UK (particularly in vocational, professional and further education), are much more prescribed and regulated. Frequently very badly. With very little understanding on the part of awarding and validating bodies about what it is like to study on their programmes. (See here on who writes syllabi, if you've not been there already.

* my terms rather than Perkins'.

** This excerpt concerns the explication or deconstruction of the chair desk (chair with flap-over writing surface) based on Luttrell (2004) (full source on Perkins p.238; author referred to here as "Wendy")

A chair and a desk are fused into the same convenient unit, the desk component a rather small platform upon which the student can rest a book or a notepad. Books usually can be stored under the seat. Wendy provokes people to realize that this very ordinary instrument of education embodies numerous tacit assumptions and expectations that deserve a second thought. [...]

[...][T]he conventional chair-desk favors right-handed stu­dents; the writing platform is almost always to the right. The work­ing surface is not very large, so apparently students are not expected to coordinate multiple sources of written information or develop complex representations. Also, the chair-desk gets in the way of students forming working circles and deprives them of common desk space, as when five or six pupils sit around a table. Learners work alone! Normally chair-desks come in one size for a classroom. One-size-fits-all!

And there is more...

*** (Update 29 August) I now discover that this idea originates from Nicholas of Cusa (1401-1464). See here for a brief but more detailed exposition than mine, and a discussion of how he attempts to use it as a proof of the existence of God, but the writer claims eventually proves exactly the opposite.

References

Perkins D N (2009) Making Learning Whole; how seven principles of teaching can transform education San Francisco; Jossey-Bass

Ramsey I T (1967) Religious Language London: Macmillan

On the impossibility of philosophical progress

The link is to an enjoyable, accessible and iconoclastic article by Eric Dietrich, entitled "There is no Progress in Philosophy". (The first four sections are the most entertaining; the remainder is more technical, but still not particularly hard going.) From the Abstract:

Except for a patina of twenty-first century modernity, in the form of logic and language, philosophy is exactly the same now as it ever was; it has made no progress whatsoever. We philosophers wrestle with the exact same problems the Pre-Socratics wrestled with. Even more outrageous than this claim, though, is the blatant denial of its obvious truth by many practicing philosophers. [...] The final section offers an explanation for philosophy’s inability to solve any philosophical problem, ever. The paper closes with some reflections on philosophy’s future.

This is in contrast, of course, to the achievements of science.

If you accept the argument (which I think I do with some reservations), it is interesting to speculate whether the same can be said of the rest of the humanities, albeit in a weaker form. It is fair to argue that no progress has been made in the study of literature, for example, partly on the contingent basis that determining what constitutes "progress" in such a field is a philosophical question. Of course the stock of literature is ever-increasing, so we may have quantitative growth if not qualitative. I take it that the sterile deviation of "theory" (now apparently in retreat) is evidence that attempts at "progress" can only achieve the feat of disappearing up the proponents' own nether regions. A similar argument applies to the study of history (but again not to the creation of history)... As Alan Ryan observes in today's Times Higher Education (a propos of a nanced discussion of the relationship between teaching and research):

The corpus of available Greek literature that has escaped the ravages of time is finite and scholars have just about all of it under their belts. Interpretations of that finite corpus are another matter; they are, if not infinite, certainly indefinitely many. Nor is there any particular technique likely to yield insights that will be definitive, irresistible, part of a cumulative project of explaining everything there is to explain about Greek literature. Physicists may fantasise about finally reaching the "theory of everything", but it is unimaginable that anyone will produce the definitive way to read Aeschylus.

This is of course not good news for the practitioners of the humanities, which are under threat in the academy yet again. But are these disciplines about "making progress"? Or are they the stuff of Oakeshott's "conversation across the ages" (quoted here by Mike Love)?

As civilized human beings, we are the inheritors, neither of an inquiry about ourselves and the world, nor of an accumulating body of information, but of a conversation, begun in the primeval forests and extended and made more articulate in the course of centuries.

But like all conversations (including the arguments of philosophers which are Dietrich's starting point):

"In a conversation the participants are not engaged in an inquiry or a debate; there is no 'truth' to be discovered, no proposition to be proved, no conclusion sought. They are not concerned to inform, to persuade, or to refute one another, and therefore the cogency of their utterances does not depend upon their all speaking in the same idiom; they may differ without disagreeing. [...] It is with conversation as with gambling, its significance lies neither in winning nor in losing, but in wagering. Properly speaking, it is impossible in the absence of a diversity of voices: in it different universes of discourse meet, acknowledge each other and enjoy an oblique relationship which neither requires nor forecasts their being assimilated to one another."

And that is a delight. But does it make sense to try and "professionalise" it? And what are the implications for higher education of accepting such an argument--namely, that the humanities are not "subjects" in the same way as other subjects which do make progress?

(The original liberal arts, in the trivium and quadrivium, for example, are more focused than the usage adopted today in the US traditional liberal arts college; one might argue that only philosophy got a look in, under the heading of "logic" or "dialectic". So the historical argument for their centrality to the curriculum, weak as it already is, doesn't wash. And the study of English Literature is positively new--the University of Cambridge only appointed its first endowed chair in this dubious area of study in 1911, although interestingly there was a chair at the University of Glasgow from 1862.)

There is of course a recurrent debate in educational circles about knowledge and skills--which I am not going to reference because of its ubiquity. It's not merely a matter of liberal arts versus practical and vocational arts, lively though that discussion is. It is about how one goes about cultivating the higher reaches of critical understanding. Is it a matter of cultivating the skills of critical thinking first, with the knowledge base as an underpinning resource? Or is it a matter of transmitting the knowledge base, so that students are equipped to make judgements on the basis of real knowledge--and trusting that the skills will emerge?

False dichotomy of course. Both-and rather than either/or. But Bloom implies that the way to the skills is through the knowledge. And if the point of the knowledge is ultimately that the skills of creative thinking are engendered ("Creating" is the highest stage in the Krathwohl and Anderson revision of the Cognitive Domain) then it may not matter that the knowledge base itself is not going anywhere at a scholarly or cultural level. It is going somewhere for a particular learner.

And--just possibly--there may be something to learn from its substance regarded substantively rather than instrumentally, for its own sake rather than in the service of some other objective.

Jim Hamlyn touches on some associated questions here.(Although as I have revised this post, we may have diverged.)

Reference
Oakeshott, M. (1962) "The Voice of Poetry in the Conversation of Mankind," Rationalism in Politics and Other Essays. London: Methuen, 197-247.

The original pointer to the Dietrich essay was from the Browser. Many thanks.

On evidence-based study tips

I don't often simply post a link on this blog, but this is a digest from the British Psychological Society of study tips with links to sources--very useful, in preparation for a new academic year.

On hostages to fortune

I'm quite pleased with myself. It appears that after forty or so years in the further and higher education system, I am still naif. (The masculine--if connotationally effete--form of "naive". Not the same as "naff", although that may also apply...)

How do I know? Some time I may tell the tale of the great Course Review (but, like that of the Giant Rat of Sumatra, the world is not yet ready). But my bit of the fallout is to revise the Course Handbook. There are two hundred or so revisions required.

• Since many of them are uncontentious technicalities such as updating the names of committees to this year's fashion, why can't the administrators--who know the answers--just correct them, instead of telling academics they are wrong and they need to look up the latest regulations?
  
  
• It's not surprsing the committee wants changes. The course has now been running for fifteen years, and we have amended the handbook every year to accommodate its growth and perpetual updating. It was last formally reviewed in 2007, with no comments on the handbook or regulations. Because no-one read them. For years I've been slipping in asides and jokes and odd footnotes--and I admit many of them have been self-indulgent and even confusing for students who don't share my odd sense of humour. No-one has noticed because no-one has read it. (Tip for authors of this stuff. Plant an "easter egg" in the middle of the verbiage, and see whether anyone finds it. More.)

But beyond the legitimate stuff (and of course there is some) and other issues which are primarily attributable to the culpable negligence of a succession of dubiously competent "senior" staff ...

...my principal concern is the number of occasions on which I have been peremptorily instructed to insert the standard university material, rather than what we developed for ourselves to suit our course in the light of our experience. Invariably (and I do know what that means, and I use the term advisedly) the standard bumf is a qualified, weasel-worded, diluted fudge of our undertakings.

(One exception! Although I may argue with the approach and style of the Library's guide to citation [it's prescriptive without sufficient attention to the underlying pricniples], it is both compact and comprehensive, and I have happily replaced our own amateur guidance with it.)

In particular, we have been instructed not to refer to an independent support website (because it "might confuse the students")

Critical remarks about items on the reading lists are not allowed: but all the texts are curate's eggs, we know (including mine) And we can't refer to a "curate's egg" or latin abbreviations such as "q.v." or "inter al." because they too might confuse the students!

This is a deeply patronising and downright insulting approach to mature students.

If it is naif to believe that, then so be it.

On welcoming change

The link is to a piece by Steven Levitt at the Freakonomics blog.

He's just learned a new approach to putting (in golf). That kind of change is often resisted, as I explored here. Levitt does acknowledge a minor downside, but no sensation of loss, even after, by his own admission, he has been barking up the wrong tree for the putting proportion of 5,000 hours of golf.

Is it a matter of mind-set? And is sport an activity which promotes an incremental mind-set?

On being condemned to expertise

I've just been reading Matthew Syed's excellent Bounce: The Myth of Talent and the Power of Practice (2011, Fourth Estate*). I'm not a great sport fan, or music or chess buff, and those are the fields he discusses most, but he has an entertaining approach to Ericsson, and Dweck and other usual suspects.

I may be perverse but I do wonder about the opportunity cost of acquiring such expertise. In other words, what is the trade-off between what these people could have done with their childhood and adolescence, and what they ended up doing with it? I'm sure that they learned a great deal about perseverance and commitment and mind-set; but... Often it wasn't their idea--the Williams sisters and Tiger Woods and many more had to forgo many "normal" experiences of growing up because they were too busy and too focused--and it was not nexessarily their vision which drove them. See the controversy provoked by Chua's (2011) Battle Hymn of the Tiger Mother London; Bloomsbury.

See also this post and its links

* I'm not signed up to Adsense or any other scheme, and I have decided not to link to Amazon any more. If you want it you can find it--preferably at an independent local, physical bookshop. I'm prompted to this by a belated epiphany. I had occasion to go to Ampthill yesterday and came across Horatio's bookshop (and artist's materials purveyor...). Quirky and independent--and I'm sure endangered (although according to the website it started in 2009). And what about Topping's in Bath and Ely? Or even grumpy County Town Books in Bedford... Use them or lose them! Yes, they're not as cheap, but buying the book is only a small part of your investment in it--consider how long you spend reading it.

On learning in a technological age

No, this is not about the use of technology to enhance learning and teaching--it's a different angle.

Nicholas Carr's (2011) The Shallows: How the Internet is Changing the Way We Think, Read and Remember (London: Atlantic Books) is a well-written popular account of what it says in the title. Naturally it is selective and tends to assume its conclusions from the outset, but its fair.

There have been a couple of substantial blog posts this week, too, exploring similar issues;

• from Ed Yong at Not Exactly Rocket Science on "The extended mind – how Google affects our memories" and
  
• from The Neurocritic on "The Google Stroop Effect?" (read it to find out what a stroop effect is!)  

The Yong article seems to have been the source for a number of shorter pieces in the serious newspapers.

The very basic underlying argument is a variation on Dr Johnson (1775); 

Knowledge is of two kinds. We know a subject ourselves, or we know where we can find information upon it. 

Broadly it suggests that since the net (or Google as its metonym) has made it so easy to know where to find out, it has made it unnecessary to hold the information ourselves. There is a further argument that just as Socrates argued against writing in the Phaedrus (274e-275b);

...writing is inferior to speech. For it is like a picture, which can give no answer to a question, and has only a deceitful likeness of a living creature. It has no power of adaptation, but uses the same words for all. It is not a legitimate son of knowledge, but a bastard, and when an attack is made upon this bastard neither parent nor any one else is there to defend it. The husbandman will not seriously incline to sow his seed in such a hot–bed or garden of Adonis; he will rather sow in the natural soil of the human soul which has depth of earth; and he will anticipate the inner growth of the mind, by writing only, if at all, as a remedy against old age. The natural process will be far nobler, and will bring forth fruit in the minds of others as well as in his own.

...so the adoption of technological extensions to human capabilities ultimately undermine those abilities. Calculators replace the ability to do mental arithmetic, for example. It's an old argument.*

And to a certain, nuanced, extent it is true.  As it has been through the ages. The introductions of writing, of printing, of local printing, etc. have all had their impact. They have changed what it means to "learn". The challenge for education... Sorry! Scrub that cliche! So how have they changed what it means to "teach"?

(I recividistically** and opportunistically try to weave too many themes into a post, but this is about "reflection" and this does reflect how I think, for better or for worse...)

My colleagues and I have just been subject to a (insert derogatory adjective of your choice but don't forget nugatory) review of a course. One perfectly proper and reasonable question focused on the assessment strategy. "Why don't you use a wider variety of assessments? Quizzes? Timed tests?..." etc.

We didn't answer in these terms, but it did occur to me that the choice of assessment is an epistemological issue. What kind of knowledge/skill/value do you think you are testing? And on this kind of professional course, sheer memorization as assessed by a multiple choice test does not matter very much.***

It's a matter calling for thoughtful consideration, not knee-jerk answers.

[Although the other book I've just been reading--which poses similar questions from a very different angle--is Matthew Syed's (2011) Bounce: The Myth of Talent and the Power of Practice London; Fourth Estate, which touches on similar material to that I've discussed earlier.

What I don't see in much of the popular literature--and the textbooks, indeed--is much recognition of how fundamentally different the learning issues are in different disciplines and contexts.]

---

 * ...and of course my discovery of these links and references relied heavily on the availability of the linked material online.

** I'm not sure if this is a "proper" word (it only gets three hits on google, at the time of writing, in the adverbial form) but I commend it as "in the manner of a repeat offender".

*** which is not to say it doesn't matter a great deal on other professional courses, such as medically or perhaps legally-based programmes, where there is indeed a body of knowledge to be acquired for instant access--"there's a fracture of the thingy-bone and a puncture of the whatsit-artery. Sorry! On the tip of my tongue! I'll just go and look them up..."

and--great minds think alike corner--here is another piece.

On insight

The link is to an excellent article from Edge magazine. It's a conversation with Gary Klein on the nature of intuition and the decision-making of experts.

Judgments based on intuition seem mysterious because intuition doesn't involve explicit knowledge. It doesn't involve declarative knowledge about facts. Therefore, we can't explicitly trace the origins of our intuitive judgments. They come from other parts of our knowing. They come from our tacit knowledge and so they feel magical. Intuitions sometimes feel like we have ESP, but it isn't magical, it's really a consequence of the experience we've built up.

• There's enough material here for an entire course
• In particular, it poses interesting questions about the much-vaunted notion of reflection.
• Because it is an interview piece (although we only get Klein's contribution) it is very accessible. It's even exciting in parts!

(It's also available as a video on this page.)

On---I'm exasperated about silly arguments!

This one is about pedagogy vs. andragogy. If you need primary sources; See here and here.

I have been teaching for forty+ years (whether my students have been learning for the same period is of course contestable).

I have never yet encountered any exclusively behavioural learning (even with our wonderful Westies) or "cognitive" or "humanistic" or any other label.

And as for pedagogy versus andragogy... It's a spurious distinction whose primary achievement was to raise the profile of Malcolm Knowles. But read his piece here. Is that about treating people as grown-ups?

What these theorists never (OK! rarely) discuss is what is being learned. You want to learn a foreign language? At some point you are going to have to memorise a lot of vocabulary. You want to throw pots or improve your tennis? You need to practise, and there are people who can do it much better than you and you would be stupid to ignore their advice. You want to study political philosophy (OK, this is the most difficult)? Most people, of course, don't study it. They go straight to teaching it in the saloon bar... But...

And that is to say nothing about STEM subjects (Science, Technology, Engineering and Maths), where it is self-evident that there is an enormous (and growing) body of knowledge into which you must be inducted before you can make any contribution.

As I complain here, andragogy is not an approach to teaching--it's a brand. And one which gets in the way of making appropriate decisions about how to teach a particular subject/topic/skills to a particular group of learners.

On angels, pinheads and APA

I've been here before, and here. And I'm marking again, so the old referencing bug-bears come out of hibernation.

The obsessional scholasticism of the "style" manuals fascinates me, and this post from the APA blog apparently takes the biscuit. Read it before continuing.

What really irritates me is of course that it privileges a US convention, that colons are followed by capitals: that is not the British style. But the convention imposes an arbitrary divide: it's no longer a matter of style, taste and preference, but of right/wrong, good/bad. (Which may translate into assessment grades and even immediate career prospects.) [Check out the arguable solecisms of this paragraph.]

There are several standards in play here. I hold to my opinion that we should not get our knickers in a twist about material up to Master's level. (Including such colloquial phrases).

That is what we might call domestic level citation. It's a matter between student and tutor/supervisor. Its test is whether the t/s can trace evidence back to source. In the fairly rare event that an assessor does such a trace, it will be done manually; with considerable tolerance of punctuation, case, and formatting. All my complaints about obsessional and pedantic compliance requirements stand, up to this point.

But I'm reluctantly changing my mind beyond that point.

I don't use EndNote or any other citation management software: I don't understand what most of its fields mean, and the prospect of importing a lifetime's references is unattractive in any case. And I don't need it. But I can see why many people do need something like it. Preparing literature reviews or surveys for publication is made so much easier; such tasks go beyond the domestic level and call for industrial strength solutions. And once data has to be "normalised" (made consistent) for use in a database, such arcane minutiae as the place of a colon matter. The question of whether the colon is followed by an upper-case letter could determine whether the whole title is treated as one field, or as two--title and sub-title.

Such is the stupidity of machines that we have to do their bidding.

Incidentally, in one of those earlier posts I noted that Jude Carroll;

"...went so far as to say that this obsession with "correct" referencing was a phenomenon of the last ten years, and implied that it was symptomatic of a crisis of confidence on the part of academics in their own authority in a post-modern world."

But perhaps that is in itself a humanistic perspective, and the same time-scale could be accounted for by the rise of the machines?

But the Oxford comma? (And here.) I don't think there is any excuse for getting worked up over that.

On classic sociology; they don't write them like this any more

I'm reading Becker et al. (1961) The Boys in White; student culture in medical school. It's the first time I've actually read it. I have of course raided it before, searching the index for a gobbet of material to use for a particular purpose, but I've never had the time to get into it. It is after all over 450 pages, and even now I am not reading from beginning to end.

At one level I am disappointed--I got it out of the library in order to raid it yet again, for a blog post yet to appear on professional socialization as the hidden curriculum, in the light of a recent study of how medical students' empathy declines through medical school--and I couldn't find what I expected to find. But then... I began to read more leisurely, taking the book on its own terms rather than imposing my own imperious demands on it, and I rediscovered a lost genre.

From the Lynds' Middletown studies (1929 on) to Becker and co.'s own Making the Grade (1968) there is a whole seam of big, sprawling, accessible and humane case studies*, more like current anthropology than sociology, which may be theoretically "naive" to current researchers for whom fieldwork is a (not entirely) necessary obstacle on the shortest possible path between idea and rateable article for the REF score... **

I have no idea how these people pitched for the funding to do the research, but it certainly wasn't by predicting what they would find out before they started looking for it, as current researchers have to do writing their bids. They did "grounded theory" before Glaser and Strauss invented it (I suppose it could be argued that it only had to be formulated as a "theory" because it was then becoming necessary to legitimise proposals with a "theoretical", even "scientific" base).

But these works are rich. They are full of--often unnecessary, by current standards--transcribed interviews, and observational anecdotes. Some of the subjects become old friends--oh, here's Jackson again! Wasn't he complaining about not being able to see the actual surgery on his obs/gyn rotation?

What's happened? Partly this kind of common-sensical description has been taken over by a generation of academics scrabbling to be more "scientific" (or more likely more "post-modern" aka incomprehensible) than thou. Partly it has drifted into the realms of reportage: the rise of popular non-fiction has probably lowered the stock of accessible scholarly writing.

It's all moot. But we'll all be the poorer if it disappears in any form; and there was a lot to be said for academic rigour and a concern for balance.


* I'm sure someone will argue with these arbitrary chronological boundaries; I concede in advance that I can't be bothered to research the field properly. It's potentially a whole academic career...

** I'm referring to US studies, but the UK tradition is not to be sneezed at, see for an intro (if you can find it) Frankenberg R (1967) Communities in Britain London; Penguin. And belated thanks to Frankenberg for accepting my first (actually, only) respectable sociological article in 1971 when he edited the Sociological Review.

On learning by naming

To-day we have naming of parts. Yesterday,
We had daily cleaning. And to-morrow morning,
We shall have what to do after firing. But to-day,
To-day we have naming of parts. Japonica
Glistens like coral in all of the neighboring gardens,
          And to-day we have naming of parts.

Reed, Henry. "Naming of Parts." New Statesman and Nation 24, no. 598 (8 August 1942)

(Do read the rest of this very funny, but also cutting and sometimes beautiful, long poem about the absurdity of war, from the link above.)

I was reminded of this as I drove back from another teaching observation this morning, (the same circumstances as this earlier one). I'm happy to report that the student is making good progress, but she is still stuck with another hopeless course. The Wolf Report on vocational qualifications makes no bones that the quality of many of those qualifications is dubious, they are not fit for purpose, and effectively deceive young people into believing that they are going to lead somewhere. What I observed this morning clearly met those criteria. It was a Level 3 National Diploma in "Sports Development and Football"--a spurious concoction of a curriculum, even by vocational education standards, clearly not well understood by students or even the tutor. (To be fair, I must concede that today's lesson on planning activity sessions for children did have some practical merit--despite being enirely classroom-based.)

Its recurrent theme was matching up all activities with "the" four Benefits of Exercise. Not three, not five, not twenty-two, but four. It appeared that these were so doctrinally significant that they had to be taught dogmatically; they could not be discussed and discovered. And then they had to be written down and incorporated into the (written) assignment. (Without attribution--despite this being Level 3, which is just one level below first-year undergraduate level, the question, "Who says?" was never raised. No wonder freshers are thrown by their university experience...)

And there was no Japonica in a garden outside, but it came to me as I drove back that I had sat in on a session about naming of parts. At this level, it appears (on the basis of the previous observation and class discussions) that most of the learning is about attaching an approved label to a concept or experience. Rather like the magical belief in the potency of naming as a means of gaining power over an object, the assumption is that to name is to know, and that is all that is needful.

At least in Reed's case, the naming was clear labelling of concrete objects, even if one of them "you have not got"...

This is the lower sling swivel. And this
Is the upper sling swivel, whose use you will see,
When you are given your slings. And this is the piling swivel,
Which in your case you have not got. The branches
Hold in the gardens their silent, eloquent gestures,
          Which in our case we have not got.

On frameworks

I'm sure the stats facilities of Blogger could tell me how many people have dropped by this blog, once, and then moved on. I do it all the time, dipping into blogs via RSS feeds. The really interesting ones I subscribe to, but I have to admit that many of them (principally those thrown up by my "teaching+learning+college+practice+teach+reflect" search string) are full of pious jargon-ridden bulls**t. Usually immaculately referenced.

Occasionally I come across sites like this one, or Don Clark's. I'm impressed by the amount of study and thought and experimentation which have gone into Don's ADDIE model. I wish I could be as systematic and organised. I wish I could bring together such an eclectic range of reading as does the author of the other linked site (although it does feel like a required assignment).

A while ago, I was trying to put something together on planning for teaching, and I made some effort to research the topic. Every textbook has something to say about it, and indeed the first pages I wrote for my main site were about how to do it. (You can see it is ancient--I still believed in "learning styles" then!)

In higher education, the most popular approach seems to be Biggs' "constructive alignment" (and here). But see David Jones' blog for a dissenting view-- I'm in awe of his scholarship, too. I'm just not convinced by any of them.

I don't believe anyone actually works like this; that they proceed step by step through a planning checklist or algorithm and eventually end up with a lesson plane or scheme of work of impeccable provenance which they actually implement (other than when they are being observed for assessment or inspection purposes, of course).

If I try to do that, I come up with something which is deeply boring and/or superficial, and I find myself back-tracking to change earlier stages in order to come up with something I can actually see myself teaching.
Which is a little embarrassing, because I teach this stuff!

And that's another problem, because students keep pressing me for a clear planning procedure.

So I was relieved to come across this (sort of) confession by Dan Ariely. It's not about teaching as such (but it is about how he teaches a subject in response to those demands), it's about marketing, but there are similarities of simplistic prescriptions;



Ariely's full blog posts with comments is here.

(Er--just don't tell my poor students who have to jump through these irrelevant hoops, particularly as they are submitting their final assessments! Although, to be fair, some of the simpler stuff can be justified on the basis of scaffolding, at the start of a career.)

On "Toy Stories"

Simply the most entertaining hour of TV for (in my limited experience) years. It does take an hour to watch, it's unashamedly "boys' toys", and probably inaccessible outside the UK, but enjoy!

On academic inhibition

A few days ago I wrote quite a long post about teamwork, nearly posted it, and then didn't, cut a large chunk and reinstated some of it, and then finally posted it yesterday and immediately had second thoughts...

It made no mention of teams in other contexts, of sports teams for example. (Because I know nothing about them.) It did not discuss Belbin on management teams (often misapplied and contestable as it is). The list of what is missing is endless.

It is also highly self-indulgent; much of it is about team membership in my own not-very-exciting career, and there is little reason to suppose that anyone else will be interested in that. At least I had the decency to put that beyond a jump break.

Why am I bothered? Even to the extent of boring stiff my few readers with this kind of meta-reflection? After all, the blogosphere is precisely a place where people notoriously pontificate about stuff they known nothing about, make stuff up, are unremittingly egocentric. There are no rules about citing evidence for assertions and claims, there is no requirement to confine oneself to relevant argument, or to keep within one's realm of knowledge and competence.

But those qualities are precisely those I bang on about all the time as an academic, and seek to instil in students. And it is their absence which leads me to be cautious about accepting students citing from the web in general and the blogosphere in particular...

And I have internalised them rather too much, to the extent of discounting the value of any other kind of writing, particularly on my own part. Even to the extent of gradually ceasing to voice opinions on anything on which I am not an expert--hence, on practically anything at all.

I'm planning a book; working title, "The Secret Life of the Classroom". It's proceeding rapidly. Backwards. It's not original research, but intended to be research-based and I suppose scholarly but accessible. It will be based on material I have taught for years--but the further I get with planning, the further away I am from actually starting, because in order to be credible I realise how much more work I need to do and how much I don't know.

Still, I have the time, I hope. So watch this space in a year or two, or five...

On a team

I was in prison the other day (just visiting for a teaching observation).

My student and I were just beginning our post-session discussion when one of his colleagues knocked on the classroom door to remind him that the morning team meeting would begin in a couple of minutes. He asked whether I wanted to attend. I wasn't sure it would be appropriate, because I was an outsider, but I needed an escort to get back to the gatehouse and no-one would be available until after the meeting, so I acquiesced.

I was so pleased I did. It only took ten minutes, and nothing exciting happened; some nagging about locking doors (of course), some reporting back on "incidents" last week which some people may not have known about, with a clarification of reporting procedures...

There were a dozen or so people present, most of them standing up, in the dingy outer office. About four or five people spoke in response to the chair's invitation for anything to share.The "manager" was not even there (and I'm sure she's not happy with her formal label--she was a student of ours several years ago) but it all happened regardless. I have no idea whether anyone took notes. I hope not, because there is nothing more toxic to teamworking than having an eye to an audit trail.

I was the outsider, and stayed physically on the periphery. But I felt totally at home.

It may be that working as a teacher, in a prison, focuses one's thinking on the clash of educational and custodial values.  (Although it does occur to me that these meetings may be part of the standard operating procedure and that the prison officers on the wings do the same thing... I don't know enough to have a view on that.)

But what really struck me was the sense of team membership. Interdependence counts in an uncertain environment (although in practice of course prison education is much less hassled than in any open environment). It's the culture I have spent most of my working life in, and although not a demonstrably sociable person, it's one I value enormously.

And it is almost totally absent in most teaching in higher education*. (Big generalisation, carefully qualified, and of course not properly evidenced... B-?) It exists in research teams, sometimes very powerfully, but not much in respect of  teaching.

I think there is probably more of a sense of being a team amongst staff teaching on a professionally-- rather than academically--oriented programme. The modules of a professional programme are jigsaw pieces rather than free-standing units**, and (apart from notorious instances where the staff are at vicious loggerheads with each other) the staff need consciously to work together. Numbers of students and of staff are probably more manageable, too.

On academic programmes, particularly those such as humanities where there are few prerequisites, the structure (and options) of the course typically follow the individual interests of the faculty. ("Fred's retired, so we'll have to drop 'Origins of the novel' this year, but Gabby needs some hours, and so we could offer 'Queer theory and early 20th century poetry' instead...")

Academe has an ambivalent relationship with teamwork. It's both a highly individualistic environment and highly mutually dependent (and the extent depends a lot on the discipline--the more equipment you need, the more you have to depend on each other). But it's a tremendous and yet little explored resource.

Incidentally, real Teamworking has absolutely no connection at all with what Human Resources departments think it is, with their obscene expenditure on external "team-building" consultants whose only impact is to unite the potential "team" in visceral hatred of HR and probably management in general. It has to be a bottom-up phenomenon (and of course as such it can--for better or for worse--interfere with top-down initiatives) but when it comes together it is transformative.

On living in a different world

1. A little while ago I was helping my son to do some basic DIY involving screws. He's coming up for 30, and I was amazed to find him muttering to himself the the mantra; "Righty-tighty, lefty-loosy" to ensure he screwed them in correctly.
2. Last year, at a study day for our in-service vocational teaching students about threshold concepts, the engineering special interest group suggested that "Righty-tighty, lefty-loosy" was an important threshold concept for their students to acquire. 
3. Just now, confronted with a rather odd mixer tap (faucet) in the bathroom where I am staying, and trying to balance the flows, I found myself having to use it, too. (There were two tap heads mounted horizontally opposing each other at the base of a common outlet pipe.)

But where has it come from? Why do students of 16+ years have to be taught it? Isn't it just more complicated than the metaphor we have "always" used--clockwise and anti-clockwise? Yes it is, but I've just realised that these students grew up in a digital era. I read somewhere in the last few days the claim that 60% of people check the time on their mobiles, even after they have just looked at their watches. That struck me as rubbish, but it does suggest that the analogue clock face is no longer the universal trope it once was. (Hey! I finally used that word! Probably never again.)

On the police getting younger...

...but not so young they need to be nannied like this?

Just what goes through the so-called "minds" of the people who devise this rubbish? And do they give a thought to what message it sends to rank-and-file officers about how their seniors view them?

On a prediction come true...

Harold Camping's response to the failure of his eschatological prediction of the "rapture" (pardon my ignorance, but this is a term which appears only to been used in the past ten or so years, associated with the amazingly/bizarrely successful "Left Behind" series of novels) is exactly as might have been predicted by Festinger et al.

But Mr. Camping said that he's now realized the apocalypse will come five months after May 21, the original date he predicted. He had earlier said Oct. 21 was when the globe would be consumed by a fireball.

Saturday was “an invisible judgment day” in which a spiritual judgment took place, he said. But the timing and the structure is the same as it has always been, he said.

“We've always said May 21 was the day, but we didn't understand altogether the spiritual meaning,” he said. “May 21 is the day that Christ came and put the world under judgment.”

(source here: retrieved 25 May 2011; my emphasis)

But what strikes me most forcibly is that he took the "spiritual/non-empirical" way out. He took the Pauline (Paul-eye-ne) option. Jesus was open to empirical claims and tests. The most important was that he would rise from the dead: even Paul took this on board ( I Cor. 15:14), but it remains unclear about what this meant/means. Jesus claimed a gospel of liberation:

he hath anointed me to preach the gospel to the poor; he hath sent me to heal the brokenhearted, to preach deliverance to the captives, and recovering of sight to the blind, to set at liberty them that are bruised... (Luke 4:18 [AV]) 

But there was precious little evidence of it happening (not his fault: I remember a vox pop interview in South Africa at the time of the first free elections, and the interviewee, who had been queuing for hours to vote, being asked what she expected from the ANC and Nelson Mandela. She recited a long list of perfectly reasonable but utopian aspirations. How long would all this take? asked the interviewer. "I'll grant them three months.")

If you can't deliver what you have promised--promise even more, but in less specific terms...Or those further removed from testable reality... Until no-one can tell whether or not you can deliver.... Prophets and politicians and perhaps pedagogues all do it.

Sometimes--not at all often, I hope--it's the best thing to do. Better than setting yourself up to repeat the same error five months down the line.