Monthly Archives: December 2007

Assessing Assessment (thanks, ADM)

Perhaps because the semester is over and faculty are thinking about evaluations more generally, some interesting threads on assessment have appeared lately in In Socrates’ Wake, Easily Distracted, and Blogenspiel.  One of the first things that struck me about these posts was how differently this language of assessment and accountability plays out, depending on where you stand in the hierarchies of the discipline or professional status: public universities vs. private colleges, Research 1 vs. everyone else, tenured faculty vs.  tenure-track, lecturers or grad students.   Assessment looks different to people in each one of these locations, and offers different opportunities and risks.  This shouldn’t surprise anyone, but it should make us bite our tongues and think a little harder before speaking out on behalf of others.  After all, though assessment is usually framed as an insiders/outsiders debate, the interests of those “inside” are in fact multiple and may in fact conflict.  Some quick takeaways:

1.  Another Damn Medievalist notes that

all of the assessment of us as faculty is predicated on the idea that students are here to learn. It’s based on the idea that students do learn, and remember. Even though we like to cringe, because so much of assessment and accreditation seems to focus on whether students are getting value for money, it *is* also based on the idea we try to impress upon our students: what we teach is important, and a BA/BA is not just a piece of paper that means a better entry-level job. But what if that’s all the students want? What if they aren’t so worried about the experience, about savouring and remembering what we try to teach them? What if, as one of my students said to me, they just aren’t that into it and really just didn’t feel like getting the A they could have got, because all they needed to graduate was 70%? For most of my students at SLAC, that seems to be the rule.

So assessment gauges individual teacher performance on the basis of student performance: so far so good, but this mimetic approach may also ignore how systemic disparities of student motivation and/or skills will interfere with this model.  As she notes, “All kidding aside, how do we assess places like SLAC in a way that is fair to the good faculty and to the good students? If a campus tries to push a reputation as ‘selective’, then how do we integrate the results for those students who came in on waivers?”  My answer would be, such an assessment will not be “fair” unless these factors are somehow represented as well.  But this inevitably demands that those being assessed work with those doing the assessments, to make sure the questions, criteria, etc. reflect the local circumstances.  And in my experience, it also means that those below a certain threshold of job security/status will never have a voice within the assessment process. 

2.  The ISW discussion was valuable, I think, because it introduced the distinction between standardized assessments (multiple choice tests designed for all students of a particular discipline, for example) and quantitative assessments (rubrics etc. that would allow faculty to exercise qualitative judgments that could in turn be translated into quantitative measures).  Since both philosophy and literature lack a definitive canon that students could be universally tested on, I think the assumption of standardization is the biggest problem with these kinds of proposals, which come most often from non-academics.  It may be different in other disciplines, though.  Those who spent some time in the sciences would know more about this than I do.

3.  Finally, Tim Burke’s discussion advocates pretty convincingly for more transparency and demystification of higher ed, and argues against a “regulatory machine administering tests, enforcing rigorous common standards, hauling professionals up before a bureaucratic star chamber every four years?”   Fair enough, but which do you think your state legislature would prefer?

Best,

DM

Advertisements

MLA, I’m on my way . . .

For anyone interested in getting together for some drinks or coffee or whatever, I’ll be at MLA between the 27th and 29th.  Contact me here or offlist at dmazella at uh.edu.

And here’s the announcement of our Friday evening panel, for anyone interested:

 375. Cultural Studies and Eighteenth-Century Studies in the Classroom

7:15–8:30 p.m., Mississippi, Sheraton Chicago

A special session

Presiding: Laura Rosenthal, Univ. of Maryland, College Park

Speakers: Dwight Codr, Tulane Univ.; Thomas P. Di Piero, Univ. of Rochester; Laura T. Engel, Duquesne Univ.; Carl H. Fisher, California State Univ., Long Beach; David Samuel Mazella, Univ. of Houston, University Park

It looks like a great lineup, and I hope to see some of you there. 

Things have been quiet around here for the last few weeks, but if you’re giving a paper or chairing a panel, let us know.

UPDATE: here’s the Literature Compass Blog pre-MLA round-up, which mentions us and a bunch of other panels.  And here’s the MLA convention listing, for easy searching.

Best,

DM

Is the crisis in academic publishing also a crisis in book reviewing?

Miriam Burstein of the Valve and Little Professor has just put up a very useful post about Profession ’07’s collection of responses to the “Report of the MLA Task Force on Evaluating Scholarship for Tenure and Promotion,” both of which, I think,  deserve to be widely discussed.  I was a little surprised, in fact, that the Report generated so little comments, but perhaps this is all old news to people.

 Burstein noticed something that caught my eye, as well, while I was reading the Report: how the devaluation of book reviewing had led to some perverse results for academic publishing and the dissemination of scholarship generally.  Departments tend not to count such service, journals themselves are cutting down on review space, while fewer and fewer senior scholars can take time from their “official” (meaning counted) publications and research to read, let alone discuss and publicize, major critical works outside their own fields of specialization.  Here is Burstein’s take:

on the subject of book reviewing, the committee worries that “senior scholars” have largely jumped ship, leaving the decks to “graduate students or junior scholars in the field, who may not be sufficiently specialized in the subject or treatment of the book to draft anything other than a summary” (56).  Understandably, the committee worries about the effects of this shift on the tenure prospects of young faculty.  But allow me to make a very cranky and entirely anecdotal observation, based on what is no doubt insufficient experience (one year working at Modern Philology).  During my time wielding a blue pencil, I found that faculty at all ranks were capable of producing mediocre to excellent reviews.  However, the only true clunkers came from established scholars. And by “clunkers,” I mean reviews dreadful enough that I could use them as spicy anecdotes during job interviews (leaving the scholars in question anonymous, of course):  reviews loaded with grammatical errors, reviews that pimped the scholar’s own work and ignored the book purportedly under discussion, reviews that were almost entirely incoherent, reviews that appeared to have been written by freshmen.    Junior faculty have professional reasons–beyond whatever sense of obligation to the discipline they feel–to invest themselves in writing serious book reviews.  Senior faculty, not as much.  Again, it’s not that senior faculty always wrote the worst reviews, but that the worst reviews were always by senior faculty.

Burstein observes, in effect, that our usual expectations of performance are reversed when it comes to institutionally devalued work: we assume that the most honored and highly rewarded faculty do the best job when it comes to reviews, but this assumes that they will ignore the way incentives are structured in their departments, or how reputations are maintained in their fields.  A dangerous assumption, I think.

We also talked about this problem at our NEASECS panel earlier this year, when I mentioned that for all the potential problems introduced by academic blogging (logrolling, backscratching, insufficiently critical dialogue, or conversely, off-topic or overly polemical discussions inadequately vetted by non-experts), I saw many of the same problems with print reviews nowadays, with the added problem that they tended to appear years after books went out of print.  So I do think that academic bloggers can play some role in disseminating information about new academic research, if not the entire role.  But having done both, I can easily see the problems with both forms of dissemination.

 What I haven’t seen anywhere, however, is an acknowledgment that the problems with academic reviewing are only a part of a more general decline of book reviewing, as newspapers and periodicals of all kinds are folding or substantially reducing their review sections for general readers.  There have already been a series of warnings about the decline of newspaper reviews, which I regard as the disappearance of a generally unnoticed infrastructure of print culture that we have taken for granted since the eighteenth century.  These changes might very well help to contribute to the “death of the reader” that we have been hearing so much about lately.   Indeed, if we are justified in talking about something as general and epochal as “print culture,” academics are no more immune to its vagaries and historical shifts than anyone else.   (UPDATE: for Caleb Crain’s blogging follow-up to his excellent NYer piece, “Twilight of the Books” referenced above, see here).

(UPDATE #2: Hmm, this isn’t what I had in mind (h/t 3quarksdaily).

DM 

18th century blogging round-up: miscellaneity

Here’s a very unsystematic (make that, distinctly miscellaneous) selection of eighteenth-centuryish blog posts from the past few weeks.  If anything else has appeared lately that has caught your eye, let me know, and I’ll put it up and discuss it at great length, I promise.  (Note: the 18th Century Graduate Reading Group has just put up its Fantomina posts, but I’m saving my response for another day.  But go take a look)

In any case, here are some highlights:

1.  Courtesy of 3quarksdaily, the TLS review of the first volume of the new Buffon edition published by Champion and edited by Cremiere and Schmitt.  What Cobb’s review emphasizes is how much the major scientific advances of the Enlightenment owed to the mature print culture of the mid-18th century: Buffon’s L’Histoire naturelle volumes were beautifully written, systematizing, serially published works (36 volumes published over 39 years, while serving as a stylistic (and business?) model for the Encyclopedie), featuring superb engraved color illustrations, and attracting an international lay-audience eager to follow volume after volume of Buffon’s insights into the whole of “human knowledge about the natural world.”  An astounding achievement, and communicated in a manner impossible to imagine for another era. 

2.  Greg over at Slawkenbergius’ Tales has been knocking out a series of intriguing posts about science, rationalist historiography of the irrational, and modes of history and their historians (I like the appreciation of Pocock best), all united by his ongoing interest in Enlightenment historiography  Always nice to read a historian who knows his Borges, but still insists on writing about real people. 

3.  Obligatory contemporary politics connection: How many of the signers of the Declaration of Independence were clergymen?  Don’t ask Mike Huckabee.  Instead, visit J.L. Bell’s always astounding Boston 1775, and get the straight poop.  (Answer: 1, the one-time president of Princeton College, John Witherspoon).

4.  And here’s Alice Boone (of Ben and Alice) describing her history of reading and rereading Virginia Woolf’s A Room of One’s Own, first in her freshmen year seminar, then in subsequent years, while attempting to develop her own view of Margaret Cavendish, and one that could stand up to Woolf’s indelible image of her in the Room.  A valuable set of reflections on teaching and canonical revision.  And don’t forget her interview with Sophie Gee, author of the Scandal of the Season.

More to come.

DM

Satire and the fake first-person voice

As usual, satire is quicker to identify and exploit the conventions and styles of new genres than any other literary form.  Where Jonathan Swift once emptied out and reanimated genres like the projector’s pamphlet or the periodical-pastoral poem, some other Jon Swift has mastered the art of reviewing books he hasn’t read for Amazon.com.  Another good example is the fake-blog, like the “fake Steve Jobs,” who probably writes better and more amusing prose than the real Steve Jobs.  Now we have an entire constellation of fake blogs, courtesy of NewsGroper, who feature a little troop of insiders dishing on celebrities with blogs like Fake McCain and Fake Al Sharpton (which sounded authentic enough to fool MSNBC).  And one of my favorite examples of this kind of fake first person writing remains Jesus’ General.

One of the keys to this kind of online writing is the strategic mystery surrounding its sources: the satire always works better, I think, when there is genuine uncertainty regarding its origins and therefore its purposes.

DM

Marilia Svinicki’s Learning and Motivation in the Classroom

I haven’t done a teaching post in a while, so I decided to highlight Michael Cholbi’s excellent twopart series at In Socrates’ Wake, which discusses how to motivate post-secondary students first of all to abandon their existing understandings of a subject, and second, to develop the skills and self-confidence necessary to enter into what I’d call the disciplinary understanding of that subject matter. 

Svinicki’s argument should remind us that the learning process for our students often consists as much in abandoning earlier conceptions as it is acquiring new ones.  This kind of insight into our students’ tradeoffs should make their resistance to our teaching more explicable, and might also explain why students will cling to earlier, more rudimentary models of understanding (say, the 5 paragraph essay form) that may seem utterly unsuited to the tasks that we assign. 

Cholbi emphasizes the risky nature of this learning process for philosophy undergraduates, and picks up on Svinicki’s notion that the beginning student must learn how to model herself on others, while also being given the ability to “fall intellectually without being hurt.”  Yet as Svinicki also suggests, the student who attempts to model herself on the self-designated expert (i.e., you, the instructor) often finds that her half-articulate understanding of some exchange during classtime will instantly evaporate the moment she has reached the door.  Here’s Cholbi’s comment:

I find Svinicki’s student comment — “but I understood it when you did it in class!” — to be very familiar. For example, student papers can reproduce arguments discussed in class, but it is often apparent from students’ inability to analyze the argument, pose objections, etc., that their level of understanding is not as strong as they had anticipated.

This is a very familiar problem for me as well, especially when I am attempting to teach students some piece of counter-intuitive critical theory, and one of the main reasons why I assign students at this level group research projects and presentations, so that their modeling and feedback comes not just from me but from their peers, as well.  It’s also why these kinds of assignments work better incrementally than all at once.

But Cholbi raises an interesting question at the end of his piece: how should we reward risk-taking in our grading of student responses?  Should the safe and unambitious paper, neatly typed and with perfect MLA form, win out over the ramshackle and frequently out of control paper that might possibly have an interesting thought in there, somewhere, in its smeary pages?  How much of our grading involving student intellectual risk-taking is either a reward or a punishment for our own undergraduate behavior? 

[incidentally, don’t miss David Morrow’s response and bonus link to the Onion at the end of the piece]

DM

Now this

just doesn’t happen very often: Verlyn Klinkenborg has written a piece for the New York Times that connects the eighteenth century with speculative fiction. See “When Doris Lessing Meets Lady Mary Wortley Montagu” (Dec. 8/07) for an interesting read about the ways in which we tend to position writers from the past, and how re-imagining them can offer new insights. The wry Montagu is particularly suitable for this somewhat whimsical treatment, while conversely being enough of a heavyweight to survive a comparison with Lessing.

[Xposted]