fighting the last war: UPDATED

[image from http://en.wikipedia.org/wiki/Maginot_Line%5D

The new AAUP Journal of Academic Freedom came out this week, and I was happy to see that David Porter, a Psych professor and former administrator, had responded to the journal’s earlier essays by Powell and Champagne about assessment.

Porter does a pretty good job of countering the caricatured version of assessment found in Powell and Champagne, when he uses his own experience as a faculty member, administrator, and accreditation liaison to argue that

1) assessment is an integral part of learning (and hence education), 2) assessment is a necessary function of effective and adaptive organizations, and 3) involvement in assessment activities is particularly important for the AAUP and its members. Assessment is about creating a culture of evidence that is much more than merely collecting piles of data and accumulating a multitude of meaningless measures . . . (3) [my emphasis]

Porter moves the discussion from Powell and Champagne’s “culture wars,” where the humanities are always besieged, to a “culture of evidence,” in which social scientists bravely, carefully, patiently do battle with a dominant culture’s myopia to correct educational institutions.  This is a smart rhetorical move, since it raises the question of who is truly the insider and who the outsider here, and who is really representing the interests of Truth in these debates.  But is this really a debate worth having?

In some sense, we haven’t really gotten past the problem of mutual disciplinary caricatures and endless reaffirmation of tribal identities, when what we really want to know is why assessment so often fails to attain its stated institutional and educational goals.  Powell and Champagne ignore assessment as it is professionally practiced, in order to affirm the superiority of the humanities over the social sciences (e.g., Powell 12; Champagne 15). Porter, perhaps encouraged by his interlocutors’ crude self-descriptions, does some fingerpainting to dismiss Freud and Marx and their followers as insufficiently “scientific” (where’s the falsifiability?) (or, more tellingly, excessively interested in “conflict”) (18-19).

In neither Porter’s nor the Powell/Champagne approach do I find an acknowledgement that the market forces overtaking the contemporary university are affecting all its component disciplines, or that the reflection, and action consequent on reflection, necessary to contest this problem will probably need to be similarly comprehensive. Why would anyone concerned about a problem of this scale rule out of court potential allies and the kinds of evidence they could bring to the larger debate?

Finally, and this is a point I think I owe to my debates with my rhet/comp colleague JZ in my own department, I think that that much of the anti-assessment discourse in the humanities is indebted to the longstanding left critique of bureaucratic rationality from the ’60s and ’70s.  (And, indeed, much of this critique still holds true, in particular contexts; cf., for example, this).

But at this point in US history, after several decades of political reaction to the Great Society, do we really believe that the reason for non-responsive government is that favorite of conservatives and neo-libs: i.e., “bloated bureaucracy”?  Or is it the capture of once-public institutions by unaccountable financial and corporate interests? This is why I think that much of the anti-assessment discourse in Powell/Champagne’s work is a Maginot Line designed to defend us against a political opponent who no longer exists.

If we look at the intertwined history of assessment and accountability from  the vantage-point of post-Occupy American politics, what we would say is that the accountability movement, in its obsession with finding “efficiencies” in public higher ed spending, has used the long-term disinvestment in public institutions by state and federal governments to weaken internal governance and to force universities to partner with corporate interests at every level. In the meantime, neither movement has developed any cogent response to the single biggest threat that I see to the future of higher education in this country, which is the mounting student debt load necessary to attain a college degree.

So public costs are a matter of obsessive public debate, and these debates simply reinforce the drive toward disinvestment. In the meantime, the leaders of our so-called educational establishment have nothing to say about the growing alliance of finance and higher education, or how this privatization is slowly undermining the public credibility of educational institutions.

Apparently, as long as students and families are paying ever-growing amounts to private entities to gain an education, no state legislators or administrators can be bothered to intervene. So how much money would you contribute to a school run by Mark Yudof? Or Linda Katehi?  (And, indeed, no accountability advocate I’ve read seems to worry about how efficiency and privatization might impact US higher ed’s unique ties to philanthropic giving, which is unparalleled in other national systems)

In the context of the incredible debt-load that students now carry, we can now safely assume that the accountability movement has done nothing to reduce costs to students (as opposed to taxpayers), since reducing public investments in higher ed has only driven students and their families into the maw of  the (under-regulated) for-profit schools and the financial industry.

As far as I’m concerned, no savings, no give-backs wrung out of public institutions and their labor force can keep pace with the disinvestment accumulated over 30 years, and now capped off by falling state revenues during our Great Recession.  And as Krugman has pointed out in the context of the Austerity Debates, the rhetoric of controlling costs is not about “debts or deficits at all; it’s about using deficit panic as an excuse to dismantle social programs.”

So what is the way forward with accountability and assessment, under the conditions I’ve just outlined?  I’ve published some thoughts about this before, so I won’t rehash, but I think that any defense of the contemporary research university will have to take in the perspectives of all the disciplines housed inside it, as well as the full range of students we teach, and will have to develop a professional, reflective, publicly accessible view of its teaching activities to be communicated to the multiple publics we address.  Porter’s emphasis on “learning organizations” (11) sounds like a variation on my own proposal of Schon’s “organizational learning” in the university context, but I think it will have to take place in the criss-crossing, sometimes irritating discursive loop-de-loops and cul-de-sacs we are all familiar with in academic life.  In other words, no reason to over-idealize academic debate in the context of our disciplinary “tribes.”

I also think that if Porter wishes for assessment to gain the trust and respect necessary for it to be successfully integrated into our “learning organizations,” the assessment experts will need to take some responsibility for the conditions under which it is imposed.  When he writes,

Rather than building community, the hierarchical imposition of externally generated, but ill-conceived, assessment programs and inadequate protocols fractured academic communities. These initiatives also distracted and frustrated educators throughout the organization. Most of these problems, however, appear to be a consequence of the misuse of assessment principles and processes rather than deficiencies in assessment itself. (21-22)

So what ethical and professional responsibility do assessment experts like Porter take when their processes or results are misused by upper administrators or external stakeholders?  In what ways do they defend the scientific rationality of their enterprise or the deliberative process around the dissemination of its results?  How much do assessment experts attend to the organizational learning of the units whose behavior they analyze?  And how reflectively do they analyze their own role in the inevitable distributions of power and hierarchy throughout educational institutions?

In effect, to call his (and his tribe’s) long-term goal the installation of a “culture of evidence” gives the game away, because it means that the goal is not so much about instilling a respect for universal scientific reason across the university community, but trying to see whether his specific culture’s goals and values might be shared by the others in the university.  And I don’t see any reason why they shouldn’t, so long as we felt that they helped to strengthen the university as it pursues its mission.

Trying to defend not just one’s discipline but the contemporary university, as mixed as it is, means making some attempt to understand the other disciplines’ practices and what they are attempting to do; it also means developing a professional and reflective view of one’s own instructional activities and curriculum, so that students can begin to understand your discipline’s knowledge structures; finally, it means having some account of how your discipline and your university are affecting the lives and livelihoods of the people who help to support you.  How unreasonable does that sound?

DM

UPDATES:

  • If you want a really enraging example of how privatizing a public good completely distorted it, check out this piece by Andrew Leonard about Sallie Mae’s role in the student loan debacle.
  • According to this piece in College Guide, Iowa is the latest state to devise new budgetary gimmicks to reduce its support of in-state college students.
  • Sherman Dorn offers a much better and more patient demolition of David Brooks (sorry, no link for you, DB) on accountability than I could ever do, and gets into some of the real complexities it poses, especially for accreditation.

If others have additional links or sources they’d like to bring into the discussion, put them into the comments and I’ll put them up here. Thanks, DM

About these ads

6 responses to “fighting the last war: UPDATED

  1. Laura Rosenthal

    Good points. I remember thinking, similarly, when the first two articles you mentioned came out that they felt a little dated, although you are offering here a more complicated analysis. I think, though, that accountability advocates have contributed to thinking about the issue of student debt, although perhaps indirectly. One theme of *Academically Adrift*, for example, is the balance between attention to student learning and expensive “extras” like sports teams and the synecdotal ‘climbing wall’ that drive up the price of higher education.
    Also, I don’t think all assessment advocates share Porter’s social science perspective (although some do). Peter Ewell calls assessment “craft research”; I have also heard it called “action research” for the same purpose—not meant for publication or advancing the field, but for making improvements in a local, idiosyncratic context.

  2. Dave Mazella

    I think the Academically Adrift volume is going to force a more precise analysis of engagement, but I’m not totally satisfied with their analysis, either, which is based on a standardized instrument, the CLA I think, that raises its owns questions. A friend of mine at UT showed me some data they’re working on that supports the notion of extracurricular activities boosting student engagement generally; from what I’ve seen, the higher the engagement level, the better the chances for success and timely graduation. Given the students that we (at UH, and nationally) are more likely to get in the next 10-20 years, my concern is the number of hours students work while they carry academic loads, and how that cuts into engagement.

    The best descriptions I’ve found of what we need are in the work of Michael Quinn Patton and “developmental evaluation,” a framework which is explicitly concerned with evaluating complex situations and responding in timely and effective ways. How much this has penetrated the world of academic assessment, I don’t know. What I do know is that the cognitive and organizational complexity of educational institutions and activities tremendously complicates any efforts towards evaluation.

  3. Laura Rosenthal

    Certainly more research about the issues they raise is needed. My only point was that their book offers one example of how student debt is addressed in the context of accountability. I think it’s very much a part of the larger conversation even if it’s missing in certain particular accounts. Also, like anything else, the accountability movement is diverse and aappealing to different people for difference reasons, so one is unlikeley to find a single response, cogent or otherwise.

  4. Dave Mazella

    Sure, more research is needed. What I’m looking for in both the research and the policy discussions is an acknowledgment of the role played by states’ disinvestment in the cost side of the equation. It’s not like rainfall; deliberate policy decisions, for example, switched federal support from outright grants to loans to students in the ’80s, but the impact of that wasn’t truly felt until states stopped contributing to their university systems, and forced students to take on increasing amounts of the cost via debt. Most of the mainstream discussion is focused on productivity (=tenure, research, “lazy professors,” ivory tower) rather than long-term state support.

  5. dave porter

    Dave, Thank you for your review of my recent essay in the AAUP’s Journal of Academic Freedom. From what you have written here and elsewhere, I can see there is a great deal of overlap in your perspective and the one I tried to articulate in the essay. Just as war is too important to be left to generals and the corporate tycoons of the military-industrial complex, assessment urgently needs the participation of informed, dedicated, and diverse faculty members. In my opinion, faculty members are the primary owners of any sustainable higher education assessment program.

    I regret the social scientific tribal connotations and any slight to the contributions of qualitative research alluded to in your review. As with your experience with your QEP and establishing a Center for Teaching Excellence at UH, I’ve found inclusive conversations among diverse faculty about substantive educational issues at many levels to be of great value. In fact, after hearing Erik Olin Wright speak recently about real utopias, I’d have to include Sociological Marxism http://www.ssc.wisc.edu/~wright/SocMarx.pdf ; http://www.ssc.wisc.edu/~wright/ERU.htm among the approaches that may have the most to offer educational assessment programs.
    I found your comments about the funding of higher education and the serious problem of student debt to be right on target. As I am sure you are already aware, graduating with a baccalaureate degree by the age of 24 is largely determined by one’s family income ( http://www.decisionsonevidence.com/2012/03/is-higher-education-perpetuating-class-hierarchies/ ).
    I’m not sure how much you know about Berea College, but I thought you and your readers might be interested in this unique institution. Founded in 1855, Berea College was the first school south of the Mason-Dixon Line to offer educational opportunities to both black and white women and men. A distinctive characteristic of the college today is that it only admits students with significant financial need (i.e., basically those who are Pell eligible) and requires every student to work at least 10 hours per week throughout their time at the college. About 75% of the annual $22K cost of education is paid mostly by a billion dollar endowment, but about 25% of the annual tuition costs come from private donors and state and federal grants. Another remarkable characteristic of the college is that the average debt of its graduates (slightly less than $6K) is lower than any other college or university in the nation (http://projectonstudentdebt.org/state_by_state-data.php). About 20% of our students are African American and another 10% were born outside the United States. We have sustained a graduation rate near 65% for most of the last decade, and rank among the top 8% of colleges and universities in terms of the proportion of our graduates who earn doctoral or terminal professional degrees. Named by the Washington Monthly in 2011 as the top liberal arts college in the nation in serving the broader needs of society http://bcnow.berea.edu/2011/08/berea-college-1-in-washington-monthly%E2%80%99s-2011-college-rankings/ , Berea College also had the distinction of being one of the dozen schools invited to the White House by President Obama last December to examine ways to reduce the cost and increase the productivity of higher education. Hopefully, the development of an even more effective and inclusive assessment program under a new administration will contribute further to Berea College’s continuing accomplishment of its distinctive educational mission.

  6. Dave Mazella

    Hi Dave,

    Thanks for the treatment of Powell and Champagne, and for engaging with my response. I wasn’t particularly worried about the disciplinary differences, which I think will be a given when people try to engage in these kinds of broader discussions. I think focusing on the defense of the university and the students help all of us to surrender or at least modify some of the disciplinary, tribal gestures we sometimes fall into. And I can say from experience that collaboration is the quickest way to begin learning humility about the knowledge and skills maintained by practitioners in other disciplines.

    As I hinted in my response, I feel that the more interesting question is why assessment, when practiced within the institutional contexts we’re familiar with, seems regularly to fall so far short of the potential it would seem to have to change things, and routinely so. In other words, assessment seems as if it should be able to change practices, but it seems only rarely to accomplish this. Why not? What kinds of institutional changes would be necessary to realign practices so that they could be better, more regularly informed by the evidence contained in assessments? Or what kinds of alterations in assessment practice would be necessary for the results to be used more regularly and reflectively?

    In other words, most universities struggle with the meta-assessment, or the assessment of assessment, necessary to feel that it is fully integrated into the process of improving the creation and teaching of the disciplinary curricula, which to me seem as unreflective as ever, with a few notable exceptions. Would you agree with me about that?

    There is certainly an obstructive or anti-intellectual response from disciplinary faculty to contend with, but there are clearly questions about how academic leaders use, or don’t use, the results; there is also the complete indifference of most external stakeholders to the evidence- and research-based paradigms of teaching and learning that assessment properly belongs to.

    However, since we have most say over the faculty role in instruction, we can look at the faculty side of this and to some extent reshape it. The argument I developed, which I see you’re independently elaborating, is that assessment can only play this role when departments, colleges, and universities are consciously organized and run as “learning organizations.” At my institution, and in my CTE, we’re attempting to address this shortfall with a more conscious emphasis on reflective teaching as part of one’s professional development, to be developed via Wergin’s notions of departmental engagement. We’ll see how this goes.

    Finally, thanks for the info and links about Berea and the Decisions blog. I think faculty at all these kinds of institutions need to be able to communicate with one another, because effective teaching is so much about working effectively within specific contexts, and what we learn is often produced by reflective comparisons across contexts. So I am glad to hear that Berea seems to be doing so well graduating its students; we could all learn something from Berea’s example.