Tuesday, November 14, 2017

Universities are underinvesting in efforts to improve the quality of teaching

My friend and colleagues, Michael O’Hare (a Professor at UC-Berkeley), points out in a recent paper entitled The 1.5% Solution: Quality Assurance for Teaching and Research that major research universities underinvest in continuous improvement of their teaching efforts. Given that universities have only two primary tasks -- teaching and research – they ought to be willing to invest as much in improving the quality of their teaching as they do in providing an elaborate infrastructure to support basic and applied research. But, that doesn’t seem to be the case.

O’Hare calculates that major universities devote something like $300,000 to present a semester-long course (i.e. student time, rooms, professor’s salary, web, teaching assistants, etc.). This is what it takes to ensure that faculty and students are present in the right place, at the right time, with the resources they need. He assumes, for planning purposes, that a course is taught to 50 students; faculty at research universities carry a three-course-per-year teaching load, teaching is half a professor’s academic year time, and fringes and benefits are included. To increase student learning by 5%, therefore, O’Hare estimates that it ought to be worth spending $45,000 per year, per professor, to improve the quality of teaching and student learning. Unfortunately, nothing close to that is currently being spent.

O’Hare suggests that universities ought to invest 1.5% of their faculty payroll in quality assurance to improve teaching performance – in much the same way that almost every industry invests in quality assurance as it seeks to improve its efficiency and effectiveness.

O’Hare points out three ways that any and every university department could try (at very modest cost) to improve the quality of its teaching. These follow closely what other segments of the economy have learned about quality improvement. While teaching is not the same as producing most other products or services, I’m convinced (after almost 50 years as a teacher at MIT) that the most basic quality assurance strategies do apply equally well to the university.

Instructors should talk more with each other. You might not believe it, but it is very rare for MIT faculty to sit in on each other’s courses to observe and offer advice on possible ways of improving teaching. And, similarly, faculty members almost never compare notes before classes begin on what they are proposing to cover in their classes and how they intend to go about teaching the material. Everyone is presumed to be a subject matter expert; although why this is presumed to carry over into teaching expertise is beyond me. If a Department made it a policy that every faculty member should expect one of their colleagues to sit in on at least one of their class sessions each semester, then no one would feel singled out. While such assignments could be made absolutely randomly, I don’t see a problem asking the faculty to choose the colleague they want to have sit in. In an after-class discussion, I would hope that the person observing would suggest (1) things I saw you do that I’m going to try myself (and why), and (2) things I’m going to suggest you might find helpful. I don’t think such reports need to be submitted in writing to the relevant Department, but it might be valuable if the person being observed wrote a short summary of what they heard and how they intended to take the feedback on board.

Instructors should make a greater effort to help students learn to teach each other more effectively. Faculty are used to giving students formal feedback (i.e. graded tests and quizzes) on how well they have mastered the material presented in class. It seems to me that faculty could also observe each student giving feedback to a fellow student, and translate that into an occasion to help every student get better at giving constructive feedback and advice to their peers. We need to make it easier for our students to learn from each other. In one of my classes, I ask a few students to make six minute oral presentations -- in a hypothetical work situation -- drawing on what they have learned that week in class. As soon as they are done, every student in the class is asked to use a one page printed template highlighting five or six aspects of the presentation to provide the presenter with immediate feedback. In addition to noting what was done well and what could be improved, each student provides several sentences of commentary. This is all done in five to seven minutes. Each presenter that gets 25 separate sources of feedback on their presentation. This has nothing to do with their grade. Everyone in the class makes at least three oral presentations throughout the term. Each bit of feedback is not anonymous. We always say students learn as much from the other as from their professors, but what do we do to make sure that happens? Nothing. I think faculty should commit to make sure that students learn (as part of every course!) how to help their fellow students learn as much as they can from the class. It should be the faculty member’s responsibility to instruct and support students as they help each other learn. I think that academic departments should insist that faculty make an effort to get better at doing this.

Academic departments should measure everything they do on a continuous basis. There’s nothing new about this idea. Arthur Demming pointed out many years ago, in the context of industrial activities, that anything not measured is not likely to be improved. What to measure, though, in the context of university teaching, is not clear. Most universities currently measure student satisfaction immediately at the end of a semester-long class. More than anything else, this tends to gauge the popularity of the professor. I’ve rarely seen student course evaluations lead to improvements in teaching strategy or performance. What else might be measured? It seems obvious it would be a good idea to measure student knowledge about course material before and after each segment of a class, as well as before and after the entire course. This works if a class is mostly aimed at helping students master substantive knowledge. But, if a class is supposed to teach students how to do something, it makes more sense to give students simulated opportunities to see whether they have mastered the relevant skills. Digital simulations are expensive to build, but they work. Face-to-face role play simulations are not expensive to create, and they work as well. When groups of students in a class play the same game separately, comparisons of the results and student reflections on the experience can give faculty a clear idea of what they are conveying effectively and what needs improvement. I’ve found that saving the last three minutes of a class to ask students what they took away from the session often generates surprising responses. It certainly helps me recalibrate when what they report don’t correspond with what I thought I was teaching! I’m in favor of asking each faculty member what they intend to measure so that they can improve their teaching performance. A university department should provide technical support to make this happen. Then, with the relevant data in hand, each faculty member should commit in writing to experiments or reforms in their next round of teaching, along with a clear indication of what they will measure next time.

I know that there will be substantial resistance to these three simple ideas. Non-tenured faculty will be worried that admitting there is room for improvement in their teaching may somehow jeopardize their reappointment. Tenured faculty have little or no incentive to invest in getting better at teaching. To date, most faculty members at most research universities have not been asked to focus on teaching their students to teach their classmates. This will be seen as an (uncompensated) expansion of the faculty’s role and responsibility. Most faculty won’t know how to do this. Departments will complain that arranging a system of faculty visits to each classroom is a new administrative task for which they are unprepared. Systematically measuring teaching performance (and improvements in teaching performance) is not something that academic administrators know how to do. Nevertheless, I would argue that University leaders should pursue Professor O’Hare’s 1.5% solution to the problem of improving teaching effectiveness. There’s really no good excuse for not getting better at what we do.

Monday, September 18, 2017

Consensus building in the Age of Trump: Strategies for the ADR Field

What’s it like in The Age of Trump?

What’s special about the Age of Trump?  I would point to two things. First, our political leaders (not just the President) no longer feel an obligation to represent all the people in the district or state that elected them. Now, they only feel accountable to their “base.” This is a relatively new occurrence (not just in the United States, but in other democracies as well).   It used to be that after politicians were elected they felt some obligation to represent the interests of all the people in their district or state.  As a result, we now have districts or states (or countries!) where 49.9% of the electorate has no representation.  This makes them feel angry, anxious and defense.  It also makes them feel combative.

The second thing that has changed, and it is related to the first, is that many elected and appointed officials don’t care what evidence or  arguments anyone on “the other side” presents. They won’t allow themselves to be convinced by what anyone outside their base has to say. This means that those in control of the levers of power can pursue whatever agenda they choose, without having to explain or justify their actions in a manner that “an independent observer” would agree is reasonable. This adds to the outrage, and even desperation, of those who feel shut out and unrepresented.  They are especially angry that scientific evidence can be ignored entirely.

So, in the Age of Trump, many people who have not felt powerless before feel powerless now.  They are befuddled by the changes that have occurred in the rules of the game. In the past, they assumed (maybe somewhat naively) that their elected leaders would choose the common good over narrow partisan interests; and, they counted on being able to advocate for what they believe by presenting credible evidence. Now they assume these things won’t happen.

Special challenges for Consensus Builders and other ADR professionals

ADR professionals operate in ways that are intended to ensure fairness – to ensure that all voices are heard and all interests are taken into account when disagreements arise.  In a decision-making or governance system that rejects the idea that the interests of all groups matter, ADR professionals are not quite sure what part they are supposed to play.  The reason those of us in the ADR field have worked hard to add facilitation, mediation and arbitration to public and private efforts to deal with differences, is to enhance the fairness, efficiency, stability and wisdom of decisions that must be made. In the judicial, executive and legislative branches, at every level of government, we have spent decades demonstrating that adding a professional neutral can, in fact, save time, save money and produce better outcomes (and give stakeholders greater control over what happens to them).  In the Age of Trump, ADR professionals now wonder how they can do their job if some of the parties don’t care what the interests of the other parties are; or, some parties feel no obligation to listen to or present credible evidence to support their claims.  Many ADR professionals are extremely upset about these changes. Some are so upset they feel compelled to invest their personal time in political efforts to put things back the way they were. When this involves advocacy, though – even when the professionals involved are operating as private citizens --  it threatens our most important professional asset – our neutrality.

Neutrality is central to the value we add as ADR professionals. Our neutrality allows us to earn the trust of all sides in any dispute.  It also means we can operate in the interstices between the parties and, in so doing, carry messages and provide cover for parties to come together without appearing to be weak. My contention is that many ADR professionals are so upset by what is happening in the Age of Trump that they are ready to risk their perceived neutrality.  While I understand their motives, I am convinced this would be a disaster for the profession.

Increasing demand for ADR assistance in periods of heightened conflict    
The Age of Trump has certainly generated new conflicts of various kinds.  When everyone is escalating their efforts on behalf of their own point-of-view, and more people feel entitled to act in the own interests regardless of the interests of “the other side,” there ought to be increasing demand for our services.  So, in these times, we ought to be able to make a greater contribution (in part because no one else is offering to reconcile those in conflict or pursue problem-solving strategies in spite of the conflicts that exist).  To succeed in the current context, however, will require several things:

1.   First, we have to remind our potential clients that our goal is not to stamp out conflict.  Rather, if they find themselves stalemated and unable to take unilateral action, we can help them find agreeable ways forward in which no one has to give in.  

2.   Second, if well managed, conflict can lead to produce change. Conflict is not a bad thing.  As others have noted, it is the engine of change.  We can help manage conflict in a constructive way.

3.   Third, the fact that parties are inclined to express their interests and concerns with more passion in the Age of Trump, is not a problem for us. In some ways, it should make our work easier. We need to know what the interests and priorities of each party are so we can help them formulate mutually beneficial agreements. We do this by supporting the parties in their search for trades (across issues they value differently) that produce outcomes better for all sides than their BATNAs.

4.   Finally, we need to be sure that our clients understand that our job is not to get anyone to change their beliefs or change their mind.  We try to help parties reach mutually advantageous agreements in spite of their differences.  We do not allow our own point of view or our own preferences to intrudce.

Harmonizing Interests through dialogue vs. assisted problem-solving

A segment of the ADR profession has been moving in the direction of facilitating dialogue. Indeed, there are many who think we should devote a substantial portion of our time to helping Red and Blue (and others who have conflicting values) learn to talk with and understand each other more effectively.  I’m personally not convinced that dialogue for its own sake should be a high priority for the ADR profession.  I don’t think greater understanding is going to lead to harmonization of conflicting values and interests.  Perhaps we can help people with diametrically opposed views hear each other, but I’m not sure that’s as important as working out agreements in specific contexts.  I think we should emphasize problem-solving -- generating “a workable peace” when some action needs to be taken -- rather than devoting time to generating a deeper understanding of the sources of disagreement.  I don’t think Red and Blue need to believe the same things to find ways of taking action.

The key is to convince as many stakeholders as possible that there is a way to meet their interests in a manner that will get them more than what no agreement (stalemate) guarantees, and more than they are likely to get if they continue to battle.

Coming back to neutrality

As I have already said, we must be absolutely diligent about maintaining our neutrality – no matter how strongly we feel personally – if we want to make a case for the value we add. I’m convinced that the way we act in our personal lives may shape how we are perceived in our professional roles. While each of us has opportunities to take direct political action in our personal lives, remember that if you sign a petition, march peacefully, write op eds, or lobby for your point of view, there is no way anyone on the other side will accept you as a dispute resolution professional they can trust. We need to think very carefully about how we carry ourselves in public. I promise you that whatever actions we take in our personal lives will be noted. Being perceived as neutrals in the Age of Trump is, in my view, the key to contributing to conflict resolution in these difficult times.

[Based on the keynote presentation I made to the Biennial Conference of the New England Association for Conflict Resolution (NEACR) in Waltham, Massachusetts on 9/8/17.]

Sunday, August 13, 2017

Big Data, Urban Science and the Search for New Ways of Improving Life in the City

 Imagine you had all the data you could possibly want about a city. I’m talking about real time readouts of all inputs and output, along with documentation of public satisfaction levels with all aspects of city life.  Further, presume you had access to similar information for other cities as well, so you could see trends and patterns. What might you do with all this information to help improve the quality of life in the city? 

There are some urban planners and city administrators who believe that if they had that kind of information they would be able to figure out when and how to spend public money, allocate equipment and personnel and restrict and support private efforts most efficiently.  They also think they could use the same data to anticipate certain tipping points so that traffic snarls could be avoided before they happen, public works staff (including police and fire) could be deployed where they would be most needed, and infrastructure repairs could be managed with minimal disruption. With such information, the presumption is, water, electricity and other city resources could be priced and deployed on a real-time basis in the most cost-effective way possible. Empty housing and commercial space could be repurposed almost immediately, and priced to match the city’s urban development objectives. Taxes and fees could be collected electronically while feedback from residents could be shared with officials on a continuous basis. With the right kinds of electronic monitoring (including sensors of all kinds), sufficient data collection, investment in high-level analytic capabilities and appropriately trained staff, a city could become a “smart city.”  All this digitized data could be displayed in visual form -- across multiple platforms – so that everything would be easy to read and understand.

Of course, knowing what problems a city faces, and even understanding what’s causing them, is not the same thing as being able to respond effectively. The availability of real time data, even with the most advanced application of artificial intelligence, won’t make it clear who ought to do what, in what order, in what way and for whose benefit. These are political choices: “is” does not lead directly to “ought”

One school of thought, promoted by some economists and engineers, assumes that the goal of city management should be to maximize efficiency – eliminate waste and stretch every dollar as far as possible.  They want to make sure public that tax revenue, fees and intergovernmental transfers are allocated in the most cost-effective fashion. If the goal is to collect trash, arrest criminals, clean-up air and water pollution, or fight climate change, money shouldn’t be wasted in the process.

A second school of thought, inspired by ecologists and advocates of sustainable development, believes that every dollar of public spending should be used to meet economic, environmental and social needs simultaneously, in ways that takes account of long-term needs. Efficiency in the short-term isn’t as important to these thinkers as long-term sustainability (which includes meeting the needs of both current and future generations in as fair a way as possible). All the data in the world won’t make it clear what ought to be done. In a democracy, such choices need to be made through a messy process of reconciling conflicting interests and values in which the population participates directly. Efficiency isn’t always the highest priority goal.  

My own university, MIT, is thinking about launching a new undergraduate degree program in Big Data and Urban Science. This would bring together faculty from urban planning, information science, electrical engineering, city design and the applied social sciences to prepare undergraduates to build and operate smart cities. Other universities, as described in a recent issue of the Chronicle of Higher Education, are a step ahead.  NYU, Northeastern, Carnegie-Mellon, John Hopkins, University of Illinois, University of Rotterdam and others have already launched undergraduate and graduate degree programs that seek to merge teaching about big data and urban studies. 

As faculty in all of these programs try to decide what skills and knowledge they want a new generation of urban scientists to master, there are six questions I think they need to confront:

1.     What do we mean by a city?  (Are they talking about activities that take place within a municipal boundary, or will they focus on a larger set of regional, national and international forces that shape urban life more generally?)
2.     Do they think that privacy is a concern? (Do they assume that any and all information that can be collected, should be collected? And should this information be available to anybody who wants to use it?  Can people or organizations opt out and keep information about themselves private?)
3.     How will we fend off cyber-attacks on critical urban infrastructure that have already begun? (If urban science means greater centralization of information and data management, won’t that increase vulnerability to attack?)
4.     Who do they think should be in charge of designing and managing big data systems and setting the standards used to make judgments about what’s working well and what’s not? (Will this be a managerial task assigned to various government agencies, or will elected officials be accountable for how all this information is collected, analyzed and interpreted?  Will new laws be required to ensure that individual and organizational rights are protected? Will this require federal, state or local legislation? Where will enforcement responsibility sit?)
5.     Will they be working from and toward an idealized model of an efficient city, or will they work to preserve historical and cultural diversity and variation? (I presume that students from all over the world will want to participate in these programs? Won’t the differences in culture, laws, and history require very different ways of applying the new urban science in each country? Should we assume that this is basically a technical education and teach a kind of “one-world view”? Or, would that be a terrible mistake?)
6.     Are the universities involved aiming to prepare public employees (whose job it is to serve the public interest), or are they training experts who will sell their services to the highest bidder?

The City As a Place vs. the City as an Idea

Most efforts to model urban dynamics assume that a city can be described as a series of “stocks” and “flows” within a set of boundaries.  While there are always important “feedback loops,” many of which are likely to cause unexpected consequences, most modeling (and forecasting) efforts begin by postulating a set of boundaries.  But, what if cities, as many urbanists contend,  are largely a product of a great many extra-territorial (even global) forces? Capital or data flows originating in other parts of the world may have as much of an effects as  economic and social forces originating in the city. Moreover, if we say that the operation of many of the sub-systems in a city reflect the ways that groups of people or institutions think about things – their perceptions -- how do we include these in the models we teach students to build?  The city is a physical space affected by global geological and ecological forces. It is also an idea shaped by millions of individual perceptions. Will it be possible to make sufficiently simplified models of the city to generate useful insights and predictions?

What’s Confidential and What’s Not?

Assume we can answer the first question, and we know which data are required to make a city substantially smarter.  Gathering some of these data will require tapping into otherwise secure or private data sources.  And, even if we argue that individual identities will be scrubbed, should people and organizations be forced to give up their privacy in the name of the greater good? In the United States, we are currently watching a version of this question play out as a National Commission on Elections and Voting demands that all 50 states to turn over voting data that may reveal how specific individuals voted. When it comes to financial records (including tax returns and credit card expenditures), even if these data are crucial to modelling how a city is doing, should individuals have a right to keep such personal data confidential?  What ethical obligations will we impose ona new generation of big data analysts or urban scientists?

Cyber-Security in the Urban Realm

Urban infrastructure is already under attack from hackers who seek to hold energy, medical, water, sewage and other systems hostage. Each new layer of encryption added in response just ups the ante. These systems are vulnerable because individuals are not as conscious of their cyber-security responsibilities as they should be.  Solutions will require further technological innovation and investment, but that won’t be sufficient.  What will we teach a new generation about cyber-security and how to maintain it?  And, given the vulnerability of critical urban infrastructure to global attack, should that affect what we guarantee with regard to privacy and control over personal data?

Knowledge, Power and Authority

Let’s say a city has put together a comprehensive data gathering, analysis and visualization operation. Who will have access to the raw data?  Who will have the right to publish analyses of the information that has been collected?  Will the city be willing to share assessments of things that residents think are going badly?  Will those who want to challenge current office holders be allowed access and permitted to publish any analysis they like?  Who will make decisions about how data should and should not be interpreted?  It’s my assumptions that managers of smart cities will have “to do” lists that far exceed their resources.  Setting priorities (often in real time) will require quick decisions, faster than the public can follow. If all big data about cities were open sourced, would that allow more citizens to be involved in helping to make decisions that are going to affect them?  While real time referenda might be possible, is that how cities should set priorities and make judgments?

Is There an Ideal City?

Urban planners are very place-oriented; data scientists are not.  Urban planners want to preserve the special historical and cultural features of each city.  Data scientists, on the other hand,  are looking for rules of thumb to describe the most efficient ways of delivering goods and services in general. They might inclined to disregard inefficiencies that are a by-product of local history, culture or values.  There’s no point collecting data if there’s no intention to use it, but putting all these data to use means measuring how things are going compared to some benchmarks.  Should benchmarks be unique to each community?  Or, is the goal of merging big data and urban science to create “ideal benchmarks” (based on studies of many cities over time)?  Should urban science be practiced differently in different cities, let alone different countries?

Public Sector vs. Private Sector Careers 

Urban planning education in North America has been provided by major colleges and universities for more than 80 years.  The majority of graduates of such programs aspire to work in the public sector or in civil society (e.g. NGOs or public interest organizations).  This is true regardless of where students originate.  Of course, some graduates find private sector jobs, either temporarily or permanently in consulting firms or corporations. Whether they are headed to the public or private sector, students studying urban planning tend to focus on ways of meeting the needs of the poor and the disadvantaged; they start with a theory of market failure and look for ways of using public-private partnerships, regulation, public investment or political advocacy to meet the needs of those for whom the market tends to fail.  The engineers and scientists likely to be drawn to these new urban science programs may not be so public sector oriented.  Will the new urban scientists/big data managers who graduate from these programs be public sector/civil society or private sector oriented?

I see the emergence of interdisciplinary urban science programs around the world as a good sign. Merging the capabilities of scientists and engineers with applied social scientists, designers and urbanists interested in the life of urban residents would be a positive development.   We need to provide all the help we can to people in cities trying to make adjustments and reforms that reflect a clear-headed awareness of the complex dynamics they face. I worry, though, that some universities moving in this direction may pay too much attention to the advice of economists and management gurus obsessed with numerical trends, who are willing to focus on correlation because they don’t have the tools to understand causal dynamics. I hope that the applied social scientists and urban planners will succeed in ensuring that progressive values like concerns about fairness and sustainability at the core of the training of a new generation of urban scientists. I’m certainly glad that most of the people involved in these new efforts appear to be committed to blending schools of thought that have operated separately for too long.