Anyone working to build a learning organization will, sooner or later, run up against the challenge of “proving” the value of what he or she has done. Without some form of assessment, it is difficult to learn from experience, transfer learning, or help an organization replicate results.

But assessment strikes fear in most people’s hearts. The word itself draws forth a strong, gut-level memory of being evaluated and measured, whether through grades in school, ranking in competitions, or promotions on the job. As writer Sue Miller Hurst has pointed out, most people have an intrinsic ability to judge their progress. But schools and workplaces subjugate that natural assessment to the judgment of teachers, supervisors, and other “experts,” whose appraisals determine promotions, wealth, status, and, ultimately, self-esteem.

Assessing Learning

Is it possible to use assessment in the service of learning? Can assessment be used to provide guidance and support for improving performance, rather than elicit fear, resentment, and resignation? This has been a guiding question at the MIT Center for Organizational Learning for several years, as we have struggled to find a reasonable way to assess learning efforts. The motivations are essentially pragmatic: our corporate affiliates need some idea of the return on their investments, and we as researchers need a better understanding of our work.

To create a new system of assessment, we started by going back to the source — to the people who initiate and implement systems work, learning laboratories, or other pilot projects in large organizations. We then tried to capture and convey the experiences and understandings of these groups of people. The result is a much-needed document that moves beyond strict assessment into the realm of institutional memory. We call it a “learning history.”

The Roots of a New Storytelling

A learning history is a written document or series of documents that is disseminated to help an organization become better aware of its own learning efforts. The history includes not just reports of action and results, but also the underlying assumptions and reactions of a variety of people (including people who did not support the learning effort). No one individual view, not even that of senior managers, can encompass more than a fraction of what actually goes on in a complex project — and this reality is reflected in the learning history. All participants reading the history should feel that their own points of view were treated fairly and that they understand many other people’s perspectives.

A learning history draws upon theory and techniques from ethnography, journalism, action research, oral history, and theater. Ethnography provides the science and art of cultural investigation — primarily the systematic approach of participant observation, interviewing, and archival research. From journalism come the skills of getting to the heart of a story and presenting it in a way that draws people in. Action research brings to the learning history effective methods for developing the capacities of learners to reflect upon and assess the results of their efforts. Finally, the tradition of oral historians offers a data collection method for providing rich, natural descriptions of complex events, using the voice of a narrator who took part in the events. All of these techniques help the readers of a learning history understand how participants attributed meaning to their experience.

Each part of the learning history process — interviews, analysis, editing, circulating drafts, and follow-up — is intended to broaden and deepen learning throughout the organization by providing a forum for reflecting on the process and substantiating the results. This process can be beneficial not only for the original participants, but also for researchers and consultants who advised them — and ultimately for anyone in the organization who is interested in the organization’s learning process.

Insiders versus Outsiders

One goal of the learning history work is to develop managers’ abilities to reflect upon, articulate, and understand complex issues. The process helps people to hone their assessments more sharply by communicating them to others. And because a learning history forces people to include and analyze highly complex, dynamic interdependencies in their stories, people understand those interdependencies more clearly.

In addition, the approach of a learning history is different from that of traditional ethnographic research. While ethnographers define themselves as “outsiders” observing how those inside the cultural system make sense of their world, a learning history includes both an insider’s understanding and an outsider’s perspective.

Having an outside, “objective” observer is an essential element of the learning history. In any successful learning effort, people undergo a transformation. As they develop capabilities together, gain insights, and shift their shared mental models, they change their assumptions about work and interrelationships. This collective shift reorients them so that they see history differently. They can then find it difficult to communicate their learning to others who still hold the old frame of reference. An outside observer can help bridge this gap by adding comments in the history such as, “This situation is typical of many pilot projects,” or by asking questions such as, “How could the pilot team, given their enthusiasm, have prevented the rest of the organization from seeing them as some sort of cult?”

Similarly, retaining the subjective stance of the internal managers is important for making the learning history relevant to the organization. In most assessments, experts offer their judgment and the company managers receive it without gaining any ability to reflect and assess their own efforts. The stance of a learning history, on the other hand, borrows from the concept of the “jointly told tale,” a device used by a number of ethnographers in which the story is “told” not by the external anthropologist or the “naive” native being studied, but by both together. For these reasons, the most successful learning history projects to date seem to involve teams of insiders (managers assigned to produce and facilitate the learning history) working closely with “outside” writers and researchers hired on a contractual basis.

Results versus Experience and Skills

Companies today don’t have a lot of slack resources or extra cash. Thus, in every learning effort, managers feel pressured to justify the expense and time of the effort by proving it led to concrete results. But a viable learning effort may not produce tangible results for several years, and the most important results may include new ways of thinking and behaving that appear dysfunctional at first to the rest of the organization. (More than one leader of a successful learning effort has been reprimanded for being “out of control.”) In today’s company environment of downsizing and re-engineering, this pressure for results undermines the essence of what a learning organization effort tries to achieve.

One goal of the learning history work is to develop managers’ abilities to reflect upon, articulate, and understand complex issues.

Yet incorporating results into the history is vital. How else can we think competently about the value of a learning effort? We might trace examples where a company took dramatically different actions because of its learning organization efforts, but it is difficult to construct rigorous data to show that an isolated example is typical. Alternatively, we might merely assess skills and experience. A learning historian might be satisfied, for instance, with saying, “The team now communicates much more effectively, and people can understand complex systems.” But that will be unpersuasive — indeed, almost meaningless — to outsiders.

In this context, assessment means listening to what people have to say, asking critical questions, and engaging people in their own inquiries: “How do we know we achieved something of value here? How much of that new innovation can we honestly link to the learning effort?” Different people often bring different perceptions of a “notable result” and its causes, and bringing those perceptions together leads to a common understanding with intrinsic validity.

For example, one corporation’s learning history described a new manufacturing prototype that was developed by the team. On the surface, this achievement was a matter of pure engineering, but it would not have been possible without the learning effort. Some team members had learned new skills to communicate effectively with outside contractors (who were key architects of the prototype), while others had gained the confidence to propose the prototype’s budget. Still others had learned to engage with each other across functional boundaries to make the prototype work. Until the stories of these half-dozen people were brought together, they were not aware of the common causes of each other’s contributions, and others in the company were unaware of the entire process. The learning history thus included a measurable “result” — the new prototype saved millions of dollars in rework costs — but simply reporting a recipe for constructing new prototypes would be of limited value. At best, it would help other teams mimic the original team, but it wouldn’t help them learn to create their own innovations. Only stories, which deal with intangibles such as creating an atmosphere of open inquiry, can convey the necessary knowledge to get the next team started on its own learning cycle.

The Strength of the Story

Some learning histories have been created after a project is over. Participants are interviewed retrospectively, and the results of the pilot project are more-or-less known and accepted. Other histories are researched while the story unfolds, and the learning historian sits in on key meetings and interviews people about events that may have taken place the day before. “Mini-histories” may be produced from these interviews, so that the team members can reflect on their own efforts as they go along and improve the learning effort while it is still underway. But such reflection carries a burden of added discipline: it adds to the pressure on the learning historian to “prove results” on the spot, to serve a political agenda, or to justify having a learning history in the first place.

To create an ongoing learning history, an organization must embrace a transformational approach to learning. Instead of simply learning to “do what we have always done a little bit better,” transformational learning involves re-examining everything we do—including how we think and see the world, and our role in it. This often means letting go of our existing knowledge and competencies, recognizing that they may prevent us from learning new things. This is a challenging and painful endeavor, and learning histories bring us face to face with it. When the learning history is being compiled simultaneously with the learning effort, then the challenge and pain of examining existing frameworks is continuous. But to make the best of a “real-time” learning history, admitting and publicizing mistakes must be seen as a sign of strength. Uncertainty can no longer be a sign of indecisiveness, because reflecting on a learning effort inevitably leads people to think about muddled, self-contradictory situations. Much work still needs to be done on setting the organizational context for an ongoing learning history so that it doesn’t set off flames that burn up the organization’s good will and resources.

Currently, there are almost a dozen learning history projects in progress at the Learning Center. In pursuing this work, we no longer talk about “assessing” our work. Instead, we talk about capturing the history of the learning process. It is amazing how this approach and new language changes the tenor of the project. People want to share what they have learned. They want others to know what they have done — not in a self-serving fashion, but so others know what worked and what didn’t work. They don’t want to be assessed. They want their story told.

George Roth is an organizational researcher with the MIT Center for Organizational Learning and a consultant active in the study of organizational culture, change, and new technology introduction.

Art Kleiner is co-author and editorial director of The Fifth Discipline Fieldbook, and author of the forthcoming The Age of Heretics, a history of the social movement to change large corporations for the better.