Cognition, Affect, and Learning
The Role of Emotions In Learning
Barry Kort, Ph.D.
Affective Computing Research Group
MIT Media Lab
Affective Computing Research Group
MIT Media Lab
Our species, Homo Sapiens, is the being who thinks. But we are also the beings who learn, and the beings who experience a rich spectrum of affective emotional states.
This article presents research and theoretical models relating emotions to learning and cognition.
This article presents research and theoretical models relating emotions to learning and cognition.
- Basic Concepts and Terms
- Role of Emotions In Learning
- Analogizing Motion (Time, Distance, Velocity and Acceleration) to the Learning Process (Time, Knowledge, Learning, and Emotions)
- The Bardic Arts
- Multi-Layer Storybook Character Model
- Clancy's Theorem
- Insightful Learning
- Multiple Interlinked Economies
- The Socratic Method
- Bibliographic References and Additional Resources
Google's Knol Project introduces the term of art 'knol' as a unit of knowledge. As we learn, we acquire 'knols' much like pieces of a gigantic jigsaw puzzle and arduously weave them into a "fabric of knowledge" in which all the pieces fit neatly together and a "big picture" emerges. The "big picture" which is hidden from view until we are nearly done is called "insight".
There is a dynamic process called "learning" that takes place as we build a knowledge base. The rate of learning is generally not steady. We learn in fits and starts and occasionally have to "unlearn" a misconception or two. That is, our "learning curve" does not gracefully ascend but has "wiggles" in it as the learning process advances and retreats.
There are identifiable emotions that appear along the way. Among these we especially note curiosity, fascination, confusion, anxiety, surprise, perplexity, frustration, chagrin, despair, hope, satisfaction, and elation.
In this article we will craft a model of the interplay of emotions and learning and extend the model to encompass drama theory as well.
This article will cover the following topics:
Anyone who has ever bothered to look at a human face knows full well that facial expressions correspond to identifiable emotions such as anger, surprise, disgust, sadness, confusion, and joy. But facial expressions can be quite nuanced, and some of them are clearly linked to affective states that arise in the course of learning. If you’ve ever been “in a pickle” you might appreciate how that looks to others.
Let’s begin by defining basic terms and concepts that arise in the context of emotions and learning.
Basic Concepts and Terms
We use the term ‘affect’ to subsume the spectrum of affective emotional states and feelings that one can experience while engaged in the learning process. You can see a sampling of them on this slide, together with some nuanced terms for degrees of intensity…
Modeling emotional axes is far from a settled question in Psychology and Cognitive Science, but we’ve sought to identify those affective emotional states (and axes) that seem to be most relevant to the learning process — both for voluntary learning (the intrepid learner) and for involuntary learning (the reluctant learner).
There is copious evidence that we all experience at least some emotions whilst engaged in the learning process. Our goal is to construct a coherent model of the interplay of emotions and learning.
Cognition — using one’s noodle to think and solve life’s large and little problems — is clearly a learned behavior. We learn from academic sources, from practical firsthand personal experience, from cautionary tales, stories and dramas, and (hopefully) from our disheartening, embarrassing, lamentable, and crushing failures.
In the above slide, you will notice that each emotional axis is divided into two halves — a positive valence (pleasurable) half, and a negative valence (unpleasurable) half. But most activities will activate emotions on more than one such axis. And it’s quite possible that we can experience an emotional tug-of-war, say between anxiety or dread on the one hand, and curiosity, anticipation or hopefulness on the other. That is to say, we are often obliged to mix the bitter with the sweet.
But we want to dig deeper into the interplay between emotions and learning. We want to go beyond poetic metaphors to a more comprehensive and scientific model relating emotions to cognition and learning.
And that’s where we’re headed. So put on your seat belts, because we are going on an emotional roller coaster ride into the Calculus of Emotions and Learning.
One of the more useful slides to consider when a learner is caught in the emotional roller coaster ride is this one…
In the above slide, we can see the ups and down of the rollicking learning curve. We call this ‘Non-Monotonic Learning’ to remind ourselves that sometimes we acquire erroneous beliefs and misconceptions which we eventually have to discard. In that slide, I’ve superimposed a ‘Frowny Face Scowl’ and a ‘Happy Face Smile’ to point out where the emotional states are variously negative valence (unhappy) and positive valence (happy).
Later on, we’ll dig a little deeper into the specific emotions that typically arise in these alternating phases. What I hope our readers can see from this slide is that there is some mathematics here. We are dividing the roller-coaster learning curve into sections where the curvature is variously concave (downward-turning) or convex (upward-turning). In the Calculus, this corresponds to the sign of the Second Derivative. A curve is turning downward when the Second Derivative is negative; a curve is turning upward when the Second Derivative is positive. We’ll return to this observation shortly.
Learning is defined as the acquisition of knowledge over a span of time. But what, exactly, is ‘knowledge’?
For our purposes, we will adopt a scientific notion of knowledge as Concepts, Mental Models, Theories, Beliefs, Hypotheses, etc.
That is, we intend to reckon how our Belief Systems evolve and become increasingly refined, sophisticated, accurate, and insightful.
We all know from direct experience that our learning is not error-free. We adopt misconceptions and erroneous beliefs, and struggle with them for a time before diagnosing and correcting our mistaken beliefs.
While we can easily forget knowledge, we sometimes are obliged to discard beliefs that we have discovered in the fullness of time to be so much codswallop. Thus our store of knowledge (or reliable beliefs) can rise and fall over time.
Sometimes the hills and valleys of the learning curve are gentle rolling slopes, and sometimes there are precipitous cliffs. We all long for those ‘Eureka’ moments where everything suddenly becomes crystal clear. And we dread those belief-shattering passages where reality rears its ugly head and reveals to us that our cherished bright ideas are so much rubbish.
The roller coaster ride travels through any number of emotional states along the way — curiosity, fascination, surprise, anxiety, confusion, bewilderment, frustration, anguish, chagrin, hope, perplexity, elation, satisfaction, and confidence. Part of the story is just coming up with a comprehensive list of words to name all these affective emotional states.
To make our job easier, we classify the emotions shown in Figure 3, above, into a handful of Emotions Axes:
Role of Emotions In Learning
Readers who have some background in math will appreciate where we are going with this model, because we are going to analogize the ups and downs of the ‘Learning Curve’ (Figure 4) to the properties of a generic mathematical function that exhibits a similar roller-coaster shape.
If you study the motion of a real roller-coaster, you find that you need to keep track of four basic parameters — Time, Distance (or Position), Velocity, and Acceleration. These four parameters are not independent. Velocity is the Time-Derivative of Position, and Acceleration is the Time-Derivative of Velocity.
Before we can go on with our Theory of Emotions and Learning, we need to remind ourselves of the relationship between Time, Distance, Velocity, and Acceleration in the trajectory of a moving object.
When I give this presentation in person, I use a little puppet figure — Montana Mouse — to illustrate simple Galilean Motion, and review the elementary model of motion, as suggested by Galileo.
In the Calculus, when we model simple motion, we can construct a Phase Plane Diagram, in which Velocity is plotted against Acceleration. If you add Distance as a third axis, perpendicular to the Phase Plane, you get a three-dimensional Phase Space Model of motion, parametric in time.
Those readers who have some grounding in Calculus or Physics or Engineering may recall using Phase Plane Diagrams or Phase Space Diagrams once upon a time.
For everyone else, just bear with us, as this diagram will still be helpful, even without a full appreciation of the underlying math. Try to become comfortable with reckoning a journey in terms of parameters like Time, Distance, Velocity, and Acceleration, because we are going to analogize those to their corresponding elements in the Theory of Emotions and Learning.
Analogizing Motion (Time, Distance, Velocity and Acceleration) to the Learning Process (Time, Knowledge, Learning, and Emotions)
This section presents the technically most difficult and challenging part of the Theory of Emotions and Learning -- the mathematical analogy that undergirds the Phase Plane Diagram, a concept that is central to the Calculus. For readers who are less familiar with these concepts from mathematics, it will be worth your time to explore this way of reasoning about the roller-coaster ride of the learning curve (Figure 4, above), because we are borrowing from the original thinking of Galileo and Newton. In Figure 6, below, we examine the four phases of any cyclical process:
Let’s begin by reviewing our analogy between a Learning Journey and an ordinary journey.
In ordinary Galilean motion, we typically include the following terms in our mathematical model of motion: Time, Distance, Velocity, and Acceleration.
When Newton invented the Calculus to systematize the study of motion, he also got around to naming the next term after Acceleration. Newton called it Jerk. I kid you not. When your car lurches, that’s called Jerk in the Newtonian Calculus.
So now, let’s set up our proposed analogy between ordinary motion and a learning journey…
Table 1. Analogy Relating Ordinary Motion to Learning Journey
Now in the Phase Plane Diagram for Ordinary Motion, we plot Velocity (on the vertical axis) against Acceleration (on the horizontal axis).
The Phase Plane divides everything into Four Quadrants. In Quadrant I (the upper right quadrant), Acceleration is Positive (you have your pedal to the metal), and that means your Velocity is Forward Motion and Increasing. In Quadrant I, the Trajectory through the Phase Plane is Upwards.
In Quadrant II (the upper left quadrant), you have your foot on the brakes (or retro rockets). That means you are slowing down your forward motion. The Trajectory through Quadrant II is Downwards.
In Quadrant III (the lower left quadrant), you are firing your braking thrusters so intensely, you are now beginning to gain speed moving backwards. The Trajectory through Quadrant III is still Downwards.
In Quadrant IV (the lower right quadrant), you once again turn on your forward thrusters, which arrest your backward motion. The Trajectory through Quadrant IV is now Upwards again.
Thus a learner cycles around the Phase Plane in a characteristic trajectory as the rate of learning variously accelerates or decelerates.
Does your head hurt yet from all this mathematical thinking? Imagine how Galileo and Newton must have felt working it all out for the very first time since Zeno originally fired his mind-boggling arrow and threw down the gauntlet.
But I digress.
Back to our analogy…
Recall that we analogized Velocity to Learning and Acceleration to Emotions. So now we just plug in those labels into our conventional Newtonian/Galilean Phase Plane Diagram. And that’s how we arrive at Figure 6.
In Quadrant I, we have positive valence emotions (like curiosity) as we investigate, explore, and begin to build our initial, tentative, mental models. Gosh this part is fun!
Eventually we have enough new ideas in our heads to be able to anticipate and make predictions about how things work. But wait! What’s this? Do mine eyes deceive me? I just saw something surprising and unexpected, something I didn’t predict. Whazzat?
Zat is Quadrant II. Now we are really confused. We need to diagnose the discrepancy between our jejune beliefs and simple-minded expectations and what’s really happening out there in that subtle and perplexing world.
Suppose we’ve really blown it. Suppose we’ve adopted some cockamamie theory like ‘the world is flat’ or ‘the rival gang down the street has a huge stockpile of secret weapons hidden in the dumpster behind the Burger King’.
Trying to navigate the world with an error-ridden map can be downright frustrating and exasperating. Arrgggh!
Eventually we’re going to have to diagnose, admit, and discard such erroneous beliefs. With chagrin in our bosom and embarrassment on our faces, we move along to Quadrant III and pitch the bogonic beliefs.
Lemme tellya, this is the Dark Tea Time of the Soul. No one ever wants to admit they are traversing the Pits of Quadrant III. Oy vey.
But lo! There is Hope. It’s over there → in Quadrant IV.
And, so with our indomitable faith, determination, and steely grit, we get over our humiliating mistakes and hunker down to fresh research.
Einstein said, “If we knew what we were doing, we wouldn’t call it Research.”
Research is what I do when I don’t know what I’m doing.
I do a lot of research.
Welcome to Quadrant IV.
And what do we find after a quarter century of research?
We find the very slide we are studying -- the Phase Plane Diagram of the Learning Journey of the Intrepid Researcher.
Or to quote our old friend, Archimedes, “Eureka! We have found it!”
Which brings us home to Quadrant I -- the Joy of Discovery Learning.
Round and round we cycle, time and again, climbing the arduous learning curve depicted in Figure 6 -- the roller-coaster of life’s learning journey.
Each time around the loop, we have a net gain in total cumulative knowledge. If you add the Knowledge Axis, perpendicular to the Phase Plane of Emotions and Learning, our Trajectory looks something like the Helix of Figure 7:
“Onward and upward!” says Professior Excelsior.
What a wild ride, eh?
The Bardic Arts
Throughout most of human history, cultural knowledge was passed from one generation to the next by telling stories. It’s only recently that we invented the classroom model of education. And, sadly, we lost much of the power of story when we invented the classroom instruction model.
This is especially true in science and math, where storycraft is almost entirely unheard of.
So let’s fix that. Let’s gin up a Theory of StoryCraft, and look at how we might reclaim the Bardic Arts as an educational vehicle.
Now the first thing we’re going to need in a story is some Characters.
What’s a Character? It’s a highly simplified model of a human being.
So let’s see what might go into a basic Character Model for a storybook character. Take a look now, at Figure 8:
The explanation that goes along with that figure is the next chapter of our story about the Theory of StoryCraft.
A story is an anecdote drawn from the culture. A well-crafted anecdote or story has value both as an amusement and as a source of insight into the world from which it is drawn. Since the plural of ‘anecdote’ is data, if you gather a large enough collection of anecdotal stories you ought to have a data base sufficient to model the culture from which the stories are drawn.
But first we need a functional model of StoryCraft. Before we can outline the plot structure of our story, we need to define the Characters who will play out our nasty little drama. The two primary characters in any drama are called Protagonist and Antagonist. Let’s construct a simple model of them. Each character in a drama has a characteristic psychological profile. We propose modeling this profile in terms of eight basic psychological parameters: Fears, Emotions, Backstory, Issues, Beliefs, Practices, Desires, and Intentions, which drive the character’s Actions/Reactions and Dramatic Life Story. This model concords with both Jungian analysis and existing models of Characterization in Drama Theory.
Multi-Layer Storybook Character Model
So, to elaborate, we have this Shreklisch Onion Layer Character Model, from the inside out:
Table 2. Layers of the Character Model
The Innermost Fears are the most fundamental (and deeply hidden) elements of the character’s psychology. Swirling around the Innermost Fears are other Burbling Emotions which are generally acted out, but not necessarily mentioned by name. The character’s Undisclosed Backstory establishes the basis for their dreads and associated unresolved problems. The Undisclosed Backstory explains the unresolved Burning Issues from the character’s past. The Sacred Beliefs are the first important cognitive component of the character’s psychology. Beliefs can be accurate or inaccurate as well as riddled with gaps. Associated with Beliefs are Derivative Practices, which correspond to the way the character handles (or fails to handle) recurring situations. Through the lens of the character’s Belief System, he envisions his or her Heart’s Desires. The Desires are the nominal inverse of the Fears and Dreads. The character believes that the object of desire will protect him from his dreads and alleviate his suffering. Among all his Heart’s Desires, he undertakes to achieve at least one of them, and that becomes his Avowed Intention, his Holy Grail. That pursuit is the Opening Act of the unfolding Shreklisch Drama.
In order for the drama to get off the ground, the Protagonist and Antagonist must have Complementary Psychologies. The Desires and Intentions of the Protagonist must arouse the Fears of the Antagonist, and vice versa. This sets up the primary feedback loop that propels the conflict and the drama. To run a drama to Tragedy, it suffices that the characters consume their resources without resolving their conflict. Of all tragedies, perhaps the most complex are the Greek Tragedies which we may call Agonistic Drama. To run the drama to Comedy, something astonishing must happen. The characters must solve the Mystery of their long-suffering Agony and discover the system model we just presented. And, of course, so must the audience.
The central theorem of Drama Theory is called Clancy’s Theorem, after one of Tom Clancy’s book titles. But it’s also inspired by one of Shakespeare’s titles as well — Much Ado About Nothing.
What this means is that in order for the story to have dramatic continuity, each action by some character has to be an emotion-driven reaction to some preceding action by an antagonist, and the reaction has to provoke a downstream response from one of the other characters.
Now, if we look at Figure 10, we can now see how the Bardic Arts emerge. Each scene in the story is an anecdote in the context of the story so far. A coherent collection of anecdotes makes a story.
And that’s what’s been missing in traditional classroom education in subjects like Science and Math, which concentrate on just the lower part of the loop in Figure 10. Look at the poverty of Figure 11:
We have Data (which is the answer to a question no one asked).
We have Information (which is a motley collection of Question/Answer Pairs).
And we have Knowledge, which is an integration of such Question/Answer Pairs into a Fabric of Knowledge, much like assembling a Jigsaw Puzzle.
But what’s missing is Wisdom and Story. To get from Knowledge to Wisdom, we have to fold in a Value System. But that’s just a fancy name for Dreads and Desires. As soon as you have diverse characters with competing and conflicting Dreads and Desires, you have Drama.
And that’s what we need to recognize and reckon — the Function of Drama in the Learning Process. It’s not just Education. It’s Edutainment, too.
We’ve talked about Time, Knowledge, Learning, Emotions, and Shock. In a dramatic story, there are climactic moments where emotions run high. In your classic Hero/Goat Drama, the Hero is the character whose Beliefs are revealed to be accurate. The Hero has a Triumphant Belief-Crystalizing Moment. And the Goat has a Humiliating Belief-Shattering Moment (“Curses, foiled again!”).
Most of the time, in our pedestrian studies, it’s not such high drama. But the dramatic elements are there, perhaps lurking below radar. And while we don’t have to overdramatize the learning journey, it wouldn’t hurt to spice it up a tad, just to immerse ourselves in a bit of playful drama as we slog through the tedium of our studies.
References for this section:
The Science Behind the Art of Teaching Science: Emotional State and Learning(Conference Paper in PDF Format)
The Vexagon Diagram (PowerPoint Slides)
Cognition, Affect, and Learning (Online Presentation and Discussion of the Above Paper and Slides)
Affective Learning Companion Project at the MIT Media Lab
Barry Kort’s Emotions and Learning Jump Page (Where To Find Most Other Related Materials)
NSF ROLE Grant References (Comprehensive List of All References)
Bonus Musical Parody: Shreklisch Onion
Multiple Interlinked Economies
Some of the material in this section is only loosely connected to the main Theory of Emotions and Learning, but we pause to reckon it for the supplemental value it adds.
The material on the Multiple Interlinked Economies is summarized in Figure 12:
An Economy is defined as the flow of some commodity within a system.
Mostly we think of the Material Economy — the flow of Goods and Services, in exchange for little pieces of green paper. Those of you who live in other countries than the USA might have more colorful currencies than we do. Anyway, we can measure and tally the flow, and count the ‘chits’ that are in flux.
Today, we also need to consider the Information and Attention Economies. I can look at my web server log and note that about 16 people bothered to look at the photo of Montana Mouse from Monday’s installment. We can measure Information and Attention in terms of ‘bits’ and ‘hits’. (There were 10383 bits in that photo of Montana Mouse.)
The high end of the Information and Attention Economies are found in the Entertainment and Drama Economies. We can measure them in ‘skits’ and ‘fits’ I suppose. (Bart Simpson might say, “Don’t have a cow, man!”)
The next pair of economies that we consider are the Emotions and Learning Economies. This is the educational part of Edutainment. Perhaps we can measure them in ‘snits’ and ‘wits’.
And at the high end of the Emotions and Learning Economies we find the Spiritual Economy, where we might have those rare moments of insight that are divinely inspired. Let’s call that ‘grits’.
As suggested by the next figure, the ‘Body Politic’ juggles all these interlinked economies much like fiddling with a ‘slinky’. Flows and fluctuations in any one econonmy induce fluctuations and flows in the adjacent economies, and the effects ripple on down the line, ultimately linking the Material Economy on the one hand (gotta be the sinister left hand) to the Spiritual Economy on the other hand (the right hand of God, I suppose).
The point of this model is that if one is going to reckon any one economy (say Emotions and Learning), one is obliged to reckon the entire chain of interlinked economies in which the one we most care about is embedded. Everything is connected to everything else.
The Socratic Method
Finally, let’s look at the last figure, on Socratic Dialogues …
Socrates came up with a remarkably powerful technique for structuring a dialogue that facilitates learning. It’s possible to work out a general template for a Socratic Dialogue by cataloguing the kinds of questions one might ask at different junctures in a conversation in the style of Socrates. You can find several such templates at this URL…
I occasionally use these templates to structure a dialogue when I want to diagnose a particularly difficult learning passage.
Bibliographic References and Additional Resources
Readers can find previously published conference papers with most of this material at these two repositories:
Barry Kort’s Emotions And Learning Summary Page
MIT Media Lab Project Page for the NSF-Funded Affective Learning Companion Project
The SITE-2004 paper, “The Science Behind the Art of Teaching Science: Emotional State and Learning” (PDF format) is the most recent one.
An older article which appeared in the October 2002 special issue of the IEEE Journal of International Forum of Educational Technology & Society (IFETS) and IEEE Learning Technology Task Force is entitled “Theories for Deep Change in Affect-Sensitive Cognitive Machines: A Constructivist Model” (HTML format).
For scholars who are interested in reading the original peer-reviewed publications upon which this Knol is based, here is the annotated bibliography from the original NSF-funded Learning Companion Project at the MIT Media Lab:
External Representation of Learning Process and Domain Knowledge: Affective State as a Determinate of its Structure and Function is a paper presented at AI-ED 2001(Artificial Intelligence in Education) that presents a novel model of learning. This model underpins the evolution of the Learning Companion. Essentially, we believe there is an interplay of emotions and learning, but this interaction is far more complex than previous theories have articulated. This model goes beyond previous research studies not just in the emotions addressed, but also in an attempt to formalize an analytical model that describes the dynamics of emotional states during model-based learning experiences. The summary and description are also available in Word 97 format and in PDF format.
An Affective Model of Interplay Between Emotions and Learning: Reengineering Educational Pedagogy — Building a Learning Companion (PDF format) is a paper presented at ICALT-2001 (International Conference on Advanced Learning Technologies) which won the Best Paper Award. Here are the downloadable PowerPoint Slides for this talk.
Improving Pedagogy in Developing Nations (PDF format) is a paper submitted to the July 2001 Digital Nations Symposium at the MIT Media Lab.
Analytical Models of Emotions, Learning and Relationships: Towards an Affect-Sensitive Cognitive Machine (PDF format) is a paper presented at the January 2002 Conference on Virtual Worlds and Simulation (VWSim 2002) held in San Antonio Texas.
A Pedagogical Model for Teaching Scientific Domain Knowledge (PDF format) is a paper presented at the November 2002 Conference on Frontiers in Education (FIE 2002) held in Boston Massachussetts.
Theories for Deep Change in Affect-Sensitive Cognitive Machines: A Constructivist Model is an article which appeared in the October 2002 special issue of the IEEE Journal of International Forum of Educational Technology & Society (IFETS) and IEEE Learning Technology Task Force.
Communications Technology and Personal Identity Formation is a summary of a May 2000 panel discussion published in Educational Technology & Society, 2000, Volume 3, Issue 3 – Special Issue on “On-line Collaborative Learning Environments,” pp. 27-38, ISSN 1436-4522, by the panel moderator, Leslie Henrickson. The journal is published by IFETS, the International Forum of Educational Technology & Society, a forum sponsored by the IEEE Technical Committee on Learning Technology.
Most of the theoretical content of this article is original research over the past 25 years of the principle author of this Knol. However I have many collaborators to thank in helping organize this material and bringing it to light. Rob Reilly and Rosalind Picard played instrumental roles in organizing and introducing this material into academic channels, professional society channels, and funding channels. Both of them co-authored and/or carefully edited the peer-reviewed conference papers and journal articles which initially introduced this material to the academic community. I am also indebted to collaborators such as Art Graesser, Frank Thissen, Lucy Kirshner, and Dan Noren who channeled this material into their programs and projects at other institutions. Without their support, encouragement, editorial hand, and hard work within the organized channels of academia and educational outreach, this material might never have seen the light of day.
Barry Kort, July 2008
Barry Kort, Ph.D.
Affective Computing Research Group
MIT Media Lab