Posted by Michael on June 13th, 2006, in Projects
These are quotes from papers written in the context of the Oz research project.
I’m still reading these texts, so this post may be updated later.
Form “The Role of Emotion in Believable Agents” by Joseph Bates (April 1994)
“It can be argued that while scientists may have more effectively recreated scientists, it is the artists who have come closest to understanding and perhaps capturing the essence of humanity (…)”
(referring to A.I. scientists and Disney animators)
“The apparent desires of a character, and the way the character feels about what happens in the world with respect to those desires, are what makes us care about that character. If the character does not react emotionally to events, if they don’t care, then neither will we.”
“Jones says he discovered that it is the oddity, the quirk, that gives personality to a character, and it is personality that gives life.”
(referring to animator of Bugs Bunny and a programming error in one of the Woggles)
From “Believable Social and Emotional Agents” by W. Scott Neal Reilly (May 1996)
“My framework gets a lot of its power from being part of a broad agent architecture. The concept is simple: the agent will be emotionally richer if there are more things to have emotions about and more ways to express them. This reliance on breadth has also meant that I have been able to create simple emotion models that rely on perception and motivation instead of deep modeling of other agents and complex cognitive processing.”
“The goal of building believable agents is inherently an artistic one. Traditional AI goals of creating competence and building models of human cognition are only tangentially related because creating believability is not the same as creating intelligence or realism.”
“From the standpoint of believability, it is better to go with the less realistic characters which meet the audience’s expectations than to go with the more realistic characters which don’t.”
2 Comments »
Posted by Michael on June 6th, 2006, in Projects
Andrew Stern has been doing the kind of stuff that we’re looking into on these pages for years. He has been involved as a designer and developer in pet games Dogz, Catz and Babyz as well as the recent interactive drama Façade. He’s made numerous publications and presentations on the subject of autonomous characters and is a very active contributor to Grand Text Auto.
I had the pleasure of interviewing him via email. He starts by answering two questions simultaneously.
Michael Samyn: What do you think of the “paradox of the actor“, as described by Diderot in the 18th century, in relation to autonomous characters? Diderot claims that the best actor is the one who does not feel a thing but who excells at imitating only the symptoms of the behaviour of humans. Is it required for virtual actors to feel the emotions (i.e. to posses a mind) in order to express them?
Your work has always favoured artistic goals over scientific ones. Still I feel that there is a strong scientific “reflex” in the solutions you come up with. A lot of your work seems to attempt to model a character or a story from the inside out, which I consider to be the scientific approach (break things apart and put them back together). Do you feel that a scientific approach is required?
Andrew Stern: First, let’s realize that even the most sophisticated “mind” we can program with contemporary methods will be but pale imitations (greatly simplified versions) of real minds. So arguably all we can do, practically speaking, is imitate symptoms of human behavior anyway. But the spirit of your questions is, should we attempt a cognitive solution, that attempts to model mind-like processes, or should we create some other simpler, perhaps an ad-hoc solution?
The answer is, it depends on what you want your virtual actor to be capable of doing. If you’re happy with shallow interaction, say, stimulus-response interaction, then an approach such as AIML, used to make the text bot A.L.I.C.E. will suffice, which is basically a large number of inputs mapped in a relatively simple way to outputs.
A.L.I.C.E. is quite broad, and very shallow. It works for certain domains and artistic goals; in fact the creator of AIML and A.L.I.C.E. has been known to say that statistically speaking, most real human conversation is shallow, stimulus-response interaction. I see A.L.I.C.E. as, in a sense, a commentary on the shallowness of the typical human conversation; the system is successful in that regard.
But if you want you virtual actor to have deeper responses, that is, responses that take into account the history of the player’s input, that result in more meaningful, cumulative responses to your discourse, you’ll need to be keeping track of the discourse (an episodic memory), to be modelling attitudes and beliefs over time (a model of emotion, knowledge, personality), give the actors motivations (goals and plans) and so on. The models are some sort of encapsulation of the “rules” of how your actor should behave, implemented in some relatively elegant, thought-through, non-ad-hoc way.
At the end of the day, there’s just no way to fake that. And at the end of the day, most artists want more than less meaning and depth in their work.
Note behavior models (e.g. episodic memory, emotion, knowledge, personality) — which are in essence various ways of keeping track of and modulating the actor’s state as it changes over time in response to the player’s actions — can be much simpler than real minds are, and in fact may not really directly mirror any systems actually in the human mind. Nonetheless, they are non-trivially sophisticated models of dramatic behavior. In fact, because they’re modelling dramatic behavior, they will necessarily differ from biologically-inspired models of behavior, e.g. a-life driven agents.
Behavior models are also the key to creating characters that can generate behavior, that is, virtual actors that can get themselves into states and as a result perform patterns of behavior over time that weren’t explicitly pre-written by their author.
Another way to answer your questions — and this is something my collaborator Michael and I often say — is that there is no “design only” solution to creating non-trivially interactive characters and stories. To make something more sophisticated than a stimulus-response bot or a choose-your-own-adventure story, one needs to start modelling behavior.
I think modelling behavior is probably equivalent to what you’re calling truly “feeling” the emotions they are performing. Do real actors need to do that? By and large, I think many successful actors truly feel the emotions they are performing; by doing so, it allows them to generate a more authentic, robust performance.
Finally, let me say that these dramatic models of behavior, while technical in form, are heavily shaped by our artistic goals. That is, when creating these models — essentially these rules governing how virtual actors will behave — we’re always thinking about how the player perceives this behavior, are these rules going to result in the kind of performance we want to achieve for this actor, etc. As artists, we’re still very much in control of what the actor will ultimately be capable of doing or saying, even though exactly what is said and when will be determined in real-time, in response to the player’s actions. Even as these models get deeper and more generative in the future, I feel we’ll be able to retain a high level of authorial control over them.
MS: How do you feel about the concept of plot in interactive storytelling? And more specifically about the concept of a drama manager (a computer program that steers an interactive story towards an interesting plot arc)? Do you believe that a drama manager can be built to tell stories that can rival traditional literature or good theater? Or is pulp and formats the best it can do?
AS: I very much want to create systems that can generate a variety of narratives, in response on a moment-by-moment basis to what the player is saying and doing. Necessarily, this means we must abandon the ideal of a tightly-plotted story, because players are always going to be experimenting, pushing things in all kinds of directions, that will rarely allow for an extremely well-formed overall plot to be created.
That said, let’s not forget that in a real-time collaboration between the human player and the drama manager, that the drama manager — who is controlling the behavior of all of the virtual actors — is one half of the overall authorship of the experience. 50% control over the events being generated in a story is probably is enough to allow a loosely-plotted story to be created, if the overall domain of the story is set up to allow for that.
For example, in Facade, you’ve got the player vs. the duo Grace and Trip. The player can say and act any way they like; they could act like they’re dying, or act really violent, or completely absurd, what have you. The coordinated duo Grace and Trip will attempt to respond to the player, but also can believably mix in their own agenda, even believably ignore the player at times as needed (just as the player may ignore Grace and Trip if she wishes). You may end up with a absurdist story in the end where the characters each seem to be in a world of their own, but because the characters all share the same space and to some extent are reacting to one another, I’d at least call that loosely-plotted.
(Ideally the human player and drama manager cooperate and help each other out, at least some of the time.)
I’m not at all interested in making a drama manager that attempts to force the player into a particular story. If Facade seems to do that, it’s only because we ran out of time and/or energy to author a broad enough array of responses to at least respond to what the player wants to do. (This is another argument for more generative models of behavior, to help make virtual actors even more responsive to the player.)
Once you buy into the idea of allowing for loosely-plotted stories, even “bad” stories to be generated, we get rid of several of the supposed conundrums of interactive stories: 1) we get rid of the conflict between freedom vs. well-formed plot, because we’re not requiring a well-formed plot, we’re okay with a looser plot; 2) we get rid of this concern of some that too much freedom is bad thing, that we must greatly constrain players in order to ensure for a satisfying experience. I’m much more in favor of an open ended interface, that allows players to do and say anything they want, and only apply constraints on how the virtual actors will decide to interpret and work with the player’s actions. Again, let’s give the player a full 50% of the authorship of the story, and the drama manager 50%. Ideally the player and drama manager cooperate, but they don’t have to.
MS: The linearity of a story and the linearity implied by goal-oriented gameplay seem oddly compatible. This motivates people like Marianne Krawczyk (writer for God of War) to claim that they overlap, that the story of a game is expressed through goal-oriented gameplay. Obviously your narrative ambitions go far beyond anything an action game can express. Does this mean that goal-oriented gameplay should be abandoned?
AS: No, goal-oriented gameplay shouldn’t be abandoned, in fact it should be expanded and broadened. It’s only natural that players find achieving goals satisfying. But allowing players to form their own goals and be able to pursue and achieve them, in various ways, will be far more satisfying than offering players only one primary goal and one linear path to achieve that goal.
Just as importantly, we should allow players the freedom to just screw around, to play, to not pursue a goal if they wish. In total, we should be making dramatic worlds that allow for both freeform play, as well as goal-oriented play.
MS: You once mentioned that you thought of agency as one of the most important elements in interactive fiction, i.e. the things that the player can do and how this effects the other characters and the game world. Why is agency so important? Does a higher level of agency improve the story? And if so, how?
AS: I mention it all the time in my blog posts! Agency is the most fundamental property of interactive anything — be it games, web surfing, email, what have you. If you can’t have meaningful influence over what you’re doing when you take action on the computer, why is it interactive? If you’re to be led through an experience, why didn’t the author just make a movie or write a book?
But like I said earlier, I don’t think players need 100% control over the experience; an interactive experience can be a 50/50 collaboration between the human player and the system — where the system is a proxy for the human artist/programmer who created it.
Let’s not forget, ultimately, playing interactive art/entertainment is an interaction between people, via the artifact of a software system — particularly when the system is simulating (dramatic) human behavior, i.e. virtual actors.
13 Comments »
Posted by Michael on May 25th, 2006, in Projects
http://www.cs.cmu.edu/afs/cs.cmu.edu/project/oz/web/papers.html
These are my remarks on An Oz-Centric Review of Interactive Drama and Believable Agents by Michael Mateas which offers an overview of the project’s goals (1997).
Oz is a reserach project into believable agents and interactive drama. Like Drama Princess, the Oz project focuses on drama much more than scientific AI. Unlike our work at Tale of Tales, the Oz project values plot and story arc greatly. This leads them to build a piece of software called drama manager, which we consider to be a deus ex machina for plot-fetishists.
We believe that the plot belongs to lineair formats. Non-linear formats should not be structured along plot arcs. This is no different now, with interactive media, as it has been in the past, when architecture, sculpture and painting were also completely plotless narrative formats.
Interestingly, they also mention the problem that I refered to earlier as the Seduction of Realism: When attempting to marry a technical field like Computer Science with a cultural activity such as story telling, it is extremely easy to become sidetracked from the artistic goals and to begin pursuing purely technical research. (Michael Mateas)
From the same text, we learn about the “requirements for believability” in characters:
- Personality – Rich personality should infuse everything that a character does, from the way they talk and move to the way they think. What makes characters interesting are their unique ways doing things. Personality is about the unique and specific, not the general.
- Emotion – Characters exhibit their own emotions and respond to the emotions of others in personality-specific ways.
- Self-motivation – Characters don’t just react to the activity of others. They have their own internal drives and desires which they pursue whether or not others are interacting with them.
- Change – Characters grow and change with time, in a manner consistent with their personality.
- Social relationships – Characters engage in detailed interactions with others in a manner consistent with their relationship. In turn, these relationships change as a result of the interaction.
- Illusion of life – This is a collection of requirements such as: pursuing multiple, simultaneous goals and actions, having broad capabilities (e.g. movement, perception, memory, language), and reacting quickly to stimuli in the environment. Traditional character artists do not mention these requirement explicitly, because they often get them for free (from a human actor, or as a deep assumption in animation). But builders of interactive characters must concern themselves explicitly with building agent architectures that support these requirements.
While this list is very useful, it bothers me that it is still written from the assumption that the actor performing the character actually feels the emotions, drives and desires. While it would be more correct, in my opinion, to claim that the character only needs to show symptoms that can be read as if it would posses these traits.
About the difference between realism and believability:
When watching a play or film, viewers know that the characters are not “real” but that does not detract from being engaged by the character.
Oz’s Drama Manager serves the purpose of creating plot in an interactive story. To be able to do this, the Drama Manager needs certain rules, an “evaluation function”. To author this system, Michael Mateas suggests the following steps:
1. write some linear (non-interactive) story as a sequence of “important moments”
2. reverse-engineer your own thinking to figure out why you think that particular sequence is a “good” story
3. capture this aesthetic as a set of features (over sequences) in an evaluation function
4. make sure that you have (approximately) captured your aesthetic by comparing the output of the evaluation function with your own evaluation of a set of sequences (of course include the original story – the one your “really” want to tell)
I find this highly problematic. First I don’t like the typical scientific way of taking a whole, breaking it apart and then trying to put it back together. I believe that you lose the essence of the whole when you work like that. Second, not unrelated, never, in these steps, is the user mentioned. It’s all about the author and what she wants and never about what you want the user to experience, let alone how.
He then continues to describe how this Drama Manager would, at each plot point, evaluate all possible future interactions to then choose one that is rated high by the evaluation function and push the story in that direction. Not only does this strike me as a million times more difficult than creating a believable agent, I also have serious doubts about the desirability of such a system. Ultimately the best you could achieve is a formulaic story. Because you literally apply a formula to a narrative. And formulaic stories happen to be the worst that film and literature have to offer.
Does this mean that it is impossible to make narrative art with interactive media? No. It just means that interactive media are not suitable for writing novels or creating movies, much like novels and movies are not suitable for painting pictures or carving marble.
4 Comments »
Posted by Michael on May 16th, 2006, in Projects
Chris Crawford’s Storytron is a technology for interactive storytelling. Swat is a tool for writing interactive stories. I’ve been looking at the documentation of this application (Alpha version 0.6). Here’s some ideas I found interesting.
Every event in the story contains all possible types of reactions to this event, called Roles, and an if statement for each of these Roles. So the actors don’t really think for themselves, they just look at the list of Roles and pick the one for which the if statements evaluates to true. This sounds very similar to our own “AI from the outside” ideas and Richard Evans’ “Activity-things”.
For each Role, there are several ways of reacting to an event, called Options. A character chooses an Option based on its Inclination. And this is where the (artificial) intelligence of the system resides. The inclination to do each option is scripted by the writer in mathematical formulas that can relate it to the personality of the character, its history, its relationships, etc. The character will choose the option (i.e. perform the action) with the highest Inclination value. One can imagine that there must be a random number to add some spice to this relatively stiff system.
Swat’s scripting interface (or perhaps even the very use of scripting) requires that a writer breaks down everything into minute elements. Perhaps an extremely clever person can keep the enormous mental picture in his head that encapsulates every element in the scene. But in general, I feel that a lot of the little emotional things that a writer might include spontaneously would get lost in this process. What about, e.g. ambiguity of words, associations, implications of tone? All of these are extremely important for the dramatic impact of a story but how do you express these in formulas that only describe events?
The single biggest problem of Storytron, however, is how it represents the story. It does so in a subset of English, represented in a symbolic way, called Deikto. Here’s a mockup of that representation:
The part on the left says “Knifer has just issued you a threat: Give your bicycle to him or he’ll knife you.” The part on the right is how you respond.
Now, even though Mr. Crawford states that the singular beauty of interactive storytelling is not in its representation – it is in the richness, depth, variety and drama of the interactions it allows, the representation still needs to be understandable, readable at least. Are readers really expected to learn this new language in order to experience these storyworlds? And how will Deikto translate to other languages?
4 Comments »
