Andrew Stern has been doing the kind of stuff that we’re looking into on these pages for years. He has been involved as a designer and developer in pet games Dogz, Catz and Babyz as well as the recent interactive drama Façade. He’s made numerous publications and presentations on the subject of autonomous characters and is a very active contributor to Grand Text Auto.
I had the pleasure of interviewing him via email. He starts by answering two questions simultaneously.
Michael Samyn: What do you think of the “paradox of the actor“, as described by Diderot in the 18th century, in relation to autonomous characters? Diderot claims that the best actor is the one who does not feel a thing but who excells at imitating only the symptoms of the behaviour of humans. Is it required for virtual actors to feel the emotions (i.e. to posses a mind) in order to express them?
Your work has always favoured artistic goals over scientific ones. Still I feel that there is a strong scientific “reflex” in the solutions you come up with. A lot of your work seems to attempt to model a character or a story from the inside out, which I consider to be the scientific approach (break things apart and put them back together). Do you feel that a scientific approach is required?
Andrew Stern: First, let’s realize that even the most sophisticated “mind” we can program with contemporary methods will be but pale imitations (greatly simplified versions) of real minds. So arguably all we can do, practically speaking, is imitate symptoms of human behavior anyway. But the spirit of your questions is, should we attempt a cognitive solution, that attempts to model mind-like processes, or should we create some other simpler, perhaps an ad-hoc solution?
The answer is, it depends on what you want your virtual actor to be capable of doing. If you’re happy with shallow interaction, say, stimulus-response interaction, then an approach such as AIML, used to make the text bot A.L.I.C.E. will suffice, which is basically a large number of inputs mapped in a relatively simple way to outputs.
A.L.I.C.E. is quite broad, and very shallow. It works for certain domains and artistic goals; in fact the creator of AIML and A.L.I.C.E. has been known to say that statistically speaking, most real human conversation is shallow, stimulus-response interaction. I see A.L.I.C.E. as, in a sense, a commentary on the shallowness of the typical human conversation; the system is successful in that regard.
But if you want you virtual actor to have deeper responses, that is, responses that take into account the history of the player’s input, that result in more meaningful, cumulative responses to your discourse, you’ll need to be keeping track of the discourse (an episodic memory), to be modelling attitudes and beliefs over time (a model of emotion, knowledge, personality), give the actors motivations (goals and plans) and so on. The models are some sort of encapsulation of the “rules” of how your actor should behave, implemented in some relatively elegant, thought-through, non-ad-hoc way.
At the end of the day, there’s just no way to fake that. And at the end of the day, most artists want more than less meaning and depth in their work.
Note behavior models (e.g. episodic memory, emotion, knowledge, personality) — which are in essence various ways of keeping track of and modulating the actor’s state as it changes over time in response to the player’s actions — can be much simpler than real minds are, and in fact may not really directly mirror any systems actually in the human mind. Nonetheless, they are non-trivially sophisticated models of dramatic behavior. In fact, because they’re modelling dramatic behavior, they will necessarily differ from biologically-inspired models of behavior, e.g. a-life driven agents.
Behavior models are also the key to creating characters that can generate behavior, that is, virtual actors that can get themselves into states and as a result perform patterns of behavior over time that weren’t explicitly pre-written by their author.
Another way to answer your questions — and this is something my collaborator Michael and I often say — is that there is no “design only” solution to creating non-trivially interactive characters and stories. To make something more sophisticated than a stimulus-response bot or a choose-your-own-adventure story, one needs to start modelling behavior.
I think modelling behavior is probably equivalent to what you’re calling truly “feeling” the emotions they are performing. Do real actors need to do that? By and large, I think many successful actors truly feel the emotions they are performing; by doing so, it allows them to generate a more authentic, robust performance.
Finally, let me say that these dramatic models of behavior, while technical in form, are heavily shaped by our artistic goals. That is, when creating these models — essentially these rules governing how virtual actors will behave — we’re always thinking about how the player perceives this behavior, are these rules going to result in the kind of performance we want to achieve for this actor, etc. As artists, we’re still very much in control of what the actor will ultimately be capable of doing or saying, even though exactly what is said and when will be determined in real-time, in response to the player’s actions. Even as these models get deeper and more generative in the future, I feel we’ll be able to retain a high level of authorial control over them.
MS: How do you feel about the concept of plot in interactive storytelling? And more specifically about the concept of a drama manager (a computer program that steers an interactive story towards an interesting plot arc)? Do you believe that a drama manager can be built to tell stories that can rival traditional literature or good theater? Or is pulp and formats the best it can do?
AS: I very much want to create systems that can generate a variety of narratives, in response on a moment-by-moment basis to what the player is saying and doing. Necessarily, this means we must abandon the ideal of a tightly-plotted story, because players are always going to be experimenting, pushing things in all kinds of directions, that will rarely allow for an extremely well-formed overall plot to be created.
That said, let’s not forget that in a real-time collaboration between the human player and the drama manager, that the drama manager — who is controlling the behavior of all of the virtual actors — is one half of the overall authorship of the experience. 50% control over the events being generated in a story is probably is enough to allow a loosely-plotted story to be created, if the overall domain of the story is set up to allow for that.
For example, in Facade, you’ve got the player vs. the duo Grace and Trip. The player can say and act any way they like; they could act like they’re dying, or act really violent, or completely absurd, what have you. The coordinated duo Grace and Trip will attempt to respond to the player, but also can believably mix in their own agenda, even believably ignore the player at times as needed (just as the player may ignore Grace and Trip if she wishes). You may end up with a absurdist story in the end where the characters each seem to be in a world of their own, but because the characters all share the same space and to some extent are reacting to one another, I’d at least call that loosely-plotted.
(Ideally the human player and drama manager cooperate and help each other out, at least some of the time.)
I’m not at all interested in making a drama manager that attempts to force the player into a particular story. If Facade seems to do that, it’s only because we ran out of time and/or energy to author a broad enough array of responses to at least respond to what the player wants to do. (This is another argument for more generative models of behavior, to help make virtual actors even more responsive to the player.)
Once you buy into the idea of allowing for loosely-plotted stories, even “bad” stories to be generated, we get rid of several of the supposed conundrums of interactive stories: 1) we get rid of the conflict between freedom vs. well-formed plot, because we’re not requiring a well-formed plot, we’re okay with a looser plot; 2) we get rid of this concern of some that too much freedom is bad thing, that we must greatly constrain players in order to ensure for a satisfying experience. I’m much more in favor of an open ended interface, that allows players to do and say anything they want, and only apply constraints on how the virtual actors will decide to interpret and work with the player’s actions. Again, let’s give the player a full 50% of the authorship of the story, and the drama manager 50%. Ideally the player and drama manager cooperate, but they don’t have to.
MS: The linearity of a story and the linearity implied by goal-oriented gameplay seem oddly compatible. This motivates people like Marianne Krawczyk (writer for God of War) to claim that they overlap, that the story of a game is expressed through goal-oriented gameplay. Obviously your narrative ambitions go far beyond anything an action game can express. Does this mean that goal-oriented gameplay should be abandoned?
AS: No, goal-oriented gameplay shouldn’t be abandoned, in fact it should be expanded and broadened. It’s only natural that players find achieving goals satisfying. But allowing players to form their own goals and be able to pursue and achieve them, in various ways, will be far more satisfying than offering players only one primary goal and one linear path to achieve that goal.
Just as importantly, we should allow players the freedom to just screw around, to play, to not pursue a goal if they wish. In total, we should be making dramatic worlds that allow for both freeform play, as well as goal-oriented play.
MS: You once mentioned that you thought of agency as one of the most important elements in interactive fiction, i.e. the things that the player can do and how this effects the other characters and the game world. Why is agency so important? Does a higher level of agency improve the story? And if so, how?
AS: I mention it all the time in my blog posts! Agency is the most fundamental property of interactive anything — be it games, web surfing, email, what have you. If you can’t have meaningful influence over what you’re doing when you take action on the computer, why is it interactive? If you’re to be led through an experience, why didn’t the author just make a movie or write a book?
But like I said earlier, I don’t think players need 100% control over the experience; an interactive experience can be a 50/50 collaboration between the human player and the system — where the system is a proxy for the human artist/programmer who created it.
Let’s not forget, ultimately, playing interactive art/entertainment is an interaction between people, via the artifact of a software system — particularly when the system is simulating (dramatic) human behavior, i.e. virtual actors.
Posted on June 6, 2006 at 12:35 am
I found this a very enlightening interview. I think that Drama Princess has a different focus, is trying to come up with “a third way”. When Mr. Stern says that the answer to my first questions “depends on what you want your virtual actor to be capable of doing”, he has already made the choice to talk about the solution in quantitative terms. He already assumes that the problem is how to make the virtual actor do something. This is not the problem of Drama Princess. Our problem is how to make the spectator think that the actor is doing something. The solution to this problem may deviate from the scientific (or scientifically inspired) solutions quite a bit more than simply in terms of thinking of our characters as more or less stupid. Is a character on a Rubens painting more stupid than a character on a Titian painting?
I can accept that a certain form of “behaviour modeling” is required, but I’m interested in ways this modeling can occur from the outside. It’s easier in a way, to build a system from the inside, with goals and motivations and such, because you can search your own mind for this. To model them from the outside, you need to study the symptoms and try to recognize the patterns in those symptoms, perhaps even without interpreting them (leave that to the spectator). Or as all drawing teachers say: “draw what you see, not what you know!”
This does not necessarily mean that the system will be simpler. It only means that you’re modeling the thing that the spectator is going to see directly, rather than the engine of which you hope it is going to produce the things that the spectator will be seeing.
Since Mr. Stern advocates making some kind of artificial mind for autonomous characters, I wonder what he thinks of The Sims. The latter also builds minds but the characters this produces aren’t nearly as believable and endearing as Grace and Trip in Façade. I wonder what’s the difference…
Posted on June 6, 2006 at 12:36 am
[…] After interviewing Andrew Stern (soon to be published here), I realized that our approach to story is very different than most when dealing with interactive narratives. This has a deep impact on what we want our Drama Princess to become. I tended to make it easy on myself by just saying that we don’t care much about plot, but given, amongst others, the answers I got from Mr. Stern in response to this, I guess the impact of such an attitude is not clear. […]
Posted on June 6, 2006 at 3:15 am
Good stuff. Agency definetly is the most imporant thing, in fact, I hit on an idea about three weeks ago I call “conservation of agency” where the user’s sense of meaningful choice requires a balance between the local and global effects. Its a nice way to think of the design space, and it sounds like Drama Princess’ focus is even more towards the local agency side of things than my project, which has roughly the same local/global porportion as Facade, though the system will be less complex.
I agree with you that a Drama Manager isn’t nessecary, but for me I’m sort of “spiritualizing” my DM, or distributing it, so that its spirit lives in the low-level rules. To a large extent I’m going for a more ad-hoc system, but only in the sense that I sacrifice scope of the engine, I’m still going to have a (hopefully) elegant personality model for the characters, but it’ll be more specific to the magical context of the game’s setting. Also, I dig his comment on goal orientation, which I think reflect the comment I left below (which I wrote before reading this).
Basically, I think you’re on the right bead with the idea of imitation over cognitive simulation, but you still need a data structure to hold the imitations, and you need processes to operate on that data. Going about it entirely ad-hoc will be less of a service to your players, since you’ll be using a model build from your assumptions; the only different there is that the model is in YOUR head, instead of the virtual head of your agent.
So I think you should implement a basic cognitive model for behavior, but allow it to be highly parameterized by player input, more so than the Facade suite or Stron. A book I’d really recommend thats been helpful to me is “Better Characters By Design” by Dr. Katherine Isbister. It approaches this idea from a socal, “outside” view, instead of a cognitive, “inside” view.
Posted on June 6, 2006 at 8:51 am
Thanks for the reading tip! There’s an excerpt from this book on Gamasutra.
In terms of comparing our approaches, I think you are much further along the path of designing a system, while I’m still playing with ideas and concepts. That will soon change as we have scheduled this month for design. So stay tuned. 😉
But obviously, Façade is even further ahead in the process and I’d like to hear what you and other people think about it and Mr. Stern’s ideas in general.
Posted on June 6, 2006 at 10:45 am
He already assumes that the problem is how to make the virtual actor do something. This is not the problem of Drama Princess. Our problem is how to make the spectator think that the actor is doing something.
In the end, I too only care about is what the player thinks the actors are doing. I’m not motivated in a scientific way to make the internal workings of the characters be “true” or “pure”, for it’s own sake… The reason I’m advocating making virtual actors intentionally do meaningful things, is because my experience has taught me that such an approach has a greater chance of resulting in the player perceiving the actors as doing meaningful things, versus some other method of acting.
In fact, regarding implementation, I used to think more along the lines of “faking” it; I’ve come to realize that it takes too much effort to keep that house of cards from collapsing, better to move towards “really” doing it, but still of course use techniques of artifice whenever I can get away with it.
Note, even in Facade my collaborator Michael and I are not doing as much “pure” behavior modelling as you might think; much of the apparent deliberation of the characters is in fact “baked in” to the narrative itself. That is, often there’s *not* a deeper emotional model motivating the characters, it’s actually embedded in carefully written, particular, somewhat ad-hoc clusters of narrative behaviors designed to achieve certain narrative effects.
When you say modeling “from the outside”, I take that to mean you hope to supply the player with enough cues and hints to cause her to easily imagine a narrative, to fill in the blanks — a more impressionistic form of story, without explicit plot. Actually, that’s a style of narrative I highly enjoy; for example, one of my favorite directors is Eric Rohmer, whose films are famously plot-less; fiction-wise, I love Raymond Carver, for example. In future work, I plan to create interactive experiences of that nature.
However, at this point in time, at this point in the evolution of interactive narrative, I feel that approach is a retreat from the harder, fundamental problem of creating a more explicit plot in reaction to the player’s actions. I’d rather first make significant progress in creating a system that results in explicit (if loose) plots, and then once understood, relax that requirement and make characters/stories with less plot. It’s sort of like the philosophy of learning to paint abstractly: best to first learn how to paint really well representationally, before moving on to abstraction.
Michael, in your work perhaps you can skip this step of creating explicit plot; for me, I feel compelled to work on it for a while. At a minimum, it’s a fascinating problem that many, many people want to solve; also, if solved it can lead to commercially marketable work, versus less marketable plot-driven work; it’s a way to make a living at this along the way… 🙂
I wonder what he thinks of The Sims. The latter also builds minds but the characters this produces aren’t nearly as believable and endearing as Grace and Trip in Façade. I wonder what’s the difference…
The Sims does in fact have a behavior model motivating the characters, and while I think that’s effective for generating reasonably interesting behavior, I’m not at all satisfied with the particular model that was implemented (too simple), and especially unsatisfied with the dramatic performance of the model: the Sims have no specific language. While Simlish and the icon balloons are an elegant design solution, I don’t particularly enjoy it as a player. In fiction, I enjoy specific dialog of specific characters, not generic emoting. To me, the Sims are an ant farm version of people; too bland for my taste; even more bland than daytime soap operas. Also, as a player I don’t want to play God, I want to be a peer of the characters I’m interacting with. (However, I highly enjoy playing God when I author interactive dramas; if I didn’t have the ability to author my own interactive dramas, perhaps I’d need something like the Sims to find that authorial pleasure.)
btw, my collaborator Michael and I have talked to Will Wright a few times about Facade. Will thinks our approach needs to get more generative in order for it to be a successful technique. That is, currently our techniques require too much authorial work to achieve a narrative, and therefore our narratives will end up being overly limited. My answers to the original questions above essentially agree with Will, that our approach needs to become more generative; however, because we’re not interested in abandoning specific characters and specific dialog, it’s going to require an order of magnitude more sophisticated of an approach than the Sims to pull it off. (One of Will’s big strengths is working within his means.)
Posted on June 6, 2006 at 10:51 am
I mean to also suggest you look at Brandon Rickman’s most excellent Dr. K Project. He presented it at the 1999 Narrative Intelligence symposium:
http://www.lcc.gatech.edu/~mateas/nidocs/Rickman.html
Brandon blogs at Polymath, aka antimodal, here:
http://www.antimodal.com/
Posted on June 6, 2006 at 11:07 am
Thank you, Andrew. That was a very enlightening response. And a warning that does make me hesitant but will not prevent me from taking the plunge anyway (I justify this by defining what we do in a positive way: we’re not trying to make plotless stories, we’re trying to make interactive paintings, e.g.).
And when I end up with my head against a brick wall, I will enjoy your dancing around me, singing “I told you so! I told you so!” 😉
Thanks for the tips, as well.
Posted on June 7, 2006 at 7:35 pm
Hopefully I’ll be dancing with the Drama Princess, singing “oh joy, there is another way!”
The Dr. K. project listed above certainly is another approach to narrative, and one I found really interesting and inspiring.
I did want to add two clarifications to my comments above:
At the end of my answer to your 5th question I said “playing interactive art/entertainment is an interaction between people”, by this I meant the player and the human author of the software (for which the system is the proxy for the author). I should say this relationship is very stretched when the system is primarily abstract gameplay (e.g., chess, Tetris); it holds best when the system has embodied virtual actors.
Also, I’d like to modify my mentions of the term “behavior modeling” to be dramatic behavior modeling. By adding dramatic as a modifier, it distinguishes the root motivation and framework of the model to be focused on drama, leaving the “boring bits” of real life out, versus say a more cognitive model of behavior, such as the B&W creature. There is definitely much overlap between dramatic behavior and more traditional cognitive behavior, but there are important differences.
Finally, I should say regarding the Oz project, don’t take Michael Mateas’ Oz-centric review as the final word on the Oz philosophy; you really should read the Joe Bates papers from the early 1990’s, and at least wade into the theses of Loyall, Reilly, Weyrauch and Sengers. There’s a lot of really great material in there, much of which the believable agent community has yet to absorb, I think because of NIH biases. Sadly, the Oz crew never converted their old papers to pdf, making it more difficult to read them. But put in the effort!
Also, have you read Brenda Laurel’s Computers as Theater? It’s a fundamental text.
Posted on June 7, 2006 at 10:32 pm
I like the Dr K project description as well. Oddly it reminds of an idea I had for a 3D rendering effect. I wanted to make an engine that took Level of Detail to its extreme in combination with the idea of peripheral vision and the theory that humans first see what they know and then what’s really there. The idea was to render the whole scene in simple shapes and only if you you stood still, the things in the center of your vision would get more and more detail.
Can you recommend software to read .ps files? 😕
I have ordered Miss Laurel’s book. I really like her attitude and other essays that I’ve read.
Posted on June 8, 2006 at 12:34 am
This looks good, let me know if it works
http://www.ps2pdf.com/convert/index.htm
Posted on June 8, 2006 at 12:59 am
Ps2dpf.com works! Fonts are a bit pixelly but the text is still readable. Thanks!
Some files don’t convert properly.
Posted on June 10, 2006 at 1:14 pm
Hello. I find a good way to get beyond the common preoccupation with “behavior modelling” is to take a step back from the idea of a player operating a computer. Make this relationship more symmetric. Instead of the player having agency within the computer generated environment, consider what it means for the environment to have agency within the player. When you can get both sides to recognize the patterns that result from this mutual agency, then both sides can start to negotiate the meaning of those patterns.
There are a couple of caveats with this situation. For one thing, the player has to be willing to enter into this symmetric relationship, to not insist upon feedback for the sake of feedback. Second, the author of the system has to be aware that the quality of the narrative created by this relationship will be a lot stranger than an ordinary conversation between two people. The results are far different from the familiar and coherent outputs of a simulation.
Posted on June 10, 2006 at 2:13 pm
Mutual agency sounds fascinating. I have a hard time imagining it, though. Do you just mean it as a mental exercise while designing? Or as something practical that can be made? And if the latter, can you give an example of what form that could take?