Rogelio Cardona-Rivera Plays Games for Science

Sarah Webb  00:00

I’m your host, Sarah Webb, and this is Science in Parallel, a podcast about people and projects in computational science. Before we get into our new episode, I encourage you to support us and hit the subscribe button wherever you’re listening today. [Theme music in background.] We’re continuing our series on creativity in computing. And this episode turns toward games in ways that go beyond hardware and software to address the essential elements that evoke human responses and applications of these ideas.

[Theme music plays]

Sarah Webb  00:51

My guest applies the scientific method to understand game design, incorporating computer science, cognitive science, narrative, and much more.

Rogelio Cardona-Rivera  01:01

So my name is Rogelio Cardona-Rivera. I am an assistant professor of games at the University of Utah.

Sarah Webb  01:08

They’re applying this work in areas such as artificial intelligence research, Indigenous storytelling and education. Rogelio’s interest in games goes back to playing Super Mario as a child as an unconventional strategy to preserve their eyesight. After college, Rogelio was offered a tech job but decided to pursue games research as a graduate student instead, They didn’t look back.

Sarah Webb  01:30

Join us for a conversation about the emerging field of technical games research that’s rich with ideas, insights and communication strategies for anyone working on interdisciplinary science. This interview kept me on my toes in the best possible way.

Sarah Webb  01:44

Rogelio, it is great to have you on the podcast.

Rogelio Cardona-Rivera  01:47

Oh, it’s lovely to be here, Sarah, thank you for having me.

Sarah Webb  01:50

So your job just sounds like a dream job. I’m imagining that a kid and even a lot of adults would be excited about what you do. What does it mean to be a professor of games?

Rogelio Cardona-Rivera  02:05

In truth games are such an interdisciplinary field that the question will probably receive a different answer, depending on who you ask. My emphasis in games is on the technical side. So the way that I would describe it is I am a technical games researcher. And so you might also say I’m an assistant professor of technical games research. The things that I focus on are more on the computation side, so specifically, how games can be constructed to elicit specific responses, both at the hardware and the software level.

Sarah Webb  02:41

In the day-to-day sense, what are the questions that drive you?

Rogelio Cardona-Rivera  02:44

I do get to say and claim very, very deliberately I play games for science, and everyone who knows me knows that I have a backlog of games that I will very much never get to, because the game log just keeps expanding. But the kinds of research questions that I’m interested in answering are really centered on design. So we rely on games, not just for entertainment, but also for training, for learning, for education. And despite the almost like 20 years of game development and game design practice that we’ve accrued, it’s still very much is a field that doesn’t rely on lessons learned. And so we don’t overtly know if we want to create, for example, a game that elicits a sense of suspense or a game that elicits a sense of surprise, what are the structural elements that you have to introduce to elicit that response. We kind of just reinvent the wheel every time. And I’m hoping that the work I do changes that to some degree that it makes it more of a systematic practice something that’s predictable in the long run.

Sarah Webb  03:51

So that there are building blocks that you know, if you want to do this, you put this piece in there? If you want it to do that, you put another piece in there? Is that how that works?

Rogelio Cardona-Rivera  04:01

Exactly that. The kind of metaphor I rely on is I’m looking for the equivalent of my basic LEGO bricks for building games, right? The LEGO Company has created has done a phenomenal job at creating like these, like base blocks that like are, you know, combinable in a bunch of different ways. But at some level, it stops, right. So you have the basic block that you can’t decompose that block further. I’m looking for the equivalent on a computational level for games: What are the building blocks that I can combine in specific ways to achieve specific effects?

Sarah Webb  04:41

So how did you get started with this?

Rogelio Cardona-Rivera  04:44

It was a roundabout way of doing it because I really didn’t get started into studying games and like understanding them systematically until grad school, actually. I did my undergraduate at the University of Puerto Rico at the Mayaguez campus. I did it in computer engineering, and I got into computer engineering because I personally believed in the transformative power of games. But I ended up not doing anything games-related, not even as a hobby, until grad school. I kind of had that crisis moment when I was about to graduate. I had a job offer from Microsoft, when I was looking at the terms of the job offer, I just thought to myself, like, is this really what I want to do?

Rogelio Cardona-Rivera  05:18

And so at the very, very last minute, I, on a whim, decided to apply to four different graduate schools that I knew had game programs, or game-related research, really. And I said to myself, Okay, well, you know, if I’m really wanting to make a thing out of this, that was the time, and I got into graduate school in 2010, at North Carolina State. And at the time, I said to myself, I’m not still sure if the Ph.D. is really right for me. I might just get a master’s and go into the games industry and try my luck there. And then I ended up really liking research. And so a combination of I knew that I wanted to do games, I ended up really liking science, and the scientific method led me to where I am today.

Sarah Webb  05:57

Take me back to games and what they meant to you originally, I want to know the deep roots of it.

Rogelio Cardona-Rivera  06:03

 I don’t know how many people can actually say this, but I actually have my parents to thank for this. So as a kid, I this is maybe right when I was about to enter kindergarten. My parents took me to an eye doctor, an ophthalmologist, and the ophthalmologist diagnosed my right eye with amblyopia, so lazy eye, and the situation was so grave that my  doctor at the time told my parents that I was a candidate for surgery, that if nothing happened between now and the projected surgery date, I would certainly need it. Otherwise, I would just have reduced vision out of one eye,

Rogelio Cardona-Rivera  06:05

We were looking at alternate options, and so the doctor basically said, Well, one way that we might be able to mitigate this is, in essence, having a bunch of like eye-training exercises. And so my parents in their wisdom, and realizing that I would be hard-pressed to be convinced to do eye-training exercises every day, what they basically did was they said, We’re gonna get you a Super Nintendo. And so I got eye patches, and I patched my good eye, and forced my lazy eye to try to track the different characters on the screen. And I remember playing Super Mario World on the Super Nintendo for hours upon hours. And to the point that the next checkup visit with the ophthalmologist, the pre-surgery screening, he said, I cannot believe the dramatic change in terms of the motor control. And I was able to avoid surgery. I’ve always been drawn to games that have story components and motivate me to find out like what’s going to happen next. But I think if I had to point to one instance, in my life that love for games started, it was back then.

Sarah Webb  07:47

 That’s  really wild– considering that probably there’s a subset of parents that are all “You’re gonna destroy your eyesight!”

Rogelio Cardona-Rivera  07:57

In fact, that was quite the opposite. I can literally say that it saved my eyesight and saved me from surgery. It’s yeah, it was phenomenal.

Sarah Webb  08:05

But you also mentioned story. Tell me about where story came in for you, too.

Rogelio Cardona-Rivera  08:14

As a kid, you know, I had a lot of exposure to my dad used to read bedtime stories, like, every night, and he’d make the voices, and I would just be enthralled and just really interested in finding out what happens next. Between that and just watching a lot of cartoons as a kid. The episodic format of cartoons was really fun, because it’s effectively a new story every week with the same character. So like, there is some kind of continuity. But you know, cartoon characters get into all kinds of crazy high jinks and things happen. And there’s drama, and there’s suspense. And there’s mostly goofy chaos. But I’ve always been drawn to stories as a child kind of through television first. And then as I grew older, like into movies, and specifically Disney, my parents were fortunate enough to be able to to take my brother and I to Disney World in Orlando. And seeing stories come to life– it felt like pure magic to me. So I think that’s where maybe the love affair for stories comes from. It’s just being exposed and seeing stories in different formats in different media and kind of just letting yourself be part of this alternative possible world for you.

Sarah Webb  09:23

On your research website, you talk about this vision for establishing a science of game design. And you’ve already alluded to some of the pieces that go into that we’ve been talking about: narrative, computing is why we’re here, cognitive science, other things, artificial intelligence. I guess, let’s talk about some of these other ideas. Was cognitive science something that you were always interested in? Where did that come into your interest?

Rogelio Cardona-Rivera  09:48

 Right, I think part of my interest in cognitive science started in my undergraduate and in a sense I have my dad specifically to thank for this on my dad. Dad is an accountant and has a Ph.D. in finance. And I grew up hearing about accounts receivable way before I even knew what that actually meant. But then, you know, when I get into undergraduate, there were like these electives that you have to take to complement your undergraduate degree in engineering. And so I ended up taking a psychology course on a whim. And I ended up taking a microeconomics course on a whim. And I found just really, eerily shocking, the degree to which we could predict, both at like the personal scale in psychology and kind of the microscale in economics, what people will do given conditions that they encounter in their daily life, right.

Rogelio Cardona-Rivera  10:44

So psychology has kind of built up this, this massive body of knowledge to understand when you show this stimulus, you should get this response. In economics, it’s kind of like that, but in aggregate, right. If you fix this price, then this is how it will affect supply and or this is how it might affect the demand for the product, given a certain supply level. It felt like a game in that sense of like, oh, like these are like different dials on a machine that I can play with. And if I increase more of this, then I should see more of that, and so on. And in graduate school, as I said, like it was the first time that I started thinking about games as a discipline in its own right. And it kind of clicked for me that that’s what’s happening. That even coming from a computational perspective but thinking through what is needed to build these games, right at the software at the hardware level, I realized you can only get so far without considering the human element here.

Rogelio Cardona-Rivera  11:36

There’s only so much success that you can achieve without deeply considering what the psychology behind the gameplay, what does it mean for you to be psychologically engaged in play or psychologically engaged in a game? Right? And I think that that’s kind of where I realized, oh, yeah, if I don’t look at cognition, I’m missing this really massive and key piece of the puzzle, that  will only let me get so far. So I just kind of rolled with that. And I said, Well, practically speaking, I need this. And then the deeper I got, the more I realized, wow, we know a significant amount of things, which in and of itself is also kind of scary, because knowing how your brain ticks means that I can manipulate things that you see to get your brain to tick in a specific way. And that’s also where my ethical concern comes from for like games and game design.

Sarah Webb  12:27

So where did your interest in computing start?

Rogelio Cardona-Rivera  12:31

My interest in computing came from just wanting to figure out how things work and breaking a bunch of things in the process, I must have broken so many devices as part of growing up, I would then with what little I could glean from, you know, tearing things apart, I would just come up with these stories in my head of oh, this is how this thing likely works. And I would use that to fuel my own LEGO creations. And I would say, Okay, well, this, this LEGO achieves like lightspeed travel by taking in energy from the sun and harnessing it into this engine thing that I somehow assume worked. But maybe this is a confession, I actually didn’t program until like my third year of undergraduate, which is bizarre to me, I guess, because I was looking at my peers. And there were people who were programming C code since they were 13, 14. So in many ways, I actually felt really behind. Well, one thing I will say is that my access to a computer, I think, really fueled it, I was playing games on an Apple II device. And then after, after the Apple II, I played games on a Windows 95 machine. And just, again, thinking of these as portals into other alternative worlds, was just a compelling thing that I wanted to know more about, but I knew remarkably little about it going into my undergraduate,

Sarah Webb  13:49

Where we started was you basically took a leap and said, Okay, I’m not going to take this job at Microsoft. I’m going to go pursue this thing that I think is really going to be cool and interesting, and see where it leads me. So talk a little bit about that leap and what that journey has been like, sure,

Rogelio Cardona-Rivera  14:07

I like to say that I am distracted by a bunch of shiny objects and dovetailing into the curiosity bit that I mentioned earlier. I’ve just been a deeply curious person about a bunch of different things. And for no rhyme or reason other than just, I kind of want to know how this works. It is both good and bad. I will be the first to tell you that I feel like sometimes I suffer from having a lack of focus in one specific way. But invariably, I have benefit from being able to tie these just completely disparate concepts into one thing. And games actually is a remarkable home for that kind of thinking. It’s both deep and broad, but you get a lot of benefit from thinking more broadly than deeply. And I haven’t even touched upon AI which is something else that I fold into my own research and is a key part of the work that I do, which I didn’t get exposure to until grad school. And there with AI, I think a lot of things kind of fell into place.

Rogelio Cardona-Rivera  15:05

Because on the one hand, AI is a remarkable tool to help us think through the structure of the mind. And the prevailing theory or metaphor in cognitive science is that, to some degree, the human mind is like a computer, in the sense that it processes inputs, it performs some kind of mental computation, and then produces outputs, whether it be additional thoughts, or utterances or body movements, etc. That metaphor is not perfect. If you really start to attack the metaphor, that kind of breaks down. But at the time, as a grad student, I did not expand my thinking to that point. And I said, Okay, well, if you think of the mind as a computer, how far can you get? And in the case of game design, it kind of served as a really great metaphor of saying, like, Okay, well, I am building a game. That’s the thing that I’m wanting to do. And I’m wanting to build this game, because I want to achieve this experience for the player that’s like a narrative arc for the player. So what do I have to put as input to this, so that when it is received by players as input to them, they actually think through the things I care about or feel the things I want them to feel or enact the things I want them to enact, Right?

Rogelio Cardona-Rivera  16:23

So a lot of game design is about shaping the behavior that you kind of want to see in players because you think that that behavior will be fun in some sense, will be exciting for the players to experience in some sense. So AI, as a software tool, was really helpful. Because in AI, there were a bunch of different methods that you could apply to achieve different effects. AI is almost like a library of input-output patterns. And that helps me think through what are the kinds of input-output patterns that I could predict using software and computation. That was really like the unifying force is really like thinking about like AI as a potential vehicle, or conduit, to just model the different facets that I care about to achieve specific game designs.

Sarah Webb  17:12

AI is related to computational science. And so what’s the role of computational science in all of this?

Rogelio Cardona-Rivera  17:21

For context, in you know, as part of my graduate school experience, I was elected to be a Department of Energy Computational Science Graduate Fellow,

Sarah Webb  17:28

That’s the DOE CSGF that we often mentioned on this podcast. Science in Parallel is a media outreach project of that program.

Rogelio Cardona-Rivera  17:36

And, at the time, it was because I wanted to apply high-performance computing to modeling the aspects of the human mind, in essence, to approximate the computational power of the mind, it made sense to me at the time to say, well, we’re going to need just a lot of computing power. We’re going to need to really orchestrate a bunch of like high-performance computing systems to even approximate what the mind does. The thing is, as I progressed in my career, and as I graduated with my Ph.D., I realized that the computational science, while there is a really strong component that’s related to high-performance computing, computational science is more about thinking about other phenomena along computational terms. So like you want to, for example, model computational fluid dynamics. And so you construct this very elaborate high-performance computing simulation, to just track all the degrees of freedom that get changed as part of the simulation. But there’s a critical thinking step that happens before that which says, okay, in order to be able to take this phenomenon that’s out there, it’s really complex, and be able to actually simulate it, you need to be able to understand it well enough that it becomes amenable to simulation. I need to be able to track at least the following kinds of inputs, manipulate them as follows, and then produce the following kinds of outputs. So that thinking step is kind of what was, in my mind, glossed over as a grad student.

Rogelio Cardona-Rivera  19:01

And so where computational science falls for me now is different from how I used to think about it. Where I’m at right now is shifting kind of the emphasis all the way up the abstraction, in what phenomena I’m targeting to begin with. You can think of game design as non-natural phenomena, or artificial phenomena. Artifactual, because you’re built, you’re building an artifact of some sort. And so if you think about, okay, this is an artifact that has been constructed by people, right, could you apply computational science to understand the process of game design or the process of designing in general. And that’s, that’s where computational science now sits for me is saying, yes, in computational science, we want to simulate phenomena in terms of programs and computer software. That’s fantastic. Can we do the same with artificial phenomena, instead of just like limiting ourselves to natural ones? I would say the answer is yes, we can do it. The problem was always in the details, right? What do you encounter along the way, what the method of inquiry is, I think remains the same. And I think it’s powerful to apply it to things that are human constructed.

Sarah Webb  20:13

I want to hear about some examples of this. What are some of your favorite projects, ones that you think are case studies for all these ideas that you’ve been talking about?

Rogelio Cardona-Rivera  20:23

So one project of the three that I have going on is a project funded by the National Science Foundation, in their Robust Intelligence program. So I earned a CAREER award to study this project, looking at a specific kind of AI called automated planning, to simulate the process of human narrative comprehension. So there’s a lot of words there. On the one hand, the human side of it is for the purposes of this research project, it is not sufficient to look at an input-output model. And what I mean by that is, if I have an AI system that can parse a lot of text and can reproduce answers to questions I have, you might anthropomorphize that and say, like, yes, that computer program understands that. You can’t see me but I’m using scare quotes here understands the narrative that it was it was input.

Rogelio Cardona-Rivera  21:24

But for the purposes of this project, it’s not sufficient to do that. Because what I really care about is, can I simulate the process through which people arrive at answers to questions and like hides the the other part of the emphasis, which is really like plan based simulation of narrative comprehension? That means you’re able to answer questions about the story that is both overtly told to you and the story that you infer as a consequence of what has been told. That kind of exemplifies the science of game design, specifically focusing on the narrative component, which is, if I want to understand what you understand, and I want to be able to give some kind of guarantee over the story, for example, at a minimum, you should be able to understand the story by the end of its telling, then, this kind of research would help in that regard. Because not only would we be able to predict that, we would be able to predict where you don’t understand and further how you arrive at those conclusions. It’s an input-output model plus a behavior component. So I need to know what is the mapping of input to output that allows you to arrive at an answer to a question I have about the story. That’s one project.

Rogelio Cardona-Rivera  22:31

A second project that I have going on is also funded by the National Science Foundation. This is in collaboration with two colleagues over at Utah State University, Melissa Tehee and Brianne Litts, looking at creating computational models of Indigenous storytelling practices and looking specifically at Indigenous practices within the state of Utah. And in partnership with an ex-chairman of the Western band of the Shoshone nation, Darren Parry, who is now actually a professor at the U. That project recognizes outright that the available software that we have to tell stories, is Western-coded in the sense that what we think of as a story is latent in the software that we have to produce stories for others. And this is maybe evident most crisply, in the computational artifacts to tell stories, like for example, the iMovie creation software, or the equivalent of the Windows software is, to create stories, you’re met with like the timeline view of narrative, right? So what that is already telling you is that first of all, time is organized linearly from front to back. And further, the narrative content is intended to be clips that are spliced together. And you are responsible for tracking what the continuity is across the clips. But the atomic building blocks are these event chunks that are captured in clips. And as a fun fact, in iMovie, I think that the smallest clip you can have is like 0.4 seconds, which also tells you something time doesn’t exist below point four seconds. So

Sarah Webb  24:19

Well, it sounds exactly like what I have to do when I’ll edit this podcast.

Rogelio Cardona-Rivera  24:23

Oh, right. Exactly. Exactly this right. So like, there’s what Sarah, what is the minimum time that you recognize as like actual time.

Sarah Webb  24:31

I don’t even know, but I’m not usually putting together a 0.4-second clip.

Rogelio Cardona-Rivera  24:37

Right? So the software that you’re using to kind of compose these narratives, already has an implicit value system attached to it, whether you’re consciously aware of it or not. And so the emphasis of the grant is figuring out okay, this kind of storytelling assumptions around like the structure of story and what you can do with that runs contrary to the storytelling practices that that the Western Shoshone us in telling their stories that convey tribal knowledge, tribal lessons, tribal culture. And so if we want to enable computation as a tool that that tribal partners and tribal folks can use, then we have to rethink the software from the ground up. And in that regard, it’s thinking of what are the affordances of software that will let Indigenous partners envision telling stories through the software in a way that’s more fluid than the friction that they encounter right now. Right? So this is another example of the science of game design, because instead of it being audience-focused solely, it’s also creator=focused. So we want to build the systems that afford the systematic creation of story structures, that also respects the creators’ mindset. I would hope that we don’t presume to tell Indigenous storytellers how they should tell stories. In fact, we would want the software to reflect what they already know and facilitate that in some way.

Sarah Webb  26:01

I’m imagining a potential counterexample of a circle rather than a line or something like that, where you could come back.

Rogelio Cardona-Rivera  26:07

Exactly right. So one of my colleagues, Brianne Litts had earlier work to demonstrate that if we just present tools that currently exist to young Indigenous storytellers, who are keen to use technology, they already put their hands up and say, I don’t even know how to tell a story like this. It just feels very unnatural to me to splice it like this. And, Sarah, what you’re mentioning, the conceiving of time as circular, not linear, is one thing. Another thing that we’ve discovered is the importance of a sense of place. So no stories take place in a vacuum, as a kind of element in the storytelling practice is critical. You’d be hard pressed to name a movie-editing software that lets you say, Okay, this happens in this specific place and uses that as a constraint for things that happen around it. So yeah, absolutely, like having affording a different framework as a way to discuss it.

Rogelio Cardona-Rivera  27:01

The third project I’ll mention very briefly, is a more internally focused project that I have going on at the U with several partners in the college of science and the math departments were wanting to build a VR game intended to complement a math course on campus: math 1050, college algebra. It’s a service course; it sees upwards of 1400 students every year. Unfortunately, you know, math, having the stigma that it has, generally, that course suffers from a drop rate of about 50%. Students either withdraw, or they get D’s or F’s or like nonpassing grade to advance at a rate of 50%. So we’re talking about 700 students

Sarah Webb  27:43

Wow

Rogelio Cardona-Rivera  27:43

 every year. What we’re hoping to do is not use really VR as a substitute for learning, but it’s more to give college algebra an application context.  Why you would want to learn in a kind of engaging setting, and embedding the mathematical concepts into the narrative universe. So the things work in the world based on math concepts, and it will hopefully get students to realize, oh, okay, this is this math concept applied. And in solving this puzzle, they may not even be aware that they’re rehearsing the math concept as part of it. And again, this all goes back again, to the science of game design, What elements do we have to introduce to elicit the right mental model of the math concepts on a kind of like lab by lab basis?

Sarah Webb  28:29

Very cool. How far are you along with this?

Rogelio Cardona-Rivera  28:34

Oh, yeah, so we are having a major development cycle over the summer, we’re hoping to pilot the course in the fall of 2024. So we’re, it’s a tight production schedule to be candid. But I think that we’ll be able to demonstrate a proof of concept of what this could look like, even if we don’t cover all of the course concepts.  I will be the first to admit that there is a tendency, I think, in computer science to wedge technology where it doesn’t really belong or hasn’t been validated in the first place. And so I’m all about pilot studies to kind of understand, okay, is this an appropriate intervention to introduce for this purpose? And so that’s kind of like our step one.

Sarah Webb  29:13

You’ve been talking about building this science of game design. Where do you think we are In terms of establishing this science of game design?

Rogelio Cardona-Rivera  29:24

Ah, this is a this is a really good question. It’s, it is tough for me to give you an estimated any way that would make sense we’ve had roughly 20-odd years of progress in understanding games across engineering, sciences, humanities, social sciences. One of the issues that I’m wanting to tackle is that I think that there is progress, but it is not clear. To what degree we have progress that builds on each other. And it is all and it is also not clear. or what someone else’s progress means for me. And this is, in part why I am so adamant about using computation as a kind of vehicle toward toward the science of game design. It’s because in general, software programs do not permit ambiguity, right, either compiles or it doesn’t, it either runs or it doesn’t. And so when you are in the process of creating like a computer simulation of something of some phenomenon, you have to fill in the blanks for any ambiguous spots that emerge. And my hope is that, as my work evolves, we’ll be able to just even fix on what the vocabulary words are and what they mean to each other. So one of the examples that I bring up in this discussion is the use of the word mechanic in games, like, oh, this game has the following mechanics. Sorry, I don’t know if you’ve heard this word before. I was hoping you would indulge me and tell me what do you think a mechanic is in games?

Sarah Webb  30:57

I mean, I guess it would be some operating process. But that’s just me throwing something out there.

Rogelio Cardona-Rivera  31:05

No, this is great. I would bet a donut that you have hit upon a definition that has been uttered by someone else. Because I think to date, we have 117 uses of the word mechanic applied in different papers to mean completely different things. The terminological ambiguity is just to be candid, maddening, it is just frustrating to no end because I no longer know whether you and I are meaning the same thing when we’re using the same words, which I think anyone who has done interdisciplinary work will tell you this is one of like the core problems, just like establishing an Interlingua that we can all agree with. And so, you know, there are lessons learned and lessons to be had, if we could only just understand what the words we’re using are to make sure that we’re meaning the same thing. So that I think is one of the core issues that I see is I can’t answer the question of like how far we’ve come? Because we can’t even agree on the words.

Sarah Webb  32:05

In a way it sounds like you’re creating a lexicon of

Rogelio Cardona-Rivera  32:09

Yeah, I think that that’s entirely a fair characterization. It is in part a lexicon. And the hope is that over time, as the science of game design evolves, the lexicon comes coupled, not just with a description of what the building block is, or what the Lego brick is, but also what that Lego brick does to someone, right. So like, when players encounter this specific content, what will they experience at whatever level of abstraction makes sense? Will they feel something? Will they think something? Will they understand it? Will they not understand it? And take it from there.

Sarah Webb  32:39

I asked Rogelio about the size of the field of technical games research, and they didn’t have an exact number. But they noted that academic industry and government researchers are working in this area.

Rogelio Cardona-Rivera  32:51

For example, Sandia National Labs has a phenomenal group looking at the application of games to understand like strategic nuclear deterrence and looking at games as like a vehicle to understand behaviors of people in those scenarios. So I’d say that there’s a there’s a sizable group, but I owe you the exact number.

Sarah Webb  33:08

So I want to ask you a bit more about artificial intelligence, because obviously, this is something you’ve been working with for a long time. And it’s something that has moved outside of the research space and into the public consciousness. What’s sort of your reaction to that transition? What surprised you? And how is this shaping your work?

Rogelio Cardona-Rivera  33:27

This is something that I was wholly unprepared for it because it was an unexpected thing of AI becoming a household term. I am used to, for example, my parents sending me articles about finance. I am not used to my parents sending me articles about AI. It’s interesting, I’ll talk about a kind of like at a macro and then maybe dive deeper into my own work. at a macro level, I think it is forced conversations that I think needed to happen, specifically around what the ethical responsibilities and concerns are around the use of AI. And I think it also created a bunch of– I’m gonna say AI boogeymen– because there are real pressing issues that we ought to address today. And I think that those are maybe being inadvertently hidden by focusing on AI doomerism, and rise of Terminator robots, which I cannot wait for that metaphor to fizzle out and I doubt that it will. There are real concerns that we need to be paying attention to. And that is so far beyond the locus of what’s possible today, or even what’s possible within 10 years.

Rogelio Cardona-Rivera  34:35

I think it is useful to have the discussions around what AI is doing and how AI engages, like, in different aspects of society. I think that talking about AI regulation is a worthwhile endeavor, and particularly to protect people in a variety of different ways. But it’s interesting that my own research and my own scholarship, the kind of AI that I do, Isn’t perceived as AI. And the reason for this is that a lot of the AI that I do is, effectively agglomerated under the label of symbolic systems. So there is no overt machine learning components. So I’m not inputting a bunch of people’s data. It’s more that I am looking at how to think about the automatic construction of content from a very, very precise approach, and that I know at every stage what this software system is doing. So one kind of fun tidbit from the history of artificial intelligence is that originally there was contention as to whether the field should be called out in the first place. And one of the proposals that didn’t win was something to the tune of automation informatics.

Sarah Webb  35:58

AI

Rogelio Cardona-Rivera  36:00

A whole different AI. right, and my secret social campaign here is to try to get automated informatics to take root, because I think it is a more apt descriptor of what it is. Artificial intelligence obfuscates the fact that at the end of the day, we’re talking about software systems and software systems are programmed by people. The kind of AI that I do is more about automation informatics and trying to understand what are the ins and outs of what you can automate or what you should automate, without really relying on other people’s data, which I think is one of the core problems that we have. It’s irresponsible use of other people’s data to train these AI systems to achieve certain things. For the world of games, it’s interesting, I’ve had to be very careful about how I talk about my work, because a lot of the work that I did, and I still do, could be used for this thing called procedural content generation, which today, you might know it as generative AI, it is an AI system that is given a set of inputs, and is tasked with composing those inputs in a way that produces some kind of artifact at the end of it, whether it be a fully fleshed game or game content, like audio for the game, or a level for the game, and so on.

Rogelio Cardona-Rivera  37:11

So in terms of how it’s impacted me, I’ve had to be very careful to say, okay, the kind of AI, like processing that I am doing is not relying on other people’s data. It’s more like thinking deeply about the structure of a thing I want to create, and going into like a meta creation process. So I ended up building the thing manually to understand it, and then saying, Okay, what are the atomic elements that I manipulated in the construction of this thing? Can I then take a step back and build those components automatically, right, and then compose them to achieve the ultimate artifact? And AI has had a long history in games, that there’s a lot of like, automated control of characters and a lot of AI that has been around almost, it’s the foundation of the field, or even the creation of games. And it’s interesting to see people’s reactions to it.

Sarah Webb  38:03

You’ve talked a lot about interdisciplinarity. And it seems like the work you do is probably extreme, almost extreme in that way, which

Rogelio Cardona-Rivera  38:12

almost a ridiculous amount, you could say it’s alright, it’s okay. It is yeah.

Sarah Webb  38:17

But you actually get to do research, and dip your toes and all these different areas. What does interdisciplinarity mean to you in this extreme case?

Rogelio Cardona-Rivera  38:28

Wow, maybe I’ll answer this in terms of like lessons that I’ve learned along the way of like how to do this kind of work. One concrete thing is trying to keep an open mind about what the right way to proceed in any kind of problem is because I have to acknowledge that my training is in computational science and computer science. And I have a very biased way of thinking in that regard. And I don’t see that as a negative. It’s very difficult for me to not see the structures and algorithms that are latent in discussions, but not presuming that that’s the right answer, I think is a core component is being willing to be the most ignorant person in the room and just like ask  maybe the most basic questions about what does this word mean? Again, going back to the interlingua problem of can we even agree on what language we’re using? Ultimately, interdisciplinary work is about communication. If you and I are on a research team, we need to be able to communicate to each other. But a different problem is, okay, we’ve done this project; we have these insights. Who are we going to communicate this to and how different venues will have different expectations of what the right language is and what the right technical detail is? And in a sense, you’re in this learning process all the time because you want to be able to communicate the findings, but not everyone who has a stake in the conversation will have the same nuanced language that you will you have to be able to convey the findings in a way that’s interesting or relevant to someone who’s coming up as fresh. And so a lot of the interdisciplinary work is just can I find the right thing to say to get you to understand why you should care about this very, very specific problem that I have?

Sarah Webb  40:18

I’ve been doing this series on creativity and computing, you’re talking about design, you’re right here in the thick of this. So I want to know, Rogelio,

Rogelio Cardona-Rivera  40:29

 Yes,

Sarah Webb  40:29

 what creativity means to you.

Rogelio Cardona-Rivera  40:32

Oh, Sarah, this is an entire different episode, I will say that creativity is impossible to disconnect from its assessment. As in, I can deem something very creative, but if it is not judged, as creative by someone else, then is it really creative? I don’t think I can say, a definitional term because I think that it’s impossible to get away from the social component. So fully recognizing that I hate this definition, as I utter it, creativity is what is deemed as creative. If I wanted to get more specifics, I think that I’d be willing to say that something that is creative, maybe defies expectation in some way. And that expectation can come from many places, it could be your own expectation, but it could also just be an audience thing of what is the standard to do something. And so anything that deviates from the standard, could be considered creative, right? You just have a default expectation of like, oh, yeah, this is what it is. And when you deviate from that, it kind of calls your attention. And you’re like, Oh, I didn’t expect that. But the assessment of creativity, I think, is ultimately socially negotiated and constructed. That’s kind of where my head goes.

Sarah Webb  41:49

But you did hit on that novelty part of it. That idea of unexpected.

Rogelio Cardona-Rivera  41:53

So this is interesting. If you are equating novelty and expectation, can something be novel without it being unexpected. Or vice versa. Could something be unexpected without being novel?

Sarah Webb  42:05

I guess, I guess it could probably be unexpected without being novel. I don’t know. I mean,

Rogelio Cardona-Rivera  42:10

I don’t know. This is the language discussion of we throw around these terms. And not to say that you are throwing them around irresponsibly or casually. But I have found a helpful tool in my toolbox is just asking the question, What do you mean by that? Because I find that it exposes a lot of latent or tacit assumptions about the phenomenon that you’re talking about. One thing I’ll say about novelty and specific something could not be novel. But you could deem it unexpected because of what you know of someone. Right? It’s not that I didn’t expect this in general. I didn’t expect this from you.

Sarah Webb  42:43

Yes, definitely. Right. Absolutely. Absolutely.

Rogelio Cardona-Rivera  42:46

And so to the social component of like, there’s something in the assessment of creativity, or novelty, or expectation, or whatever you want to call it, that is inherently socially constructed and negotiated. And I think it’s interesting to ask these questions, not to say that just because it’s socially constructed. I don’t want to just leave it at that. I think it’s interesting to dissect that, like, why is it that.

Sarah Webb  43:05

It’s interesting, right, but you’re narrowing down on language because I realized, as I said that wait a minute.

Rogelio Cardona-Rivera  43:10

I have learned to anthropomorphize a piece of software, like a software language compiler. Okay. What did you say? What did I say?

Sarah Webb  43:16

We have talked about so many things, we could talk about so many more?

Rogelio Cardona-Rivera  43:21

Oh, yes, absolutely.

Sarah Webb  43:23

Is there something that we missed you think is important to mention before we sign off?

Rogelio Cardona-Rivera  43:29

Oh, one thing I’ve encountered in the work that I do, is someone saying something akin to? Well, you’re just trying to like codify culture. And how could you possibly codify this in the software, the things that contribute to storytelling are the things that contribute to what we find entertaining or engaging in games? And I think that that’s an interesting critique. If you were to take that literally, then yeah, it’s always a moving target. But I think that there are enough foundational things that we can point to that we can start making this practice a little bit more systematic. By practice, I mean, like the practice of narrative or game design, like an understanding at an elemental level, these small things when orchestrated together will produce this effect. The thing that I rely on is, again, drawing from like the natural sciences, F equals MA, Newton’s Second Law, is an invariant relationship that exists between you know, force, mass, acceleration, and, in essence, by manipulating one of the variables or one of the degrees of freedom, you can get the other two. And what I’m hoping to do is make enough progress to find the invariance of game design. So if you were to think of what’s the equivalent of F equals MA for game design, you might think, well, the output that I want to hit is some effect on people. And the inputs are, what is that person’s context? Or who are they contextually like, are they playing at home? Are the playing on their phone? And what are they engaging with? What are the structures that are present might let us get toward understanding people? [Theme music begins in background.] And I think that we can make enough progress along those lines to eventually establish the equivalent invariance of game design, which is what I, I hope my research does as a major contribution to games.

Sarah Webb  45:20

On that note, Rogelio, thank you. This has been so much fun. I really appreciate it.

Rogelio Cardona-Rivera  45:26

Thank you. This was lovely. I hope that this isn’t the last time that we get to chat.

Sarah Webb  45:32

To learn more about Rogelio Cardona-Rivera and their research projects and about technical games research more broadly, check out our show notes on our new website at scienceinparallel.org.

Sarah Webb  45:46

Science in Parallel is produced by the Krell Institute and is a media project of the Department of Energy Computational Science Graduate Fellowship program. Any opinions expressed are those the speaker and not those of their employers, the Krell Institute, or the U.S. Department of Energy. Our music is by Steve O’Reilly. This episode was written, produced and edited by me, Sarah Webb.