Deep Fakes and the Next Generation of Influence Operations

Wednesday, November 14, 2018
Max Taylor
Robert Chesney

James Baker Chair in Law, University of Texas at Austin

Aviv Ovadya

Chief Technologist, Center for Social Media Responsibility, University of Michigan

Laura M. Rosenberger

Senior Fellow and Director, Alliance for Securing Democracy, German Marshall Fund of the United States

Guillermo S. Christensen

Partner, Brown Rudnick LLP

This panel identifies guidelines tech companies can follow to limit their negative use and offer views on how governments should react to deep fakes, if at all. 

CHRISTENSEN: Well, good morning, everyone, and welcome to our second panel of today. The topic is "Deep Fakes and the Next Generation of Influence Operations." I think with deep fakes we’re into some very new territory for the Council and we’ll talk a little bit about a lot of very interesting issues.

First, let me introduce the panel that we have. Next to me is Bobby Chesney. Among other things, he is the blogger and chief editor, I think, of Lawfare.

CHESNEY: Ben Wittes is chief editor. Me, just a lackey.

CHRISTENSEN: Just one of the lackeys. OK. Then Aviv Ovadya, who is formerly the chief technologist with the—

OVADYA: Center for Social Media.

CHRISTENSEN: —Center for Social (Media) Responsibility but is now the founder for Thoughtful—the Thoughtful Technology Project, and we might hear a little bit more about that. And then to my far right, Laura Rosenberg (sic; Rosenberger), who’s with the German Marshall Foundation (sic; Fund).

So a couple of reminders before we start. This is an on-the-record discussion. My name is Guillermo Christensen. I’m a partner with Brown Rudnick, a law firm. I was a former CIA fellow here at the Council on Foreign Relations so I’m very happy to always be back home, as it were.

We will have a panel discussion up here at the podium for about twenty, twenty-five minutes and then we will have an interaction with the members and I look forward to a lot of very good questions and, hopefully, some controversial debate on this topic.

So as I looked into some of the history of deep fakes and some of the background for today’s discussion, it was interesting to note that with the recent one-hundred-year anniversary of the end of World War I, we have actually a very interesting historical analogy and use of deep fakes and information warfare that happened shortly after the French surrendered to the Germans in World War II. There’s a very famous episode where Adolph Hitler was filmed coming out of the train car where he forced the French to sign the surrender and he was video—he was filmed at the time, obviously, taking a very odd step back.

The Canadian in charge of propaganda—that’s what we used to call it back then—for the Canadian government noticed this and figured out that he could manipulate that foot into one of those sort of moving GIFs that you may see on the internet and made it appear that Hitler was actually dancing a jig—a happy jig—with the surrender of France, and this was played on newsreels all over the world. Obviously, it didn’t change the fate of history. Everybody already didn’t think very highly of Mr. Hitler. (Laughter.) But it added a tone of, you know, pleasure at what he was doing to the French nation that helped the allies.

So this is not something—this is not a new development, in many ways—the manipulation of video images. But, clearly, with the advent of artificial intelligence and this new development of so-called deep fakes—and we’ll also talk about shallow fakes, I think, if I may use that term—there is something that’s changing and you may already have seen some evidence of that, any of you who have watched one of the more recent Star Wars episodes.

You will have been surprised to see Peter Cushing make an appearance as the emperor. Well, Peter Cushing passed away a number of years before that movie was made and the only reason it was possible to have him in there was because of the technology that we’re going to talk about today.

So, first, I’d like to ask Bobby to talk a little bit about the technology and why deep fakes benefit from artificial intelligence and what are they, at the most basic level.

CHESNEY: Thanks, Guillermo. Thanks to all of you all for being here.

So you’re right that there’s nothing new about fraud, forgery, information operations, and manipulation of evidence, right. So as long as we’ve had information and evidence, we’ve had manipulation designed to create the wrong impression for various purposes. So what’s new?

My co-author, Danielle Citron, and I argue that what’s new is the deployment of deep learning techniques, neural network methodologies, most especially something that is, again, a regenerative adversarial network where you have two algorithms, one of which is learning and creating, the other is detecting and debunking, and they train off against each other in a rapid evolutionary cycle.

This has produced some qualitatively significant leaps in terms of quality, difficulty of detection, and, we argue, capacity for diffusion. So let me say a few words about each of that. Quality wise, it’s not like we’ve never had, as Star Wars illustrates—it’s not like we haven’t had in recent years ability to create high-quality but not actually authentic depictions of people saying or doing things they’ve never said or did. But this particular methodology is one that will yield content that will pass the ears and eye test relatively easily compared to other existing forms of digital manipulation, combined with a capacity for eluding detection by more traditional detection means.

Now, it’s one thing—as we had recently with the Jim Acosta video, which I’m sure we’re all familiar with, the allegation that the video that was distributed of the incident—the White House press briefing—where it had been sped up, that’s easy to debunk insofar as you’ve got lots of well-established credible versions of the actual video and you can just compare and contrast. So we’re not really talking about a scenario like that where it’s easy to compare a known original.

I’m talking about scenarios where we don’t have a known original to just immediately compare and contrast to. Deep fakes, when really well done—when it’s a true deep fake, not a shallow fake or a cheap fake or whatever we might then want to call it—there is a difficulty of detection that’s different from what we’ve seen in the past.

And then the capacity for diffusion—it’s great for industrial light and magic to be able to produce with months and months or years and years of effort to reproduce Peter Cushing. To be able to do it much more quickly, that’s different in kind. If you can produce something that passes the ears and eyes test that is extremely difficult to detect unless you’ve got equivalently high-end algorithms for detection, and maybe not even then, and it’s something that can diffuse and that the expertise needed to produce it or access to that technology will diffuse in various ways, whether it’s through the offering of commercial services—and there are a bunch of companies emerging in this space, companies like—to give one example, a company called Lyrebird, L-Y-R-E-B-I-R-D—commercial services, dark web services, just as you can go on the dark web and you don’t have to know how to construct a botnet or to deploy a DDoS attack but you can buy that as a service—you can rent that as a service. It will not be long before you can get high-quality deep fakes as a service even if you yourself don’t have the capacity to generate it.

Our argument is that this is going to be a further contribution to our current problems of disinformation but it’s a qualitatively different one and we think it has big implications.

CHRISTENSEN: Effectively, today, if you have a iPhone you sort of have the ability to create a—again, not a deep fake but a shallow fake, right. You can do these wonderful little face emojis—you may have seen them—where you record your face, you speak, and it looks like you are some kind of a cartoon character. So that technology is already on an iPhone. How soon before we get to the point where this technology is available—not the shallow but the deep fake—where the ability to manipulate a person’s visage, create new video, is available on someone’s MacBook or laptop and they can do that in not three months like it might take for the Star Wars creators but maybe a couple of hours for a propagandist sitting in the Ukraine?

CHESNEY: Well, I won’t offer predictions on how long it’ll be before we have apps galore that really provide true deep fake-level qualities here. But for cheap or lesser forms already there are all sorts of apps that are designed to help you with makeup, hair style, fashion. My kids have shown me many of these. It’s not the sort of thing that will persuade at the high level but this underscores a point. We, obviously, live in an environment in which manipulation of information including imagery, audio, and video already is quite possible. Augmented reality techniques of all sorts will contribute to that and put it at our fingertips.

To emphasize the deep fake piece of the problem in a way is to just shine a spotlight on the tip of the pyramid of persuasiveness where you could potentially do the most damage insofar as you’re talking about frauds or disinformation attacks that are meant to persuade even in the face of efforts to debunk. That doesn’t mean to disparage the importance of the rest of—the big part of the pyramid where the cheap or shallow fakes are that, as we’ve seen many examples of, may be relatively easy to debunk but they’re still going to cause harm.

CHRISTENSEN: Yeah. Aviv, you’ve written recently that disinformation from state and nonstate actors is just one component of this—I think you labeled it deteriorating information environment globally—and then you’ve also suggested that AI, so not just deep fakes but AI, is likely to accelerate failures in our public sphere in this area. Could you talk us a little bit through what you see and some of the dynamics and the direction that you see this taking?

OVADYA: Yeah, and just to sort of be a little bit more broad here, I guess part of that included things like deep fakes and, really, this sort of broader class of synthetic media which is, you know, you’ve got your ability to manipulate video, the ability to manipulate audio, ability to make it seem as if you’re talking to any—to a human when you’re not talking to a human—like, that entire suite of technologies where you’re—it’s sort of like that dream of that personal assistant who could do anything you want for you, but then what happens when you have that technology?

There’s all these side effects because that magical sort of science fiction realization can be applied to manipulate discourse more broadly and I think that that’s sort of what I’m trying to get at when I talk about the impacts of these advances and gains are definitely a core component of that on our information ecosystem. I think there was a second part to your question.

CHRISTENSEN: And then in terms of the risks that AI creates in this context, how would you break those out?

OVADYA: Yeah. So I think about this as sort of creation, like, how does AI affect what we can—what types of media we can create; how does it affect how we distribute media and how does it affect our consumption or belief of media. And so on creation, obviously, there’s deep fakes and then there’s those other sort of media, you know, whether it’s text or audio that I just mentioned.

In distribution, you have the—one second—you’ve got both your ability to change how content is amplified, and in particular the way the recommendation engines suggest particularly the content and/or particular events or particular people to being paying attention to across different platforms. And you also have your ability to create or to coordinate inauthentic actors and then in terms of consumption you have the ways in which you can target that.

I mean, artificial intelligence is what you use to say, oh, you’re like that person—we could use this technique, or we should show you this piece of media as opposed to that piece of media. And then there’s, similarly, you can use artificial intelligence to say, oh, here’s a piece of content that will be more persuasive or here’s—we can generate content that is more persuasive in these particular ways for that particular person. And so that’s a way that they can impact belief.

CHRISTENSEN: Laura, we’re, obviously, going to spend a lot of time focusing on the U.S. in this discussion. We seem to be in the middle of these issues in ways that so far most are still a few steps behind. But the technology is not U.S. only by any means. Artificial intelligence is, arguably, a higher state-sponsored priority in China, for example, than in the United States. What kind of dynamics are you seeing out in the Far East in China both with AI as far as disinformation and to the extent you’ve seen it in the deep fake world? If you could touch on that as well.

ROSENBERGER: Yeah, absolutely. And let me just spend one second first just to pick up on a couple threads and then I’ll move to the China and AI question. I think one of the reasons that there’s been so much attention on deep fakes is because, you know, we’re trained to basically believe what we see, right, and so when we think about manipulated video and audio content, especially when it becomes undetectable to the human eye, it could pose real challenges for taking sort of the erosion of truth to a whole different level.

What I would say on that, though, is I would not be so dismissive of the shallow fakes question in the sense that even while the Jim Acosta sped-up video was very quickly debunked, side by side, I bet if you survey the U.S. population there is a large segment of the United States that believes the altered version of the video and that’s in part because, number one, lies travel much faster than truth and so people who see altered content are actually quite unlikely to see debunking of that content, number one. And number two, with the degree of polarization we have in the United States right now, we are very predisposed to believe what we want to see and I think that that means that we need to be very worried about deep fakes but I also think that the problem is here now much more than we necessarily realize it and I think we need to sort of cast it in that light.

On the China question and what others are doing, and when I think about sort of the next generation of information operations, I think it’s really important that we not just think about information operations in the way that it has manifested from the Russian-driven model. That is one model. But when we look at how China and the Chinese Communist Party has been able to use technology—the online space—to exert information control and manipulation over its population and then we look at how AI is going to super charge that through increasingly effective and efficient surveillance and control, it poses a real challenge for democratic values and norms.

Now, right now we see this being beta tested within China’s borders and, most notoriously, in Xinjiang, where we have the combination of facial recognition technology with voice recognition technology combined with, you know, QR codes that are being placed on Uighur homes that is basically being set up to completely monitor the entire Uighur population of Xinjiang to immediately root out any dissent, completely control the information space by denial, right. It’s not by necessarily spreading some kind of false or manipulated content. It’s information operation by denial of speech in the first place and by control of that speech.

We also, of course, see in China the creation of the social credit score which, again, in its eventual version will be AI enabled. Then you combine that with the Great Firewall and what China has been able to do in terms of censorship and control. And by the way, China is starting to push out the borders of that censorship on its own platforms where they’re used outside of its borders.

Then it’s exporting digital technology to an increasing number of countries, essentially laying what I see as the digital track of information, control, and manipulation in the future. So you don’t even necessarily have to create falsehoods or false narratives or manipulated content. And by the way, the Chinese are decent at that, right. I mean, they’ve got, you know, the 50 Cent club that, you know, is basically paid—it’s humans, right—that’s paid to write commentary online that is solicitous of the Chinese Communist Party. Once you can enable that with AI and automate the whole thing, you can, again, take that to a whole different level.

Now, again, this is largely confined within the borders of China at the moment. But it certainly appears, like, in a very kind of futuristic Black Mirror sort of way that the Chinese Communist Party is basically preparing an authoritarian toolkit in a high-tech toolbox that could be exported, and I know I’m painting an alarmist picture and I’m doing that deliberately so because I do think that we need to bust out of some of what has become how we think about disinformation and understand that information operations can manifest in a wholly different way that seriously undercut both our values, norms, and interests around the world.

CHRISTENSEN: Thank you. And so if we take the U.S. approach to the extent that there is one, China—the other one that’s sort of in the middle—maybe not in the middle but out there between those two—would be the European approach, especially when you’re talking about data privacy, the use of personal information. Europe is taking a different tack than both the United States and China.

Open it to all of you. Where does the European experiment play in the use of AI and the impact of AI on information, information warfare, and disinformation and how does that—to the extent that it’s different than the U.S. and Chinese approaches? If you could comment on that.

ROSENBERGER: I’m happy to take a first crack. I mean, we don’t see significant AI development out of Europe, right. I mean, Google’s DeepMind is actually based out of the U.K. But we don’t actually see a whole lot of other European-based development of AI. That may in part—I mean, or really, you know, in the broader tech space, and there’s a lot of reasons for that.

But to kind of pull the strand back I think to get to the crux of your question, you know, AI feeds off data, right. That’s how it learns. That’s how we train it. That’s how we make it good. And the European approach by, you know, putting a pretty heavy regulatory framework around data and including things like the right to be forgotten, can constrain the use of data for AI training.

You know, in China this is not an issue. You’ve got 1.4 billion people who have no choice but to freely admit their data not only to Chinese tech companies but those are, you know, very closely intertwined and there’s a lot of good work documenting how that data flows to the Chinese Communist Party and its constituent surveillance elements.

In the United States, we haven’t set up a regulatory framework around data yet. I personally am a strong advocate of doing so. I think thinking about data privacy and data protection is a critically important question, especially when we come to dealing with broader questions of information manipulation online.

But there’s a quandary here because if we do that in a way that inhibits data flow to AI development, we’re then setting ourselves up to lose the AI arms race with China. And so I think this is a conversation that has not yet taken the kind of policy debate shape that we really need. I mean, this is a core, core question, I think, for policy makers when it relates to technology and when it relates to questions of privacy, and I don’t see that happening in any robust fashion and I think, frankly, we’re late to that game.

CHESNEY: I’ll add to that that we have this federalism system, this decentralized system, in which the federal government may not be playing the game but some states can and, you know, famously, California, as you know, has decided to enter that space in GDPR like fashion. So right now we have an interesting sort of hybrid in the U.S. where we have some elements that are moving forward with the "protect the data, maximize privacy" model. We have other elements that are not that at all, and it’s not really clear where we’re going to land.

I mean, it does seem like what you’ve got is an emerging set of blocs with Europe taking the individual privacy protection model to the maximum extent, China, obviously, going the complete opposite direction, and the United States sort of traveling this middle path, and one of the great questions over the next ten years is what will this mean—are we going to really see this Balkanization of the information environments where you can choose the American model which may become sort of the Anglo-American model with Brexit, the continental European model, and then the Chinese model, and there will be competition elements of that especially between the United States and China that Europe’s not really going to play with. They’re going to be on the consumer side, not the generator side.

ROSENBERGER: Right. That’s right.

OVADYA: There’s also the challenge of Chinese apps getting a foothold in the U.S. market, so right now TikTok is blowing up and it’s a Chinese-based company, and where is that data going, or companies like Grindr that were bought, which is basically the best information you can use if you want to extort someone, unfortunately. (Laughter.) And so there’s a lot of challenges there.

CHESNEY: You know, I would add to that the genetic 23andMe—


CHESNEY: —that sort of—that space. The amount of genetic data that we are putting into the hands of companies that actually in many instances now are owned by Chinese entities is really remarkable.

CHRISTENSEN: Yeah. One can imagine a future—maybe we don’t want to—but in twenty or thirty years where that DNA data is really used to make some seriously deep fakes, right, with the—(laughter)—so but, hopefully, well after my time. (Laughter.)

CHESNEY: Don’t count on it.

CHRISTENSEN: Don’t count on it. True.

Laura, you mentioned the fact that the United States and China, largely speaking, as blocs are leading the way on AI at different paces. Everybody else seems to be fairly far behind. When we look at the possibilities for technology to assist us in countering deep fakes, for example, and the possibilities for technological solutions to the challenge that these developments will bring to our dialogue, Bobby, are there technologies out there? Are there ways in which we can put up some walls or try to protect ourselves, things like—in the copyright space there’s watermarking. But is there—are there technologies that are being developed side by side?

CHESNEY: Absolutely. Right. So there are many technologies that are relevant for this conversation. When Danielle and I have spoken about the deep fakes question, the first thing people tend to say is, well, if technology is the cause of this problem isn’t it also probably the solution and, certainly, it’ll be part of the solution. Two different technologies that are really relevant for shallow or deep fakes, one is detection technologies and we could talk about the—the general consensus is that it’s an arms race.

The capacity to detect is growing as well and you occasionally will see stories—there’s been a flurry of them in recent years—saying ah, well, deep fakes—not so good because it turns out people don’t blink in the videos they create. Well, that’s super easy to fix, and as soon as people pointed that out all the databases that were feeding the generators were adapted to make sure you included normal human blinking in the—in the feed. And now the newly-generated deep fakes include blinking. Or if it’s the subtle skin colorations caused by blood circulation—whatever it is—once it’s known what the tell is, the databases can be modified to account for that. So there’s that arms race on the detection side.

Much more interesting is the digital provenance issue set. And I was looking through the list of attendees; I don’t know if Tara from Truepic is here. Yeah. So Truepic is an—sorry. Uh-oh. (Laughter.)

CHRISTENSEN (?): What just happened? (Laughter.)

CHESNEY: Truepic is doing really cool work. It’s one of several companies in the digital provenance space and what this means is coming at the problem from the supply side, trying to increase the extent to which there’s authentication-based technologies—watermarks, if you will—built right into the content at the point of capture, and it’s really cool when it works.

The dilemma from a policy perspective—the reason this isn’t a panacea yet and might be hard to get to is if this is really going to work to minimize fraudulent video, audio, and photography, you need ubiquitous uptake, digital provenance, watermarking in all the capture devices. You got to get it in iPhones. You got to get it in pixels. You got to get it in everything that’s got a microphone or a lens. That’s going to be an uphill battle. And then you need that also to be ubiquitously adopted by at least most platforms, major and minor, as at least a filter where they’ll flag content, alert users to the lack of authentication of that kind, if they do allow it, or if we really want to go there I suppose you could make it a necessary condition for upload in the first place.

There are a million reasons why this may not be a situation we’re going to get to any time in the next many years. So we shouldn’t view it as a panacea. It’s probably part of the solution and, for my part, if we’re going to go the digital provenance is the answer route, I hope that what the platforms do is not to bar uploading of content that lacks it but simply some sort of user-accessible flagging so you can take it into account. Maybe it affects the algorithm of how things get circulated as well.

CHRISTENSEN: Yeah. Before we get to the final question on what other regulatory or government initiatives could address the problems of deep fakes, are there positive scenarios we can imagine for the use of deep fakes? I mean, one that I can imagine myself is representing, in the educational setting, characters for children to view historical developments that were never filmed as a way to bring them into the story better. But short of that, I struggle outside of the Hollywood context to see what are the positive scenarios for the use of these and my follow-on question to that would be if not, should there be more of a regulatory approach to keeping them out of the information flow that we all live in.

OVADYA: So I’ve got answers to both of those, at least. So positives—well, many of you take video calls. Sometimes you don’t always look your best. What if you always looked your best no matter what you were wearing or not wearing? So there is that. (Laughter.) There is that. But there is also what if you wanted to speak to someone in their own language with your voice and your intonation translated perfectly, right? Those are—those are positives. What if you want to have—actually have that seamless interaction with that science fiction-level virtual assistant? So there is—there is this sort of—there are—there’s not nonzero benefits to these technologies and to the things that come out of them.

On the regulatory side, can we actually put this genie back in the bottle? I don’t think there is an easy way to create regulation that says you cannot use a particular type of technology. You can regulate a platform where there’s centralized control. But when it comes to a base technology it’s like saying you cannot, you know, broadcast things on radio waves. It becomes—it becomes a lot more challenging, especially when everyone actually has radios anyway. You all have a computer. You all have a million computers, and you can’t—we can’t just say you can only do some things on a computer without completely abrogating our personal sovereignty over our devices, which that’s a much longer conversation.

ROSENBERGER: Yeah. I would just say as a fundamental principle truth is essential for democracy. Truth is essential for a functioning democracy, and so the more we enable not only sort of manipulation of information but that we create scenarios in which truth does not seem to exist anymore, I think we’re doing real fundamental damage to the underpinnings of our democracy. So that’s sort of a fundamental principle for me.

However, I also believe deeply that free speech is fundamental in our democracy and that includes falsehoods, right. Whether we like it or not, that includes opinions we don’t like. And so that’s why I’m—in part why I’m skeptical of regulatory approaches to this particular piece of the puzzle and I also agree with the view that I don’t know that this genie can be put back in the bottle or that you can say certain kinds of technology are out of bounds.

Now, I do think that there are ways—you know, as we think about broader questions of regulation around technology in general and the obligations of technology companies, I do think that there are real questions there that can be, you know, at a more basic level, constructed, even if we think about just the basic premise of Section 230 of the Communications Decency Act.

You know, Senator Wyden talks about how, you know, that law was always intended as having both a sword and a shield, right. It was to shield the companies from liability but also to give them a sword to be able to police what happens on their platforms. Now, these platforms have taken on a role that was probably never envisioned by Section 230, right. Section 230 certainly enabled their creation in the first place but now I don’t think it’s exactly, you know, anything like this could have been envisioned back in 1996. But I do think that there is a space there for companies to be thinking about this and for the broader regulatory discussion to continue to enable that.

The last piece I would say, though, is when it comes to regulating any particular kind of technology, technology is always going to move faster than law and regulation, and the minute we start trying to regulate or pass legislation about particular pieces of technology we’re probably going to be, like, fighting the last battle and we’re going to already be on the next war and, you know, it’s going to be a losing race. So that’s another reason why I’m skeptical of that approach for this.

CHESNEY: (She asked ?) about the beneficial uses. When Danielle and I wrote our paper—which, by the way, if you’re interested in reading, like, the longest, more-detailed-than-you-want paper about this, Google “Chesney deep fakes” and you’ll get a copy of it—we had a section where we felt was important to list the beneficial uses, and we kind of struggled. We mentioned education, art, and so forth. Aviv has definitely given me hope that I can stop having to attend any VTC or other meeting and have my virtual me show up and say smart things and look good at the same time. So that’s clearly—(inaudible). (Laughter.)

A different one that we learned about, I mentioned the company Lyrebird. In the original draft of our paper we had some criticism of this company because we thought some of the claims they made on their website about their corporate responsibility struck us in kind of a negative way, and then I felt so bad about it later when I learned about one particular kind of pro bono public activity they engaged in. They’re using their algorithm-generated voice simulation technology to restore and give original voice to ALS—Lou Gehrig’s Disease—victims who’ve lost their voice and that—you know, a dear friend of mine died from that not that long ago and I just was really struck by, wow, that’s a use of avatar technology that is so powerful.

In terms of other responses, I mean, it’s completely correct that the legal and regulatory solutions to the general problem of fraudulent imagery or audio or photography are—it sounds good in theory but you have difficulties of attribution and finding the original actor. You’re much more likely to have effect if you put pressure on the platforms themselves. But the platforms would then quite rightly say, OK, but you better define it really specifically what it is we’re not allowed to or not supposed to circulate and you better define it in a way that’s reasonably possible for us to scale up the filtering and detection and removal of that—go. That’s a really hard problem.

We know that from the hate speech context. We know that from the—as the first panel said, the terrorism speech or the incitement speech context. This is a general problem the platforms are struggling with and the first panel I think nicely laid out all the practical difficulties of scaling up to perform that function. And it ultimately comes around to a question of will they on their own as a matter of political and market pressures or will they, because they get pushed to it by our government or others, develop a sufficiently robust process and a sufficiently transparent, by the way, process—you want to have some idea how this works—to decide what content gets pulled and what doesn’t in an environment in which people don’t agree about what sort of frauds cross that line. What’s political satire? What’s a bit of lighting manipulation? You know, think about how many political commercials you’ve seen where the candidate for the other side is depicted always off kilter, grainy, gray, voice sounds like it’s an echo chamber. Is that a deep fake or is that a shallow fake or is that just fair bounds of sharp-elbowed political content? I don’t know that we’re going to actually have solutions to that.

OVADYA: If I can add to that for a second. Yeah. So I don’t want to imply that I don’t think regulation doesn’t have a part to play in these broader set of issues and even in the issue of deep fakes and I think that—or, like, we call synthetic media, more broadly. I think that the frame here is sort of around the cost dynamics to some extent. What will make it harder to spread something that is a weaponizable fake? What is something that will make it cheaper to show—like, to spread something that’s actually real and true?

And so that’s the underlying dynamics and you can do it on both sides and you can have regulation, like, in a variety of different areas—things that affected the creation of content, the distribution, the ways in which people form beliefs and the ways in which it impacts our actual specific institutions whether they be courts or, you know, journalism. All of these have different places where they’re vulnerable and where they can—they can become stronger.

And I also want to just touch on the Lyrebird. So they actually provide a good example of this where one of the things that I admire about them is that they require you—if you wanted to—right now if you wanted to make a copy of a voice, they give you some particular sequence of words and you need to repeat that sequence of words in order to train the model. But that also provides a verification that you, the human whose voice is being used, was the human that, like, essentially gave consent to their voice being used, right. If I give a random string of words and say, you know, generate a voice that can then be used to say anything else, that provides a degree of difficulty for copying that voice where you would have to, like, somehow find those particular words, resplice them together in order to train this model otherwise. And so it creates a barrier to entry. It decreases—it increases the cost to creating that fake voice model which can then be used to say anything.

CHRISTENSEN: And that’s—

OVADYA: And so that’s an example of something a—a regulation you might advocate for if you’re creating technology that allows this.

CHRISTENSEN: And that’s one reason why you should never say my voice is my password ever again, as you see in the movies. Doesn’t work. Too easy to copy.

CHESNEY: Verify me.

CHRISTENSEN: So we’ve now reached the—I think which is the highlight of any Council session, which is when the members get to throw questions at our panel. I hope you’ll ask some hard questions. As always, please stand up, identify yourself, ask a concise question, if you will, please, and other than that, I will open up the floor for the members. And if I may, since we already put you on the spot, if you have a question then, please, let’s go.

Q: Thanks so much for the shout out. My boss is really going to like that. So my name Tara Vassefi. I just started with Truepic last week.

And my background is actually—(laughter)—my background is actually as a lawyer I was working on basically collecting, verifying, and archiving digital evidence for human rights cases and war crimes prosecutions, and, obviously, that process of verification is very robust and there’s a lot that goes into it and I’m having trouble kind of shifting away from, you know, the hours that goes into, you know, making evidence meet a(n) evidentiary standard for a criminal prosecution to, you know, content that’s just kind of captured by your phone and user generated.

And it think there is a concerning conflation between, yeah, authentication and verification. There’s a lot of—there’s a notion that if something is authenticated then what the thing is saying is true. And so I’m trying to figure out what’s the best way to apply some of that, you know, high evidentiary standard of verification for user-generated content and distinguish that, yes, if we can prove that this image was taken at a certain place at a certain time and, you know, barometric pressures and all these other things—these cool things that Truepic does—then how do we kind of translate that into OK, well, what is the person saying and what is actually verifiable in the content or the behavior that’s manifesting. So I’d love to hear your thoughts on that.

CHRISTENSEN: So does AI have a role to play in this? I think it does. But—

OVADYA: Well, I guess I’m not one hundred percent sure I understand the question here. You’re asking—and if someone else believes they do feel free to chime in—but you’re asking if—like, I guess, the authentication in this case being—meaning this is the person, this is the phone, this is the location—verification meaning this event happened. Is that—that’s the distinction that you’re trying to—

Q: The content of what it is that is being authenticated. So, like, is there a crime being committed or—

OVADYA: Right.

Q: —is the person who is saying these things and we know that they’re saying it what—if they’re—what they’re saying is actually true, does AI have a role or, you know—

OVADYA: I see.

Q: —can we use some of this technology not just to prove that the thing hasn’t been manipulated but also to show what we really want to get out of why we’re viewing these things, which is the truth behind what’s being captured.

CHRISTENSEN: So fact checking, in the modern parlance.

OVADYA: Right. Yeah. I mean, I think that AI is far, far away from that in our current—you know, it’s hard enough to do content moderation when it’s necessary at the scale of billions of people. When you’re talking about verification of this form, my guess is that you definitely, especially in a human rights context, you want humans involved in that for the foreseeable future and I think that the first places that we’ll see this improving, at least having a little bit, is going to be in the content moderation space.

CHRISTENSEN: Let’s see if we can get another question. The gentleman in the back.

Q: Bob Boorstin, Albright Stonebridge.

None of you mentioned India. You’ve said U.S. You’ve said Europe. You’ve said China. India is trying, and I’ll put emphasis on that word, to develop a third way when it comes to information. Why didn’t you mention it?

CHRISTENSEN: I’ll take that as my responsibility but why—let’s talk about India. What do we see in the Indian context and, for that matter, are there other players—Israel, I know, has a very vibrant industry around AI and these things—are there other players that are coming out with a different approach besides the ones we’ve talked about that we haven’t mentioned?

CHESNEY: I’ll just say I’m not familiar with the details on India so I don’t talk about it, out of ignorance. But my main concern when I began writing in this area was for what it means for the United States and what does it mean for the U.S. government, our lawmakers, our regulators, our intelligence community, our military, and local authorities. So it was intentionally framed as an America-specific project, in my case.

OVADYA: I guess and I’ll add that India is very interesting in that it is, you know, the most populous democracy and it’s—there’s still a large nonliterate population in India. And so when you have YouTube being a major player in India and many people using it not being able to read or write, the implications of this sort of technology become even more staggering and just the ability to sort of fact check, you know, when you cannot read or write in that context.

So I think that’s one of the interesting wrinkles that it provides in addition to, as you may have heard, some of the—some of the ways in which mass violence has been linked to social media or messaging within that context, and India is exploring some interesting regulatory options. But I’m not as familiar with the details there.

ROSENBERGER: Yeah. Similarly, I’m not as familiar so I’ll definitely have to look into it. But there are, of course, many other players in this space. You know, you mentioned Israel. Of course, Russia as well. I mean, Russia is far behind but Putin has made similar comments to Xi Jinping: to paraphrase, in short, whoever masters AI will rule the world. I mean, that’s essentially what they have both said in different verbiage.

Now, again, Russia is far behind. But there’s actually—you know, technology is increasingly an area where there are signs of cooperation between Beijing and Moscow. So there’s been a lot of attention to the recent joint military exercises that occurred between China and Russia. But the same day of those exercises there was a major deal inked between Chinese and Russian tech firms that will allow for things like data transfer and other kinds of tech sharing and collaboration. So I do think that there’s a number of other countries in this space that are important to be watching both on the tech development side and on the—you know, how they’re approaching the data questions.

CHRISTENSEN: Please. You have a mic coming up there. Yeah.

Q: Thanks. Kim Dozier, contributor with The Daily Beast and CNN.

So I’m picturing a nightmare scenario, because that’s what reporters do, where a government agency in the Middle East—government news agency—releases a piece of video purporting to show the American president calling for turning a certain country into glass. Now, I, as a reporter trying to decide whether to run that video or not, we do all sorts of things right now where we get stuff that we see on social media and we try to cross-reference it and make sure it’s true.

So, say, I, here, would run it on The Daily Beast site but it’s running across the Middle East. U.S. intelligence comes out and says, it’s not real and here’s how. But, of course, no one in the Middle East believes U.S. intelligence. So what body right now, either at the U.N. or some of the cyber cooperation bodies that we’ve created—who has the international sort of wasta influence to be believed?

CHESNEY: I don’t think anyone does. I think that if you look at parallel scenarios such as the DPRK attack on Sony—the cyberattack on Sony Pictures Entertainment—the U.S. government comes forward with an attribution—a completely correct attribution, and so many people didn’t buy it and so many people critiqued it.

So as your question suggested, if it comes from us and it favors us, then it’s not going to be credited if it’s an internationally salient issue like that. I certainly don’t see much in the way of international bodies that are, A, likely to have credibility and, B, likely to generate consensus on the issue. Something like a presidential statement may have—like the Acosta episode, have the virtue of so many other cameras and microphones on the scene that it is at least easy to, for what it’s worth—and I completely agree with Laura’s point that the truth—the debunking never catches up with the original lie—but you can at least close that gap and create the smaller impact.

It’s situations like, oh, I don’t know, the Helsinki Summit where nobody’s in the room where it happened and we don’t know what was said, and if someone comes out with a highly—high-quality audio that appears to indeed be the president’s voice making promises to Putin that, you know, we are not going to do X if you do the following in the Baltics, that could have real repercussions that could cause real things to happen and it’d be hard to debunk.

ROSENBERGER: Yeah, I agree with Bobby that I don’t think that thing exists right now. There are some interesting proposals that are being batted around, particularly led by the private sector, so Microsoft in particular. Brad Smith has been very engaged on thinking through whether there are some private sector collaborative functions that can be brought together both in terms of attribution, sharing signatures, obviously, which already happens but in terms of joint attribution largely on the cyber side. But I think they’re also thinking about the possible info ops applications of this.

There’s been some thinking—and I’m blanking at the moment on who authored this paper—on even looking to some kind of independent body that could be stood up along the lines of—they use the IAEA as a model. I think it’s an imperfect parallel. But something like that that could create some kind of verification and attribution function. Again, I think these are, largely, aspirational. I think they will require an enormous amount of political will.

I think you’re always going to have the problem of even if you have some independent commission that’s backed by, you know, very powerful private sector entities with a lot of resources behind them, you know, you’re going to have the problem of nation states that may have an interest in casting doubts on some of these questions and so then you’re still always going to have this tension there.

And so I’m not sure it’s perfect but I do think there are some interesting proposals out there that are worth thinking about and, frankly, even if I think some of them are not super practical I’m really glad that people are thinking about these issues and I think we need to be doing some more of that.

CHRISTENSEN: And the reality is is that by the time the response comes out several embassies will be turned into dust or some god-awful other disaster will have happened. So—

CHESNEY: Or the vote will have happened.

CHRISTENSEN: Or the vote will have happened. Right. So, let’s see—the gentleman over here, please.

Q: Thank you. My name is Nathan Fleischaker. I’m with the DOD.

So we spend a lot of time talking about the technical ability to make realistic videos or audio. Is there much AI being used to figure out how to—what the video should say or the audio should say? So, like, looking at how do we say—how does somebody say something that’s going to incite the most violent or incite the most rifts. So it seems like there’s a make something realistic, but also what does that realistic thing accomplish. And how do we figure out what—how do we or how does the adversary figure out what that thing should be saying?

CHRISTENSEN: Is that in terms of persuasiveness? Like, what would work the best with that culture or that language group, that location?

Q: Correct. Yeah. So if this is about influence or influence operations how do we make things that are most influential. So one part is making sure it’s realistic but then what does that realistic thing actually say to have the most influence or to do whatever the objective is.

CHRISTENSEN: Yes. In a way, does AI—can AI be used to help shape someone’s message so that it’s most impactful.

OVADYA: Yeah. I mean, I think one of the pioneers in this space is Pixar where, you know, they put a lot of people in a room and, basically, track their emotions and, like, tweak that until it really, like, hits as hard as it can, right. And right now on your iPhone you’ve got your, you know, complete 3-D scan of your face in real time and you have a computer there that can, you know, generate video in real time and you can imagine something that literally goes through that process for you, and so that’s a world that we’re entering. I haven’t seen this actually be weaponized in any meaningful way. I think there is a lot of work to be done to make that happen, which hasn’t been as well researched yet, which I think is probably a good thing. But that’s a world that we could be entering into.

CHESNEY: I think the world of marketing is very much in—all about micro targeting as a means to figure out which distribution channels will give you the highest leverage for your message. And so that’s not quite what you’re asking but I think that’s the place where AI currently provides the most bang for the buck—micro targeting.

OVADYA: Right. Yeah.

ROSENBERGER: I would just say, really quickly, just candidly, since you’re asking from the DOD perspective, I’ve been quite clear that I don’t believe the U.S. government should be engaged in these kinds of operations ourselves. So I just want to stipulate that up front because I think it’s very important. If we are trying to defend democracy, engaging in a race to the bottom of what is truth is not where I think we should be. So just to state that.

However, broadly speaking, I’m thinking from the adversary perspective in what they’re doing and trying to anticipate that. You know, the things that we see from disinformation, more broadly, I think very much apply similarly in the deep fake space. And so it’s things that hit on highly emotive issues. It’s things that speak to outrage. It’s things that speak to preconceived positions, right. So using micro targeting to identify what people may believe, reinforcing those beliefs, or just taking them one or two notches further is particularly effective and, then again, you know, basically seizing, exactly as the Russians have, on any divisive issue in our society is probably, you know, I think, broadly speaking, the categories that are most effective.

CHRISTENSEN: Up here. I think you’d be getting the counter U.S. government rebuttal at this point. (Laughter.)

ROSENBERGER: I’m fine with that.

Q: Glenn Gerstell, Department of Defense.

I’m not going to rebut that, but I do—Laura, would like to follow up on your comment before about—you commented that truth is, of course, the foundation stone for democracy. How do we deal with the fact that all the solutions that are proposed or all of the solutions I’ve heard that have been proposed to the challenges presented by AI involve in some way some curtailment on liberty?

In other words, in order to protect our democracy—I’m making this an exaggerated statement but just for purposes of getting the point across—in order to protect our democracy against the challenges posed by some of these new technologies we need more regulation, more curtailments of liberty, restrictions on anonymity, et cetera. I’m not advocating it. I’m just asking for the question of how do we deal with this paradox or irony. Thank you.

ROSENBERGER: I think it’s a really important question. You know, my own view is very much that we—in protecting democracy, we have to protect democracy and, in fact, strengthen it. And so I’m not in favor of steps that would intrude on First Amendment rights. I will go down swinging for my political opponents’ ability to have views that I disagree with. But I do think that there are steps. Number one, I don’t necessarily think that regulation alone equals curtailment of liberty. There are regulatory frameworks that actually in many ways can enhance freedoms and liberties and so I think it’s all about how we craft those.

Two is I think we’ve talked about this a lot in terms of content, which is how these conversations often go. But a lot of the most interesting approaches to rooting out the—particularly speaking from the sort of Russian disinformation angle, one of the most interesting approaches to dealing with that has absolutely nothing to do with the content that they’re pushing. It has to do with the origin and the underlying manipulative or corruptive behavior whether that’s coordinated inauthenticity, whether that’s covertness in terms of misrepresentation of who people are.

I would distinguish misrepresentation from anonymity. I think those are two different things. And so there are ways to think about this. Facebook talks about it as coordinated inauthentic behavior. I think that’s one frame. I think—you know, I’ll just make one very interesting example here is Facebook recently, just yesterday, announced more details of their most recent takedown that they conducted right before the midterms and one of the things they talked about that was in the content there was a whole bunch of content about celebrities.

Now, they removed that not because of anything about the content itself. The reason celebrity content was there—and we see this consistently in the Russian disinfo operations—is it’s building an audience, right. These operations are only effective once you have a following. How do you build a following? Well, you share interests with the people. You do that by hopping on trending topics. You do that by talking about celebrities or TV shows or things like that. It has nothing to do with the content. It has to do with what the intention is and the underlying behavior and the origin in that instance. And so I think the more we can think about it from that perspective and less about the content, then we don’t get into the same free speech quandaries that we’re talking about.

The last point I would just make is when it comes to, again, going back to Section 230 and the terms of service that the platforms have, I mean, they actually quite a bit of free rein under our current regulatory framework to enforce their terms of service, pretty much all of which prohibit this kind of coordinated inauthenticity and manipulation.

OVADYA: Oh, adding to that, so there’s this—the content approach. There’s a behavior approach and then there’s also sort of this underlying infrastructure that enables this stuff to happen, right. And so that’s the marketplaces that allow you to sell the activity. I think you alluded to that earlier. And, for me, one of the clear places where regulation might be able to jump in is saying if you are selling accounts at scale that is illegal. There is—I have yet to hear of a strong legitimate reason to be selling thousands of Facebook accounts or Twitter accounts. If someone can tell me that I’d love to hear it. Come up to me afterward. (Laughter.)

But those sorts of marketplaces, and that can be international. I mean, I think that that’s a place where you might even be able to get international agreement. So that’s, like, a very concrete infrastructure layer, behavioral—I mean, a regulatory approach. But there’s also—just going back to the deeper question of, like, democracy, liberty, freedom, I think that these are the same challenges that our Founders faced.

When you’re trying to lay out how do you balance these powers, how do you limit government, how do you limit, like, what are the tradeoffs that you’re making in order to sustain a democracy, we have to be cognizant that that is not a question where—the answer to that question is in light of the capabilities of the people, right.

When you’re trying to talk about what is the capacity or what should government be able to do, what should it not be able to do, what should people be able to do, what should they not be able to do, that’s in light of the powers that people have, and if I can create a megaphone that people can hear all over the world maybe it’s a little different than I can just talk to people directly around me, and there might be different properties around how that operates in that new regime.

CHESNEY: I just wanted to add that I very much am with Laura on the solutions here need to be in the nature of not suppressing speech, not the—sort of the European model of let’s make information go away but, rather, let’s have more information. And you’ll never close the gap entirely but more information is the better solution.

Now, part of that also leads to something I hear about a lot in these discussions, which is, well, shouldn’t we just be trying to educate ourselves—get people more sensitized to the risk of this sort of deception so they understand that their—our natural, indeed, hereditary inclination to trust our eyes and ears can trick us now in the digital world in ways you may not expect.

And that’s true up to a point. But I think one of the most easily missed aspects of this debate is if you do a lot of that—if you really pound the drum about deep fake capacity being out there—you’re going to open space for people to get away with things that actually were captured in legitimate audio and video. In the paper we call this the liar’s dividend, which is just a clunky way of saying that instead of crying fake news, people will cry, well, deep fake news even though there’s video of them saying or doing something that they shouldn’t have done.

CHRISTENSEN: Right. And on that last closing point, I think we, obviously, just touched the surface here of a fascinating topic. But it is time for the session to wrap up. I want to thank this fantastic panel for their time and great contributions and I invite you to join us for a coffee break before the third session begins at 11:15.

Thank you very much. Please join us. (Applause.)


Top Stories on CFR


During Kenya’s state visit, the United States should work toward building a more resilient model of U.S.-Africa partnerships.



Ebrahim Raisi was more loyal to hard-line Supreme Leader Ali Khamenei than previous presidents, and whoever succeeds him is likely to be just as conservative.