--- title: "Battling Deepfakes and Synthetic Voices in Voice Biometrics" url: "https://symnexconsulting.com/video/battling-deepfakes-and-synthetic-voices" date: "2023-05-25" author: "Matt Smallman, Haydar Talib" category: "Maximise Impact" description: "Explore the growing challenge of deepfake and synthetic voice attacks on Voice Biometrics systems and learn how to mitigate the threat." video: "https://stream.mux.com/4aYDQzHGj6H007HWbTaGrtTvkh1hsHkXUtyHdcnyeDSE.m3u8" --- # Battling Deepfakes and Synthetic Voices in Voice Biometrics _Explore the growing challenge of deepfake and synthetic voice attacks on Voice Biometrics systems and learn how to mitigate the threat._ In this insightful webinar, we explored the growing challenge of deepfake and synthetic voice attacks on Voice Biometrics systems. With the rapid advancements in artificial intelligence and machine learning, malicious actors are finding increasingly sophisticated ways to bypass security measures, making it crucial for organisations to stay ahead of these emerging threats. This session covered the following key topics: - Understanding deepfakes and synthetic voices: Learn about the technology behind these deceptive audio manipulations and their potential impact on voice biometrics security. - Identifying vulnerabilities: Discover the weak points in current voice biometrics systems that make them susceptible to deepfake and synthetic voice attacks. - Detection and prevention techniques: Explore state-of-the-art methods and best practices for detecting and mitigating deepfake and synthetic voice attacks on voice biometrics systems. - Building resilience: Learn how to develop robust voice biometrics strategies that can adapt to the evolving threat landscape and stay ahead of malicious actors. Watch the full session to equip you with the knowledge and tools to safeguard your Voice Biometrics systems from these emerging threats. ## Chapters - 0:47 Objective and Context - 2:15 Introducing Haydar Talib - 4:57 What are the origins of text-to-speech technology? - 7:24 How do modern text-to-speech systems create synthetic voices? - 11:10 What are the boundary conditions for synthetic speech to be a threat to Voice Biometrics? - 12:11 How good are the latest synthetic voices? - 15:22 Are they good enough to defeat Voice Biometrics? - 22:11 How much target audio is needed to create a synthetic or deep fake voice? - 25:31 Are they quick enough to be able to maintain a realistic conversation? - 27:43 How to mitigate the threat from deep fakes to Voice Biometrics systems - 30:26 How can presentation attack detection prevent deep fakes? - 33:49 How can watermark detection prevent synthetic voice attacks? - 40:34 How can other authentication methods reduce the threat from synthetic voices? - 46:44 Summarising the mitigations available to protect Voice Biometrics from synthetic speech and deep fake attacks - 53:36 Question: How to manage the impact of press articles on customer confidence? - 55:34 Question: How to respond to customer concerns about the threat to Voice Biometrics? ## Transcript [00:00:00] **Matt Smallman:** Hi. Good afternoon, everyone, and thank you very much for joining us this afternoon for our Modern Security Community session. My name is Matt Smallman. I'm the author of Unlock Your Call Centre, tactfully positioned here just behind me. And just introduce myself briefly. My job really is to help organisations remove those time-consuming, frustrating, and pointless security processes that we're all too familiar with. [00:00:24] Through the Modern Security Community, we've been doing our best to improve understanding and build the case for better security processes, particularly in consumer-facing applications. So I'm delighted that you've been able to join us this afternoon for our special session, where we'll be looking at battling deepfakes and synthetic voices, subtitled Safeguarding Voice Biometric Systems. ### Objective and Context [00:00:47] **Matt Smallman:** The objective of this afternoon's session really is to increase understanding of this deepfake threat to voice biometrics, discuss their real-world implications, and potential mitigation approaches. This is an incredibly popular topic, but we have specifically scheduled this session after a number of previous ones, because I think it's really important to get it in context of both how voice biometrics works as a technology, but then also the entire range of vulnerabilities that voice biometrics and other call centre security technologies are prone to. [00:01:21] So if you haven't already, or you don't have a great understanding, I'd encourage you to check out the previous sessions. We had myself and Ian McGuire running a Beginner's Guide to Voice Biometrics, which is a really interesting session, and really some interesting and amusing anecdotes from Ian as well. And then a couple of weeks ago we ran Understanding and Mitigating Voice Biometrics Vulnerabilities. [00:01:44] Both of those are available on the Modern Security Community website, and I particularly encourage you to check them out because we've improved our video player, which should enable you to jump exactly to the right sections in the things that you want to know about. So on the right-hand side you'll see all of the key sections in the video highlighted, and just click on those and it will take you to that section. [00:02:03] So you don't have to sit through 60 minutes of Scottish jokes in order to find the nugget of insight that you want. So that's the prelude. ### Introducing Haydar Talib [00:02:15] **Matt Smallman:** I also want to introduce our special guest in this afternoon's session, Haydar Talib, who's joining us from Nuance Communications. Do you want to give a quick intro, Haydar? [00:02:25] **Haydar Talib:** Sure thing, Matt. Hi, everybody. Thanks for having me on today. Really interesting topic. So my name is Haydar Talib, and I manage the security and biometrics research team at Nuance, which, as some of you may know, is now Microsoft company. We have a long history in the space of voice biometrics. We've been developing voice biometrics technology for over two decades. [00:02:51] And for reasons that will become more obvious as Matt and I have the discussion on the topic with deepfakes, we've also expanded the technologies that we developed in this space of secure authentication and fraud detection, namely in the space of anti-spoofing technologies, as well as technological factors that would be complementary to voice biometrics as well. So my team are the group that develops the underlying technologies that are used in our solutions. [00:03:25] **Matt Smallman:** Thanks very much. So, this afternoon's session, we're going to go through three main areas, really. We're going to talk about the evolution of text-to-speech and where the origins of this deepfake and synthetic voice threat. We're going to talk about how that manifests itself, then, as a threat to voice biometrics, and then we're going to discuss the range of mitigations and the ways to think about how to counter this threat when implementing these and other technologies. [00:03:50] At the end, we'll have an opportunity to look at potentially the broader implications of synthetic voices and what we as an industry, and as a set of experts in that field, might be able to do to help society as a whole tackle some of the challenges that are inevitably going to come down the road. This is a really popular and interesting topic that lots of people have thoughts, concerns, and interests about, so we would love to hear your questions. And I will moderate those and drop them in at the right sections in the session. [00:04:21] There are two ways you can do that. You can use the chat feature in Zoom. If you use the chat feature, though, bear in mind that everyone else will be able to see your name and details. Or you can use the Q&A feature, where only I can see who's asked a question. And I won't attribute it to any particular organisation when I'm doing that. So I'd encourage you to use both of those. The more interactivity we get, the better. Otherwise, Haydar and I are gonna just drift off into a conversation about different aspects of this that we find interesting. [00:04:50] But what we really want to hear from is the things that are worrying you, keeping you up at night, and how we might help address those issues. Okay. ### What are the origins of text-to-speech technology? [00:04:57] **Matt Smallman:** So, making a start, then. Just to find the right buttons here. So this is the Kempelen speaking machine. This was from 1783 to 1784. A guy called Kempelen, who's a really interesting scientific inventor, came up with this. And effectively it's mimicking the human voice. It's mimicking the way in which the human vocal cords creates voices. [00:05:24] And it's quite [laughs] amazing to think that it was almost 250 years ago now that this was created. But this guy's also famous for coming up with a thing called the chess Turk, which was a complete fraud. So that was where a machine was built in order to play chess, and actually it was a man hidden underneath. So humans have been interested, naturally, in replicating human speech since time immemorial. [00:05:51] There are incidents of people theorising about how this could be done going back to before the current era. So this is nothing new, and you can see why there's that interest. But really it's accelerated in the last few years. This is the 1939 World Fair, where Bell Labs demonstrated the VODER system, which was a pseudo-mechanical electrical system for producing those phonetics that are key to the human voice. [00:06:22] Not entirely practical, but for anyone who's following the final series of Mrs Maisel on Amazon Prime, you might recognise it from a previous episode. But even going back to the 1980s, the technology in systems required to recreate even the most simplistic human voice were really, really challenging. This is the amount of kit that Stephen Hawking had to carry around on his wheelchair in order to create that really, really terrible artificial voice that you'll recognise from his work over the last decade. [00:06:57] Obviously it got smaller and reduced over time, but this is just the 1980s version. Now, clearly, over time, and in recent years, and, in fact, months, the technology has improved dramatically. And just hat-tipping to some of the organisations involved in there, and potentially some of the both good and potentially bad actors in here. And also want to touch on why this is so important. ### How do modern text-to-speech systems create synthetic voices? [00:07:24] **Matt Smallman:** Well, the first thing is the emergence of deep neural networks. So, as opposed to effectively hand-crafting the vocal cords necessary in order to create different words and creating in vast dictionaries of the way in which every single word is said, by exposing AI, machine learning and deep neural networks to large repositories of human speech, we've been able to get the computer to effectively figure out the way in which to replicate human speech, without us having to do all the hard work in between. [00:07:53] And that has created a dramatic increase in both the quality and the speed in which these voices can be created. Why is that of interest commercially? Well, if you think about the cost it might take to get a Hollywood actor back on set to rerecord a certain piece of dialogue, and the amount of delay to the schedule that might cause to next year's Hollywood blockbuster or Marvel series film, then it's absolutely worth the time and money to do that. [00:08:22] If you thought about how you might want to create more and more high-quality audiobooks, how you might want to localise your services in many different languages, how you might just want to improve the quality of your interactive voice response service, there are a whole host of completely legitimate commercial applications for better, and higher quality, and even cloned synthetic voices. I think the one that sticks in my mind most interestingly is, just last week, Apple announced that they'll be introducing a service in the next generation of iOS specifically designed to help people who are at risk of losing their voice capture elements of their voice, such that they can sound like themselves in the future. [00:09:06] So there is a huge genuine commercial, accessibility, and even public desire and need for these kind of services. So they are not going away, but there are some key characteristics that I think we'll come back to in our sessions later on that are interesting, that can be picked up on again. [00:09:28] **Haydar Talib:** I think it's worth highlighting here, and I think you touched upon this a little bit, Matt, but the primary use cases of these technologies actually has little or nothing to do with voice biometrics at all. So I know we're going to be looking at these through the prism of the voice biometrics world. But this point is going to come back in a subtle way shortly. The goal is to produce natural, human-sounding speech synthesised by these advanced neural network-based technologies. But the ultimate goal, the objective function they're optimising for, is not one that is necessarily targeted at voice biometrics, so. [00:10:08] **Matt Smallman:** Yeah. So that's a really, really important factor. I think the other things to bear in mind as you think about the origins of this technology are just where that learning's taking place. So there are only a finite number of extremely large audio, spoken-word repositories under which these systems are trained. And the training process for these systems, a lot of these systems today rely on what we call pretrained models, and they spend a huge amount of computational power, effort, and money to produce these pretrained models, on which a small overlay is effectively created to make the voice sound a little bit different. [00:10:48] So the fact that there is a corpus that's used to create it in the first place, and the fact that there are a finite number of these pretrained models, both, and though those pretrained models are very expensive and time-consuming to create, give some clue as to how we might be able to think about mitigating this threat as it evolves over the next few years. Cool. ### What are the boundary conditions for synthetic speech to be a threat to Voice Biometrics? [00:11:10] **Matt Smallman:** So I just want to talk for a few minutes, really, about what I see as the... 'Cause I mean, this stuff is not new. Yeah, so I've been implementing voice biometric systems for more than a decade. Synthetic voices have existed for exactly that same period of time. This has always been what we would call a horizon risk. It is something that it was reasonably inevitable, at some point, synthetic voices would be good enough to start fooling the human ear, and they might become good enough to start fooling biometric systems. [00:11:39] So as we've been thinking about that threat over the last decade, I've identified what I call three key boundary conditions. And, oh, I just put my slides back up. So they are quality, the quality of the audio that's being produced; the amount of audio required in order to produce that, both the content of it and the duration of it; and the speed of that production. And we're going to have a little look at each of those in the next few minutes, 'cause I think Haydar's got some really, really, really interesting examples. ### How good are the latest synthetic voices? [00:12:11] **Matt Smallman:** So actually, let's go to those now. Let's talk first off about the quality aspects, okay? So where do we go for this? So, here we go. We're gonna try and run this test online. So Haydar, and, in fact, I have as well, has created a number of synthetic versions of his voice. So we're gonna play a little game. [00:12:31] And if I can just launch the quiz? So you should have had launched a quiz, and I'm going to play seven different audio samples, and if you could use the quiz function to tell us whether you think they're real or fake. And then we'll come back to those at the end. So here's sample number one. [00:12:50] **Haydar Talib:** Hi. This is Haydar. [00:12:51] **Matt Smallman:** What do people think? Let's have your answers. Oh... Well, this is an interesting experiment, this. Okay. [00:12:59] Indeed. [00:13:00] And the answer is... It's a fake, which 92% of you got correct. So well done [laughs], everyone. Next sample. [00:13:10] **Haydar Talib:** How can we help you today? [00:13:11] **Matt Smallman:** Oh, there's some hesitation going on. Not everyone's voting quite as quickly. Okay, so 75% of people think that's a fake, but in practice it's real. Let's try another one. [00:13:25] **Haydar Talib:** Yes. Of course, I could. [00:13:26] **Matt Smallman:** Oh, okay. Apparently some people are having difficulty accessing the quiz, 'cause if you hit submit, then it disappears. So I'm sorry about that. So, yeah. So if you, [laughs] you can put your comments in the chat if you like. So that was sample three, which the one person left in the quiz, because I didn't explain it very well, apologies, said was fake, but in practice is real. Okay. We'll just do one or two more. Let's do this one, because it tells you a bit about Haydar's personality. [00:13:58] **Haydar Talib:** Marvel superhero films are not as good as they used to be. [00:14:00] **Matt Smallman:** Just throw some answers in the chat. It's a fake. There's some reals. Oh, there's reals, fakes. So, oh, fakes, fakes. Well, oh, there's a consensus emerging for real. But it is, in fact, fake. [00:14:19] Yeah. [00:14:19] Okay. So, given the quiz has failed, we'll move on and not necessarily cover all of the rest of those samples, but I think that makes some really interesting points, Haydar. Do you want to jump on those before we carry on? [00:14:31] **Haydar Talib:** I mean, [laughs] I won't make anyone feel ashamed for their performance here. I think the key point is, indeed, these technologies have advanced to the level where they are extremely difficult to detect by the human ear. And in this case, you all have just heard me speak for a few moments. [00:14:53] But a lot of these fake synthetic voices, clones of myself using some of the more advanced technologies available today, can indeed produce what seem to be true-to-life spoofs, insofar as the human ear is concerned. And I think that's a key point to bring in at this point. And, I know, Matt, you wanted to segue into maybe the technological side of things. How do they look? [00:15:22] ### Are they good enough to defeat Voice Biometrics? [00:15:22] **Matt Smallman:** Yeah. So, there's a couple of things that flow out of that. So one is, like, you're listening to this on a reasonably high-definition Zoom call, which isn't a great comparison for sometimes the quality of audio we get in the phone channel as people call, speak to the contact centre. But, more importantly, I wanted to come back to a slide that we used in a previous session, in the introduction to voice biometrics, and this talks about this decision-making. [00:15:49] Like, voice biometrics is a probabilistic technology. It's not deterministic. So every single test produces a score, and we use this nought to 100. And everyone else uses different ways of scoring it, but nought to 100 works for this base. And you'll see the vast majority of genuine speakers sound like themselves, but they never really sound perfectly like the enrolled samples. So we see, up here, we see the genuine speakers. [00:16:12] A high proportion of those tend to score really well. Most other imposters sound nothing like the real person they are. So the majority of imposters score far over to the left, but they never score quite zero, because they're still humans. They're probably speaking English. There's some elements of commonality in there. And what synthetic voices are really doing is trying to create something in this middle ground. [00:16:35] And we've drawn the curve here for illustrative purposes, and Haydar, I'm sure, can tell us about some results. What we tend to see with synthetic voices is that, as Haydar says, they're designed to replicate the elements of the voice that humans think are most distinctive, those that we would recognise in a movie, or a podcast, or when we're listening to a localised version of a service. So when we're trying to listen to our great-aunt's voice that's been protected over time because of her loss of voice. [00:17:06] So synthetic voices are moving the curve, but they're not moving it completely overlapping with genuine, and this is the result of some real-world experiments. What we actually see is, yes, there are a significant proportion that get over that threshold line, but you could move the threshold. But there are also a huge chunk that still aren't good enough in that case to make the threshold. Haydar, I think you've done some experiments on this. Do you want to give us some- [00:17:30] Yeah. [00:17:30] ... real-world insight? [00:17:31] **Haydar Talib:** [00:17:31] I mean, I can get into a little bit of storytelling on how we would interpret a graph like this. And I know you put this up for illustrative purposes, but I would say this is a good illustration of what the state of the art might look like today. But there's a deeper story here that I think is interesting to highlight. And it's a story of fraudsters trying to impersonate legitimate individuals, right? [00:18:01] So the introduction of voice biometrics was crucial in making sure that your customers with whom you interact are really who they claim to be, the voice on the other line, as it were. And the goal of any fraudster, or non-legitimate person, is to try to be confused for the legitimate account holder, let's say. Historically, right, like, Matt, you were saying we've all been in the voice biometrics business for, like, about a decade here. [00:18:32] And 10 years ago, if you look at, say, technology back then, the two bumps that you showed, the imposter and the genuine, they were a lot closer to one another 10 years ago. So the fact that we can show it now and they look far apart, even though, as you said, it's not zero. There's that little overlapping region somewhere in the middle. You've drawn your threshold line vertically. [00:18:58] These are like the pragmatic realities, and we're not even talking anything about synthetic voices or deepfakes yet. We're only talking about human malicious actors trying to commit crime against innocent customers. So the bumps used to be a lot closer to one another, meaning the trade-off between security and convenience was a lot more of a difficult decision to make 10 years ago. But the same AI technologies that have benefited advances in different areas, in the human-like, natural TTS engines, have also benefited voice biometrics technology itself. [00:19:40] And that's what's actually enabled those two bumps to separate further and further apart. Those trade-offs between allowing somebody who is not who they claim to be to be authenticated wrongly versus the convenience and security that you provide to legitimate customers, those trade-offs are a lot easier now. Like, the overlapping region is still there probabilistically, but it's very, very small, and that is actually thanks to AI advances in deep neural networks and deep learning of recent years. [00:20:09] I would say the major revolution as far as voice biometrics was concerned started approximately in 2018, and we've just been reaping the rewards, developing advancements in those technologies. Every year you can see that in the domain of voice biometrics. But where does the spoofing come in, all of a sudden? Well, what it does is, you can actually look at it as moving the bump. The red, which are bad guys, the fraudsters, if you will, are now, again, closer to the green bump, right? [00:20:41] So, yes, that for them seems like they're gaining ground, if they're actually even able to use these kind of technologies and synthetic TTS voices, and use them effectively. But the other way of looking at it is that it's certainly not 100% overlap. So even before we talk about some of the other technologies that we're going to talk about shortly, I think what's interesting here is, yes, the bump has moved back closer to perhaps what the difference might have been a few years ago. Again, not easy to achieve that even with tools today, but it's certainly not 100%. It's not a foolproof way to try to fool a modern state-of-the-art voice biometrics system. [00:21:25] **Matt Smallman:** And I think that's true when we look at those media cases. Inevitably the media showed us the videos of the one-and-done successes, and we know in practice that that's not what happens. There were dozens, if not more, attempts to successfully attack those applications. So- [00:21:44] Yes. [00:21:45] ... this is what really happening. And I think this talks to, if we are breaking down the threat into these different categories, this means that it's starting to approach technical feasibility. Yes, voices can, constraints aside, be created that are starting to be good enough in a proportion of cases to beat a voice biometric authentication system, the way they are currently configured in some applications. ### How much target audio is needed to create a synthetic or deep fake voice? [00:22:11] **Matt Smallman:** There are two, however, other boundary conditions I think it's worth coming back to talk about, and the second of those is about the amount of audio required to produce it. So I think I talked in the opening segment about the... I think when we were trying to create synthetic voices even maybe five, six years ago, we'd have to put a recording artist in the studio for like a week [laughs] to create a generalised voice that we could use in a typical financial services application. [00:22:35] In practice, in many cases we just use prerecorded samples, because it just wasn't worth the effort. And that has come down significantly, so I think all of the targets that we've seen in the media reports have talked about in the range of 90 to 15 minutes of audio being used to create these samples. So that starts to become practical. That starts to become practical if somebody is a high-profile individual, has a lot of YouTube videos available online of them; then there starts to be a high-quality of audio available to them. [00:23:08] And the, or high volume of audio. And the other thing that's improved recently is the ability to transcribe that audio effectively and automatically, because it's not enough just to have samples. We need to know what they're saying, I mean, because that's what the input's required to train these models. So, thinking outside the voice biometrics arena directly, we have seen in the US a couple of instances of parents being extorted money by synthetic voices of their children being used to suggest that they'd been kidnapped, which is, I think, a topic we'll come back to later. [00:23:46] Maybe the bigger threat too of this technology is beyond our scope and is to the consumer rather than to the enterprise. So the amount of audio required has dramatically reduced, and there are, and Microsoft's strength Nuance is one of the players in this, there are models in existence now requiring five to 10 seconds of audio to create a human-like sample. [00:24:12] But I think, from some of our own research, we can say that, whilst they're not available in the wild, the samples that we do have access to suggest that, whilst they might sound okay, they're probably not gonna defeat a biometric system today. I don't know if you got more to add on those? [00:24:26] **Haydar Talib:** Yeah. No. That's exactly right, and that example of the grandparent being taken advantage by a very clever fraudster who somehow has impersonated their grandchild, possibly using something like this. And it goes back to that quiz you just ran for everybody, right? What sounds real or natural to the human ear is not necessarily good enough to fool the machine, right? [00:24:50] So the voice biometrics technology, for sure, the Venn diagram has a big overlapping region between the two, natural-sounding speech, what voice biometrics is doing, but it's not a full overlap. Meaning the voice biometrics, again, a state-of-the-art voice biometrics technology, is looking for a plurality of characteristics in the frequency domain that are related to voice, associated with voice, but not necessarily in the particular recipe or combination that these TTS technologies are producing the speech. So it's those kind of things that, yes, make them sound good to the human ear, but not necessarily good enough to fool a voice biometrics engine. [00:25:31] ### Are they quick enough to be able to maintain a realistic conversation? [00:25:31] **Matt Smallman:** That's right. Thank you. And the final category, then, is boundary conditions that we continue to track, and the one that's least met today is really about speed of output. So these systems, even the best state-of-the-art systems, often require 30, 40 seconds to generate a new phrase or utterance. We are not yet at the stage where I can type and the words will come out, because actually you need to know what the next word is in order to figure out how to make the current word sound correctly. [00:26:03] So there is almost an in-built delay in how these systems will work. Our human minds takes care of without even worrying about the ums and uhs that I inevitably drop into sessions like [laughs] this. But that's something that isn't really available yet. I suspect it's a function of computational power, and it will emerge over time. But that means that systems that are predictable in terms of the input that they require are far more vulnerable to this form of attack than those unpredictable systems. [00:26:34] And the biggest unpredictable system in the world is the call centre agent, and what they might ask, and what they might do next. So there is still some reduced feasibility as a result of the way in which those interactions take place. Yes, in an IVR, where the interactions are reasonably predictable, it's far more likely that a fraudster could pre-prepare the required utterances. But when they get into a conversation with a human, being able to maintain a consistent human conversation is pretty challenging without significant amounts of efforts. So, do you have anything to add on that area, Haydar, or? [00:27:11] [00:27:11] **Haydar Talib:** No. I think you've covered it. You're absolutely right. [00:27:14] **Matt Smallman:** So just to summarise where we are on the threat-spectrum state, yeah: absolutely, these forms of attack are feasible. There are constraints on fraudsters' ability to execute them, but there is almost certainly a more significant constraint on their intent and desire to use them, which we'll come back to later when we look at this threat in context to the whole. But right now, we're just talking about the theoretical vulnerability of this technology. ### How to mitigate the threat from deep fakes to Voice Biometrics systems [00:27:43] **Matt Smallman:** So we want to move onto next is thinking about how we might mitigate this threat. And there are a couple of areas in which it's worth thinking about. We'll wrap these up in a summary at the end, but I think of this in three buckets, and we'll dig into each of these. The first is what I call the first line defence, and that is the biometric system itself. So, like, a customer who is not enrolled is far more vulnerable than a customer who is enrolled, because I don't... [00:28:10] To sound like them, I don't need to pass any test. So that's the first line of defence. But then again, I think, as we've talked about in the sessions, the curve isn't exactly overlapped with a real customer. There are many attempts that fail to meet the required threshold, and that in its own right is detectable. [00:28:35] Implementing simple rate-limiting methodologies, either permanently locking voice prints in accounts or temporarily locking them, restricts a fraudster's ability to exploit them. As does tuning and calibrating your system to be optimised for your particular environment, your particular acoustic characteristics, and your particular customers. So the biometric system itself is really the first line of defence, and I think does actually provide more security and safety than media reports might have suggested. Haydar, I'm sure you've got some thoughts there. [00:29:12] **Haydar Talib:** Yeah. I mean, certainly I don't want to [laughs] start a debate about journalistic integrity here or anything. But for sure, some of those articles perhaps lean a little bit towards the sensationalisation, I think, is where we're going with this. They are indeed showing the one attempt that succeeded, and, indeed, that can certainly be something that is demonstrated. [00:29:36] It is also recalling the point that even the fake audios that you all heard of myself, well, those were generated with my full participation, and I was able to iterate and reiterate on the voice until it sounded true to life, providing as much audio of myself as the system needed. So it's like stacking the odds. [00:29:57] Everything that you do like that, that is not something that necessarily, realistically, a fraudster might be able to do or pull off with ease, gives an advantage, obviously, towards the example of running the synthetic speech attack. And then, finally, if you can try it any number of times and only play back the one time it fooled the system, as it- [00:30:20] Hmm. [00:30:20] ... were, then, of course, you're painting a certain picture that, again, is not necessarily realistic. [00:30:26] ### How can presentation attack detection prevent deep fakes? [00:30:26] **Matt Smallman:** That's great. Thanks for that. And the second bucket, then, and you will definitely have more comments on this, is what I would call feature detection. So there are a couple of characteristics of this kind of attack that maybe today we haven't prioritised detection of, but that are certainly feasible to detect. So the first of those is, we'll look at each of these in turn, but they are the fact that the audio has to be played back through some form of device. [00:30:51] And presentation attacks are something that we've been dealing with in the voice biometrics space for a while. And these attacks are, in fact, no different from other presentation attacks. And they have to be generated, and they have to be played back using speakers and microphones in most cases. The second area is watermarking, an increasingly emerging field, which we'll come back to. And then, finally, as we talked about a little bit earlier, is actually detecting the characteristics of the text-to-speech engines themselves that are creating these voices, whether that's the processes they use or the corpuses under which they're trained. [00:31:26] So if we just look at each of those in a bit more detail, then. So I think playback, presentation attacks, is obviously something we've had to deal with for as long as voice biometric systems have existed, and there are distinct characteristics in some cases that can be detected that provide an opportunity to prevent cursory and trivial attacks. And those technologies have also improved at rates of knots, given the advances in AI and DNN. So, Haydar, how are your latest playback and recording detection processes? [00:32:04] **Haydar Talib:** I mean, I won't get too far into the specifics, but I think, philosophically, so, everything I said earlier about voice biometrics, that was only considering it on its own. But just speaking from my personal experience, and where I've been working for the last number of years, as I mentioned earlier, we've actually gone on to develop additional technologies. And the reason for that is because, as strong as voice biometrics might be on its own, it's just safer, and perhaps even wiser, to deploy technologies without allowing for, let's say, a single point of failure, right? [00:32:44] So what you want to do is now look into all the different things you can do to potentially ensure robustness of your solution as a whole, right? So no longer think of voice biometrics as something on its own, but think of voice biometrics complemented by additional security features or additional factors that can ensure robustness. So playback detection is what we would refer to as the feature that would help with detecting presentation attacks. That is part of our, the way we frame this is under the anti-spoofing module, which also includes detection. I think we're gonna touch upon these other- [00:33:25] Okay, yeah. [00:33:26] ... technologies as well, but some of these other features that we aim to detect as well. On the playback specifically, I mean, yes, it is basically, at this point, a deep neural network-based technology. I would say, for all of our features in our suite of factors, they're all fundamentally based on deep neural network methods. [00:33:49] ### How can watermark detection prevent synthetic voice attacks? [00:33:49] **Matt Smallman:** Cool. The other category, I think, that's quite interesting, and emerged were these two categories emerging, is watermarking. So if we just think back to that slide with all the logos on a few minutes ago, like, none of those organisations want to be associated with this kind of activity. And to some extent they've all been caught by surprise by how their technology, which they invented, and matured, and developed for completely just commercial reasons, has been misused in the way in which it has. [00:34:20] ElevenLabs, most famously, whilst they were actually the service that was used to create most of the recent press articles, in practice they have also been used to create deepfake videos and voiceovers of famous celebrities in very awkward and disturbing situations. And they have been forced to change their ways of operating in order to do that, in order to reduce the damage to them and their brand. [00:34:50] And, increasingly, we see organisations, nearly everyone on that slide actually would stand up and tell you about their responsible AI practices, about recognising the potential harms in doing what is necessary to mitigate those risks. So watermarking is something we've been talking about, again, for probably as long as I've been involved. But it's something that's come to fruition literally in the last six months, as organisations have recognised that they really need some traceability and accountability for the voices that they're creating. There don't appear to be- [00:35:21] Yes, I agree- [00:35:21] Oh, go on, Haydar, yeah. [00:35:22] **Haydar Talib:** ... I agree with that. Sorry to cut in. I think, yeah, you're touching upon an important theme here, right? So the providers of the technologies need to continue to recognise their responsibility, and provide potentially powerful capabilities that may be abused outside the intention of the original intent of the technology, but they have a responsibility too, nonetheless. And the watermarking is indeed one way that the providers of these TTS lifelike technologies, something they can do to help, I guess, the community as a whole continue to operate in a trustworthy manner. [00:36:00] But I would say that we don't rest on our laurels, right? So, as technology providers, or those who deploy these kind of technologies, we essentially can't take for granted that the watermarks will exist. So we still have to continue collectively developing these countermeasures and essentially staying up to date with voice biometrics evolution, with anti-spoofing measures, assuming that there are no watermarks. But certainly, when they are there, that will definitely help in the overall defensiveness of these kind of solutions. [00:36:35] **Matt Smallman:** Yeah. And I think that there are some challenges with watermarks. Right now, though every provider has them, not every provider is making them publicly available, if they're, or even privately available to produce this technology. So I think we're in a bit of an awkward period where this is emerging. There's regulations being proposed in Europe. There's regulation being discussed in the US. I think this will settle down in the medium term, so that these kind of commercial sources are likely to be protected in this way. [00:37:04] And we've already seen improved consent-based processes with people like Resemble. I think ElevenLabs doesn't require an audio consent statement, but pretty much everyone else now requires audio, like the real speaker to read out that they are happy for a voice to be created. And that is manually checked before a voice is created. So processes are emerging that will constrain the abilities and sources for their... That's not to say, however, there won't be ways to circumvent those, and that there might be sources that don't use that technology. [00:37:34] So that leads us on to that final question, Haydar, which is around: actually, can you detect the characteristics of either the pretrained models, or the way in which they're developed, or the way in which they're overlaid, or the corpuses themselves? How effectively are you able to do that? [00:37:51] **Haydar Talib:** I'll speak broadly here, right? So that's a whole area of research, and it's a whole area of technology and development. I mean, it's certainly something we've included in our practice for the better part of a decade. So, in a sense, again, just speaking through the lens of my own experience in this industry, and obviously working at Nuance, the problems are not new in and of themselves. What's really new here is the revolution in the quality of these types of attacks, right, the synthetic speech and so on. [00:38:27] Perhaps even the access to data that is far more prevalent now than it was a decade ago when we first started in this area. Certainly there's a bit of a cat-and-mouse-game aspect to this, right? And it goes back to that idea of, if you're reading between the lines, you will hear we have said things like modern systems and- [00:38:49] [laughs] [00:38:50] ... contemporary systems. The technologies for voice biometrics, for synthetic speech detection, playback detection, the things we develop that make the solution as a whole continue to be effective and robust, those themselves evolve along with what we're seeing in the TTS technology evolution. So I think it does become important to understand how do you keep pace with those technologies. So, for sure, there are the vendors and providers of the technology themselves, it's in their best interest to keep evolving their own solutions, [00:39:27] make sure that they're not vulnerable to things like sophisticated TTS attacks. But, yeah, it's important to stay current in terms of developing the technologies, but also in deploying the technologies, or staying up to date with the current thing. And, yes, unfortunately, it's a little bit of a cat-and-mouse dimension to that, if you really look at it at the technical level. [00:39:50] **Matt Smallman:** And I think it's reached a point that, fortunately, it coincides with the move of a large number of implementations to the clouds. But I liken this to going back to the early noughties, updating your virus definitions once a week just to make sure you got the latest patch for the- [00:40:07] **Haydar Talib:** [00:40:07] Yeah. [00:40:07] **Matt Smallman:** ... for the latest things that emerged. Because I think, to some extent, for a period of time, whether that's a few months or a few years, this will be a game of Whac-A-Mole. It's not that these things aren't detectable and preventable, it's that actually the speed of that cycle, observe or, and take, and act on it, is going to be critical. So I think that, for me, would be one of the major takeaways for organisations battling with this risk right now. ### How can other authentication methods reduce the threat from synthetic voices? [00:40:34] **Matt Smallman:** Just conscious of the time, and people may have questions. We've got one very interesting question already come up in the chat, which I will save because I need to read it properly. The final bit I want to talk about is, I think, and, for me, this is actually quite positive, what these range of attacks has shown us is that voice biometrics is not impenetrable. It never was. It was always a probabilistic system. [00:40:58] There were always risks to it. But they have shown us that it's not the golden bullet, or the silver bullet, I think, is probably the correct analogy, to every security problem. It needs to be implemented as part of a layered, and considered, and deliberate approach to security. And therefore, when you combine voice biometrics with other forms of detection, and I'm not talking about going back to passwords and PINs here, [00:41:24] I'm talking about leveraging the continued advance in technology, things like network authentication, things like device identification and management, such that, without requiring any additional customer effort, you can add these factors together. And I think Haydar described it as like a security web rather than a chain. Like, you shouldn't rely on, like, breaking the chain will break the system. But actually, if you have a web, then you can respond depending on where the weight of the attack is. [00:41:52] **Haydar Talib:** [00:41:52] Yeah. [00:41:53] **Matt Smallman:** And I know, Haydar, you actually, and I didn't know this until earlier in the week, you actually have a patent yourself on ConversationPrint- [00:42:00] Right. [00:42:00] ... which is an approach to this. I don't know if you want to give us a bit more detail on that whilst you're on there? [00:42:05] **Haydar Talib:** Yeah. I'd be happy to. I'll try to keep it brief, but happy, if you have additional questions, to talk more about this. But I think what you're referring to, ConversationPrint, what something... So, indeed, as I said earlier, we were conscious of the evolution of these technologies in general as a whole, and spoofing was absolutely part of it. But we were always looking at the bigger fraud picture, and the philosophy is, while the more things you're using as part of your authentication technological solution, the more things you're looking at at the same time, the harder and harder it becomes for somebody who has malicious intentions to bypass any one of the sub-technologies, if you will. [00:42:44] So now voice biometrics, while still a critical part of the solution, can be complemented by other things. And one of the things we started to do a few years ago was, analogously to how the voice biometrics tries to identify the acoustic characteristics of somebody's voice based on the audio signal, we developed something called ConversationPrint, which actually starts to do, analogously, something similar, but focusing on the language that somebody employs. [00:43:12] So not the language you're speaking, literally, but the expressions you use, the vocabulary you tend to use, the grammatical constructions of how you put your phrases together when you're in a conversation with somebody. That, as it turns out, also helps to identify us individually. So now, when you combine this as a new dimension to the voice biometrics, well, all of a sudden, you're looking at a much richer set of features using complementary technologies, making it a lot more difficult, potentially, for anyone with malicious intention now to try to fool the whole, as opposed to one specific type of technology. [00:43:54] **Matt Smallman:** That's great. That's great. And I guess that's most effective when you've got a chunk of a call rather than two, three, four seconds. [00:44:02] [00:44:02] Yeah. [00:44:02] You need more. That's 'cause there's less data, I guess, in a- [00:44:06] **Haydar Talib:** [00:44:06] Yeah. [00:44:06] **Matt Smallman:** ... conversation the best- [00:44:08] **Haydar Talib:** In a practical, yeah. On the practical side, what you're doing is basically now you're transcribing in real time, if you're talking about telephonic conversations, 'cause the same technology, by the way, can apply in chat as well. [00:44:19] **Matt Smallman:** [00:44:19] Yeah. [00:44:19] **Haydar Talib:** But for the purposes of, a modern voice biometric system can probably get you a good authentication result with as little as one or two seconds of speech. That is not... That is barely a full sentence, so you'd be surprised what you could do with one full sentence. But, generally, something like ConversationPrint, we prefer to have a certain number of sentences strung together and get a sequence of phrases. That way, before rendering a more and more confident decision, if you will. [00:44:47] **Matt Smallman:** Oh, that's great. And I think that just reinforces, and actually I'm pretty excited about a range of developments in this space. If you think about the phone call itself, like, yes, there is the way in which the person speaks on the phone call. There is also the things they say when they're on the phone call. There is the way in which the phone call gets to you, and there's the noise and the signal that's in the silence in between what people are saying. [00:45:11] And all of those have characteristics that can be used for authentication, with different degrees of confidence, and fraud detection, different degrees of confidence, yes. But none of them require the customer to go through any extra efforts. And none of them have the same vulnerabilities as knowledge-based authentication or some other authentication scheme. So I think it's a really exciting time to think about how we expand. We already have the phone call, so what else can we do with it that increases security whilst not having a detrimental effect on the customer experience? [00:45:43] So I think we're in for an interesting few years as this evolves. I do, just conscious of time, again, I'll ask for any questions on the chat. I just want to zoom back out, because we've gone right down the rabbit hole of deepfake and synthetic risks. [00:46:00] So, first off, one of the questions that's come in the chat has been around the different ways in which this kind of technology could be used, and the different methods of attack that could be used. [00:46:11] And, clearly, we've seen a couple of those in the press. In practice, there are probably four or so modes of attack. We're not going to lay those out today, because I think that would be slightly irresponsible, but if you want to reach out afterwards, and if we can qualify who you are, we can probably have an offline conversation about those specifically. [00:46:36] But taking it back to context, then, if I just pull up my last slides. Oh, yeah. Sorry. Yeah, we... ### Summarising the mitigations available to protect Voice Biometrics from synthetic speech and deep fake attacks [00:46:44] **Matt Smallman:** Just finally, to summarise what we're talking about in terms of mitigations. [00:46:47] We've got that first level of defence, biometrics, getting it tuned correctly, maybe adjusting the threshold as appropriate, maybe for different customers, different types of interaction. Rate limiting, in order to reduce the opportunity to try and test this kind of service. [00:47:02] Looking at different detection characteristics, or detection services, and systems, and processes that are available in order to pick up those deepfakes, whether that be through a recording, whether through watermarks or TTS detection, and bearing in mind that those things are going to evolve pretty rapidly over the next couple of years, and therefore there's a need to keep on top of them. And then, finally, other factors, as we talked about: using information from the network, [00:47:25] using information that we get about the device that's being used, and, finally, the things that are being said. And all of these other factors that can improve the security of the overall process. So there are a range of mitigations available, but they do require some considered implementation and integration. This is not about a single-factor authentication solution anymore. This is about weighing the risks of various detection and authentication factors. [00:47:53] Zooming right back out, though, and thinking just about the fraudster threat overall. So this is the slide I used in a session a couple of weeks ago, when we looked about vulnerabilities as an overall. In order to create a threat to a system, it requires the vulnerability to exist, that fraudsters have that capability, but, most importantly, for the fraudster to have the intent to compromise the system in that way. What we've talked about today is that capability, and, to some extent, the vulnerabilities, and how we can mitigate those vulnerabilities. [00:48:25] We haven't really talked a lot about the intent. Whilst we have in the wilds seen a handful of test attacks against these kind of systems, they are only that. Today, the vast majority of callers to contact centres and to enterprises across the world are still protected by knowledge-based authentication processes, which are, quite frankly, trivial to compromise through social engineering, phishing and other methods. [00:48:52] So, yes, whilst voice biometric systems may be vulnerable to this form of attack, as yet we do not have a clear intent or exploitation of that at any scale, nor do we expect it whilst those other methods of authentication remain. So it's simply too easy to compromise someone's password. It's trivially easy now to compromise their SMS or online credentials. So I think it's important to, yes, acknowledge there's threats, continue to monitor it, make sure that you have met appropriate mitigations in place, evaluate the risk, as we all should. [00:49:27] But we should really bear in mind that the intent doesn't yet exist, because nearly every organisation who's implementing this technology today has a process by which, if a customer fails biometrics, they go to an alternative process. And that alternative process probably doesn't have a lot of scrutiny on it. It probably has very similar rules set up around what transactions can or cannot be done on it. So it's only when we remove some of those, and the proportion of customers who are involved in these systems increases to the majority, that there's an incentive to attack the system head-on. [00:50:03] That said, though, there are also far easier organisations to go to to compromise it, and potentially easier channels to go to. So whilst we have talked about the theoretical vulnerability, this is still not trivial. And we expect that it will never be trivial to automate this form of attack. And therefore, whilst it will be irresponsible to say, "It's never gonna happen," it needs to borne in mind and evaluated against the complete spectrum of risks that an enterprise is faced with. [00:50:37] Voice biometrics still provides a quicker and easier and significantly more secure authentication method than traditional knowledge-based authentication, and we mustn't lose sight of that, particularly those ease-of-use and efficiency aspects that are the reason why a number of people in this call have implemented these technologies. And, finally, then, yeah, just, when we look at voice biometrics, there are a spectrum of other risks that needs considered. [00:51:05] And the one I keep coming back to is this bypass risk that I see in many organisations have not appropriately addressed. When you fail the biometric, you probably fall back to a security process that has far more holes in it and is far easier to exploit than going and creating a synthetic voice for your customer. So, with that, I'd like to open the floor to any questions. [00:51:24] We're coming up on time, but this is a popular topic, so I'm sure there'll be more questions than the one I had. And apologies, I can't answer that specifically on this call, but I don't think that would be particularly responsible, but we will in follow-up if you want to. Gonna pause, then. Haydar, do you have anything you want to add at this point? [00:51:47] **Haydar Talib:** No. I think, hopefully, this was illuminating, right? We gave, I think, a pretty complete overview on different aspects. I think those last points you mentioned really get us out of the rabbit hole, reminding everybody. And also the work that you yourself do, Matt, working with organisations, I think is incredibly valuable. [00:52:08] I think that probably the key takeaway here, as even a session like this, the goal is to hopefully inform everybody and provide a reasoned and reasonable, realistic view of what things look like today. What are the actual risks? Yes, we can't rest on our laurels, but, on the other hand, we have to assess everything with the right level of magnitude for every item that we consider. [00:52:38] **Matt Smallman:** Yeah. And I think that this very much reminds me of the... I've always been nervous when implementing this technology when you don't have the, like, as we talked about those curves, inevitably this technology is subjected for success, yeah, because customers themselves test this technology, and they test it with their friends and relatives. And those friends and relatives are far more like them than the population as a whole. [00:53:01] And they test it enough that the law of probability suggests that somebody's gonna get through at some point, particularly when we're dealing with millions and millions of users. And organisations that haven't had to deal with that I always worry about, because they haven't really understood the technology. They still have a perception that it's completely bulletproof, and I think, as we've talked about here and these press articles highlight, it's not. [00:53:26] So, but it is still significantly better, and it is still more secure in general, so I think that's an important point to come back to. I've had one more question. ### Question: How to manage the impact of press articles on customer confidence? [00:53:36] **Matt Smallman:** Yeah. I may be not quoting it correctly. I think that there is a genuine point that somebody's raised, and this is the knowledge-based authentication. It's the human psychology aspect of this, yeah. So, now that there's been these high-profile attacks, does that reduce people's confidence in the technology and therefore their propensity to enrol? [00:54:01] And I think that's a delicate line for an enterprise to walk, because, in many cases, the law requires that we take the risks, that we are ultimately responsible if our customers are defrauded. Consumers are well protected in most jurisdictions, and we are responsible for most losses. And we'd love them to move to more secure processes. And actually, for most enterprises about security, an aggregate level rather than as an individual level. [00:54:32] But if lots of individuals choose not to use our most secure methods, then that can have a large aggregate effect. And so, when we've talked about incentivising enrolment and encouraging customers to enrol in voice biometric systems, we've always stopped short of just saying how bad and how insecure existing processes are, and how much fraud goes through those systems. And I think that's, maybe there's an element of human psychology here that we've missed that we should probably... [00:55:02] Maybe the fraud that happens in that space has become such the background noise that it's this new exciting forms of attack that have started to worry people. And I absolutely have seen, nearly every client has been on the phone talking about what should our Q&As be back to customers in this situation. And I think it is an awkward question, and we need to navigate it carefully. But if you've got any more specific guidance people. Haydar? No? [laughs] Okay. [00:55:32] **Haydar Talib:** No. I think you nailed it, Matt. Yeah. ### Question: How to respond to customer concerns about the threat to Voice Biometrics? [00:55:34] **Matt Smallman:** Yeah. I think we always try to stick with reasonably generic responses in these cases, like: we have many methods of security; not all of them are visible; we evolve into that; we still believe that this is the most secure method, and we're continuing, and we wouldn't be recommending it if we didn't think that was true. But it does remain people's rights in most jurisdictions to opt out of the use of technology if they choose to do so. But they may bear greater risk of it being defrauded if that was the case. Whether you actually explicitly state that in that way, it is another matter. [00:56:06] **Haydar Talib:** Well, again, I would just say that it's even this series of sessions that you've set up, Matt, and this community are super important, because it's only through reasoned information and education that everybody develops a well-calibrated view of what the risks are and the magnitude of those risks. Whereas, yeah, indeed, if you just see an article somewhere posted, then it sensationalises things. The public will fall into a quick panic. But that's not unique to our space either. That's something we're seeing. This year, I think, has been bonkers in terms of the amount of- [00:56:44] [laughs] [00:56:44] ... revolutionary advances in AI and how it might displace and disrupt all over. So, for sure, there are things to pay attention to, and there are things that are changing in a very real and material way. But I think the challenge is gonna be keeping that information, keeping the dialogue going. And I would say, Matt, what you're doing here with this forum is exactly the right approach. Set up the community. Let's talk about these things. Let's get people who are experts in the various facets to discuss and share information in that sense. [00:57:16] **Matt Smallman:** That's great. So thank you very much, Haydar, for your contribution today. Really, really useful. Really interesting to have an expert on to give us the lowdown on the detailed technology as well. So thanks so much for your contribution. [00:57:29] **Haydar Talib:** [00:57:29] Yeah. [00:57:29] **Matt Smallman:** Thank you, everyone who attended as well. We are taking a little bit of a break now with the Modern Security Community, so we will be back on the, ooh, 22nd of June. We'll be holding a private members-only roundtable discussion, of which I'm sure this will be one of the topics that people wish to talk about. So that will be going up on the website very shortly, and I'll be reaching out to a specific group to talk about how they want to participate in that. So again, thank you very much for joining. Haydar, thank you for contributing. [00:58:02] Sure. [00:58:03] And we'll speak to you all soon. Oh, we got a clap. Hooray. Thank you, bye-bye.