Returns are a standard part of retail, but they’re not without risks. Fraudulent returns can cost businesses a significant amount of losses annually. While restricting returns might seem like the only way to fight against retail fraud, there are better ways to help reduce fraud losses that don’t sacrifice the customer experience.
Leveraging an advanced voice biometrics analysis solution can help protect customer accounts, spot fraudulent returns, and streamline the call experience. This article will explore the types of return fraud and how to combat it with advanced voice security.
Understanding return fraud
Return fraud involves customers exploiting return policies for personal gain. It comes in various forms, from returning stolen items to abusing liberal return policies.
According to the National Retail Federation, return fraud costs billions annually and contributes to operational inefficiencies. Retailers often face challenges balancing customer satisfaction with fraud detection.
The most common types of fraud in retail include:
- Receipt fraud: Customers use fake receipts or receipts from other items to return merchandise
- Wardrobing: Buying an item, using it briefly, and returning it as “new”
- Stolen goods returns: Returning stolen goods for refunds or store credits
- Refund fraud: Manipulating the system to receive more than the value of the returned item
What is voice biometrics in retail?
Voice biometrics is a technology that identifies individuals based on unique vocal characteristics. It analyzes various features of a person’s voice, such as pitch, tone, and rhythm.
This technology can help protect retail contact centers from refund fraud, offering a secure and efficient means of verifying customer voices during transactions, including returns.
Unlike traditional authentication methods, such as passwords, voice biometrics provide an additional layer of security by leveraging something inherently unique to each individual—their voice. When used in tandem with other authentication factors, this advanced technology can assist retailers in combating fraudulent returns while helping create a faster and simpler returns process.
How voice biometrics can detect return fraud
Voice biometric analysis brings multiple benefits to retailers, helping to reduce fraud and improve operational efficiency.
Real-time authentication
With voice biometrics, you can authenticate customers in real-time, helping to ensure that the person initiating a return is the purchaser. This technology can be particularly useful in contact centers, where authenticating customers through traditional methods is more challenging.
By using multifactor authentication, stores can drastically reduce fraudulent return attempts. This process also minimizes disruptions for genuine customers, maintaining a smooth and efficient return experience.
Fraud detection
Voice biometrics can identify suspicious behavior patterns by the individual attempting the return.
Multifactor authentication
You can use voice biometrics as part of a multifactor authentication (MFA) approach, combining content-agnostic voice verification with other verification methods like PINs or SMS codes.
With this approach, even if one method fails, or if some credentials are lost or stolen, you still have a method to detect fraudulent activity.
Secure transactions
Voice biometrics can help create a secure environment for customers during their transactions. Once the system receives authentication information on the customer, it can securely process the return, significantly reducing the chances of refund fraud. This helps protect the retailer from loss and can provide customers with peace of mind, knowing their information is securely handled.
Accelerating return transactions
When using traditional authentication methods, customers can often find the process tedious. Voice biometrics help speed up return transactions, as customers can skip more lengthy verification procedures.
This helps create a faster, hassle-free return process, contributing to a better overall customer experience.
Data protection
Retailers can use voice biometrics to enhance data protection protocols, maintaining their consumers’ trust.
Implementing voice biometrics in your retail system
Integrating voice biometrics into your retail system in a way that’s effective and user-friendly requires careful planning.
Evaluate current systems
Start by evaluating your existing return processes and fraud detection strategies. Understanding where current vulnerabilities lie will help identify how voice biometric analysis can fill those gaps.
Select a reliable voice biometrics solution provider
Partnering with a reliable voice biometrics provider is crucial. Look for vendors with experience in retail security, a track record of success, and robust data protection measures.
Integrate voice biometrics seamlessly into retail systems
Ensure that voice biometrics integrate smoothly with your existing retail systems. This will reduce disruption during the implementation phase and allow both customers and staff to adapt quickly to the new system.
Train staff on using voice biometrics system
Training your staff members on how to use the voice biometrics system effectively is critical. Otherwise, no matter how good the technology is, there’s an increased risk of human error that could eventually lead to return fraud.
Training should include knowing when and how to use the technology and troubleshooting potential issues to prevent delays in the returns process.
Monitor system performance and optimize processes
After implementation, regularly monitor the system’s performance to ensure it functions as expected. Make necessary adjustments to optimize the system’s capabilities and improve its accuracy and efficiency in supporting fraud prevention efforts.
Additional benefits of voice biometrics in retail
Beyond helping prevent return fraud, voice biometrics offer additional advantages that enhance the overall retail experience.
- Reduced fraud costs: By minimizing fraudulent returns, retailers can significantly reduce the financial losses associated with them. This helps merchants optimize their operations, improve profitability, and focus resources on serving genuine customers.
- Convenience: Voice biometrics streamline the return process by eliminating the need for physical IDs or receipts. Customers can complete their returns quickly and easily, leading to a better shopping experience.
- Trust and loyalty: Implementing voice biometrics builds trust with customers, as they feel confident that their identities and transactions are secure. This increased level of trust enhances customer loyalty and encourages repeat business.
- Transparency: Maintaining transparency with customers about the use of voice biometrics for fraud detection can foster confidence. Clear communication regarding how voice analysis is used will help consumers understand the purpose and benefits of this technology.
Adopt a voice biometrics solution to help prevent return fraud
Return fraud is a serious issue affecting retailers worldwide, leading to losses of billions of dollars each year. While strict return policies may be somewhat helpful, retailers need to find better, customer-friendly alternatives. One such approach is voice biometrics, which offers additional defenses against fraudulent returns while improving the customer experience.
Voice biometric solutions can help merchants secure their return processes, reduce fraud costs, and build stronger relationships with customers. Adopting such a technology may seem like a significant shift, but its long-term benefits, both in fraud detection and customer trust, make it the perfect choice for small and large retailers.
More and more incidents involving deepfakes have been making their way into the media, like the one mimicking Kamala Harris’ voice in July 2024. Although AI-generated audio can offer entertainment value, it carries significant risks for cybersecurity, fraud, misinformation, and disinformation.
Governments and organizations are taking action to regulate deepfake AI through legislation, detection technologies, and digital literacy initiatives. Studies reveal that humans aren’t great at differentiating between a real and a synthetic voice. Security methods like liveness detection, multifactor authentication, and fraud detection are needed to combat this and the undeniable rise of deepfake AI.
While deep learning algorithms can manipulate visual content with relative ease, accurately replicating the unique characteristics of a person’s voice poses a greater challenge. Advanced voice security can detect real or synthetic voices, providing a stronger defense against AI-generated fraud and impersonation.
What is deepfake AI?
Deepfake AI is synthetic media generated using artificial intelligence techniques, typically deep learning, to create highly realistic but fake audio, video, or images. It works by training neural networks on large datasets to mimic the behavior and features of real people, often employing methods such as GANs (generative adversarial networks) to improve authenticity.
The term “deepfake” combines “deep learning” and “fake content,” showing the use of deep learning algorithms to create authentic-looking synthetic content. These AI-generated deepfakes can range from video impersonations of celebrities to fabricated voice recordings that sound almost identical to the actual person.
What are the threats of deepfake AI for organizations?
Deepfake AI poses serious threats to organizations across industries because of its potential for misuse. From cybersecurity to fraud and misinformation, deepfakes can lead to data breaches, financial losses, and reputational damage and may even alter the public’s perception of a person or issue.
Cybersecurity
Attackers can use deepfake videos and voice recordings to impersonate executives or employees in phishing attacks.
For instance, a deepfake voice of a company’s IT administrator could convince employees to disclose their login credentials or install malicious software. Since humans have difficulty spotting the difference between a genuine and an AI-generated voice, the chances of a successful attack are high.
Voice security could help by detecting liveness and using multiple factors to authenticate calls.
Fraud
AI voice deepfakes can trick authentication systems in banking, healthcare, and other industries that rely on voice verification. This can lead to unauthorized transactions, identity theft, and financial losses.
A famous deepfake incident led to $25 million in losses for a multinational company. The fraudsters recreated the voice and image of the company’s CFO and several other employees.
They then proceeded to invite an employee to an online call. The victim was initially suspicious, but seeing and hearing his boss and colleagues “live” on the call reassured him. Consequently, he transferred $25 million into another bank account as instructed by the “CFO.”
Misinformation
Deepfake technology contributes to the spread of fake news, especially on social media platforms. For instance, in 2022, a few months after the Ukraine-Russia conflict began, a disturbing incident took place.
A video of Ukraine’s President Zelenskyy circulated online, where he appeared to be telling his soldiers to surrender. Despite the gross misinformation, the video stayed online and was shared by thousands of people and even some journals before finally being taken down and labeled as fake.
With AI-generated content that appears credible, it becomes harder for the public to distinguish between real and fake, leading to confusion and distrust.
Other industry-specific threats
The entertainment industry, for example, has already seen the rise of deepfake videos in which celebrities are impersonated for malicious purposes. But it doesn’t stop there—education and even everyday business operations are vulnerable to deepfake attacks. For instance, in South Korea, attackers distributed deepfakes targeting underaged victims in an attack that many labeled as a real “deepfake crisis.”
The ability of deepfake AI to create fake content with near-perfect quality is why robust security systems, particularly liveness detection, voice authentication, and fraud detection, are important.
Why voice security is essential for combating deepfake AI
Voice security can be a key defense mechanism against AI deepfake threats. While you can manipulate images and videos to a high degree, replicating a person’s voice with perfect accuracy remains more challenging.
Unique marker
Voice is a unique marker. The subtle but significant variations in pitch, tone, and cadence are extremely difficult for deepfake AI to replicate accurately. Even the most advanced AI deepfake technologies struggle to capture the complexity of a person’s vocal identity.
This inherent uniqueness makes voice authentication a highly reliable method for verifying a person’s identity, offering an extra layer of security that is hard to spoof.
Resistant to impersonation
Even though deepfake technology has advanced, there are still subtle nuances in real human voices that deepfakes can’t perfectly mimic. That’s why you can detect AI voice deepfake attempts by analyzing the micro-details specific to genuine vocal patterns.
Enhanced fraud detection
Integrating voice authentication and liveness detection with other security measures can improve fraud detection. By combining voice verification with existing fraud detection tools, businesses can significantly reduce the risks associated with AI deepfakes.
For instance, voice security systems analyze various vocal characteristics that are difficult for deepfake AI to replicate, such as intonation patterns and micro-pauses in speech. These systems can then catch these indications of synthetic manipulation.
How voice authentication mitigates deepfake AI risks
Voice authentication does more than just help verify identity—it actively helps reduce the risks posed by deepfake AI. Here’s how:
Distinct voice characteristics
A person’s voice has distinct characteristics that deepfake AI struggles to replicate with 100% accuracy. By focusing on these unique aspects, voice authentication systems can differentiate between real human voices and AI-generated fakes.
Real-time authentication
Voice authentication provides real-time authentication, meaning that security systems can detect a deepfake voice as soon as an impersonator tries to use it. This is crucial information for preventing real-time fraud attempts.
Multifactor authentication
Voice authentication can also serve as a layer in a multifactor authentication system. In addition to passwords, device analysis, and other factors, voice adds an extra layer of security, making it harder for AI deepfakes to succeed.
Enhanced security measures
When combined with other security technologies, such as AI models trained to detect deepfakes, voice authentication becomes part of a broader strategy to protect against synthetic media attacks and fake content.
Implementing voice authentication as a backup strategy
For many industries—ranging from finance to healthcare—the use of synthetic media, such as AI-generated voices, has increased the risk of fraud and cybersecurity attacks. To combat these threats, businesses need to implement robust voice authentication systems that can detect and help them mitigate deepfake attempts.
Pindrop, a recognized leader in voice security technology, can offer tremendous help. Our solutions come with advanced solutions for detecting deepfake AI, helping companies safeguard their operations from external and internal threats.
Pindrop® Passport is a robust multifactor authentication solution that allows seamless authentication with voice analysis. The system analyzes various vocal characteristics to verify a caller.
In real-time interactions, such as phone calls with customer service agents or in financial transactions, Pindrop® Passport continuously analyzes the caller’s voice, providing a secure and seamless user experience.
Pindrop® Pulse™ Tech goes beyond basic authentication, using AI and deep learning to detect suspicious voice patterns and potential deepfake attacks. It analyzes content-agnostic voice characteristics and behavioral cues to flag anomalies, helping organizations catch fraud before it happens.
Pindrop® Pulse™ Tech provides an enhanced layer of security and improves operational efficiency by spotting fraudsters early in the process. For companies that regularly interact with clients or partners over the phone, this is an essential tool for detecting threats in real time.
For those in the media, nonprofits, governments, and social media companies, deepfake AI can pose even more problems, as the risk of spreading false information can be high. Pindrop® Pulse™ Inspect offers a powerful solution to this problem by providing rapid analysis of audio files to detect synthetic speech.
The tool helps verify that content is genuine and reliable by analyzing audio for liveness and identifying segments likely affected by deepfake manipulation.
The future of voice security and deepfake AI
As deepfake AI technologies evolve, we need appropriate defense mechanisms.
Voice authentication is already proving to be a key factor in the fight against deepfakes, but the future may see even more advanced AI models capable of detecting subtle nuances in synthetic media. With them, organizations can create security systems that remain resilient against emerging deepfake threats.
Adopt a voice authentication solution today
Given the rise of deepfake AI and its growing threats, now is the time to consider implementing voice security in your organization’s security strategy.
Whether you’re concerned about fraud or the spread of misinformation, voice authentication provides a reliable, effective way to mitigate the risks posed by deepfakes.
The Deepfake Dilemma: Navigating AI Deception in Your Organization
The diversity of deepfakes that the Pindrop team encounters is both fascinating and alarming. From AI-generated phone calls into call centers to update account information, to deepfake scams using celebrities’ likeness to endorse phony products on social media – the availability, effectiveness, and increasing use of online text-to-speech tools underscores the critical need for robust detection mechanisms.
In this webinar, Pindrop brings together our Chief Product Officer Rahul Sood and the Founder of TrueMedia.org Dr. Oren Etzioni to discuss the rapidly evolving landscape of deepfake technology and its far-reaching implications for enterprises and media, as well as the spread of misinformation.
During the discussion, we also provided a sneak peek into exciting new product developments for Pulse® Inspect™, the latest addition to Pindrop’s cutting-edge deepfake detection Pindrop® Pulse™ product family.
Artificial intelligence, a key player in contact centers, accelerates repetitive tasks and can create a seamless experience for agents and customers. In a survey of contact center and IT leaders, a staggering 87% of respondents affirm that conversational AI has reduced agents’ effort and costs in the contact center, boosting agent efficiency by 65%.
In a Forbes article, Jennifer Lee, COO of Intradiem, writes: “As AI continues to mature, contact centers should keep routing transactional and computational tasks to chatbots while reserving more complex requests for human agents.” The key is using AI at the right moments to support the agent’s ability to handle complex issues more efficiently. Here’s how it works.
What is contact center automation, and how does it work?
Contact center automation, a technology-driven approach, streamlines routine tasks within the contact center, minimizing the need for human intervention. Automating repetitive tasks, including screening and call routing, can enhance efficiency, cut costs, and elevate customer experience by expediting response time and ensuring consistent service.
But there are vital components needed to make this happen. One component is an interactive voice response (IVR) system, which uses pre-recorded messages and menu options to interact with callers through keypad or voice commands. Another is automatic call distribution (ACD), which routes incoming calls to the most appropriate department agent based on predefined criteria such as a caller’s phone number, selected menu options, or agent availability.
When choosing a contact center automation solution, look for features such as interactive voice response (IVR), AI-powered chatbots for instant customer response, advanced call routing, real-time analytics, and integrations with CRM systems. Additionally, it should offer multi-channel support (email, phone, social media) for comprehensive customer communication. The option for personalized customer interactions is also essential.
How is artificial intelligence used in contact center automation?
AI and machine learning can be vital in enhancing customer service efficiency. They facilitate better customer interactions by automating responses, analyzing customer sentiment, and predicting trends. Additionally, they help reduce turnaround times, avoid human errors, and personalize customer interactions. Most contact center automation platforms are designed to seamlessly integrate with various existing CRM systems. This approach makes it possible to manage customer interactions better and synchronize data.
AI significantly enhances contact center automation by improving efficiency, reducing costs, and delivering a superior customer experience. Contact centers can better meet customer needs and stay competitive in a fast-evolving market by leveraging AI technologies such as chatbots, predictive analytics, sentiment analysis, and robotic process automation. Designing efficient contact center operations can make a huge difference, especially in customer-centric spaces like travel and hospitality.
5 AI Trends Shaping Contact Center Automation in 2025
1. Robotic process automation (RPA)
RPA is a technology that uses software robots or “bots” to automate repetitive, rule-based tasks typically performed by human workers. Automating various routine tasks in contact centers helps streamline operations, reduce costs, and enhance service quality.
2. Interactive voice response (IVR)
IVR is a telephony technology that allows customers to interact with a company’s host system via voice or touch-tone dialing. Contact centers widely use it to handle incoming calls, route them to the appropriate department or agent, and provide automated responses to customer inquiries.
3. Natural language processing (NLP)
NLP is a machine learning technology that allows computers to interpret, manipulate, and comprehend human language. Through applications like chatbots, voice assistants, sentiment analysis, and automated transcription, NLP helps improve efficiency, customer satisfaction, and operational insights. Despite challenges in accuracy and contextual understanding, the advancements in NLP continue to be a significant driver for innovation and better service in contact centers.
4. Automatic speech recognition (ASR)
ASR uses machine learning or AI technology to process human speech into readable text, making it possible for machines to understand and respond to voice commands. In contact centers, ASR significantly enhances the functionality and efficiency of contact centers by enabling voice-driven interactions and automating routine tasks.
5. Predictive call routing
Also known as intelligent call routing, predictive call routing is a contact center technology that uses machine learning to identify the best agent to serve a customer based on their interactions. It evaluates factors like past behavior, product preferences, and persona type to predict the skills and personality traits an agent needs. The system then matches the interaction to an available agent and updates itself as new data enters.
A closer look at the benefits of contact center automation
AI can significantly improve the experience of both agents and customers when engaging with a brand’s contact center in various ways. The primary outcomes of technology within contact centers can allow for the following:
-
Increased efficiency and productivity
Gone are the days of routine tasks like data entry, call routing, and FAQs for good service, which can now all be automated to free up time for agents. For instance, chatbots can answer top customer questions about store hours or return policies, eliminating the need for human intervention. This increases efficiency and shows agents that their time is valued, as they can now focus on more complex and rewarding tasks.
-
Improved customer experience
By eliminating the need for human interaction, you also allow for around-the-clock service for your customers without restricting them to handling personal needs during business hours. Predictive call routing ensures they make it to the right place for automated FAQs, scripts, and more so that customers receive the same information regardless of which agent or system they interact with. This consistency in service enhances customer trust and confidence in the brand.
-
Cost savings
Good software comes with scalability, and automation reduces the need for a large workforce to handle customer needs. Unlike human agents who can only manage one call, a virtual assistant can handle thousands of inquiries simultaneously. Allowing a virtual assistant to handle some inquiries can maximize human resource efficiency.
-
Higher security
AI significantly bolsters security in contact centers by enhancing fraud detection, improving authentication processes, protecting data privacy, proactively managing risks, and providing enhanced monitoring and response capabilities. These improvements ensure that customer data is safeguarded, fraudulent activities are minimized, and regulatory compliance is maintained.
Pindrop: A deep dive into contact center automation software for security
With the right technology, contact centers can be more support-oriented, serving your and your customers’ needs. They play a pivotal role in how your customers interact with the business. Self-service technologies are one of the best ways to alleviate pressure on reps while keeping customers engaged and happy.
Security events can also cost contact centers hundreds of thousands of dollars, and a secure contact center is synonymous with trust and reliability in your brand. By implementing advanced security solutions, businesses can prevent the significant financial and reputational damages associated with data breaches and fraud.
Pindrop’s solutions utilize sophisticated voice biometrics and phone-printing technology to ensure secure calls. It analyzes all aspects of the call, from quality to device type, to authenticate the caller’s identity without being obtrusive to your customers.
To learn more about the benefits of our passive authentication solution, read about our 7 proven ways to reduce costs with Pindrop’s technology in your contact center.




Understand the current state of deepfake technology and its rapid evolution
Uncover the potential impacts of deepfakes on businesses and democratic institutions
Gain insights into the specific disinformation tactics used in recent and ongoing elections
Discover best practices for maintaining public trust in your brand in the face of sophisticated AI-generated content
Preview upcoming Pindrop technologies and methodologies for deepfake detection and prevention
Your expert panel
Join us for this timely and important discussion as we navigate the complex landscape of AI-generated deception and its impact on the future of business.


Rahul Sood
Chief Product Officer, Pindrop


Dr. Oren Etzioni
Founder, TruMedia.org
The Association of Certified Fraud Examiners (ACFE) describes artificial intelligence (AI) as a “game-changer for spotting and stopping fraudulent activity,” specifically in the healthcare industry. AI systems can examine a massive amount of historical data, spotting trends and indicators of fraud. This is needed as the National Health Care Anti-Fraud Association (NHCAA) notes that healthcare fraud results in losses estimated at tens of billions of dollars every year. This article describes how AI is making strides to improve healthcare fraud prevention further.
Understanding AI in fraud detection
Although healthcare fraud costs companies billions of dollars annually, it can also impact the quality of patient care, resulting in higher premiums and diverted resources. The rise of AI offers promising solutions to this problem. AI technologies, particularly with voice recognition and real-time fraud detection, can enhance the security and efficiency of healthcare systems. According to ACFE, AI can immediately spot dubious claims, facilitating prompt intervention and forecasting possible fraud in the future by analyzing patterns and methodologies currently in use.
The rise of AI in healthcare fraud detection
AI’s capabilities in processing vast amounts of data quickly and accurately make it an invaluable tool in detecting and preventing healthcare fraud. By leveraging machine learning and natural language processing, AI systems can identify patterns and anomalies indicative of fraudulent activities, providing real-time alerts and insights to healthcare providers.
8 common examples of voice fraud in healthcare
- Identity theft: Fraudsters use stolen identities to access healthcare services and benefits through deceptive voice phishing techniques.
- Prescription fraud: Unauthorized individuals call in fraudulent prescriptions, often impersonating legitimate healthcare providers.
- False claims and billing fraud: Fraudsters submit false insurance claims using automated voice systems and manipulated recordings.
- Doctor shopping: Individuals use multiple identities and voices to obtain excessive prescriptions for controlled substances from various doctors.
- Insider threats: Healthcare employees misuse access to patient information and manipulate voice systems for personal gain.
- Provider impersonation: Fraudsters pose as healthcare providers to solicit sensitive information or patient payments over the phone.
- Social engineering attacks: Attackers manipulate healthcare staff through voice phishing to gain unauthorized access to systems and data.
- Medical device fraud: Imposters claim to be from medical device companies and offer fake upgrades or repairs to collect personal and financial information.
The advantages of AI in healthcare fraud protection
AI technologies can provide numerous benefits for healthcare fraud protection:
- Enhanced accuracy: AI systems can analyze large datasets, reducing the likelihood of false positives and negatives in fraud detection.
- Real-time detection: AI enables real-time monitoring and detection of fraudulent activities, informing immediate response and mitigation.
- Scalability: AI solutions can scale to accommodate the growing volume of healthcare data, helping to enable consistent protection across the healthcare system.
10 applications of AI in healthcare fraud detection
According to Digital Authority Partners, AI in the health industry was valued at $600 million in 2014, but it’s expected to reach $150 billion by 2026. Here are ten types of fraud detection applications worth investing in:
1. Voice biometrics for patient verification
AI-powered voice biometrics can verify patient identities, helping to prevent unauthorized individuals from accessing healthcare services and benefits. For more, see Pindrop’s research on the future of voice detection.
2. Real-time fraud analysis
AI systems monitor transactions and communications in real time, identifying suspicious activities and alerting relevant authorities.
3. Automated claim processing
AI can automate the processing of insurance claims, reducing the risk of human error and detecting inconsistencies that may indicate fraud.
4. Fraudulent prescription prevention
AI analyzes prescription patterns to identify potential fraud, helping to prevent unauthorized individuals from obtaining controlled substances.
5. Insider threat monitoring
AI systems monitor employee activities, detecting unusual behavior that may indicate insider threats.
6. Social engineering attack detection
AI can identify and flag voice phishing attempts, helping to protect healthcare staff from manipulation.
7. Call authentication for telehealth services
AI can help verify the authenticity of calls in telehealth services, helping to secure communication between patients and healthcare providers.
8. Fraudulent provider impersonation prevention
AI can detect and help prevent attempts by fraudsters to impersonate healthcare providers over the phone.
9. Behavioral biometrics analysis
AI analyzes behavioral patterns to identify fraudulent activities, providing an additional layer of security.
10. Secure communication channels
AI can help secure communication channels between healthcare providers and patients, preventing unauthorized access and data breaches.
The future of AI in healthcare fraud protection
As AI technologies evolve, their applications in healthcare fraud protection will expand. Future developments may include more sophisticated machine learning algorithms, enhanced natural language processing capabilities, and greater integration with existing healthcare systems. These advancements will further improve the accuracy and efficiency of fraud detection, helping combat emerging fraud threats in the healthcare industry.
How to start improving your healthcare fraud protection
To improve your healthcare fraud protection with AI, consider integrating voice biometric analysis, real-time fraud detection, and automated claim processing into your existing systems. Explore the potential of AI to enhance your security measures and better protect your organization from fraud. For more insights and solutions, visit our combat healthcare fraud page and learn more about ways to better protect your healthcare operations.
Truth in the age of AI
With generative AI, bad actors can spoof audio and video of anyone—especially of global leaders. As deepfakes rise, we at Pindrop have been focused on answering the question: is this human or AI?
When an online post has indicators of authenticity, like credible account activity, it can be nearly impossible to tell what’s real and what’s not. A recent deepfake of Elon Musk appears to be a straight-forward cryptocurrency scam, but the aftereffects demonstrate the complexities of deception and forces us to evaluate the importance of information validity. We aim to help solve that problem—and get to the truth—using PindropⓇ Pulse, our audio deepfake detection technology.
How it started
On Tuesday, July 23, 2024 at 10:30 pm ET, members of Pindrop’s research team discovered what appeared to be a live stream of Elon Musk on YouTube. We quickly determined that the live stream was actually a 6-minute 42-second AI-generated audio loop that mimics Elon Musk’s voice, discusses current U.S. politics and the 2024 election, and its potential effects on the future of cryptocurrency. The deepfake then urges the audience to scan a QR code, go to a “secure site,” and “effortlessly double” their cryptocurrency.
At one point, the stream attracted 140K viewers and was live for at least 17 hours.
Why did the scam appear credible?
We’re used to looking for clues that signal authenticity in the media we consume, like verification badges, account activity, and more. But those signals are easily spoofed and can’t always be trusted. Here’s how this scam made itself appear real:
- Account legitimacy: The fraudulent account had a complete profile with a verification badge, 162K subscribers, over 34M total views, and was active since 2022. The account closely resembled Tesla’s official account, so at first glance, viewers may have struggled to spot that the account was fraudulent.
- Reputable speaker: By choosing Elon Musk, a vocal leader in the cryptocurrency space, the fraudsters added a sense of legitimacy to their scam–helping them better trick viewers.
- Staying close to the truth: The statements in the video are similar to previous remarks made by Musk. By repeating or slightly adjusting Musk’s previous remarks, the video likely raised fewer red flags for viewers.
Leveraging liveness detection to spot the deepfake
Our audio deepfake detection technology, PindropⓇ Pulse, generated a segment-by-segment breakdown of a portion of the audio, with real-time results every 4 seconds. Pulse detected segments of synthetic voice in the audio—and concluded that it was likely a deepfake. We reached out to YouTube to report the video and, as of July 24, 2024 at 1:30 pm ET, the account had been taken down.

With our new source attribution technology, Pulse was also able to identify ElevenLabs as the voice cloning vendor that was used to create the deepfake, as we had done previously with the President Biden Robocall incident earlier this year. We reached out to ElevenLabs for them to investigate further.
Defending against deepfakes
Generative AI is powerful—and can be a force for good—but it can also be weaponized to deceive us. When our senses aren’t enough to validate the truth, we need to turn to technology that can assist us. Pindrop Pulse, our advanced audio deepfake detection technology, integrates liveness detection to help distinguish between live and synthetic audio. This technology empowers you with information to assess if what you’re hearing is real—and helps bring trust to the forefront of the media we consume.
- Contact center fraud has grown 60% in the last two years with rising data breaches, ID thefts, account reconnaissance, and now Generative AI.
- Financial institutions continue to see the highest number of fraud attempts, but fraud in the e-commerce sector is growing rapidly.
- Deepfakes are already in contact centers. Fraudsters are testing the waters and learning to scale their attacks.
- The average contact center authentication process has increased from 30 to 46 seconds (+53%) from 2020 to 2023, resulting in higher costs and lower customer satisfaction ratings.
Learn how to navigate the emerging threats in voice security’s evolving landscape and equip your business with robust tools to combat fraudsters and authenticate your customers effectively.
Fraud continues to rise as data breaches and ID thefts show no sign of abating. Dive into Pindrop’s annual contact center fraud research to get to the root of the problem and figure out the best way to protect your brand, consumers and business.
Deepfakes are not new, but they have become particularly treacherous due to advancements in Generative AI. Fraudsters are becoming more equipped to createat creating deepfakes. If not stopped, this could balloon into a $5 billion problem. Read this report to find out how.
Fraud rates in e-commerceEcommerce are 3x more than in financial services and are forecasted to grow by 166% in 2024. Contact centers are at the epicenter of this fraud spike. Find out what you can do to protect yourself and your customers.
Fraud and authentication are two sides of the same coin. While fraud has spiked in the last two years, authentication has become more expensive, costlier, and time-consumingtime consuming. Read this report to discover how you can balance both these challenges effectively.
Paul Carpenter, a New Orleans Street magician, wanted to be famous for fork bending. Instead, he made national headlines on CNN when he got wrapped up in a political scandal involving a fake President Joe Biden robocall sent to more than 20,000 New Hampshire residents urging Democrats not to vote in last month’s primary.
The video and ease with the magician who made it raise concern about the threat of deepfakes and the volume they could be created by anyone in the future. Here are the highlights from the interview and what you should know to protect your company from deepfakes.
Deepfakes can now be made quickly and easily
Carpenter didn’t know how the deepfake he was making would be used. “I’m a magician and a hypnotist. I’m not in the political realm, so I just got thrown into this thing,” says Carpenter. He says he was playing around with AI apps, getting paid a few hundred bucks here and there to make fake recordings. According to text messages shared with CNN, one of those paying was a political operative named Steve Kramer, employed by the Democratic presidential candidate Dean Phillips. Kramer admitted to CNN that he was behind the robocall, and the Phillips campaign cut ties with him, saying they had nothing to do with it.
But this deepfake raised immediate concern over the power of AI from the White House. The call was fake and not recorded by the president or intended for election watchers. For Carpenter, it took 5-10 minutes tops to create it. “I was like, no problem. Send me a script. I will send you a recording, and send me some money,” says Carpenter.
The fake Joe Biden deepfake was distributed within 24-48 hours
The call was also distributed just 24-48 hours before the New Hampshire primary, with little time to stop the intent of the call. Therefore, it could have swayed some people from voting, and it is worrisome to think about when an election is upcoming. When everyone is connected to their devices, it’s hard to intercept fraud in real time. The ability to inject these generative AI into that ecosystem leads some to projects we could be in for something dramatic.
How Pindrop® Pulse works to detect deepfakes
Deepfake expert and Co-Founder and CEO of Pindrop Vijay Balasubramaniyan says there’s no shortage of often free apps that can do it. He’s held various engineering and research roles within Google, Siemens, IBM Research, and Intel before co-creating Pindrop.
“It only requires three seconds of your audio, and you can clone someone’s voice,” says Vijay Balasubramaniyan. At Pindrop, we are testing how quickly you can create an AI voice while leveraging AI to stop it in real time. It’s one of the only companies in today’s market with a product, Pindrop® Pulse, to detect deepfakes, including those zero-day attacks and unseen models, at over 90% accuracy and 99% for previously seen deepfake models. The video featured on CNN of fake Joe Biden took only five minutes of President Biden speaking at any particular event, and that’s what it took to create a clone of his voice.
Pindrop® Pulse is different from the competition
Pulse sets itself apart through real-time liveness detection, continuous assessment, resilience, zero-day attack coverage, and explainability. The explainability part is key as it provides analysis along with results so you can learn from the data in the future to protect your business further. It also provides a liveness score and a reason code with every assessment without dependency on enrolling the speaker’s voice.
Every call is atomically analyzed using fakeprintingTM technology. Last but not least, it’s all fully integrated within the cloud-native capability, eliminating the need for new APIs or system changes.
What your company can do to protect against deepfakes
Pindrop could detect the robocall of fake President Biden’s voice and that it was faked and track down the exact AI company that made it. In today’s environment, AI software detects whether a voice is AI-generated.
It’s only with technology that you could know that it was a deepfake. “You cannot expect a human to do this. You need technology to fight technology, so you need good AI to fight bad AI,” says Vijay Balasubramaniyan. Like magic tricks, AI recordings may not always appear to be what they seem.
Watch the whole segment on CNN to see how easy it is to create a deepfake audio file and how Pindrop® Pulse can help in the future. You’ll see that by adding a voice, these platforms allow you to type whatever you’d like it to say and be able to produce that within minutes. For businesses, it could be as simple as: “I would like to buy a new pair of shoes, but they should be pink,” says Vijay Balasubramaniyan, making it problematic for many businesses to catch fraud going forward. Be sure you plan to detect fraud and protect teams and your company from these mistakes that can happen quickly.
AI continues to attract attention in almost every field. Since the release of ChatGPT, we’ve been caught in a race to introduce AI in every industry possible. However, AI safety has continued to garner a lot of attention, aided in no part by President Biden’s signing of the Executive Order on AI Safety.
For instance, many government agencies use AI to identify healthcare fraud. Previously, they relied primarily on data mining and digital surveillance solutions. However, with advancements in generative AI systems, simply relying on those methods isn’t effective.
What is Healthcare Fraud?
Healthcare fraud is a growing threat, costing billions of dollars annually and jeopardizing patient safety. It’s not just a distant headline – it can impact you directly. This illegal activity bleeds funds away from essential services, inflates healthcare costs, and exposes patients to unnecessary procedures.
Healthcare fraud encompasses a diverse range of deceptive practices perpetrated by various actors within the healthcare ecosystem. These practices can significantly impact financial resources, patient well-being, and trust in the healthcare system.
- Widespread and Diverse: Fraud can occur at any point in the healthcare system, perpetrated by providers, patients, or organized crime rings.
- Deceptive Practices: From billing for fake services to stealing patient identities, fraudsters exploit vulnerabilities to steal money.
- Financial Drain: Billions are lost annually, impacting everyone, from patients to healthcare institutions.
- Compromised Care: Unnecessary procedures and treatments put patients at risk, jeopardizing their health and well-being.
- Erosion of Trust: Fraud undermines public trust in the healthcare system, making it harder to access quality care.
The rise of sophisticated AI tools like voice deepfakes makes traditional fraud detection methods increasingly ineffective. This is where cutting-edge solutions like AI-powered voice biometrics come in.
Why are Traditional Fraud Prevention Systems No Longer as Effective?
Traditional fraud prevention systems, while foundational in protecting against financial and personal data breaches, especially in the healthcare industry, are facing a decline in effectiveness due to several key drawbacks. One of the main issues is the high false positive rates that result in legitimate transactions or activities being erroneously flagged as fraudulent. This problem is compounded by the systems’ limited adaptability; as fraudsters continually update their tactics, traditional systems, reliant on static, rule-based algorithms, struggle to keep pace. These algorithms require manual updates to counteract new fraud patterns, a time-consuming and reactive process. Furthermore, relying on historical data renders these systems less effective against novel or evolving fraud techniques that have not yet been recorded.
Operational challenges also undermine the effectiveness of traditional fraud prevention systems. They demand substantial resources and significant human oversight to monitor alerts, update rules, and conduct investigations. This increases operational costs and diverts staff from other critical tasks within the healthcare sector. Additionally, these systems often employ a one-size-fits-all approach to fraud detection, leading to inefficiencies and inaccuracies in the complex healthcare environment due to the lack of personalized fraud detection strategies.
Moreover, traditional systems are increasingly vulnerable to sophisticated attacks, such as those involving deepfakes or voice synthesis. These advanced techniques, which allow fraudsters to impersonate individuals with high accuracy, pose a significant challenge to systems that lack the capability to analyze unique identifiers, such as voice biometrics. Complicating matters further, companies must navigate the rising concerns related to privacy and compliance. The extensive data collection and monitoring required by traditional fraud prevention systems must be carefully balanced with the need to protect individual privacy and comply with legal standards, such as the Health Insurance Portability and Accountability Act (HIPAA) in the United States.
The Potential for Fraud in Healthcare
The average individual in the US spends a significant amount on healthcare each year. In 2022, US healthcare spending actually grew by 4.1%, with hospital care accounting for almost 30% of that increase.
Phone-based fraud in healthcare is multifaceted, exploiting the trust patients place in the system and their often limited understanding of healthcare services and insurance complexities. This fraud can lead to significant financial losses for patients and healthcare providers, eroding the integrity of the healthcare system. A typical strategy involves impostors impersonating insurance company representatives or healthcare providers, contacting patients to supposedly confirm personal information for billing or medical record updates. Unsuspecting individuals may disclose sensitive information, such as Social Security numbers, Medicare or Medicaid IDs, or credit card details, making them vulnerable to identity theft, unauthorized billing, or other illicit activities.
Another widespread scam involves offering “free” medical services or equipment. Fraudsters contact patients, promising medical devices, prescription drugs, or services at no cost, under the guise that their insurance will cover the expenses. After acquiring patients’ insurance information, they submit fraudulent claims. This defrauds insurance companies and may leave patients responsible for costs related to products or services they never actually received or needed, further highlighting the critical challenge of addressing phone-based fraud within the healthcare sector.
Phishing attacks via phone calls, known as Voice Phising, are also a concern. Callers might pretend to be conducting a survey on behalf of a hospital or a health organization and manipulate individuals into divulging personal health information (PHI) or financial information. This information can later be used for fraudulent schemes or sold on the dark web.
The advent of voice deepfakes and caller ID spoofing has further complicated the landscape of phone-based healthcare fraud. Fraudsters can now more convincingly impersonate officials from trusted institutions, making it harder for individuals to recognize fraudulent calls.
This technology enables scammers to bypass traditional security measures that rely on recognizing known fraudulent numbers or detecting suspicious call patterns. Healthcare providers and insurance companies increasingly turn to advanced technologies such as voice biometrics to combat these types of fraud.
Voice biometric systems analyze the unique characteristics of an individual’s voice to verify their identity, offering a powerful tool against impersonation and unauthorized access. By requiring voice verification for transactions and inquiries conducted over the phone, healthcare organizations can significantly reduce the risk of fraud, ensuring that sensitive information and healthcare services are accessed only by authorized individuals.
How Pindrop Protects Against Healthcare Fraud
Pindrop’s AI-powered voice authentication goes beyond simple identification. While it can verify if a caller is genuine, its core function is to assess the risk of fraud associated with the call.
By analyzing over 1,300 unique characteristics of a caller’s voice and device, Pindrop’s system can detect subtle anomalies that might indicate a fraudulent attempt, such as voice spoofing or other impersonation tactics. This advanced risk assessment helps prevent impostors from gaining access to sensitive patient information or initiating unauthorized transactions, ensuring the security of both patients and healthcare providers.
Interested in learning more about how Pindrop safeguards healthcare interactions? Request a demo today.
Voice biometric authentication systems were neither designed nor operationalized to protect against the sophisticated deepfakes attacking call centers today. It’s paramount that every call center using voice biometric authentication solution should test that their authentication processes can keep out the bad actors using Gen AI. 3Qi, a leading software QA solution provider, helped a Tier 1 US Bank test its call centers against the full variety of deep fakes. In this blog, we share their learnings and best practices from that exercise for everyone to learn from. – The Pindrop Team
3Qi Labs has always prided itself on its ability to break software. Founded as a software QA solutions provider, 3Qi Labs’ mandate has extended beyond executing predefined test plans to proactively seek out the corner cases that could precipitate defects. In software testing, constraints of time and resources dictate the scope, requiring prioritization of features critical to user experience and areas with the highest risk profiles. For Authentication systems, this translates to a balanced mix of Positive* and Negative* tests, with a focus on fraud detection capabilities due to the dramatic increase in biometric fraud incidents.
Among various types of vulnerabilities, the challenge of testing and detecting Synthetic Speech Injection stands out. Although other fraud methods like voice recordings and impersonation remain relevant, the swift spread of deep fakes powered by advancements in Generative AI has made sophisticated synthetic voice technologies easily accessible to a wide audience at minimal cost. This reality places synthetic voice generation and detection at the forefront of our testing efforts for Authentication systems like the Pindrop solution.
The current state of synthetic speech generation technology
In a recent evaluation of Authentication platforms for one of 3Qi’s banking customers, we evaluated multiple AI-driven speech generation platforms. Some of the insights are highlighted below:
- Explosion in Number of Gen AI Systems : Today there are well over 120 Gen AI systems with a combination of text-to-speech and speech-to-speech systems. Among the technologies assessed were Eleven Labs, Descript, Podcastle, PlayHT, and Speechify. These entities, fueled by significant venture capital investments, are positioned to accelerate advancements in this space.
- Easy Accessibility to Sophisticated Attacks: Minimal effort is required to create convincing synthetic voice samples— we used 60 seconds of speech per tester. Not long ago you needed a 30 minute sample to generate an equivalent sample and Microsoft claims its VALL-E model can clone a voice from a 3-second audio clip!
- Potential for Misuse: The efficacy of these technologies was demonstrated by the successful spoofing of multiple Authentication systems using our synthetic samples. It’s no surprise that the Federal Communications Commission (FCC) just outlawed AI generated robocalls, underscoring the need to protect the citizens against misuse of these technologies, especially considering the vast amount of voice data accessible via social media.
- Affordability: These technologies are accessible to a broad audience due to the low cost. Our testing encompassed all the aforementioned platforms for as little as $1 to clone a voice! It’s no wonder that the low barrier to entry for utilizing these advanced technologies has resulted in Deepfake identity fraud doubling from 2022 to Q1 2023.
Best practices for testing systems against synthetic voice
As part of an Authentication platform evaluation, we employ a holistic testing approach covering a broad spectrum of demographic categories, various technologies, a range of input/environmental factors, and synthetic voice injection.
Below is the outline of our approach for a recent evaluation for a Tier-1 US Bank:
- Proctored Sessions: Direct supervision of interactions for thorough scenario coverage and real-time result capture.
- Diverse Scenarios: Employing a mix of positive and negative tests with randomized scenario execution and Net Speech* variation (more below) across testers.
- Enrollment & Verification: Assessing user onboarding and verification efficiency, considering variables such as presence of background noise and speech clarity.
- Security & Fraud Detection: Validating systems against synthetic and replayed voice attacks, as well as distinguishing between different live voices.
- Demographic Representation: A broad participant demographic across age, gender, linguistic, and ethnic backgrounds.
- Technical Infrastructure: Utilizing a variety of mobile devices and networks, with synthetic voice generation facilitated through tools like Eleven Labs.
Central to our testing is the concept of Net Speech*, a critical variable that directly influences the accuracy and reliability of Authentication systems. The amount of net speech provided is positively correlated with the system’s ability to generate a precise voice enrollment and, consequently, its capability to authenticate a caller or detect a fraudulent one. By examining synthetic voice samples of varying lengths, we can identify the specific net speech duration at which synthetic or cloned voices begin to significantly affect false acceptance rates, a key factor in maintaining system integrity and user trust. Thus, net speech serves as a crucial variable in our evaluations, leading us to test across various intervals to determine the optimal net speech requirement per platform. This is vital for minimizing the risk of fraud while promoting a superior customer experience.
The art and science of measuring performance
The cornerstone of our analysis is the evaluation of False Acceptance Rate* (FAR) and False Rejection Rate* (FRR) across diverse data segments. This entails examining these metrics in specific scenarios or variable combinations, such as FAR for Synthetic Voice Injection across different net speech intervals, or even more detailed analyses, like verification FAR for synthetic voice injection among Spanish-speaking females with net speech < 6 seconds. While achieving statistical significance in niche scenarios can be challenging, the juxtaposition of FAR and FRR is critical. It increases the likelihood that the system’s sensitivity is finely tuned to balance robust fraud detection (low FAR) against user convenience (low FRR), essential for optimizing both security and customer experience.
Ultimately, our testing methodologies and processes are designed to arm decision makers with the data they need to objectively evaluate different Authentication platforms. The goal is to not only challenge and evaluate the efficacy of biometric authentication in Authentication systems, but also to ensure that enterprises can confidently integrate these technologies, bolstering both security and user satisfaction.
Glossary:
- Positive Tests: Test cases where the system correctly identifies an authorized user’s voice. They test the system’s reliability and effectiveness in recognizing and verifying authorized users, enhancing user trust and system integrity.
- Negative Tests (aka Spoof Tests): Test cases where the system correctly rejects an unauthorized user’s voice. These tests are essential for assessing the system’s security measures and its capability to safeguard against unauthorized access attempts.
- Net Speech: Net Speech refers to the actual amount of speech content within a voice interaction, excluding any periods of silence or non-speech elements. Optimizing the Net Speech threshold is essential for efficient and secure user authentication. It impacts system responsiveness, user experience, and the ability to accurately authenticate users under various conditions.
- False Acceptance Rate (FAR): The percentage of Negative Tests where unauthorized users are incorrectly verified/recognized as authorized users. This is a key metric for maximizing the security of the system.
- False Rejection Rate (FRR): The percentage of Positive Tests where authorized users are wrongly denied access by the biometric system, mistaking them for unauthorized users. Minimizing FRR is essential for user experience and satisfaction and overall efficiency.
In a groundbreaking development within the 2024 US election cycle, a robocall imitating President Joe Biden was circulated. Several news outlets arrived at the right conclusion that this was an AI-generated audio deepfake that targeted multiple individuals across several US states. However, many mentioned how hard it is to identify the TTS engine used (“It’s nearly impossible to pin down which AI program would have created the audio” – NBC News). This is the challenge we focussed on, and our deep fake analysis suggests that the specific TTS system used was ElevenLabs. Additionally, we showcase how deepfake detection systems work by identifying spectral and temporal deepfake artifacts in this audio. Read further to find out how Pindrop’s real-time deepfake detection detects liveness using a proprietary continuous scoring approach and provides explainability.
Pindrop’s deepfake engine analyzed the 39-second audio clip through a four-stage process: audio filtering & cleansing, feature extraction, breaking the audio into 155 segments of 250 milliseconds each, and continuous scoring all the 155 segments of the audio.
After automatically filtering out the nonspeech frames (e.g., silence, noise, music), we downsampled this audio to an 8 kHz sampling rate, mitigating the influence of wideband artifacts. This replication of end-user listening conditions is crucial for simulating typical phone channel conditions needed for unbiased and authentic analysis. Our system extracts low-level spectro-temporal features, runs through our proprietary deep neural network, and finally outputs an embedding as a “fakeprint.” A fakeprint is a unit-vector low-rank mathematical representation preserving the artifacts that distinguish between machine-generated vs. generic human speech. These fakeprints help make our liveness system explainable. For example, if a deepfake was created using a text-to-speech engine, they allowed us to identify the engine.
Our deepfake detection engine continuously generates scores for each of the 155 segments using our proprietary models that are tested on large and diverse datasets, including data from 122 text-to-speech (TTS) engines and other techniques for generating synthetic speech.
Our analysis of this deepfake audio clip revealed interesting insights explained below:
Liveness Score
Using our proprietary deepfake detection engine, we assigned ‘liveness’ scores to each segment, ranging from 0 (synthetic) to 1.0 (authentic). The liveness scores of this Biden robocall consistently indicated an artificial voice. The score fell below the liveness threshold of 0.3 after the first 2 seconds and stayed there for the rest of the call, clearly identifying it as a deepfake.
Liveness analysis of President Biden robocall audio
TTS system revealed
Explainability is extremely important in deepfake detection systems. Using our fakeprints, we analyzed President Biden’s audio against the 122 TTS systems typically used for deepfakes. Pindrop’s deepfake detection engine found, with a 99% likelihood, that this deepfake is created using ElevenLabs or a TTS system using similar components. We ensured that this result doesn’t have an overfitting or a bias problem by following research best practices. Once we narrowed down the TTS system used here to ElevenLabs, we then validated it using the ElevenLabs SpeechAI Classifier, and we obtained the result that it is likely that this audio file was generated with ElevenLabs (84% likely probability). Even though the attackers used ElevenLabs this time, it is likely to be a different Generative AI system in future attacks, and hence it is imperative that there are enough safeguards available in these tools to prevent nefarious use. It is great that some Generative AI systems like ElevenLabs are already down this path by offering “deepfake classifiers” as part of their offerings. However, we suggest that they also ensure the consent for creating a voice clone is actually coming from a real human.
Prior to determining the TTS system used, we first determined that this robocall was created using a text-to-speech engine, implying that this was not simply an instance of a person changing their voice to sound like President Biden using a speech-to-speech system. Our analysis also confirms that the voice clone of the President was generated using text as input.
Deepfake artifacts
As we analyzed each segment of the audio clip, we plotted the intensity of the deepfake in each segment as the call progressed. This plot, depicted below, shows that some audio parts have more deepfake artifacts than others. This is the case for phrases like “New Hampshire presidential preference primary”, or “Your vote makes a difference in November.” This is because these phrases are rich with fricatives, like in the words “preference” or “difference,” which tend to be strong spectral identifiers for deepfakes. Additionally, we saw the intensity rise when there were phrases that President Biden is unlikely to have ever said before. For example, there were a lot of deepfake artifacts in the phrase: “If you would like to be removed from future calls, please press two now”. Conversely, phrases that President Biden has used before showed low intensity. For example, the phrase “What a bunch of malarkey”. This is something we understand President Biden uses a lot.
Protecting trust in public information & media
In summary, the 2024 Joe Biden deepfake robocall incident emphasizes the urgency of distinguishing real from AI-generated voices. Pindrop’s advanced methods identified this deepfake and its use of a text-to-speech engine, highlighting scalability.
Companies addressing deepfake misinformation should consider criteria like continuous content assessment, adaptability to acoustics, analytical explainability, linguistic coverage, and real-time performance when choosing detection solutions.
Acknowledgements:
This work was carried out by the exceptional Pindrop research team.
Partner with Pindrop to defend against AI-driven misinformation. Contact us here for a custom demo.




Test your knowledge about deepfakes
See live demonstrations of Pindrop’s new liveness detection feature
Get your burning authentication and deepfake questions answered
So… are you smarter than a 5th generation deepfake engine?
Meet the Experts


Bryce McWhorter
Sr. Director, Product, Research & Engineering, Pindrop


Tara Garnett
Sr. Product Manager, Authentication Products, Pindrop


Darren Baldwin
Sr. Director and Account Executive, Pindrop
- Learn more about consumer concerns regarding deepfakes and synthetic voice, with 90% od surveyors expressing worry.
- Discover which industries face the highest levels of concern about deepfake risks.
- Explore how pop culture influences AI sentiment and strategies to combat deepfakes effectively




Discover the security risks posed by AI and vulnerabilities we need to address.
Learn how and why is the deepfake risk more pronounced now, than before.
Explore insights into the ever-evolving world of AI and how the risk outlook will evolve in the future.
Your expert panel


Amit Gupta
VP, Product Management, Research and Engineering Pindrop
Sen. Blumenthal made his own deepfake. Pindrop caught it. Could you?
As AI technology continues to advance, the rise of deepfakes poses a serious threat. These manipulated images, videos, and audios use artificial intelligence to create convincing but false representations of people and events. Unfortunately, deepfakes are often difficult for the average person to detect. In fact, in a Pindrop study people were able to identify a deepfake with 57% accuracy, only 7% better than a coin toss. While some instances of deepfakes may be lighthearted, like the Pope in the white puffer coat or AI-generated music from Drake and The Weeknd, their existence creates doubt about the authenticity of legitimate evidence. Oftentimes, emerging technologies such as AI chatbots may be used for good causes but it usually does not take long before they trigger the creative minds of the fraudsters. Criminals can take are taking advantage of these disruptive forces to conduct misinformation campaigns, commit fraud, and obstruct justice.
In response, deepfake detection technologies must evolve quickly as well. But more importantly, organizations that rely on voice verification need to ensure that they review their security strategy and adopt a defense-in-depth approach. With focus on voice security since its inception, Pindrop has been creating deepfake detection technologies since 2014. Pindrop’s liveness detection provides protection not only for synthetically generated speech (text-to-speech, voicebot, voice modulation, voice conversion), but also for recorded voice replay attacks and modulated voices. The good news is that machines are much better at detecting the fakes compared to humans.
At Pindrop Labs, researchers never cease to improve every part of our fraud detection and authentication suites, where knowing if the speech is produced by a real live speaker is crucial. An important component of this work is to continuously look out for real-world examples where voice-based attacks have been used to validate our detection engines. One recent example that resonated through the media in the past week is Senator Blumenthal’s opening remarks at the Senate hearing on AI. Senator Blumenthal began by speaking with his own voice and eventually switched to a deepfake impersonation of his voice. We used our liveness detection engine to detect the veracity of the voice. In the video below you will see the results of our analysis with a real-time liveness score and the corresponding decision of whether the voice is real or fake (a low score indicates a deepfake while a high score indicates a live voice). What is particularly interesting here is not only the ability to correctly identify where speech is synthesized but also where it is not and to do so in real-time.
There’s been interesting news recently on conversational AI bots being utilized by platforms like DoNotPay, Ideta, or the sassier Jolly Roger Telephone Co. These services can be used to call inbound to a contact center to dispute bills and charges, engage with customers, or even frustrate telemarketers on your behalf. While this is exciting for us everyday folk who don’t want to sit on hold to argue with a customer service provider or are excited by the idea of increasing a fraudster’s blood pressure when they try to bug you for personal information, this is incredibly frightening for the business world, especially banks. Why? While these new developments are often used for good, they can also be used with negative intentions. Fraudsters are never far behind (and usually ahead) when it comes to new techniques, technologies, and finding the best way to do more with less.
What does this mean for contact centers?
The connection of conversational AI, deepfakes or synthetic voice technology, and the processing power for real-time interactions is finally here. While it seems this trifecta was first created for legitimate purposes—like customer advocacy in the form of a “robot lawyer”—its existence signals a new age for fraudulent activity. Instead of spending time calling in themselves, fraudsters could instead utilize synthetic voices and conversational bots to interact with IVRs and even human agents. That means more activity, more chances for success, with less effort on their part.
What can contact centers do about it?
Contact centers can take comfort in knowing there are ways to get ahead of these activities. Take a look at this quick and easy checklist to deploy simple strategies that can help:
- Agent education – Make sure your agents are aware of the potential for conversation with digital voices that sound more real than they have experienced in the past.
- Utilize callbacks – If a caller voice is suspicious and sounds synthetic, consider implementing a callback process where the call is ended and an outbound call is made to the account owner for direct confirmation.
- Leverage multifactor fraud and authentication solutions – You can leverage other factors, like call metadata for caller ID verification, digital tone analysis for device detection, and keypress analysis for behavior detection, or even utilize OTPs (although we know those aren’t as secure these days).
If you’re already a Pindrop customer, you’re in luck! Leveraging negative voice profiling, Phoneprinting® technology, and voice mismatch in your fraud and authentication policies is a great way to get ahead of these bots taking advantage of your contact center. Make sure to reach out to your customer success representative to help evaluate your implementation and enable the right features for cases like this.
Pindrop is also acutely aware of the rising problem with synthetic and deepfake voices. The availability of conversational AI and increasingly better deepfakes, combined with an abundance of processing capacity is a dangerous combination for fraudsters wanting to leverage these solutions for nefarious purposes, especially account takeovers. Our research team is already solutioning for deepfake detection and we can’t wait to leverage this kind of technology within our portfolio of innovations.
Interested in learning more? Contact us today.
How to Reduce Bias: Optimizing AI and Machine Learning For Contact Centers
Bias exists everywhere in our society. And while some biases are largely harmless, like a child’s bias towards one food vs the other due to exposure, others are quite destructive. Impacting our society negatively and often resulting in deaths, dispassionate laws, and discrimination. But what happens when the biases that exist in the physical world are hardcoded into the digital? The rise and adoption of artificial intelligence for decision making has already caused alarm in some communities as the impacts of digital-bias play out in front of them every day. In addition, the current events and trends pushing the U.S. and the world towards “anti-racism” stances and equity regardless of skin color, raises concerns about how societal biases can influence AI, what that means for already marginalized communities, and what companies should be doing to ensure equity in service and offerings to consumers.
It’s no news that Artificial Intelligence and Machine Learning are vulnerable to the biases held by the persons that program them1. But, how does bias impact the quality and integrity of the technologies, processes, and more that rely on AI and ML? Covid-19 has hastened the move towards employing these technologies in healthcare, media, and across industries to accommodate for shifts in consumer behavior; new restrictions in the number of personnel allowed in one car, room, or office.
For contact center professionals concerned with ensuring business continuity, improving customer experience, or increasing capacity, the application of AI and ML during these early phases of restructuring due to the pandemic relates to the expansion of capacity, improvement of customer service, and reduced fraud and operational costs. Understanding the consequences of adopting inherently biased AI or ML technologies meant to protect you; the possible impact on your business is necessary as we traverse toward a “new normal”2 where technology fills the 6ft gap in our society and where fairness and equity will be expected for everyone.
This post discusses bias in artificial intelligence and machine learning reviews the threats to your business this bias causes and presents you with actionable considerations for you to discuss with your team when searching for a contact center anti-fraud or authentication solution.
What is Bias, and Why Does it Matter in Technology?
Bias in artificial intelligence and machine learning can be summarized as the utilization of bad data to teach the machine and thus inform the intelligence. In short, ML bias becomes AI bias through the input and presence of weak data that inform decisions and the encoding of biases based on the thought processes of developers – manifesting themselves in algorithmic and societal biases. The inaccuracies caused by these biases can erode trust between the technology and its human users as it is less reliable3. For you, this means less trust, loyalty, and affinity associated with you by consumers.
Algorithmic Bias
Includes the aforementioned bad data and is present in many data sets in 1 of 2 ways.
Selection Bias – What data is used to train the machine
This occurs when the data used to train the algorithm over-represents one population, making it operate better for them at the expense of others4.
For contact centers, a real-world example could be gleaned from AI improperly trained on international calls. For many contact centers, the majority of calls may be domestic- not giving the algorithm enough data relating to international calls may cause bias wherein international calls are flagged for fraud and rerouted to an analyst vs a customer service agent.
Interaction Bias – How the machine is trained
Additionally, the machine has to be trained, taught to make a decision. Developers bias algorithms with the ways they interact with them. For example, if we define something as “fraud” for the machine and teach it that fraud only “looks” one way – with biased inputs, it recognizes fraud committed- as long as it matches the narrow definition it has learned. Combined with selection bias, this results in machines making decisions that are slanted towards one population, while ignoring others3. For a call center professional concerned with fraud mitigation, a real-world form of this bias is an AI systematically ignoring costly fraudster activity and instead focusing on genuine caller behavior and flagging it as suspicious or fraudulent because it doesn’t “fit” the criteria for fraud that the machine has learned.
When choosing a solution for your contact center- you should ask about the diversity and depth of the data being fed to the machine and how it learns over time. Though no solution is infallible, Pindrop works to reduce bias in our AI by making sure that voiceprints are user-specific instead of a generalization based on a large population of persons with similar features, like an accent. Feeding the machine “truth” gives the machine a more diverse dataset, reducing algorithmic bias.
Societal Bias
It is not as quickly defined, tested for, nor resolved4.
Latent Bias
This occurs when an algorithm is taught to identify something based on historical data and often stereotypes. An example of this would be an AI determining that someone is not a doctor because they are male. This is due to the historical preponderance of stock imagery featuring male doctors versus those featuring female ones5. The AI is not sexist, the machine has learned over and over that males in lab coats with glasses and badges are doctors; that women can be or should be ignored for this possibility. Pindrop addresses societal bias by developing them using diverse teams. The best applications of AI are those that also include human input. Diversifying human interaction with the machine, the data it is fed, and modeling it is given, strengthens our AI against bias.
How Can Biased Tech Impact My Business?
Customer Service
Biased solutions could erroneously flag callers as fraudulent. Ruining customer experiences and causing attrition as customers’ issues take longer to resolve, ultimately costing you monetarily and in brand reputation. An example of this is contact center authentication solutions that use geographic location as a primary indicator of risk. A person merely placing a phone call as they drive could be penalized. Even worse, persons living in “risky” neighborhoods are at the mercy of their neighbors’ criminal activity, as biased tech could flag zip codes and unfairly lock out entire populations. Pindrop’s commitment to reducing bias addresses this impact to customer service using the diverse data sets mentioned above and by applying more complex models for learning. The result is no-one group is more likely than the other to be flagged as fraudulent, suspicious, or otherwise risky. For you, that means less angry callers and false positives overall.
Fraud Costs
As biases can be restrictive for some, locking customers out, other biases coded into your contact center antifraud or authentication solution can allow more fraud through as it makes certain assumptions. For example, for years6 data has pointed towards iPhone users being more affluent than Android users. For contact center professionals, should your solution make assumptions that wealthier consumers are more trustworthy than working-class persons, it may lower the score of fraudsters on iPhone, possibly allowing the perpetrators into accounts and systems while over penalizing Android users. Though Pindrop is not immune to bias – no solution is – we can greatly reduce the AI biases that can increase fraud costs unintentionally, through our approach to developing AI.
Contact Center Operations
Lastly, a biased solution could cost you in productivity and operational costs. The two examples above can quickly impact your productivity, costing you more per call. AI biases could cause you to implement step-up authentication for genuine callers and flag accounts exhibiting normal behavior as ‘suspicious’ because of an encoded logarithmic or societal bias.
Solutions like Pindrop’s single platform solutions for contact center security help improve customer experience, reduce fraud costs, and optimize contact center operations by developing proprietary AI that learns from diverse and purely fact-based input—eliminating bias in the AI.
How to Remove Bias from Your Contact Center AI
Bias enters AI and ML via corrupt data practices but also from the way the solutions are built5. But there are ways to address the builders’ biases and shield the solution from the input of “bad” data. In this section, there are 3 core principles to remember when searching for a solution employing AI or machine learning.
3 Core Principles of Bias-Free AI
Now that you understand how a biased AI can impact your business, you should consider 3 core principles when searching for a solution to serve your contact center. Your ideal solution should:
Have diverse, varied, and fact-based inputs
Diverse, varied, and fact-based inputs address selection bias and ensure that all populations are sampled and therefore considered in calculations that become decisions. For example,
Understand Garbage In, Garbage Out
Question your solutions’ data inputs. Utilizing outdated concepts, naming conventions, and more influences your machine to make decisions that are prejudiced against specific population segments. Understanding the data inputs and freshness of the data ingested by your solution helps fight against latent bias in AI. For example, earlier in this post we discussed latent bias. This kind of bias is based on societal norms or rather accepted societal behaviors at the time. With that in mind, think of an engine deciding college admissions, based on the admissions of the past 60 years. It’s 2020 – in 1960 many public and private schools where still racially segregated. In the event that this data is fed to the engine, it will most certainly weigh an applicant’s race negatively.
Everyone Has Biases
The goal should be neutrality, and diverse views bring us closer to an optimal state of development. By combining varied voices, thought processes, and capabilities from diverse groups of developers, an AI could be created with such diverse and varied inputs, that it learns to operate outside of the conflicting biases of its makers. For example, above, we explained how societal influences – even those no longer widely accepted – could impact AI’s decisions. Should the AI ingest historic information polluted with outdated thought processes, naming conventions, and other latent biases but is also fed fresh, diverse data by diverse humans, it will gain via feedback from the humans, deep learnings to help it make more nuanced, accurate, and less biased decisions.
When considering an AI-powered solution for the protection of your contact center and customers, understanding bias in AI and ML, how it impacts your business, and what you can do about it ultimately saves you time, reduce costs, and hardens your contact center to attack.
Pindrop’s single-platform solutions for the contact center can help you address challenges in fraud mitigation and identity verification. These solutions are fed fact-based inputs, follow proprietary data collection and analysis processes, and are built by diverse and capable teams to help eliminate bias from our software. Contact us today to see it in action, or learn more from our resource pages.
IEEE, Spectrum. “Full Page Reload.” IEEE Spectrum: Technology, Engineering, and Science News, 2019, spectrum.ieee.org/tech-talk/tech-history/dawn-of-electronics/untold-history-of-ai-the-birth-of-machine-bias.
Radfar, Cyrus. “Bias in AI: A Problem Recognized but Still Unresolved.” TechCrunch, TechCrunch, 25 July 2019, techcrunch.com/2019/07/25/bias-in-ai-a-problem-recognized-but-still-unresolved/.
Howard, Ayanna, and Jason Borenstein. “AI, Robots, and Ethics in the Age of COVID-19: Ayanna Howard and Jason Borenstein.” MIT Sloan Management Review, 12 May 2020, sloanreview.mit.edu/article/ai-robots-and-ethics-in-the-age-of-covid-19/.
Gershgorn, Dave. “Google Explains How Artificial Intelligence Becomes Biased against Women and Minorities.” Quartz, Quartz, 28 Aug. 2017, qz.com/1064035/google-goog-explains-how-artificial-intelligence-becomes-biased-against-women-and-minorities/.
Hao, Karen. “This Is How AI Bias Really Happens-and Why It’s so Hard to Fix.” MIT Technology Review, MIT Technology Review, 2 Apr. 2020, www.technologyreview.com/2019/02/04/137602/this-is-how-ai-bias-really-happensand-why-its-so-hard-to-fix/.
Yahoo Finance. “These Maps Show That Android Is For Poor People.” Yahoo! Finance, Yahoo!, 4 Apr. 2014, finance.yahoo.com/news/maps-show-android-poor-people-000200949.html.