The Burden of Detecting Deepfakes Shouldn't Fall on Your Employees

A contact center agent answers a call from what sounds like a trusted partner. The caller references prior conversations, uses the right terminology, and speaks with confidence. Nothing feels obviously wrong. The agent's decision about what to share or approve depends largely on what they hear.

In this case, the outcome hinges on human perception. The decision depends on what someone sees or hears, and on the long-standing assumption that those signals reflect reality. That assumption no longer holds by default.

The question isn't whether AI-generated voices and faces exist. They do, and they're improving rapidly. The important question is whether organizations and institutions have accounted for the fact that these systems now operate directly within human workflows. They influence customer service interactions, hiring decisions, access controls, financial approvals, public communications, and personal relationships. They're embedded in everyday decision-making.

This is where the ethical and operational stakes rise. AI-generated and manipulated media exploit trust. Employees, managers, customers, and citizens are now expected to make authenticity judgments in environments where synthetic media can be highly convincing and produced at scale.

Every high-stakes voice or video interaction ends with a human decision

A contact center agent decides whether to reset an account. A recruiter decides whether a candidate is credible. A finance team member decides whether to approve a transfer. Human judgment has always been the final control in these workflows, and for most of the history of enterprise security, that was a reasonable place to put it.

What's changed is the quality of the inputs feeding that judgment.

Synthetic voices can replicate a known caller with enough fidelity to pass a biometric check. Manipulated video can place a convincing face on a screen during a live interview. Generated personas can carry supporting documentation, online profiles, and work histories that hold up to casual scrutiny. The person making the decision is exercising the same judgment they always have. The difference is that the information in front of them can now be fabricated.

The OECD AI Principles recognize this directly, calling on AI actors to implement controls that address risks arising from uses outside the intended purpose, including maintaining the capacity for meaningful human oversight.

The NIST AI Risk Management Framework identifies identity, security, and human decision-making under time pressure as contexts where AI systems can cause material harm, and emphasizes the need to manage misuse in operational environments.

Partnership on AI's framework notes that synthetic media is often highly realistic, wouldn't be identifiable as synthetic to the average person, and that as the technology becomes more accessible, its potential impact increases.

Traditional fraud exploits technical weaknesses. AI impersonation exploits trust. Human judgment now operates on inputs that can't be verified by the person receiving them. The judgment is the same as it always was. What's changed is the inputs feeding it.

The human cost to employees and consumers

When employees encounter AI-enabled deception, they're asked to make judgment calls. Is that really the executive on the line? Is that candidate who they appear to be? Every interaction carries a new layer of uncertainty. Over time, that uncertainty wears people down. It creates mental strain and a persistent worry about making the wrong call.

Customers experience something similar, but from the outside in. When they can no longer be confident that a voice, a video message, or a support call is genuine, their trust begins to fray. Institutions start to feel less reliable, not because every interaction is fraudulent, but because of the possibility. That possibility is enough.

When organizations leave the detection gap open, the risk lands on the workforce.

Deepfake detection can't be solved with awareness training

Training addresses human error. Synthetic media isn't human error. It's a machine specifically designed to defeat human perception. Those are fundamentally different problems, and they require fundamentally different responses.

Telling a contact center agent to listen more carefully, or a recruiter to look more closely at a video feed, doesn't address the problem. It reassigns it. The agent and the recruiter are already doing what human perception allows. The issue is that generative systems are built to operate within the limits of what humans can detect, and to improve as those limits are tested.

NIST's AI Risk Management Framework is direct on this point. Transferring unacceptable risk to operators or end users isn't a risk management strategy. It's a failure of one. When an organization responds to AI impersonation risk by training frontline staff to spot it, the organization places the risk on the people least equipped to absorb it and least responsible for the systems that created it. Transferring the risk is not the same as managing it.

IEEE guidance on accountability reinforces the same principle. Responsibility for AI-related outcomes must be traceable and clearly assigned across designers, deployers, and organizations. It can't be offloaded onto the individuals operating inside those systems.

Expecting individuals to reliably detect increasingly sophisticated synthetic media without systemic support isn't risk management. It's an organization choosing not to act, and framing that choice as a training program.

Ethics frameworks place the responsibility on organizations, not individuals

Leading AI ethics frameworks treat human dignity and freedom from manipulation as core governance requirements, not aspirational principles. This is where that requirement becomes operational. The question isn't whether an organization values those commitments. The question is whether the workflows it runs reflect them.

IEEE's standards call for accountability mechanisms that ensure AI-related harms can be traced back to responsible parties within an organization, rather than absorbed by the individuals operating those systems.

When a contact center agent makes an access decision based on a voice that turns out to be synthetic, and no detection system was in place to flag it, the accountability gap isn't the agent's failure. It's an organizational one.

The IEEE framing makes that explicit. If the system doesn't support a responsible outcome, the organization that designed the system bears the responsibility.

The OECD AI Principle 1.3 calls for AI systems to be robust and for risks to be managed appropriately to the context. In workflows where a single interaction can authorize a transaction or reset credentials, human judgment alone doesn't meet that standard. In workflows where a single interaction can authorize a transaction, reset credentials, or grant system access, leaving detection to human judgment fails the standard. There's no proportionate control in place.

Deepfake detection built into live workflows is the operational expression of commitments an organization has already made on paper. If the ethics framework says people should be protected from manipulation, and the workflow leaves them exposed to it, the gap between policy and practice is the risk.

Detection belongs inside the workflow, not on top of it

Deepfake detection has to verify authenticity at the input level, at the moment trust is exercised, rather than leaving individuals to absorb it downstream. Embedded detection runs inside communication channels, identity workflows, and contact center systems.

Consider the changes for a contact center agent when detection runs within the call platform rather than in a separate tool. The agent doesn't need to make a judgment call about whether the voice on the line sounds off, because the system flags the signal before the conversation reaches a decision point. RealCall analyzes inbound audio in real time inside Genesys Audiohook, returning a Trust, Suspicious, or Manipulated verdict before the call reaches an agent. The agent doesn't carry the risk. The workflow does.

That shift matters because the pressure in a live interaction is real. An agent handling hundreds of calls processes each one quickly, assuming the caller is who they say they are. Built-in detection in the call flow gives the agent something human perception alone can't provide. It gives them a moment to pause. That pause can be enough to stop a rushed approval, credential reset, or account transfer from proceeding before the legitimacy of the interaction has been confirmed. In that space, verification becomes intentional, and a minor flag is caught before it becomes a significant incident.

The same principle applies across hiring interviews, identity onboarding sessions, and executive communications. When the detection layer sits inside the platform where the interaction happens, the person conducting the interaction is no longer the last line of defense.

ISO/IEC AI standards establish expectations that AI-related risks be identified, mitigated, monitored, and governed across the system lifecycle. Building detection into operational workflows, with defined escalation paths, evidence preservation, and integration into incident response processes, is how those governance expectations translate into practice.

Protecting people is now the ethical standard

Employees and customers shouldn't be left to navigate sophisticated deception on their own. Organizations should take responsibility for the risks posed by AI-enabled impersonation.

Taking responsibility means protecting employees from being placed in impossible judgment scenarios where they must decide, in real time, whether a voice or a face is authentic. It means protecting consumers from invisible impersonation that erodes confidence in legitimate communication. It means designing systems that assume deception will occur and absorb that risk institutionally rather than pushing it onto individuals.

If businesses continue to treat deepfakes as isolated fraud incidents rather than a structural shift in how trust works, they place the burden on individuals to detect what machines are designed to obscure.

Organizations that take education seriously, embed protective controls, and align detection with operational workflows will preserve trust. Those that rely on instinct alone will continue to expose their workforce and customers to escalating risk.

When deception becomes automated, protection has to become institutional.

---

FAQ

Can employees be trained to detect deepfakes?

Awareness training addresses human error, but synthetic media isn't human error. It's a machine designed to defeat human perception. Generative systems operate within the limits of what humans can detect and improve as those limits are tested. Training reassigns the risk to frontline workers without reducing it.

Where should deepfake detection live in an enterprise workflow?

Detection has to verify authenticity at the input level, inside the platforms where the interaction happens. For contact centers, that means inside the call platform itself. For video meetings, inside the meeting tool. For media uploads, inside the upload pipeline. Detection layered on top of the workflow doesn't reach the moment of decision.

What ethics frameworks address AI impersonation risk?

The OECD AI Principles, NIST AI Risk Management Framework, IEEE accountability standards, and ISO/IEC AI standards all address it. Each places responsibility for AI-related harms on the organizations that deploy AI systems, rather than on the individuals operating inside those workflows.

How does embedded deepfake detection protect employees?

When detection runs inside the platform where the interaction happens, the agent or interviewer doesn't have to make a real-time judgment about whether a voice or face is authentic. The system flags the signal before the conversation reaches a decision point. The workflow carries the risk instead of the person.