Separate detection claims from evidence you can test in your own systems

A security team watches a vendor flag a synthetic voice in a demo. Before choosing a platform, the team needs to know what that result proves for its own calls, meetings, or uploads. These five questions turn deepfake detection claims into evidence a buyer can check.

Deepfake detection analyzes audio, video, and images for evidence of AI generation or manipulation. The result informs a decision alongside identity checks, request context, and your policies.

Ask every vendor on your shortlist, including Reality Defender.

1. Do you sell the system you're grading?
2. Are you telling me it's live, or telling me it's real?
3. Which modalities can you evaluate in one platform, at the same moment?
4. What do you store, and what happens when you're breached?
5. How fast do you retest against new generators, and will you show me the results?

1. Do you sell the system you're grading?

Ask whether the vendor specializes in detection, sells it within an identity platform, or sources it from another provider. Then establish who evaluates detection and whether those results stand apart from the wider approval decision.

A combined platform may be the right fit. Its detection still needs evidence you can assess separately from identity-matching or onboarding pass rates.

A specialist's focus doesn't establish independent evaluation either.

A weak answer substitutes the wider platform's approval rate for evidence about its detection.

A strong answer names the detection provider, distinguishes internal from independent testing, and shares detection results with their test conditions and limitations.

Media analysis should strengthen existing identity and fraud controls without blurring what each control establishes.

2. Are you telling me it's live, or telling me it's real?

A moving face in an onboarding session doesn't settle whether the video was manipulated, so ask for separate evidence of liveness and media authenticity.

Liveness checks seek evidence of a living subject at capture, while deepfake detection examines the media for AI generation or manipulation. Ask the vendor to demonstrate each capability separately.

In its September 2023 presentation-attack evaluation, the National Institute of Standards and Technology (NIST) excluded digital injection attacks. Confirm the scope of any test offered as proof.

A weak answer treats a liveness result as proof that the media hasn't been AI-generated or manipulated.

A strong answer distinguishes the checks, documents the attacks tested, and explains how each result informs your workflow.

3. Which modalities can you evaluate in one platform, at the same moment?

A meeting participant turns their camera off. Audio continues, so the evaluation needs to show which checks still run and how the platform reports the missing video.

Confirm which media the product supports through your proposed connection. Ask which checks run concurrently, which run in sequence, and how the platform combines their outputs. Different uses need different inputs.

Record when media first reaches the system, how much it needs, and when your application receives a result. Test at expected traffic and media quality.

A weak answer lists supported media without showing what runs together or how the output changes when an input disappears.

A strong answer shows which checks run together, how their results combine, and how long your system waits for an answer. It also explains what happens when a stream disappears or the evidence isn't conclusive.

4. What do you store, and what happens when you're breached?

A security reviewer needs to trace what leaves an onboarding session, what's left after analysis, and who can access it.

Distinguish enrollment from retention. A system can avoid reference voiceprints and still retain submitted media, results, or operational logs, so ask about each data type separately.

Get the retention, deletion, and access terms for each data type in writing. Confirm whether the vendor uses submitted media for training and how it'll notify you of an incident.

Confirm which data your on-premises or air-gapped deployment stores and who controls it. NIST's January 2020 Privacy Framework addresses retention, deletion, and breach notification.

A weak answer gives a broad privacy assurance without explaining what happens to uploads, results, or logs.

A strong answer documents the data flow, retention and deletion terms, access controls, and incident responsibilities for your deployment.

5. How fast do you retest against new generators, and will you show me the results?

Test generators the detector hasn't trained on and media compressed by the systems you'll use.

Detection is a continuous discipline. Ask how long the vendor took to test a newly released generator and what the results showed.

Review missed detections and false alarms at the same threshold, on named datasets that include media reflecting your production conditions.

NIST's Guardians of Forensic Evidence program describes testing older and newer generation methods and media altered by compression.

A weak answer promises frequent retraining without showing whether the update improved detection or introduced regressions.

A strong answer gives the testing cadence, update triggers, model version, and dated results, including regression checks you can review.

How should the evaluation change by use case?

Choose representative media and define the action each result should trigger before comparing vendors. Set error tolerances and recordkeeping requirements with the people who own that decision.

Use the same standard for Reality Defender. Ask for evidence that matches the deployment you're considering and the media you'll analyze.