When seeing is no longer believing: The identity crisis of the AI era

Generative AI is challenging digital identity as deepfakes, cloned voices and synthetic identities force businesses to rethink how they verify trust, writes Shobona Sruthi Mohan from ManageEngine.

“What you see may be false, what you hear may be false, and only a thorough investigation reveals the truth.”

This old Tamil saying takes on a new meaning in the age of generative AI. For years, we have treated the physical presence as evidence. A familiar voice on the phone, a photograph, a video call or a signed document – all of which give us something tangible to anchor our trust in. Even online, where identity has always been somewhat abstract, these signals help us to believe that the person on the other side is real. But this assumption is becoming harder to defend.

The problem with generative AI is not simply that it can create convincing fake content. It’s that it can reproduce the very signals we have traditionally used to establish identity. A face can be generated. A voice can be cloned. A video interaction can be manipulated in real time. Documents and digital personas can be constructed around a person who does not exist. The challenge is not spotting a deepfake or AI-generated content. It is whether the evidence we use to establish trust can still be relied upon.

The face and voice are no longer proof

A 2026 systematic review of 36 studies involving more than 13,000 participants found that people identified real and deepfake faces with only 56.1% accuracy overall. Training improved performance, but only to 62.2%. AI is challenging these signals. The irony is that we spent years making digital interactions feel more human. We now have to question whether appearing human is still a useful measure of authenticity.

But the more consequential development is not simply the ability to impersonate an existing person. It is the ability to construct an identity from scratch. Fraudsters can increasingly assemble identities using a combination of genuine and fabricated information.

👉 No tech background needed to bring your business into the AI age! Secure your Google AI Professional Certificate to complete the certification at no cost. Apply now and get started!

According to the 2026 Synthetic Identity Fraud Statistics report by BIIA, synthetic identities are estimated to drive up to 80% of incidents of new account fraud. In addition, losses to businesses are expected to exceed $23 billion by 2030.

A generated photograph, fabricated personal details, manipulated documents, a cloned voice and a convincing digital footprint can collectively create a person who appears legitimate across multiple touchpoints.

The bigger problem is the trust deficit

This is where the identity problem becomes bigger than cybersecurity. Imagine a CFO receiving a video call from the CEO asking for an urgent transfer. The face is right. The voice is right. The mannerisms are right. Yet none of those things independently prove identity. Now imagine the reverse: a genuine executive makes an unusual request, but employees dismiss it because it could be a scam. As AI gets better at creating believable identities, believability stops being reliable evidence of authenticity.

This creates a trust deficit. Genuine interactions can become easier to deny, while fraudulent ones become easier to believe. The World Economic Forum article in 2026 has similarly warned that synthetic media can make it harder to distinguish fact from fiction and that even awareness of deepfakes can cause people to doubt genuine information.

For businesses, this could quietly undermine one of the most important foundations of digital operations: confidence that the person, instruction or transaction is what it appears to be.

Identity needs to move beyond what we can see

The answer cannot simply be better human detection. Organisations need to move identity assurance beyond appearance and treat the face or voice as one signal among several. Strong authentication, device identity, trusted communication channels and behavioural context can provide independent evidence that an interaction is genuine.
A request from an unfamiliar device, at an unusual time or involving an unexpected transaction can be assessed against established patterns before it is approved. The same principle applies to digital content: provenance and authenticity mechanisms can help establish where a photograph, document, video or audio recording came from and whether it has been altered.

Trust has to be built into the process

Technology also needs to be reinforced by process. High-impact actions such as financial transfers, credential changes and privileged access should require verification through established channels, rather than relying on a convincing call or message. Employees do not necessarily need to become experts at spotting synthetic media; they need to know when an interaction requires independent verification.

Identity assurance, in other words, needs to become layered, combining authentication; device and behavioural context; provenance; access controls and human verification where appropriate. AI is not necessarily destroying digital identity. It is exposing how much of our understanding of identity was built on signals that were difficult to reproduce. Seeing used to be believing. In the AI era, trust may become something we have to verify, not something we can simply see.

Image at top: Photo by Tommy van Kessel on Unsplash