Indian Voters Are Being Bombarded With Millions of Deepfakes. Political Candidates Approve

On a stifling April afternoon in Ajmer, in the Indian state of Rajasthan, local politician Shakti Singh Rathore sat down in front of a greenscreen to shoot a short video. He looked nervous. It was his first time being cloned.

Wearing a crisp white shirt and a ceremonial saffron scarf bearing a lotus flower—the logo of the BJP, the country’s ruling party—Rathore pressed his palms together and greeted his audience in Hindi. “Namashkar,” he began. “To all my brothers—”

Before he could continue, the director of the shoot walked into the frame. Divyendra Singh Jadoun, a 31-year-old with a bald head and a thick black beard, told Rathore he was moving around too much on camera. Jadoun was trying to capture enough audio and video data to build an AI deepfake of Rathore that would convince 300,000 potential voters around Ajmer that they’d had a personalized conversation with him—but excess movement would break the algorithm. Jadoun told his subject to look straight into the camera and move only his lips. “Start again,” he said.

Right now, the world’s largest democracy is going to the polls. Close to a billion Indians are eligible to vote as part of the country’s general election, and deepfakes could play a decisive, and potentially divisive, role. India’s political parties have exploited AI to warp reality through cheap audio fakes, propaganda images, and AI parodies. But while the global discourse on deepfakes often focuses on misinformation, disinformation, and other societal harms, many Indian politicians are using the technology for a different purpose: voter outreach.

Across the ideological spectrum, they’re relying on AI to help them navigate the nation’s 22 official languages and thousands of regional dialects, and to deliver personalized messages in farther-flung communities. While the US recently made it illegal to use AI-generated voices for unsolicited calls, in India sanctioned deepfakes have become a $60 million business opportunity. More than 50 million AI-generated voice clone calls were made in the two months leading up to the start of the elections in April—and millions more will be made during voting, one of the country’s largest business messaging operators told WIRED.

Jadoun is the poster boy of this burgeoning industry. His firm, Polymath Synthetic Media Solutions, is one of many deepfake service providers from across India that have emerged to cater to the political class. This election season, Jadoun has delivered five AI campaigns so far, for which his company has been paid a total of $55,000. (He charges significantly less than the big political consultants—125,000 rupees [$1,500] to make a digital avatar, and 60,000 rupees [$720] for an audio clone.) He’s made deepfakes for Prem Singh Tamang, the chief minister of the Himalayan state of Sikkim, and resurrected Y. S. Rajasekhara Reddy, an iconic politician who died in a helicopter crash in 2009, to endorse his son Y. S. Jagan Mohan Reddy, currently chief minister of the state of Andhra Pradesh. Jadoun has also created AI-generated propaganda songs for several politicians, including Tamang, a local candidate for parliament, and the chief minister of the western state of Maharashtra. “He is our pride,” ran one song in Hindi about a local politician in Ajmer, with male and female voices set to a peppy tune. “He’s always been impartial.”

While Rathore isn’t up for election this year, he’s one of more than 18 million BJP volunteers tasked with ensuring that the government of Prime Minister Narendra Modi maintains its hold on power. In the past, that would have meant spending months crisscrossing Rajasthan, a desert state roughly the size of Italy, to speak with voters individually, reminding them of how they have benefited from various BJP social programs—pensions, free tanks for cooking gas, cash payments for pregnant women. But with the help of Jadoun’s deepfakes, Rathore’s job has gotten a lot easier.

He’ll spend 15 minutes here talking to the camera about some of the key election issues, while Jadoun prompts him with questions. But it doesn’t really matter what he says. All Jadoun needs is Rathore’s voice. Once that’s done, Jadoun will use the data to generate videos and calls that will go directly to voters’ phones. In lieu of a knock at their door or a quick handshake at a rally, they’ll see or hear Rathore address them by name and talk with eerie specificity about the issues that matter most to them and ask them to vote for the BJP. If they ask questions, the AI should respond—in a clear and calm voice that’s almost better than the real Rathore’s rapid drawl. Less tech-savvy voters may not even realize they’ve been talking to a machine. Even Rathore admits he doesn’t know much about AI. But he understands psychology. “Such calls can help with swing voters.”

Brushing shoulders with politicians isn’t new for Jadoun. He used to be one. In 2015, he stood for election in Ajmer as district president of the National Students Union of India (NSUI), the youth wing of the Indian National Congress, the once formidable national party that’s now the chief opposition to Modi’s BJP.

Jadoun always had a knack for self-promotion—that’s why his friends persuaded him to run. During the campaign, he hired four shotgun-wielding gunmen to escort him around town in open-top jeeps in hopes of creating a powerful persona. “When you have gunmen with you, others start feeling envious,” he said. “Wherever I would go, these gunmen would come with me.” Sometimes they would fire off a round or two into the air. But one day he took it too far and entered a polling booth with loaded weapons, sparking a national debate on lawlessness in small-town India.

Still, Jadoun won the election, and two years later he became the state general secretary of the NSUI for Rajasthan. As NSUI leader, he gave rousing speeches in front of large crowds at rallies, posed for pictures with impoverished children, and traveled across Rajasthan to meet politicians and strengthen his network. But five years in, he felt suffocated by politics. He had dropped out of college, and he realized that while all the people around him elevated him, he was also beholden to them. “I used to get calls at 2 am to handle situations,” he says. “I realized that my time and my life weren’t mine any longer.”

The Covid lockdowns were a blessing. Stuck at home, away from politics, he started an experiment: a series of “30-day challenges” where he’d spend a month learning a new skill—playing the flute, mastering graphic design, trading on the stock market, becoming ambidextrous—to see whether that would help him figure out the next phase of his life.

In October 2020, he saw the music video for the Black Eyed Peas song “Action,” which inserted the singers’ faces into Indian movies. He started hanging out in Reddit forums and tinkering with face-swapping software. In October of that year, he set himself the 30-day challenge of gaining 10,000 Instagram followers. He created an account called The Indian Deepfaker and started reimagining clips from movies and TV shows using open source software DeepFaceLab. One put the actors from Breaking Bad into an Indian soap opera; another turned Bollywood actor Shah Rukh Khan into Iron Man.

His videos went viral. Soon he was getting DMs asking for paid commissions. For a fee he’d make a custom clip: your face grafted onto a blockbuster movie, video greetings to newlyweds from their deceased parents. It dawned on him that he could actually make a living doing this. “I was offered £800 [$1,000] for one assignment,” he says. “I lost my mind.” When Netflix hired him to work on an internal tribute video for an employee, swapping his face onto the characters in the series he had worked on, Jadoun flaunted the nondisclosure agreement on Instagram.

In 2022, Sagar Vishnoi, a sharp and slick political consultant from the small town of Chandigarh, spotted some of Jadoun’s work online. Vishnoi had pioneered the use of AI for political campaigns in India and made headlines in 2020 for creating the first high-profile sanctioned political deepfake video—for Manoj Tiwari, a BJP politician in Delhi. The 44-second monologue used lip-sync dubbing to make clips of Tiwari campaigning in English and Haryanvi—languages he doesn’t speak—saying that the opposition had “cheated” people. It was shared across 5,800 WhatsApp groups ahead of the Delhi elections, without any mention of the fact it had been made using AI.

Scrolling through Jadoun’s Instagram feed, Vishnoi instantly spotted his potential, and contacted him with a plan to use his skills for political campaigning. The duo talked for hours on the phone, swapping stories about their careers. Vishnoi told Jadoun about his work as an anti-corruption activist, and getting water-cannoned during protests. In April 2023, they joined forces, and they’ve been busy ever since.

As Jadoun and Vishnoi got deeper into legitimate political work, Jadoun started to get messages from international numbers on WhatsApp, and burner accounts on Instagram and Telegram from people who wanted to use unsanctioned deepfakes to target their political opponents. Fact-checkers have identified several prominent audio and video deepfakes targeted at influencing voter opinion in India. There have been clones of politicians saying things they didn’t, and politicians crying “deepfake” to deny saying things that they did.

Jadoun says he’s now had more than 250 messages asking for this kind of job. But he is clear about not using deepfakes to spread misinformation—partly because of what happened in November 2023, when he got embroiled in a national scandal. A deepfake had gone viral featuring Indian actress Rashmika Mandanna’s face digitally grafted onto the body of a woman dressed in a black, low-cut top—provocative by conservative Indian standards.

The story made headlines across India. Prime Minister Modi gave a speech warning against the misuse of deepfakes. A police complaint was issued to find the perpetrators. Jadoun had done some work for Mandanna in the past, and many of his Instagram followers thought he was the culprit. “The suspicion was on me because I had her facial data and everything,” he says. At one point, Jadoun received a visit from three plainclothes police officers and was briefly afraid they would ask him about the Mandanna deepfake. But they had come to ask for his help in figuring out whether a Hindi audioclip of a politician was a deepfake or not. After that, Jadoun moved away from making deepfakes for fun, and doubled down on politics. Shortly before the recording session with Rathore, Jadoun was one of three signatories to an “Ethical AI Coalition” manifesto that pledged to not use AI tech for nefarious ends.

From a one-person operation working out of his bedroom, Jadoun has now expanded to a nine-member team to help with the influx of assignments. In March, he added two floors to his home: His staff members work on the ground floor; his family lives on the second. He plans to eventually turn the second into a training institute where local teenagers can learn about AI and how to make deepfakes, and move his family up to the third floor. Polymath is also looking at other countries with elections coming up and is trying to expand globally. A client of Jadoun’s pitched his work to the team of Pierre Poilievre, the leader of Canada’s Conservative Party, who were reportedly interested in translating Poilievre’s speeches into Punjabi and Chinese. They are also eyeing the US market for the upcoming elections, pitching their personalized video messages as one of the main offerings.

Polymath is playing both sides of the aisle: While Jadoun uses his political clout and local connections to hard-sell AI campaigning to aspiring candidates, Vishnoi is creating paid workshops and awareness campaigns, teaching law enforcement how to counter deepfakes.

Jadoun’s work is just the tip of the iceberg. WIRED spoke to half a dozen companies across India working on millions of dollars’ worth of deepfake campaigns. One of them, iToConnect, conducted 25 million personalized AI calls during the two weeks leading up to the general election in the southern states of Telangana and Andhra Pradesh alone. While urban voters perceive unsolicited calls as a nuisance, the political consultants we spoke to say rural users often feel an elevated sense of importance after receiving calls from those in high positions. “Voters usually want the candidate to approach them; they want the candidate to talk to them. And when the candidate can’t go door to door, AI calls become a good way to reach them,” said Abhishek Pasupulety, a tech executive at iToConnect.

Pasupulety’s confidence in the AI approach is based on his company’s experience of the previous round of state elections in 2023. Back then, iToConnect delivered 20 million Telugu-language AI calls for 15 politicians, including in the voice of the then state chief minister. For two weeks before polls opened, voters were targeted with personalized 30-second calls—some asking people to vote for a certain candidate, others sending personalized greetings on religious holidays, some just wishing them a happy birthday.

It worked. Voters started showing up at the party offices, expressing their delight about receiving a call from the candidate, and that they had been referred to by name. They didn’t know they’d been listening to an AI. Pasupulety’s team fielded calls from confused party workers who had no idea what was happening.

The benefits of nailing this technology are obvious—personalized outreach, at massive scale. Using real humans in call centers costs about 4 rupees per call, eight times more than AI calls, says Pasupulety. Traditional unpersonalized and prerecorded robocalls are cheaper, but they’re not as engaging; according to one provider, AI calls keep recipients on the line for longer. Last-minute calls could also make a big difference to the outcome: According to the National Election Study, one in four voters in India decide who to vote for in the final few days before the polls open.

The scale and frequency of AI calls have increased considerably during this general election campaign, a seven-phase marathon that began in April and ends on June 1. Most of them are one-way blasts. In the southern Indian state of Tamil Nadu, a company called IndiaSpeaks Research Lab contacted voters with calls from dead politician J. Jayalalithaa, endorsing a candidate, and deployed 250,000 personalized AI calls in the voice of a former chief minister. (They had permission from Jayalalithaa’s party, but not from her family.) In another state, hundreds of thousands calls have been made in the voice of Narendra Modi, cloned from speeches available online, endorsing a local candidate. Up north, political consultant Sumit Savara has been bombarding candidates and other political consultants with sales pitches for his AI services. “People in the hinterland of India are not on social media,” he says. “For them, these calls will make a difference.”

Vijay Vasanth, a Congress Party politician from Kanyakumari on the southernmost tip of India, was thrust into politics in 2021 after his father, former member of parliament H. Vasanthakumar, died of Covid. In an attempt to harness his father’s goodwill, Vasanth’s team resurrected him using AI and shared a two-minute-long video on WhatsApp and Instagram asking the people of Kanyakumari to vote for his son. “He will be the rightful heir to the love, affection, and faith that you had placed in me,” the dead politician says in the deepfake. Vasanth’s team also created AI video calls with his voice and likeness, to connect with voters in the remote parts of the coastal town—places where Vasanth was unable to make in-person visits—and make it look like he was speaking to them live.

The Congress Party has emerged as the most prolific sharer of AI clones, with numerous official accounts sharing illegal voice clone endorsements by Bollywood celebrities—soliciting warnings from India’s election watchdog and at least three police complaints. Audio deepfakes are particularly pernicious, says Sam Gregory, executive director at nonprofit Witness. “When they manipulate an existing video, it is possible to track down the original and see how it has been manipulated,” he says. “The challenge with fake audio alone is that often there is no reference. There is no original to find.”

But sanctioned deepfakes muddy the waters too, even when people are happy to receive them. “Delightingly deceiving voters doesn't excuse the deception,” says Gregory. “Just because someone pulled the wool over your eyes while smiling at you doesn't change the fact that they deceived you.”

The key problems with deepfakes are lack of consent, disclosure, and content, Gregory says. “There’s a thin line here between voter engagement, candidate humanization, and deception that has a lot to do with familiarity with a given technology,” he says. We’re used to getting personalized emails and letters—but voice messages and videos are a step beyond. Jadoun may have the consent of the politicians he’s working for, but the voters they’re contacting may have no idea they’ve been talking to a clone. As India’s elections draw to close, the world’s biggest democracy is racing into an uncertain, AI-powered future.

After Jadoun finished scanning Rathore in front of the greenscreen, the politician sat us down and pulled out his smartphone. He flicked through a sea of apps until he reached the saffron logo of SARAL—a piece of software that offers a glimpse at how seamlessly AI will slot into the political machinery of the BJP. (SARAL stands for “sangathan reporting and analysis”—sangathan means “organization.”)

Using SARAL, Rathore explained, party workers can upload their activities and share the details of people they meet. “The central leadership gets real-time updates on what happened,” says Diggaj Mogra, the director of Jarvis Consulting, which created the SARAL app and oversees the BJP’s network of 20,000 call center operators.

An engineer by training, Mogra rose to prominence as one of four core team members who masterminded Modi’s 2014 win. But he prefers to stay behind the scenes. He began work on SARAL in 2019, to manage the sprawling on-ground network of BJP workers and allow the central leadership to mobilize the extensive cadre. Today, the app is the BJP’s “digital handheld office,” Mogra says. More than 4 million BJP workers use it monthly.

On SARAL, Rathore can see information about his local area and a list of thousands of people who’ve benefited from government schemes. (Whether political parties should have access to this data is a gray area, but Rathore told us it came “from the party.” Amogh Dhar Sharma, a political scholar from the University of Oxford, says there’s a long history of governments misusing such data to create customized voter messages.)

The app also gives BJP workers a chance to get noticed by party leaders: It has a leaderboard which changes based on the level of engagement. Rathore is scored on how many families he visits and how much data he collects. “I want first ranking in Rajasthan,” Rathore said. He was 12th in his region in April. Rathore is using AI calls to help meet the objectives set by the SARAL app.

In the future, Mogra hopes AI calls will help the BJP’s operations run more efficiently. Right now, the party’s massive call centers are fully occupied: engaging volunteers and discussing whether door-to-door campaigns are going to plan, checking whether workers have gotten posters and other materials, and intervening when campaign efforts falter. These follow-ups involve lots of repetitive logistics questions: Did you get the publicity material? Did you get the booth kit? “We want to replace such redundant questions with AI voice calls,” Mogra says.

He hopes to scale up the bot calls to 100,000 a day for upcoming state elections later this year. The cost savings are attractive. But he hasn’t been that impressed with the quality of the AI-based call center solutions pitched to him so far: “What they promise and what they deliver—there is a huge gap,” Mogra said. “They [promise] you a red Ferrari, and you will get a red Fiat.”

Before we left Jadoun’s home office in April, he demonstrated the fruit of his afternoon’s labors: AI agent calls, where the caller has a back-and-forth conversation with a convincing clone. The caller’s speech is turned into text, which is fed into the large language model Mistral to draft a humanlike reply, which is then turned into an audio response in the voice of whichever politician they think they’re talking to.

It wasn’t really working. What we witnessed was riddled with delays in the machine response, anglicized pronunciations, and hallucinations. On one occasion, the AI got stuck in a loop, repeating the same phrase over and over regardless of the question. On another, it introduced itself in English as “Rathore’s AI avatar.” When Jadoun tried to deploy an AI agent version of Rathore to real voters, the results were equally disastrous. “In some of the calls, the AI was hallucinating, and in one call it spoke so loudly it felt like someone was screaming,” he said. For now, even Jadoun’s peers doing million-dollar projects find interactive AI calls too unreliable to use on high-profile politicians.

But Jadoun is confident that his scrappy team of student engineers can figure it out. They are “jugaadus,” he said, referring to the Indian ways of making things work with limited resources. Each week Jadoun’s team collects call data and fine-tunes for better results. In the corner, a lanky bespectacled 18-year-old sat on a bean bag, fine-tuning a Mistral LLM and teaching it to be “polite” in answering.

A few hours after Jadoun finished scanning Rathore, a new visitor arrived at his office. It was another politician—ready to be cloned. Manoj Bhagat, a young candidate for the Congress Party, wanted Jadoun to keep his voice and image on file so it could be used to contact voters on religious holidays and other special occasions. He hoped it might help him break into mainstream politics. “I thought this was different and might attract people toward me,” Bhagat said. “Nowadays people get so many calls that they may hang up if it’s something generic. But if it starts off with their name, they would know it concerns them and will listen till the very end.”

Let us know what you think about this article. Submit a letter to the editor at [email protected].