The use of deepfake technology in disinformation campaigns has become a reality with the discovery of a pro-China campaign that used the technology to create fake news anchors. This marks the first known instance of deepfake technology being used to create fictitious people as part of an information campaign.
A.I. companies in China have been developing deepfake technology for over five years, and it has advanced steadily over the past decade. The technology, which can create characters out of whole cloth and blur the line between fact and fiction, is now easily accessible and affordable. This has made it easier to produce deepfake content at scale.
British A.I. Company Linked to Pro-China Campaign
The fake news anchors in the pro-China campaign were linked to technology made by a British A.I. company called Synthesia. Synthesia offers a software for creating deepfake avatars, which are digital twins based on hired actors. The avatars can speak in 120 languages and accents, with 85 characters to choose from with different genders, ages, ethnicities, voice tones, and fashion choices. Customers can create their own avatars based on themselves or others with permission.
Synthesia’s deepfake technology is mostly used for human resources and training videos but has also been misused for political disinformation campaigns. The software costs as little as $30 a month and can produce videos in minutes that would otherwise take several days and a video production crew.
Synthesia’s terms of service state that the technology should not be used for political, sexual, personal, criminal, or discriminatory content. The company has a four-person team dedicated to preventing its deepfake technology from being used for illicit content, but it can be difficult to detect misinformation. The CEO of Synthesia takes responsibility for anything that happens on the platform and calls for clearer rules on the use of AI tools.
Deepfake Technology and Disinformation
As deepfake technology becomes more sophisticated, it will become more difficult to identify disinformation. The discovery of the pro-China deepfake videos was made while following social media accounts linked to a pro-China misinformation campaign. The two videos featuring the “Wolf News” anchors were posted by five accounts and re-shared by two more, appearing to be part of a pro-China network.
Deepfake technology has been used in music videos, and other publicity stunts, and researchers expect disinformation peddlers to continue experimenting with AI software to produce increasingly convincing media that is hard to detect and verify.
Facebook deleted one account affiliated with pro-China deepfake videos, while Twitter did not respond to a request for comment. The use of deepfake technology is more notable than the actual impact of the videos, as they were only posted by a limited number of accounts.
The use of deepfake technology in political disinformation campaigns marks a new frontier in the ongoing battle against misinformation. The technology’s ability to create convincing digital puppets blurs the line between fact and fiction and makes it easier for disinformation campaigns to spread false information at scale. Companies that develop deepfake technology must take responsibility for how their technology is used and work to prevent its misuse. As deepfake technology becomes more sophisticated, it will become increasingly important to develop new ways to detect and verify information online.
