Influence Operations in the Cyber Realm in the Age of AI

leafykibritmahtab
,
, , , , , ,

This is the original article. This post will not pass a plagiarism review. I am sharing it now because I would like to make a point. Fear of influence operations can be part of delusional paranoia, but this doesn’t mean that this paranoia isn’t rooted in reality. In the digital age, the online world can amplify such fears. People with schizoaffective disorder and PTSD might be more vulnerable to noticing “influence operations” or spending more time online, in solitude, seeking hidden conspiracies behind words and memes. Sometimes the solution could be simply spending less time online and investing more in real-life meaningful connections. One issue that people with schizoaffective disorder might face is social withdrawal.

Under psyop realism, we all become targeted individuals under the shadowy control of the Influencing Machine – a term originally used to describe the paranoid delusions of someone diagnosed with schizophrenia. In their book Anti-Oedipus, philosophers Deleuze and Guattari describe the ‘schizophrenic’ as someone who rejects capitalism and the codes and signifiers that order it, so it’s no wonder that schizoposting – an unfiltered approach to sharing information via unintelligible text walls, memes and videos – is so ubiquitous among the terminally online, whose anti-establishment content pokes fun at these systems of power.  (26January 2023

Günseli Yalcinkaya. Dazed magazine

My fear of influence campaigns emerged during the time of the anti-judicial protests in 2023 when I was confronting the “poison machine” source source. My country seemed to be on the brink of civil war. The situation was chaotic, and I struggled to understand what was happening around me. It was then that I started to come up with symbolic, neologistic terms such as “semantic terrorism” and “virally loaded hashtags.” It was also about my personal trauma, the parts that I’m still exploring in my individual therapy.

Meanwhile, I am just struggling to survive and avoid delusions of grandiosity. I was certainly influenced by the political crisis in my country. I was afraid that my own family would reject me over my differing opinions. All my years of experience in activism didn’t prepare me for the looming threat of civil war. I am grateful for the two hours today that my depression allowed me some clarity, rather than dwelling in the grief and sorrow over the loss of myself.

Special Joint Publication by the Institute for National Security Studies and the Institute for Intelligence Methodology at the Intelligence Heritage Center, June 10, 2024

Lt. Col. T.Z. (serving in IDF Intelligence)

Institute for National Security Studies (Israel). (2023, December 27). In Wikipedia. https://en.wikipedia.org/wiki/Institute_for_National_Security_Studies_(Israel)

original link 10.6.2024

Artificial intelligence (AI) is a powerful tool for enhancing the credibility and performance of attackers on an unprecedented scale, deepening the threats and risks in the context of the information warfare.

Indeed, countries like China and Russia have invested heavily in developing capabilities in this field in recent years.

Simultaneously, experts in technology are calling to halt these developments, arguing that they pose a significant risk to humanity.

This article summarizes the principles for thinking, planning, and executing influence operations in the cyber realm in the age of AI, analyzes how AI enables a significant leap in the quality and intensity of these operations, and presents the emerging challenges from the use of these tools.

The article argues that technological advancements and the continuation of the current trend are likely to increase the motivation of interested parties to intervene and influence in ways that can undermine the perception of reality and lead to significant social and political harm. Given the potential for harm, greater awareness of the dangers is required, as well as investment in the development of monitoring and prevention tools at the national level.

The Information Age and the “AI Revolution” – A Double-Edged Sword

While dramatic improvements in quality of life have been made thanks to technological transformations and big data, the cyber realm has recently become a preferred arena for influence operations and psychological warfare.

The concept of cyber network influence attacks, aimed at disrupting the information environment of the target, has evolved to the point where it is difficult, and sometimes impossible, to detect and prevent. In recent years, there has been an increase in efforts by countries or sub-state actors to infiltrate hostile target audiences and shape their perception of reality using misinformation driven by foreign interests.

Although empirical studies have not yet presented a scientific way to measure the success of influence operations, it is evident that they are spreading faster than before under the aegis of technology and successfully causing deep social upheaval and instability.

Initially, AI served cyber attackers as an information extraction engine, enabling the processing of big data and learning about trends and narratives through which they could influence.

Today, in light of recent developments, particularly in the fields of natural language processing (NLP), AI is a powerful tool for enhancing the attacker’s credibility and performance on an unprecedented scale, deepening the threats and risks. It is no wonder that countries like China and Russia have invested heavily in developing capabilities in this field in recent years.

Simultaneously, experts in technology are calling to halt these developments,

arguing that they pose a “significant risk to humanity.”

This article summarizes the principles for thinking, planning, and executing influence operations in the cyber realm in the age of AI. The article will analyze how AI enables a significant leap in the quality and intensity of these operations (“game-changing weapon”) and present the emerging challenges from the use of these tools.

The article argues that technological advancements and the continuation of the current trend are likely to increase the motivation of interested parties to intervene and influence in ways that can undermine the perception of reality and lead to significant social and political harm. Given the potential for harm, greater awareness of the dangers is required, as well as investment in the development of monitoring and prevention tools at the national level.

Cyber Attack as a Tool of Influence

Cyber attack operations (CNA) have become an integral part of the combat doctrine of many countries worldwide, serving as a complement to conventional warfare or as standalone actions. These operations are conducted in the digital space, allowing attackers to quickly cause significant damage to the digital systems of an adversary.

Cyber attacks enable the disruption of any data infrastructure or system connected to the Internet, causing damage on a national scale, whether targeted against critical infrastructures or databases containing private and sensitive information of citizens.

The potential for damage and low cost have made cyber attacks an attractive tool. States or sub-state actors wishing to operate in the gray area, to deter or push their adversaries into action, will often choose to operate in this dimension.

Since these actions are generally considered below the threshold of war, a significant portion of cyber attack operations are carried out to apply pressure, influence a target audience, and decision-makers.

In recent years, we have witnessed the diversification and expansion of attack and influence systems in the cyber dimension. While in the past, these systems were characterized by penetration and disruption of critical infrastructures, they are now aimed at penetrating and disrupting the human mind, intending to reshape the social and political environment according to foreign interests and considerations.

The “information and digital age” enables new potential for operations whose sole purpose is influence – ICO (Influence Cyber Operations).

These operations occur in a more gray and hybrid space, within social networks, in the private and legitimate environment of billions of users worldwide.

Influence operations include the enhancement of components of soft warfare, deception, and psychological warfare, exploiting the Internet, which provides unprecedented access to target audiences.

Principles for Planning an Influence System in the Age of AI

Technological transformations in the fields of “artificial intelligence” have made influence systems a profitable effort. The variety of tools available to attackers is broad, and the ability to tailor the tools to the operational purpose and the attacker’s personal needs is available to both small and large actors.

The simpler and more familiar the tool, the easier it is to detect and accordingly, its achievements are limited.

To successfully conduct a broad influence system aimed at changing consciousness and shaping the opinions of many, attackers prepare for the operation as a special operation that includes a multi-stage process.

Each of the stages detailed below will present the opportunities that AI provides to influence attackers and the corresponding challenges that target audiences – individuals, organizations, and countries – are expected to face.

Thought, Planning, and Intelligence Gathering

Shaping public perception is a complex task that requires patience, sophistication, and significant skill. An influence attacker aiming to challenge beliefs held as truths by a large group must acknowledge that the human brain is inherently resistant to persuasion . Before discussing the tools required for action, the first phase of an influence operation is a theoretical and conceptual stage.

This stage critically examines the desired achievement, available alternatives, the risks involved, and the likelihood of failure. It is preferable at this stage not to have a single structured idea but rather a brainstorming process that results in a variety of possible plans for the desired outcome.

The attacker needs a broad database on the target audience, cultural-historical context, and current events, allowing them to direct thinking processes and shape the narrative and operation most accurately and appropriately. This thinking stage is designed to prevent imagining the victory picture before discussing the starting conditions.

Intelligence gathering and a deep, accurate understanding of the attacker’s operational environment are crucial.

An influence operation that does not fit the target audience’s mood or news cycle is bound to fail. An actor aiming to amplify an existing opinion or suggest a new one must be familiar with the characteristics and prominent trends in the discourse, especially the controversial issues and extreme stances within the audience.

An influence operation will succeed when it amplifies existing tensions within the target audience’s network. It is easier to radicalize biased thinking based on emotions and subjectivity rather than rational decision-making.

Therefore, attackers will try to deepen existing arguments, undermine trust, and minimize shared values that shape a functioning society.

In the collection phase, task forces are typically established, composed of content experts who can handle large datasets, overcome language barriers, and possess significant knowledge of the target audience’s culture, enabling them to identify unique styles often misunderstood by outsiders.

Foreign intervention in a country’s domestic affairs fails when the attacker’s information gaps about the target audience’s interests are exposed, or when incorrect use of common expressions in the language occurs. These are significant challenges with potentially enormous damage, thus the main investment of attackers is directed at this phase.

Artificial intelligence (AI) enables machine learning that efficiently handles such gaps.

The ability of a computer to learn complex content, identify patterns, and generate decision or action tools has turned the challenge of handling large data sets into a solvable problem.

A machine’s data processing speed depends on available storage, algorithm quality, and computing power—all upgradeable in the open market.

Moreover, presenting a reliable picture of large target audiences is a capability based on the fusion of large information sources—a capability reserved until recent years for state-level organizations.

Current technology allows the creation of databases tailored to the attacker’s needs, either automatically or with minimal technological investment for unique adjustments.

There are commercial companies offering data collection and organization services, and systems capable of extracting information from the internet and open social media pages. Although internet content giants attempt to limit information access, sufficient data remains online to train machines according to these systems’ needs.

Based on databases, models can be built to analyze existing trends and even generate trend prediction systems, allowing the attacker to prepare in advance with messages tailored to various developments.

One well-known and widespread model in this context is Google Trends, which offers predictive analyses about society’s state and issues related to psychological-perceptual conditions based on user searches .

Internet user information collection-based predictive models can also help analyze the quality of a narrative and its influence potential, enabling the attacker to adjust the message according to changes within the target audience.

On social networks, AI-based monitoring capabilities collect data on user interactions to better characterize audience needs and adapt the platform to commercial demand (content adaptation and adding user-required tools).

Another AI-based tool is Network Analysis, a relatively common tool helping map a community in a visual diagram based on statistical data of relationships between entities. Using this model, criteria can be defined to analyze the type and strength of connections between individuals.

The criteria can be related to biographical background (gender, age, place of birth) or attitudes and stances on specific issues relevant to narrative construction. With this tool, an influence attacker can map audience opinion distribution on various issues, easily identifying weaknesses and opportunities for influence.

Developments in Natural Language Processing (NLP) have turned language barriers into opportunities. NLP allows systems to read, write, and interpret text based on mimicking human brain activity

A system learning and teaching itself to handle languages helps monitor and prevent errors in using foreign languages by understanding dialects, abbreviations, and social network-specific styles. A notable research area in NLP is Sentiment Analysis.

The model trains to understand subjective text interpretations—attitude, emotion, suspicion, or confusion—and essentially characterize whether a text is written positively, negatively, or neutrally.

This way, user reactions can also be analyzed to determine if they are confrontational and deepen polarization. Some studies have successfully challenged models to diagnose behavior expressing depression , or to identify statements with racial, gender, or religious biases in controversial contexts.

Infrastructure Setup

The infrastructure required for a successful influence operation includes access to the target audience and the creation of identities that systematically and continuously disseminate the narrative.

There are several influence spaces where operations can take place, varying and updating according to the popular platforms used by the target audience. Influence attackers prefer to conduct the campaign within social networks (Twitter, Facebook, or Instagram) and messaging applications (WhatsApp and Telegram) because they have built-in applications and features that save resources and the need for dedicated development. These include the ability to create groups, publish a wide range of content (photos, videos, memes), express support, and respond.

Today, social networks have a community or group on any topic, and in large communities, external actors can easily integrate and become active participants and opinion leaders within a short time .

Established interest groups will set up independent web pages to strengthen the organization’s authenticity they represent and maintain direct and consistent access to the updated data they rely on.

A fundamental difficulty in using social networks is maintaining authenticity.

Most attackers create a fake identity that will serve as legitimate faces for spreading the message. Although an existing identity can be stolen, this is a potentially more dangerous move requiring advanced technological skills.

Usually, attackers will prefer to hire contractors, through the Dark-web, via intermediary companies offering services for creating and operating fake identities , or marketing companies providing exposure and content amplification services for commercial needs . In some cases, attackers will pay people from the target audience to spread messages on their behalf and real identity.

Given the growing awareness of tech giants about the presence of fake accounts within the networks, there is increasing effort to neutralize and block such activities .

Automatically created accounts often contain glaring errors such as mismatches between names and gender, code outputs appearing within user details,

or a high number of similar accounts created within a short, unrealistic time frame.

These accounts typically publish reports at a high pace, inconsistent with human activity speed.

For example, Russian infrastructures were exposed on suspicion of interfering in the 2016 US elections after participating in a campaign for African American rights .

Attackers are improving their efforts to strengthen fake identities and make them as authentic as possible by enriching the profile with photos, biographical information, and extensive activity on various platforms.

For these purposes, AI provides solutions that strengthen the attacker and complicate the work of information security professionals. If in the past, attackers chose photos of relatively unknown (low-signature) people to avoid suspicion, today’s models can generate a large army of unique “avatars” behaving like authentic entities.

Although AI can identify fake images, the same AI improves the quality of forgery using GAN (Generative Adversarial Networks) capabilities, making detection complex.

This ability is also the basis for what is known as deepfake, which can fake high-quality photos and videos, inventing non-existent people or events using “machine learning” that employs various learning models to improve result quality.

Content Creation and Narrative Embedding

Content is the fuel of influence operations. The attacker needs to gather or create content that can be systematically distributed across various platforms to capture the target audience’s attention and recruit supporters.

The primary goal of the attacker at this stage is to spread viral content, gaining extensive exposure in a short time. Researchers have analyzed that a preferred method for influence attackers is echoing messages written by legitimate users within the target audience.

This method saves the need to create new content, maintains an authentic voice and atmosphere, and avoids cultural mistakes, thus reducing suspicion of forgery. Within social media platforms, there are built-in tools for spreading messages, such as using hashtags (#) that link to similar content and help build a community or discussion, or quick editing options for photos and video clips.

Visual content is generally perceived as more viral and also complicates monitoring systems from identifying the source or analyzing anomalies . In the past two years, the use of memes has become common. Memes are an innocent tool for spreading a message on an image, now considered the most viral tool .

In summary, AI has revolutionized the cyber influence operations landscape by providing powerful tools for data processing, narrative construction, and dissemination. The ability to automate and enhance these processes has made it easier for attackers to manipulate public perception on a large scale. Consequently, there is a pressing need for heightened awareness and investment in monitoring and prevention tools at the national level to counteract these threats.

Doxing as a Form of Influence

Doxing refers to the intentional exposure of intimate or embarrassing information to influence decision-making processes. A notable instance occurred during the 2019 elections in Israel when it was alleged that the personal phone of former IDF Chief of Staff and Defense Minister, Lt. Gen. (Res.) Benny Gantz, was hacked by Iranian operatives, revealing sensitive information . The success of doxing largely depends on the authenticity and the volume of information at the attacker’s disposal to maintain long-term pressure.

Integration into Long-Term Influence Systems

Influence campaigns typically involve establishing a presence within the target platform and building trust through legitimate content before releasing the manipulative information. Attackers often integrate into the routine activities of the target audience, distributing legitimate content to gain trust before introducing their influence content.

AI in Influence Campaigns

Before the advent of AI models, long-term influence operations required significant resources and skilled manpower to adapt to the evolving landscape. AI language models like OpenAI’s GPT, Huawei’s model in China, and Alphabet’s Bard offer the capability to understand and generate text, significantly lowering the cost and time required for influence operations. These models, trained on billions of parameters and words, can generate high-quality content in various forms (text, images, videos) quickly and accurately.

Automation and Influence

Advanced language models can autonomously create content based on probabilistic calculations, mimicking human writing by selecting the next word from learned patterns. This capacity makes them powerful tools for generating influence content autonomously and potentially even initiating activities without human intervention in the future.

Echoing and Dissemination

The effectiveness of influence content is measured by its actual reach. Skilled attackers strive to make their messages viral by systematically injecting them into the target audience’s activities. In short-term campaigns, rapid dissemination aims to instill fear, whereas long-term campaigns prefer gradual echoing, initially in specific internet forums and later in broader social media, to appear credible. The success metric is when the narrative infiltrates legitimate news circles, creating a prolonged impact due to the difficulty of distinguishing and removing fake content.

AI in Echoing

AI models enhance the initial quality and credibility of influence content, ensuring it resonates strongly during the first exposure. As social media platforms evolve to detect malicious codes, attackers adapt by emulating natural online behavior, similar to how bacteria evolve to evade immune systems. Future advancements in machine learning will enable even more sophisticated and adaptable influence operations.

Creating a New Reality

The term “post-truth”, Oxford’s Word of the Year 2016, encapsulates the success of influence campaigns where emotional appeals outweigh objective facts in shaping public opinion . Efforts to inoculate the public against misinformation highlight the growing challenge of coping with a flood of false reports that exceed human processing capabilities.

Sustaining the Narrative

Successful influence operations do not end with message dissemination but require ongoing maintenance to keep the narrative in the public consciousness. The internet’s permanence makes it difficult to erase false reports, often requiring counter-influence efforts. AI and autonomous bots will increasingly identify trends and automatically generate and echo content to maintain the desired narrative.

Conclusion and Implications

The battle for influence in cyberspace threatens trust between individuals, organizations, and nations, endangering societal stability and national security. The era of “post-truth” is exacerbated by AI’s capabilities, which simplify and enhance influence operations. The rapid technological advancements and accessibility of sophisticated tools make it challenging for policymakers and international bodies to regulate these activities. Enhanced awareness and continuous efforts to promote fact-based reality are crucial to counteract the growing influence of AI-driven disinformation.

References:

Leave a Reply