ChatGPT vs Cyber Threats – The REAL Role of AI in Cybersecurity

By James Rees, MD, Razorthorn Security

In the rapidly evolving world of cybersecurity, artificial intelligence (AI) and large language models (LLMs), have become buzzwords that seem to promise revolutionary solutions. However, as with any emerging technology, it’s crucial to separate hype from reality. This blog follows the Razorwire Podcast episode on the same topic, and aims to provide a clear view of the current state of AI and LLMs in cybersecurity, drawing on insights from industry experts and real world experiences from myself (James Rees, Razorthorn MD), and our guests on this episode, Joshua Neil and Richard Cassidy.

The AI Confusion: Clearing the Air

The term ‘AI’ has become a catch-all phrase that often leads to misunderstandings, particularly in the context of cybersecurity. In order to have a meaningful discussion about its role in protecting digital assets, we need to be more specific and understand the distinctions between various AI related concepts.

Machine Learning vs AI

Many technologies marketed as ‘AI’ are actually using machine learning techniques, which are a subset of AI. While AI is a broad concept of machines being able to carry out tasks in a way that we would consider ‘smart’, machine learning refers to the specific ability of a machine to learn from data without being explicitly programmed. In cybersecurity, this distinction is crucial. For instance, a system that uses predefined rules to detect threats might be called ‘AI’, but it’s not learning or improving over time. On the other hand, a machine learning model that adapts its threat detection based on new data it encounters is continuously evolving its capabilities.

Shallow Learning vs Deep Learning

We can break machine learning down further, by distinguishing between shallow learning and deep learning methods. Shallow learning methods are often suitable for structured data like network logs or system event data. They excel at finding patterns in data where the features are well defined and the relationships are relatively straightforward. Deep learning, on the other hand, is used for handling unstructured data like images or text. These neural network based models can automatically learn to extract relevant features from complex, high dimensional data. In cybersecurity, deep learning could be used for tasks like analysing the content of emails to detect phishing attempts, while shallow learning could be more appropriate for detecting anomalies in network traffic patterns.

Large Language Models & Natural Language

LLMs represent a specific application of deep learning to natural language processing. These models are trained on vast amounts of text data, allowing them to understand and generate human-like text. In the context of cybersecurity, LLMs are particularly good at tasks involving text analysis and generation. This makes them potentially valuable for analysing threat intelligence reports, understanding the intent behind suspicious emails, or even generating human readable explanations of complex security events. However, it’s important to note that while LLMs are powerful tools for language related tasks, they are not inherently suited for all types of cybersecurity challenges, especially those involving structured data or requiring domain-specific knowledge.

Large Language Models in Cybersecurity: Potential and Limitations

LLMs have demonstrated significant potential in certain areas of cybersecurity, leveraging their ability to process and understand natural language and code structures. However, this comes with both positives and negatives that need to be carefully considered.

There are several areas with great potential:

Phishing Email Detection

As mentioned above, LLMs can be used to analyse the content and context of emails to identify potential phishing attempts with a high degree of accuracy. These models are great at understanding the nuances of language, including subtle cues that might indicate deception or manipulation. LLMs can be trained on vast datasets of known phishing emails, enabling them to recognise new variations and tactics as they emerge. They can assess multiple factors simultaneously, such as sender reputation, email structure and the presence of suspicious links or attachments.

Malicious Script Analysis

As we discussed in the podcast, LLMs have proven effective in detecting malicious PowerShell scripts by understanding the intent and structure of the code. They can analyse script behaviour, identifying potentially harmful actions like unauthorised system changes or data exfiltration attempts. LLMs are good at recognising obfuscation techniques often used to hide malicious code, a task that can be time consuming and challenging for human analysts. They can compare scripts against known malicious patterns while also identifying previously unknown threats based on behavioural analysis.

Threat Intelligence Processing

LLMs can rapidly process and synthesise large volumes of threat intelligence from various sources, a task that would be hugely time consuming for human analysts. These models can extract relevant information from security blogs, forums and reports, providing analysts with concise, actionable insights. Advanced systems can correlate information from multiple sources to identify emerging threats or attack patterns, potentially alerting security teams to new risks before they become widespread.

Incident Response Assistance

LLMs can help draft incident response plans tailored to specific scenarios, drawing on best practices and historical data. During an actual incident, they can provide step by step guidance, helping ensure that critical steps are not missed in the heat of the moment. Some systems can even simulate different response scenarios, helping teams prepare for various contingencies and improve their overall readiness for security incidents.

Limitations and Challenges

However, it’s crucial to understand that LLMs are not a panacea for all cybersecurity challenges. As Richard Cassidy mentioned in the podcast, a recent case study from a major European bank highlighted several important limitations that organisations need to be aware of when considering the implementation of these technologies.

Inaccuracies in AI Generated Information

The bank conducted a three month project to evaluate the use of generative AI in enabling SOC analysts to perform various tasks. One of the key findings was that analysts often encountered inaccuracies in the AI generated information. These inaccuracies ranged from minor errors in data interpretation to more significant misunderstandings of complex security concepts. In some cases, the AI provided plausible sounding but incorrect analyses, which could lead to misguided security decisions if not carefully verified.

Information Overload

Another significant issue identified in the study was the problem of information overload. The AI frequently raised more questions than it answered, overwhelming analysts with data. While the AI was capable of processing vast amounts of information, it often presented analysts with an overwhelming amount of content, much of which was not immediately relevant to the task at hand. Analysts found themselves spending significant time sifting through AI generated content to find actionable insights.

Efficiency Concerns

The study found that in many cases, traditional methods proved more effective and efficient than the AI driven approach. Experienced analysts often found that their own expertise and intuition, combined with traditional security tools, allowed them to identify and respond to threats more quickly than when relying on the AI system. The time spent training, maintaining and interpreting the AI system sometimes outweighed the benefits it provided.

Lack of Contextual Understanding

LLMs can struggle with understanding the specific context of an organisation’s security environment. They may not always grasp the nuances of unique network configurations, custom applications or industry specific threats. This lack of contextual understanding can limit their effectiveness in real world security operations, where every organisation’s environment is unique.

Potential for Bias and Hallucination

There’s the potential for bias and hallucination in LLMs. These models can inadvertently perpetuate biases present in their training data, potentially leading to skewed security assessments or overlooked threats. They may also ‘hallucinate’ or generate plausible sounding but entirely fictional information, which can be particularly dangerous in a security context where accuracy is paramount.

The Dangers of Misapplication

One common pitfall is the attempt to apply LLMs to tasks they’re not well suited for, such as anomaly detection in structured data. LLMs are designed to process and generate human-like text, making them excellent for tasks involving natural language. However, they’re not inherently designed to handle structured data or perform statistical analysis, which are often crucial in cybersecurity contexts.

For instance, using an LLM to detect anomalies in network traffic patterns or user behaviour logs is likely to be ineffective and potentially dangerous. These tasks typically require statistical models or purpose built machine learning algorithms that can efficiently process large volumes of structured data and identify deviations from normal patterns.

This misapplication of LLMs in cybersecurity can lead to several significant problems:

False Sense of Security

Relying on inappropriate tools can leave organisations vulnerable to attacks. When an LLM is used for a task it’s not designed for, it may provide outputs that seem plausible but are actually inaccurate or irrelevant. Security teams might believe they’re effectively monitoring for threats, when in reality, they’re missing critical indicators that a more appropriate tool would catch.

Resource Waste

Investing in AI solutions that don’t address the actual needs of the organisation can lead to significant resource waste. LLMs, especially large, state of the art models, can be computationally expensive to run and maintain. If the LLM isn’t actually solving the intended problem effectively, these resources are essentially wasted.

Overlooking Better Solutions

Focusing on AI might cause organisations to miss more effective traditional or purpose built tools. Many cybersecurity challenges can be effectively addressed with traditional tools, established statistical methods, or purpose built machine learning models. By fixating on implementing LLMs, organisations might neglect to properly implement or maintain these fundamental security tools, potentially leaving significant gaps in their security posture.

How to Use AI & LLMs in Cybersecurity

To effectively leverage AI and LLMs in cybersecurity, organisations should adopt a strategic approach that balances the potential benefits with the practical realities of implementation. Here are the key principles to guide this process:

1. Understand the Technology: Gain a solid grasp of different AI approaches and their appropriate applications. Key decision makers should have a working knowledge of various AI technologies, their strengths and their limitations.

2. Critical Evaluation: Don’t be swayed by marketing hype. Critically evaluate claims about AI capabilities in security products. Request concrete evidence of effectiveness, preferably in the form of case studies or verifiable metrics.

3. Right Tool for the Right Job: Recognise that while LLMs and AI have their place in cybersecurity, they’re not a one size fits all solution. Different security challenges may require different approaches and sometimes traditional methods may be more effective.

4. Continuous Learning: Stay informed about the latest developments and be willing to reassess your understanding as the field evolves. Encourage a culture of continuous learning within your organisation.

5. Start Small and Scale: When implementing AI solutions, consider starting with pilot projects in specific, well defined areas before rolling out across your entire security operation.

6. Maintain Human Oversight: While AI can significantly enhance cybersecurity capabilities, it’s crucial to maintain human oversight. AI should be viewed as a tool to augment human expertise, not replace it entirely.

The Road Ahead

As the cybersecurity landscape continues to evolve, AI and LLMs will undoubtedly play an increasingly important role. However, their effective integration requires a nuanced understanding of their capabilities and limitations.

The key is to approach these technologies with a critical eye, grounded in a solid understanding of cybersecurity fundamentals. By doing so, organisations can cut through the hype and leverage AI and LLMs where they truly add value, whilst maintaining a robust, multi layered approach to security.

In the end, AI and LLMs are powerful tools in the cybersecurity arsenal, but they are just that – tools. Their effectiveness depends entirely on how well we understand and apply them. As we move forward, it’s this understanding that will truly revolutionise cybersecurity, not the mere adoption of buzzword technologies.

Get in touch to discover how Razorthorn can help your organisation with AI implementation.