ChatGPT: Will AI chatbots help fight online fraud?

Kaspersky experts conducted research studying ChatGPT's ability to detect phishing links. While ChatGPT had previously demonstrated the ability to create phishing emails and write malware, its effectiveness in detecting malicious links was limited. The study revealed that although ChatGPT knows a lot about phishing and can guess the target of a phishing attack, it had high false positive rates of up to 64%. Often, it produced imaginary explanations and false evidence to justify its verdicts.

ChatGPT, an artificial intelligence language model, has been a topic of debate in the cybersecurity world due to its ability to generate phishing emails and has raised concerns about its impact on the job security of cyber experts, even despite warnings from its creators that it is too early to apply the new technology to such high-risk areas. Kaspersky experts decided to conduct an experiment to reveal ChatGPT's ability to detect phishing links, as well as test the knowledge it acquired on cybersecurity during training. The company's experts tested gpt-3.5-turbo, the model that powers ChatGPT, on more than 2,000 links that Kaspersky's anti-phishing technologies judged to be phishing, along with thousands of secure URLs.

In the experiment, detection rates varied depending on the prompt used. The experiment was based on asking ChatGPT two questions. and "Is this link safe to visit?" The results showed that ChatGPT had a detection rate of 87.2% and a false positive rate of 23.2% for the first question. The second question had a higher detection rate (93.8%), but also a higher false positive rate (64.3%). While the detection rate is very high, the false positive rate is also too high to be implemented.

The unsatisfactory results in detection were expected, but could ChatGPT help in classifying and investigating attacks? Since attackers typically mention popular brand names in their links to trick users into believing that the URL is legitimate and belongs to a reputable company, the AI language model shows impressive results in identifying potential phishing targets. For example, ChatGPT has successfully extracted a target from more than half of the URLs, including major tech portals like Facebook, TikTok and Google, marketplaces like Amazon and Steam, and many banks from around the world, among others-with no additional training.

The experiment also showed that ChatGPT can have serious problems when it comes to proving its position on deciding whether a link is malicious. Some explanations were correct and fact-based; others revealed known limitations of language models, including illusions and inaccuracies: many explanations were misleading, despite the confident tone.

As Vladislav Tushkanov, Lead Data Scientist at Kaspersky, comments,

ChatGPT certainly has the potential to help human analysts in detecting phishing attacks, but let's not get ahead of ourselves - language models still have their limitations. While they may be on par with a phishing analyst at an internal level, when it comes to reasoning about phishing attacks and extracting potential targets, they tend to have inaccuracies and produce random results. So while they may not yet revolutionize the cybersecurity landscape, they could still be useful tools for the community

Kaspersky's machine learning team is at the forefront of applying machine learning technologies to cybersecurity tasks, constantly updating Kaspersky's products with the most advanced forms of technology and information. To take advantage of Kaspersky's machine learning expertise and stay protected, the company's experts recommend:

• For corporate cybersecurity, Kaspersky Managed Detection and Response is an essential tool capable of detecting and preventing attacks in their initial stages. It uses advanced machine learning models to filter out ordinary events and sends only the most alarming ones to professional human analysts. This service enhances a company's ability to withstand cyber threats while optimizing the use of existing workforce resources.
• Providing cyber hygiene training to your staff is vital. Conducting simulated phishing attacks can also help ensure they know how to distinguish phishing emails.
• Finally, it is also recommended to use the latest Threat Intelligence to be aware of actual TTPs (tactics, techniques and procedures) used by threat actors to enhance cyber security.