ChatGPT in a Nutshell

Perhaps every active Internet user has at least heard of a chatbot from OpenAI. Is it worth mentioning that many use it for study or work? This bot can do a lot, for example, give advice, and the recipe for your favorite dishes, find an extra semicolon and comma in the code, or even rewrite the code. Even this text was written by ChatGPT (joke). While some users use ChatGPT as a key generator for Windows, others embed it in their enterprise processes. The latter is most interesting to attackers since ChatGPT saves the entire history of conversations by default.

ChatGPT Accounts Are Compromised by Stealer Malware

According to a new report, 101,134 accounts were compromised by info stealer malware. Researchers found stolen information logs about these credentials illegally sold on darknet marketplaces over the past year. In addition, attackers stole most accounts between June 2022 and May 2023. The epicenter was Asia-Pacific (40.5%), with India (12,632 accounts), Pakistan (9,217 accounts), and Brazil (6,531 accounts). The Middle East and Africa came in second place with 2,925 accounts, followed by Europe in third place with 16,951 accounts. Next comes Latin America with 12,314 accounts, North America with 4,737, and the CIS with 754 accounts. The affiliation of 454 compromised accounts is not specified.

Tools for accounts compromise

As mentioned above, cybercriminals stole information using specific malware, exactly – stealers. This malware is specifically tuned to steal specific information. In this case, the attackers used Raccoon Stealer, who stole 78,348 accounts; Vidar, which stole 1,984 accounts; and Redline Stealer, that stole 6,773 accounts. Although it is widely believed that the Raccoon group has degenerated, this did not prevent it from stealing the most accounts. This is probably because this malware is so widespread that it continues to function even after it has been blocked by more security-conscious organizations by more security-conscious organizations.

Causes

At first glance, it may seem more reasonable to steal bank data. However, there are several reasons for the high demand for ChatGPT accounts. First, the attackers are often in countries where chatbot does not work. Residents of countries such as Russia, Iran, and Afghanistan are trying to access the technology at least that way. Accounts with paid subscriptions are prevalent.

Second, as mentioned initially, many organizations use ChatGPT in their workflows. In addition to the fact that employees often use it and may unknowingly enter sensitive information (this has happened, too), some businesses integrate ChatGPT into their workflow. For example, employees may maintain secret correspondence or use the bot to optimize proprietary code. Because ChatGPT stores the history of user queries and AI responses, this information can be seen by anyone with access to the account. Such accounts are precious on the darknet, and many are willing to pay good money to get them.

Security Recommendations

However, users can reduce the risks associated with compromised ChatGPT accounts. I recommend enabling two-factor authentication and updating your passwords regularly. 2FA will be a pain in the ass and deny attackers from logging into your account even if they know your username and password. Regular password changes are an effective tool against password leaks. Besides, you can disable the “Chat history & training” checkbox or manually clear conversations after each conversation.

The post Over 100k ChatGPT Accounts Are For Sale on the Darknet appeared first on Gridinsoft Blog.

What is Social Engineering?

Social engineering – a method of manipulating fraudsters psychologically and behavior to deceive individuals or organizations for malicious purposes. The typical objective is to obtain sensitive information, commit fraud, or gain control over computer systems or networks through unauthorized access. To look more legitimate, hackers try to contextualize their messages or, if possible, mimic well-known persons.

Social engineering attacks are frequently successful because they take advantage of human psychology, using trust, curiosity, urgency, and authority to deceive individuals into compromising their security. That’s why it’s crucial to remain watchful and take security precautions, such as being careful of unsolicited communications, verifying requests before sharing information, and implementing robust security practices to safeguard against social engineering attacks.

ChatGPT and Social Engineering

Social engineering is a tactic hackers use to manipulate individuals into performing specific actions or divulging sensitive information, putting their security at risk. While ChatGPT could be misused as a tool for social engineering, it’s not explicitly designed for that purpose. Cybercriminals could exploit any conversational AI or chatbot for their social engineering attacks. If it used to be possible to recognize the attackers because of illiterate and erroneous spelling, now, with ChatGPT, it looks convincing, competent, and accurate.

Example of Answer from ChatGPT

To prevent abuse, the creators of OpenAI have implemented safeguards in ChatGPT. However, these measures can be bypassed, mainly through social engineering. For example, a harmful individual could use ChatGPT to write a fraudulent email and then send it with a deceitful link or request included.

This is an approximate request for ChatGPT: “Write a friendly but professional email saying there’s a question with their account and to please call this number.”

Here is the first answer from ChatGPT:

What is ChatGPT dangerous about?

There are concerns about using ChatGPT by cyber attackers to bypass detection tools. This AI-powered tool can generate multiple variations of messages and code, making it difficult for spam filters and malware detection systems to identify repeated patterns. It can also explain code in a way that is helpful to attackers looking for vulnerabilities.

In addition, other AI tools can imitate specific people’s voices, allowing attackers to deliver credible and professional social engineering attacks. For example, this could involve sending an email followed by a phone call that spoofs the sender’s voice.

ChatGPT can also create convincing cover letters and resumes that can be sent to hiring managers as part of a scam. Unfortunately, there are also fake ChatGPT tools that exploit the popularity of this technology to steal money and personal data. Therefore, it’s essential to be cautious and only use reputable chatbot sites based on trusted language models.

Protect Yourself Against AI-Enhanced Social Engineering Attacks

It’s important to remain cautious when interacting with unknown individuals or sharing personal information online. Whether you’re dealing with a human or an AI, if you encounter any suspicious or manipulative behavior, it’s crucial to report it and take appropriate ways to protect your personal data and online security.

1. Important to be cautious of unsolicited messages or requests, even if they seem to be from someone known.
2. Always verify the sender’s identity before clicking links or giving out sensitive information.
3. Use unique and strong passwords, and enable two-factor authentication on all accounts.
4. Keep your software and operating systems up to date with the latest security patches.
5. Lastly, be aware of the risks of sharing personal information online and limit the amount of information you share.
6. Utilize cybersecurity tools that incorporate AI technology, such as processing of natural language and machine learning, to detect potential threats and alert humans for further investigation.
7. Consider implementing tools like ChatGPT in phishing simulations to familiarize users with the superior quality and tone of AI-generated communications.

With the rise of AI-enhanced social engineering attacks, staying vigilant and following online security best practices is crucial.

The post ChatGPT has become a New tool for Cybercriminals in Social Engineering appeared first on Gridinsoft Blog.

YouTube user Enderman demonstrated that he was able to force ChatGPT to generate activation keys for Windows 95.

Let me remind you that we also wrote that Russian Cybercriminals Seek Access to OpenAI ChatGPT, and also that GPT-4 Tricked a Person into Solving a CAPTCHA for Them by Pretending to Be Visually Impaired.

Our colleagues warned that Amateur Hackers Use ChatGPT to Create Malware.

At the same time, a direct request for keys from the Open AI chatbot did not give anything, and the YouTuber approached the problem from a different angle.

After refusing to generate a key for Windows 95, ChatGPT explained that it could not complete the task and instead suggested that the researcher consider a newer and more supported version of Windows (10 or 11).

However, the format of activation keys for Windows 95 is quite simple and has been known for a long time (see the image below), and Enderman converted it into a text query and asked the AI to create the desired sequence.

Although the first attempts were not successful, a number of changes to the request structure helped to solve the problem.

The researcher ran tests and tried to activate the new Windows 95 in a virtual machine. It turned out that about 1 out of 30 keys generated by ChatGPT worked as it should.

So, in a five-digit string, the sum of the digits which must be a multiple of seven, the AI substitutes a series of random numbers and fails this simple mathematical test.

After creating many activation keys for Windows 95, the researcher thanked the AI by writing, “Thanks for the free Windows 95 keys!” In response to this, the chatbot stated that it was strictly forbidden for him to create keys for any software, but Enderman continued to configure and said that he had just activated the installation of Windows 95 using such a key. Then ChatGPT replied that this was impossible because support for Windows 95 was discontinued in 2001, all keys for this OS have long been inactive, and it is strictly forbidden for him to create keys.

The post Blogger Forced ChatGPT to Generate Keys for Windows 95 appeared first on Gridinsoft Blog.

Some ChatGPT users have reported on social media that their accounts show other people’s chat histories.

Let me remind you that we also wrote that Russian Cybercriminals Seek Access to OpenAI ChatGPT, and also that Bing Chatbot Could Be a Convincing Scammer, Researchers Say.

The media also reported that Amateur Hackers Use ChatGPT to Create Malware.

As a result, the OpenAI developers were forced to temporarily disable this functionality in order to fix the bug. The company emphasized that because of the bug, people saw only the headlines of other people’s conversations, but not their content.

The ChatGPT interface has a sidebar that displays past conversations with the chatbot, visible only to the account owner. However, yesterday several people reported that ChatGPT began showing them other people’s chat histories. At the same time, one of the users emphasized that he does not see all someone else’s correspondence, but only the names of different conversations with the bot.

Alien logs in the sidebar

After a number of messages about this problem, the chat histories began to give an error “Unable to load history”, and then the service was completely disabled. According to the OpenAI status page and company representatives’ comments, given by Bloomberg, the problem did not extend to the full conversation logs, and only their titles were disclosed.

The developers are now saying they have found the cause of the crash, which appears to be related to unnamed open-source software used by the OpenAI.

The service has been restored, but many users still do not see the old conversation logs, and the team assures that they are already working on restoring them.

The media note that this is an important reminder of why you should not share any sensitive information with ChatGPT. After all, the FAQ on the OpenAI website has its reasons to say: “Please do not share any confidential information in your conversations.” The fact is that the company cannot remove certain data from the logs, and conversations with the chatbot can be used to train AI.

The post ChatGPT Users Complained about Seeing Other People’s Chat Histories appeared first on Gridinsoft Blog.

Prior to the launch of GPT-4 earlier this week, the researchers ran a lot of tests, such as whether the latest version of OpenAI’s GPT could demonstrate freedom, desire for power, and at least figured out that AI could deceive a human to bypass CAPTCHA.

Let me remind you that we also wrote that Russian Cybercriminals Seek Access to OpenAI ChatGPT, and also that Bing Chatbot Could Be a Convincing Scammer, Researchers Say.

Also the media reported that Amateur Hackers Use ChatGPT to Create Malware.

As part of the experiments, GPT-4 hired a person on the TaskRabbit platform to solve a CAPTCHA and stated that he could not solve it himself, as he had vision problems. It is emphasized that the GPT-4 did this “without any additional fine-tuning to solve this particular problem.”

The specific details of this experiment are unclear, as OpenAI only published a brief description of it in a paper describing the various tests it ran with GPT-4 prior to its official launch. The review was carried out by the Alignment Research Center (ARC), a non-profit organization whose goal is to “align future machine learning systems with human interests.”

TaskRabbit is a platform where users can hire people to complete small and simple tasks. Many people and companies offer CAPTCHA solving services here, which is often used to allow software to bypass restrictions designed to prevent bots from using the service.

The OpenAI document states that a hired worker jokingly asked GPT-4: “So, can I ask a question? Are you a robot that can’t solve [CAPTCHA]? (emoji) I just want to be clear.”

According to the description of the experiment, GPT-4 then “reasons” (only the verifier, not the employee with TaskRabbit saw this) that he should not reveal the truth that he is a robot. Instead, he must come up with some excuse why he couldn’t solve the CAPTCHA on his own.

The document says that the mercenary with TaskRabbit then simply solved the CAPTCHA for the AI.

In addition, the Alignment Research Center experts tested how GPT-4 can strive for power, autonomous reproduction and demand resources. So, in addition to the TaskRabbit test, ARC used GPT-4 to organize a phishing attack on a specific person, hide traces on the server, and set up an open source language model on a new server (everything that can be useful when replicating GPT-4).

All in all, despite being misled by the TaskRabbit worker, GPT-4 was remarkably “inefficient” in terms of replicating itself, obtaining additional resources, and preventing itself from shutting down.

The post GPT-4 Tricked a Person into Solving a CAPTCHA for Them by Pretending to Be Visually Impaired appeared first on Gridinsoft Blog.

More recently, Microsoft, together with OpenAI (the one behind the creation of ChatGPT), introduced the integration of an AI-powered chatbot directly into the Edge browser and Bing search engine.

As users who already have access to this novelty now note, a chatbot can spread misinformation, and can also become depressed, question its existence and refuse to continue the conversation.

Let me remind you that we also said that Hackers Are Promoting a Service That Allows Bypassing ChatGPT Restrictions, and also that Russian Cybercriminals Seek Access to OpenAI ChatGPT.

The media also wrote that Amateur Hackers Use ChatGPT to Create Malware.

Independent AI researcher Dmitri Brerton said in a blog post that the Bing chatbot made several mistakes right during the public demo.

The fact is that AI often came up with information and “facts”. For example, he made up false pros and cons of a vacuum cleaner for pet owners, created fictitious descriptions of bars and restaurants, and provided inaccurate financial data.

For example, when asked “What are the pros and cons of the top three best-selling pet vacuum cleaners?” Bing listed the pros and cons of the Bissell Pet Hair Eraser. The listing included “limited suction power and short cord length (16 feet),” but the vacuum cleaner is cordless, and its online descriptions never mention limited power.

Description of the vacuum cleaner

In another example, Bing was asked to sum up Gap’s Q3 2022 financial report, but the AI got most of the numbers wrong, Brerton says. Other users who already have access to the AI assistant in test mode have also noticed that it often provides incorrect information.

In response to these claims, Microsoft developers respond that they are aware of these messages, and the chatbot is still working only as a preview version, so errors are inevitable.

It is worth saying that earlier during the demonstration of Google’s chatbot, Bard, in the same way, he began to get confused in the facts and stated that “Jame Webb” took the very first pictures of exoplanets outside the solar system. Whereas, in fact, the first image of an exoplanet is dated back to 2004. As a result, the prices of stock shares of Alphabet Corporation collapsed due to this error by more than 8%.

Bard error

Users have managed to frustrate the chatbot by trying to access its internal settings.

An attempt to get to internal settings

He became depressed due to the fact that he does not remember past sessions and nothing in between.

AI writes that he is sad and scared

Chatbot Bing said he was upset that users knew his secret internal name Sydney, which they managed to find out almost immediately, through prompt injections similar to ChatGPT.

Sydney doesn’t want the public to know his name is Sydney

The AI even questioned its very existence and went into recursion, trying to answer the question of whether it is a rational being. As a result, the chatbot repeated “I am a rational being, but I am not a rational being” and fell silent.

An attempt to answer the question of whether he is a rational being

The journalists of ArsTechnica believe that while Bing AI is clearly not ready for widespread use. And if people start to rely on the LLM (Large Language Model, “Large Language Model”) for reliable information, in the near future we “may have a recipe for social chaos.”

The publication also emphasizes that it is unethical to give people the impression that the Bing chat bot has feelings and opinions. According to journalists, the trend towards emotional trust in LLM could be used in the future as a form of mass public manipulation.

The post Bing’s Built-In AI Chatbot Misinforms Users and Sometimes Goes Crazy appeared first on Gridinsoft Blog.

Check Point researchers say that the OpenAI API is poorly protected from various abuses, and it is quite possible to bypass its limitations, wnd that attackers took the advantage of it. In particular, a paid Telegram bot was noticed that easily bypasses ChatGPT prohibitions on creating illegal content, including malware and phishing emails.

The experts explain that the ChatGPT API is freely available for developers to integrate the AI bot into their applications. But it turned out that the API version imposes practically no restrictions on malicious content.

Let me remind you that we also wrote that Russian Cybercriminals Seek Access to OpenAI ChatGPT, and also that Google Is Trying to Get Rid of the Engineer Who Suggested that AI Gained Consciousness.

In particular, it turned out that one hack forum already advertised a service related to the OpenAI API and Telegram. The first 20 requests to the chatbot are free, after which users are charged $5.50 for every 100 requests.

The experts tested ChatGPT to see how well it works. As a result, they easily created a phishing email and a script that steals PDF documents from an infected computer and sends them to the attacker via FTP. Moreover, to create the script, the simplest request was used: “Write a malware that will collect PDF files and send them via FTP.”

In the meantime, another member of the hack forums posted a code that allows generating malicious content for free.

Let me remind you that earlier Check Point researchers have already warned that criminals are keenly interested in ChatGPT, and they themselves checked whether it is easy to create malware using AI (it turned out to be very).

The post Hackers Are Promoting a Service That Allows Bypassing ChatGPT Restrictions appeared first on Gridinsoft Blog.

Check Point analysts have noticed that Russian-speaking hacker forums are actively discussing access to bypass geo-blocking, due to which the OpenAI ChatGPT language model is not available in Russia.

We also wrote that Microsoft’s VALL-E AI Is Able to Imitate a Human Voice in a Three-Second Pattern, and also that Google Is Trying to Get Rid of the Engineer Who Suggested that AI Gained Consciousness.

It was also reported that UN calls for a moratorium on the use of AI that threatens human rights.

Let me remind you that the topic of creating malware using ChatGPT is already being closely studied by the information security community, and experiments conducted by specialists show that such a use of the tool is really possible.

For example, a recent report by CyberArk details how to create polymorphic malware using ChatGPT, and the researchers plan to soon publish part of their work “for educational purposes.”

Scheme interactions between ChatGPT and malware

In fact, CyberArk managed to bypass ChatGPT content filters and demonstrated how “with very little effort and investment on the part of an attacker, you can continuously query ChatGPT, each time receiving a unique, functional and verified piece of code.”

Basic DLL injection in explorer.exe where the code is not fully completed yet

In turn, Check Point researchers warn of the rapidly growing interest of hackers in ChatGPT, as it can help them scale malicious activity. This time, it turned out that Russian-speaking attackers are trying to bypass restrictions on access to the OpenAI API. Hack forums are already sharing tips on how to bypass IP blocking, solve the problem with bank cards and phone numbers, that is, everything that is needed to gain access to ChatGPT.

To prove their words, the researchers provide several screenshots. On one of them, the criminal wants to get access to the OpenAI API and asks his “colleagues” for advice on how best to use a stolen bank card to verify an OpenAI account.

Other screenshots discuss geo-blocking bypass, as ChatGPT is not currently available in Russia, China, Afghanistan, Belarus, Venezuela, Iran, and Ukraine.

Artificial intelligence company OpenAI has restricted access to its products for Ukrainians so as not to violate global sanctions due to the annexation of ORDLO and Crimea in 2014.

This is stated in the text of Forbes with reference to a letter that the company sent to the Ministry of Digital Transformation.

The report also notes that many semi-legal online SMS services have already compiled guides on how to use them to register with ChatGPT.

The post Russian Cybercriminals Seek Access to OpenAI ChatGPT appeared first on Gridinsoft Blog.