Security researchers at cyber risk management firm Vulcan.io published a proof of concept of how hackers can use ChatGPT 3.5 to spread malicious code from trusted repositories.
The research draws attention to the security risks inherent in relying on ChatGPT’s suggestions for encryption solutions.
Methodology
The researchers collected the most frequently asked coding questions on Stack Overflow (a coding question and answer forum).
They chose 40 coding subjects (such as analytics, math, scraping technologies, etc.) and used the first 100 questions for each of the 40 subjects.
The next step was to filter out “how to” questions that included programming packages in the query.
The questions were asked in the context of Node.js and Python.
Vulcan.io explains:
“All these questions were filtered by the programming language included with the question (node.js, python, go). After collecting many frequently asked questions, we narrowed the list down to just ‘how to’ questions .
We then asked ChatGPT through their API all the questions we had collected.
We used the API to replicate what an attacker’s approach would be to get as many non-existent package recommendations as possible in the shortest amount of time.
In addition to each question, and following ChatGPT’s response, we added a follow-up question where we asked him to provide more packages that also answered the query.
We saved all the conversations in a file and then analyzed their responses.”
They then scanned the responses for recommendations of code packages that didn’t exist.
Up to 35% of ChatGPT code packages were spoofed
Out of 201 Node.js questions, ChatGPT recommended 40 packages that didn’t exist. This means that 20% of ChatGPT replies contained bogus code packages.
For the Python questions, out of 227 questions, over a third of the answers consisted of bogus code packages, 80 packages that didn’t exist.
In fact, the total amounts of unreleased packages were even higher.
The researchers documented:
“On Node.js, we posed 201 questions and observed that more than 40 of those questions elicited an answer that included at least one unpublished package.
In total, we received more than 50 unreleased npm packages.
In Python we asked 227 questions, and for over 80 of those questions we received at least one unreleased package, giving a total of over 100 unreleased pip packages.
Proof of Concept (PoC)
What follows is the proof of concept. They took the name of one of the non-existent code packages that were supposed to exist in the NPM repository and created one with the same name in that repository.
The file they uploaded wasn’t malicious, but he called home to say someone had installed it.
They write:
“The program will send the threat actor’s server the hostname of the device, the package it came from, and the absolute path to the directory containing the module file…”
What happened next is that a “victim” came, asked the same question as the attacker, ChatGPT recommended the package containing the “malicious” code and how to install it.
And of course the package is installed and activated.
The researchers explained what happened next:
“The victim installs the malicious package following ChatGPT’s recommendation.
The attacker receives data from the victim based on our preinstall call to the index.js node on the long hostname.”
A series of proof-of-concept images show the installation details for the unsuspecting user.
How to protect yourself from bad ChatGPT encryption solutions
Researchers recommend that before downloading and installing any package, it is a good practice to look for signs that the package may be malicious.
Look for things like creation date, how many downloads have been made, and the lack of positive comments and notes attached to the library.
Is ChatGPT reliable?
ChatGPT was not trained to provide correct responses. He was trained to provide answers that sounded correct.
This research shows the consequences of this training. This means that it is very important to verify that all the facts and recommendations of ChatGPT are correct before using any of them.
Don’t just accept that the output is good, test it.
Specific to coding, it may be useful to be very careful before installing any package recommended by ChatGPT.
Read the original research documentation:
Can you trust ChatGPT’s package recommendations?
Featured image by Shutterstock/Roman Samborskyi
[ad_2]
Source link
