Offensive security pros share how ChatGPT impacts their work
Could 2024 be a pivotal moment for AI in offensive security?
We know it challenges us to explore new ways to simplify our work, but how will penetration testers use ChatGPT as a tool for meaningful change?
And, most importantly, which new advancements in this space are worth keeping an eye on?
You probably have questions like these on your mind. That’s why I talked to some of the most experienced offensive security pros about how they use ChatGPT, so you can learn from their hands-on experience.
Get ready to take notes!
14 effective applications of ChatGPT for your pentesting workflow
Mădălin Dogaru
Red Teamer | Purple Teamer | Senior Threat Modeler at ThreatConnect
Madalin brings his hands-on pentesting experience to the table and shares key lessons learned from working with ChatGPT and how it made his engagements 10x better.
If we discuss practical applications, then I can speak about my experiences rather than theoretical scenarios.
My work revolves around threat modeling, red teaming, and purple teaming, and I'm using ChatGPT as Google on steroids, integrating it into my daily workflow.
For example, to understand technologies that I threat model or that I never encountered during black box engagements, here are 3 specific examples:
Technical research
I used ChatGPT to gain deep technical knowledge about several file system types, specifically CramFS, JFFS2, YAFFS2, ext2, and SquashFS.
Reading through the documentation of all of these filesystems and searching for Google links for articles with additional information are both very time-consuming.
Once I understood how to ask ChatGPT the right questions, it would give me exactly what I needed in a matter of seconds. At this point, I'm the bottleneck, as I assimilate its information slower than it does. I need a brain interface upgrade :).
Improving my execution speed
I’m constantly trying to improve my workflow efficiency and one thing I did before ChatGPT was to improve the code used in the tools I’ve written.
This exercise used to happen once a year as it was time-consuming and new projects always knocked on the door.
With ChatGPT, I can do code reviews efficiently once every 3 months. I allocate a weekend for this, ask it to give me feedback on key components of my code, perform some speed tests on several of ChatGPT’s change proposals, adjust the results based on my needs, and at the end of the weekend, I’ve even had a 40% execution speed increase.
Plan proofing
Once I’m done preparing for a black box type of red team engagement, I ask ChatGPT to attack my plan/preparation and attempt to find flaws and drawbacks in my approach.
If I believe the drawbacks that it identified apply to my context, I adjust my plan accordingly. This way, I have a lower chance of getting caught during an engagement.
Alexandru Piriu
Alexandru explains it simply:
One of the most practical ways to use ChatGPT is when performing source code reviews. Because of the chaotic ways applications were built even in 2023, ChatGPT can summarize complicated chunks of code in easy-to-understand sentences.
However, when a client gives you access to internal code, watch out to not expose client code or too much logic and to not break the NDA agreement.
Matei Anthony Josephs
Matei explains how he uses ChatGPT for one of the parts of pentesting ethical hackers like the least (to put it mildly):
Firstly, I think that one of the main areas where ChatGPT helps offensive security professionals is reporting. ChatGPT can improve the quality and reduce the delivery times of your reports. Notably, it can greatly benefit the creation of executive summaries and conclusions.
Secondly, I find that ChatGPT can expedite the learning process considerably when using the appropriate prompts. Some of my favorites are:
Explain <INSERT CONCEPT> as you would to a 5-year-old
Explain the key difference between <INSERT CONCEPT 1> and <INSERT CONCEPT 2> in bullet points
Summarize the following text in <INSERT WORD COUNT> words: <INSERT TEXT>
Thirdly, I am a bit cheeky about emails. I do not really enjoy writing them, nor reading long ones. ChatGPT is quite good at summarizing and also at building templates.
You could use the following prompt, for instance:
- I have been tasked to write an email to Fake_Name, the <INSERT ROLE> of Fake_Company about a vulnerability, which I have recently discovered. The vulnerability is <INSERT VULNERABILITY NAME>. The email should focus on <INSERT PERSPECTIVE>
This prompt may create a base to work on. Usually, there are some changes needed, but at least you do not have to start from scratch. However, for all that I've mentioned up until now, make sure not to add any confidential or sensitive information (learn more about the ChatGPT databreach).
Darius Moldovan
Darius, an active member of the cybersecurity community and experienced pentester, uses helpful examples to highlight in which areas you can max out ChatGPT’s capabilities:
The best example to use ChatGPT very easily is for information gathering or reconnaissance because when working with very large targets, you need to be efficient and optimal.
For example, you have a very large infrastructure with a lot of internal services. With ChatGPT, you can quickly make a script and check the whole infrastructure to see if service "X" or "Y" is vulnerable. As a result, you, as a pentester, will save time exploiting other services. It's all about optimization and efficiency.
Another example where you can use ChatGPT - although it is a bit harder because of the restrictions imposed (ChatGPT has some restrictions for some types of keywords like "exploitation," “malicious technique,” and so on) - is in the exploitation process. If you have a vulnerability that depends on a chain of other vulnerabilities, you can use ChatGPT to code as efficiently and as fast as possible.
Sometimes, ChatGPT is also useful to make a summary of the internal documentation of the application if it is public.
Vuk Markovic
Vuk mentions key areas where ChatGPT comes into play and how much you can rely on it, while also highlighting its limitations.
While ChatGPT won't print out anything that could be used for malicious purposes, it will always display possibilities in theory, rarely in code. For coding, red teamers and pentesters should either develop their own tool usage or visit various websites and analyze and modify the code before usage.
Social engineering: ChatGPT can craft convincing phishing emails, so it will be easier for red teamers and penetration testers to create specific phishing campaigns with specific targets.
Information gathering: It can help with the automation of collecting publicly available information about targeted organizations or well-known individuals. On the other hand, if there is a specific target within the organization that is not a publicly known figure, ChatGPT won’t know what to do, nor will it display any information.
Attack scenario development: It can help with developing attack scenarios and simulate potential threat actor behaviors to test organizations’ defenses. Though this can help only if there are some pieces missing or help red teamers and pentesters get started, it will never display the whole attack scenario nor will it give the tools to be used in the particular attack scenario.
In earlier versions, ChatGPT would display any code, build exploit, malware, effort, etc. Nowadays, with the updated version, it’s not very helpful. Personally, I like using my own tools and researching using Wayback machines, Google Dorking, and TraceLabs because they give more accurate results.
In my personal opinion, as much as I don’t like to admit it, pentesters will need to learn to code and be proficient in at least one programming language. Though that wasn’t the case before, more and more companies like to automate things, and pentesters would need to do both manual and automatic pentesting, which is taking too much time.
ChatGPT can help nowadays with automated vulnerability assessment, at most, for the actual penetration test we are doing on our own.
Frédéric Noppe
ChatGPT is a great tool to formulate phishing emails.
OpenAI has implemented safeguards to prevent the generation of malicious content by prohibiting ChatGPT from crafting communications that are aggressive, threatening, or coercive.
However, you can circumvent these protections by cleverly framing requests, such as drafting an email for a hypothetical novel, which can result in the inadvertent creation of phishing content.”
Frédéric showcases how ChatGPT helps pentesters to better manage their phishing tactics using its prompts for specific tasks.
“Furthermore, ChatGPT can also contribute to the technical aspects of phishing operations. It is proficient in composing code in languages such as HTML, PHP, or Python, requiring only minimal modifications, such as embedding specific images, to tailor the code for a phishing campaign.
To fully leverage ChatGPT's capabilities for such offensive purposes, you must employ ingenuity in formulating prompts to navigate around the platform's ethical constraints.
6 tactics offsec pros use to keep learning about AI as it evolves
Madalin highlights the importance of prioritization to do your best work.
Generally speaking, I think you always have time; you just need to remove everything that has no return on investment in your life and then prioritize what you're left with.
I don't spend my days on X or social media to search for cybersecurity posts; it’s lost time from my point of view.
I usually read about new attacks or vulnerabilities during weekends, choose the most interesting one, recreate the attack/vulnerability environment in my home lab, and then exploit it.
Now I’ve actually gained a new practical skill rather than reading about 40 techniques in a week but never having time to apply them.
In terms of methodologies and resources he uses to keep up with changes in the AI space, Alexandru points out that:
Currently, I’m using OSINT sources as well as technical papers to understand how LLMs are built. A hot topic in security for AI products is prompt injection. In order to perform such engagements, you have to acquire the fundamentals and the way these products are developed.
As for Matei, he emphasizes how he integrated ChatGPT into his workflow to enhance productivity and make room for (more) creative work.
I don't do much in this sense. I actively use ChatGPT and have been leveraging artificial intelligence to improve my workflow long before the hype with ChatGPT. I think that people should use common sense and learn by doing. Nevertheless, there is a plethora of credible resources online which present the developments in this space in bite-size chunks.
Moving on to Darius, we’re reminded of the power of building your own system to learn as much as possible about this field:
I’ve developed a good mechanism to keep myself updated with the trends in this field, but I also have to admit that it can get difficult at times. People should be aware that, in pentesting - and cybersecurity in general - not everything is as rosy as it seems.
The cybersecurity field is growing very fast, and all kinds of advanced techniques that take time to understand are emerging incredibly fast. My recommendation is to make a fairly orderly schedule of studying new techniques almost every day and reading articles.
Even books that seem outdated are sometimes useful for understanding technical details. At some point, you will see a kind of pattern that repeats for every vulnerability, with only small differences. That's why it's important to always take notes about what you are doing and practicing in your environment or a CTF competition.
Vuk reminds us about cultivating the learner’s mindset and why time management is key:
It’s complicated; sometimes there would be a lack of sleep; sometimes there would be an issue to be redone; sometimes there would be assistance in guidance; sometimes I think to myself that I became an engineer not to have communication with people, and yet today, we need to have more communication with people than people in sales. On the other hand, if I could choose again, I wouldn’t change my career for anything else.
To sum it up:
We need to stay up-to-date, so continuous learning is involved, as are certifications based on companies or clients requirements, hands-on practice (TryHackMe, Hack the Box, etc.), collaborating with diverse teams and different departments, comprehensive knowledge and understanding of networking and systems, getting familiar with new tools, research, and experimentation in a safe environment.
All of this takes time, so time management plays a big role in my life. Also, specialize yourself in one area of pentesting or red teaming, either web application pentesting and security or network pentesting and security. Both are broad topics to be covered, but with time, effort, and dedication, you will achieve a goal that you set for yourself.
As for Frédéric, he values sharing opinions with peers and reading as much as possible.
In my pursuit of the latest developments in IT, I predominantly turn to LinkedIn and other social media platforms for "fast" news. I also engage with articles from esteemed IT journals such as Heise or Golem.
Dialogues with colleagues are invaluable, often leading to the exchange of new research or papers in the realm of AI and IT security. On average, I dedicate 5 to 10 hours weekly to stay abreast of industry trends, a commitment that can escalate with the emergence of significant events influencing IT security landscapes or our client systems. In such cases, I am prepared to dive deeper into the subject matter to ensure informed decision-making while consulting my clients.
3 key limitations in using ChatGPT in pentesting
Security concerns
Pentest folks know every program is vulnerable, and ChatGPT is not an exception. It’s crucial to not share clients' confidential data with ChatGPT without their consent. The model does not really know about security awareness and may generate responses that could pose risks in a real-world security assessment.
No target-specific knowledge
Recon is the most important step in every pentest you’ll ever do, which IppSec also highlighted in the most recent podcast episode. Lack of knowledge about target-specific systems, networks, or applications is one of the reasons ChatGPT won’t replace pentesters’ jobs soon. Pentesting is often context-dependent, and understanding the intricacies of the target system (beyond issues you can categorize) is crucial.
Bridging the gap between before and now
Cybersecurity is a popular domain to be in, but those working in it often struggle to stay up-to-date because of the overflow of new information coming out every day. OpenAI’s ChatGPT model was trained using Reinforcement Learning from Human Feedback (RLHF) to follow specific instructions in a prompt and provide detailed answers, including for CVEs and recent threats.
You can feed it with URLs and then ask questions based on data from that link but that’s basically doing the research on your own and then asking ChatGPT. It’s good for a quick and easy topic but when you have something more complex I would say it’s a no-no.
How you use ChatGPT influences how relevant your work is
Yes, ChatGPT can provide helpful insights on various aspects of penetration testing and automate some of them, but we must understand its limitations to maintain the level of quality we seek to deliver.
It’s our job to combine its capabilities with our human critical thinking to deliver effective pentesting reports for our clients and improve our own workflow.
For that, we need constant reality checks on how this technology evolves and how it can effectively help us.
A big THANK YOU to all the 6 offensive security specialists who offered pro tips on how to integrate AI into your work while staying relevant in this field. We’ll make sure to update this article with fresh, nuanced opinions from other offensive security fellows.
If you want to dive deeper into the hacker’s mindset and learn about their processes, practical experiences, and more, make sure you give a listen to our podcast: We think we know.
Can’t wait to hear from you!
