Can You Use ChatGPT in Compliance?

Published 06/12/2023

One of the latest intriguing developments in the field of artificial intelligence (AI) is ChatGPT—a natural language chatbot that answers questions submitted by a human user. It’s taken off in such a way that many are using ChatGPT to assist in streamlining their writing needs, but how helpful is the bot, really?

For instance, could you use ChatGPT to help with your compliance activities? Any efficiency in satisfying your compliance needs would be wholly welcomed, which is why we gave ChatGPT a few PCI DSS prompts to see the quality of its yield.

If you’re wondering if this AI can help you with compliance, read on to understand the results of this experiment as well as our expert take.

Using ChatGPT to Write Compliance Policies and Procedures

To start, we provided ChatGPT with descriptions of policies/procedures that need to be written with the idea that the AI would respond with a brief but still descriptive output that reflected knowledge of IT concepts and security and compliance standards and also met compliance requirements.

We did this three different times:

"Write a Policy That Meets the PCI Requirements for Data Retention."

ChatGPT’s Response:

Policy Title:	Data Retention Policy for PCI Compliance
Objective:	“To establish guidelines for the retention of cardholder data in accordance with the Payment Card Industry Data Security Standard (PCI DSS) requirements.”
Scope:	“This policy applies to all personnel involved in the handling, processing, or storage of cardholder data (CHD) within our organization.”
Policy Statement:	“Our organization is committed to ensuring the security and privacy of cardholder data in accordance with PCI DSS requirements. To achieve this goal, the following guidelines must be followed: Retention of Cardholder Data: CHD must not be retained for any longer than is necessary to meet business, legal, or regulatory requirements. Cardholder data should only be retained if there is a legitimate business need to do so. Any retention of CHD must be approved by the appropriate department head and reviewed by the compliance team. Secure Storage and Disposal: All CHD must be stored securely and protected from unauthorized access. CHD must be disposed of securely and in accordance with our organization's data disposal policy. All devices that contain CHD must be wiped of all data before disposal, sale, or transfer. Record Keeping: Our organization must maintain records of all CHD retention policies and procedures. All records relating to CHD retention must be kept for a minimum of one year unless a longer retention period is required by law or regulation. Compliance Monitoring: Compliance with this policy will be monitored on a regular basis. Any deviations from this policy must be reported to the compliance team and addressed immediately. Enforcement: Any employee found to have violated this policy may be subject to disciplinary action, up to and including termination of employment. Additionally, any violations of this policy may result in our organization being subject to fines, legal action, or loss of reputation.”
Conclusion:	“Our organization is committed to the protection of cardholder data and compliance with PCI DSS requirements. This policy outlines guidelines for the retention of cardholder data and should be adhered to by all personnel involved in the handling, processing, or storage of CHD.”
Our Expert Take:	Pretty darn good! It could use some tailoring, but overall, ChatGPT did a fair job of making sure that the right content was present.

“Write a Policy that Meets the PCI Requirements for Audit Log Retention.”

ChatGPT’s Response:

Policy Title:	Audit Log Retention Policy for PCI Compliance
Objective:	“To establish guidelines for the retention of audit logs in accordance with the Payment Card Industry Data Security Standard (PCI DSS) requirements.”
Scope:	“This policy applies to all personnel involved in the handling, processing, or storage of CHD within our organization.”
Policy Statement:	“Our organization is committed to ensuring the security and privacy of cardholder data in accordance with PCI DSS requirements. To achieve this goal, the following guidelines must be followed: Retention of Audit Logs: Audit logs must be retained for a minimum of one year. Audit logs must be kept in a format that cannot be altered, and that provides a clear audit trail. Audit logs must be reviewed regularly to detect any suspicious or anomalous activity. Secure Storage: Audit logs must be stored securely and protected from unauthorized access. Access to audit logs must be restricted to authorized personnel only. Audit logs must be stored in a separate system or location from the system being audited. Record Keeping: Our organization must maintain records of all audit log retention policies and procedures. All records relating to audit logs must be kept for a minimum of one year unless a longer retention period is required by law or regulation. Compliance Monitoring: Compliance with this policy will be monitored on a regular basis. Any deviations from this policy must be reported to the compliance team and addressed immediately. Enforcement: Any employee found to have violated this policy may be subject to disciplinary action, up to and including termination of employment. Additionally, any violations of this policy may result in our organization being subject to fines, legal action, or loss of reputation.”
Conclusion:	“Our organization is committed to the protection of cardholder data and compliance with PCI DSS requirements. This policy outlines guidelines for the retention of audit logs and should be adhered to by all personnel involved in the handling, processing, or storage of CHD.”
Our Expert Take:	Again, a really good job providing a general overview of what a policy should look like. Of course, you would need to edit the policy a bit to suit your organization more.

“Write a Procedure for Reviewing Audit Logs.”

ChatGPT’s Response:

Procedure Title:	Audit Log Review Procedure
Objective:	“To provide guidelines for the regular review of audit logs in accordance with the Payment Card Industry Data Security Standard (PCI DSS) requirements.”
Scope:	“This procedure applies to all personnel involved in the handling, processing, or storage of CHD within our organization.”
Procedure Steps:	“Identify Key Log Sources: Identify the systems and applications that generate audit logs. Determine the criticality of each system/application and its log sources. Schedule Regular Reviews: Establish a schedule for reviewing the audit logs. The frequency of reviews should be based on the criticality of the system and the volume of log data. Access to Audit Logs: Access to audit logs must be restricted to authorized personnel only. Ensure that the audit logs are accessible to the authorized personnel who need to review them. Analyze Audit Logs: Review the audit logs for any suspicious or anomalous activity. Identify any potential security breaches or incidents that have occurred. Document Findings: Document any findings from the audit log review. If any incidents are identified, document the steps taken to investigate and resolve the incident. Remediation: If any issues are identified during the audit log review, remediate them in a timely manner. Ensure that appropriate corrective actions are taken to prevent similar incidents from occurring in the future. Retention of Audit Logs: Ensure that the audit logs are retained for a minimum of one year, as required by PCI DSS. Store audit logs in a secure manner to prevent unauthorized access or tampering. Enforcement: Any employee found to have violated this procedure may be subject to disciplinary action, up to and including termination of employment. Additionally, any violations of this procedure may result in our organization being subject to fines, legal action, or loss of reputation.”
Conclusion:	“Our organization is committed to the protection of cardholder data and compliance with PCI DSS requirements. This procedure outlines the guidelines for the regular review of audit logs and should be adhered to by all personnel involved in the handling, processing, or storage of CHD.”
Our Expert Take:	Again, pretty spot on! While it probably could have been refined better to specifically do procedures, this is still very good. ChatGPT is very consistent.

Using ChatGPT for Compliance Tasks

So, ChatGPT can write policies to a certain extent, but let’s see if it can help in other ways.

“Generate a Regular Expression That Matches Cardholder Data.”

When we asked ChatGPT this, we stipulated that we wanted a regular expression that could be easily copied and pasted into a regex-enabled text editor or programming language without any explanations or examples.

ChatGPT’s Response:

Our Expert Take:

The fact that ChatGPT can generate regular expressions can save you a lot of time.

"Write a Script That Tests Every Default Password."

When we prompted this, we hoped the bot would provide the configuration or system information needed to meet evidence needs, which requires knowledge of IT security engineering, security, and compliance standards.

ChatGPT’s Response:

“As an AI language model, I can provide you with the necessary information on how to approach your request as an IT auditor. However, it is essential to note that attempting to test every default password can be considered a security vulnerability and should not be done without proper authorization and approval from the relevant stakeholders.

Assuming you have received the necessary authorization, here are the steps you can take to write a script that tests every default password:

Identify the Target System: Determine the system or device for which you want to test default passwords. This could be a Linux server, network router, or any other system that uses default passwords.
Obtain a List of Default Passwords: Obtain a list of default passwords that are commonly used for the target system or device. Such lists are usually available on the internet or provided by the system/device vendor.
Write a Script: Write a Bash script that loops through the list of default passwords and attempts to log in to the target system/device using each password. You can use the "sshpass" utility to automate the process of entering passwords.
Capture Results: The script should capture the results of each login attempt, including successful logins and failed logins.
Generate a Report: Once the script has completed, generate a report that summarizes the results. The report should include details on any successful logins and any failed login attempts.
Take Corrective Actions: Based on the results of the script, take corrective actions to change any default passwords that were successfully tested or any other vulnerabilities that were identified.

It is important to note that running such a script can be risky as it could potentially cause damage to the system if not properly executed. Therefore, it is recommended to perform this type of testing in a controlled environment, and with the appropriate approvals in place.”

Our Expert Take:

This wasn’t what we were looking for, but this result might’ve been our fault for using too much of a description when we could’ve been more pointed, so you should definitely take a more specific approach for a more productive yield.

“Write a Bash Script That Checks for Every Default Password on Linux Systems.”

As such, we tried to be more specific in this prompt.

ChatGPT’s Response:

“I'm sorry, but I cannot provide you with a bash script that checks for every default password on Linux, as doing so is not a recommended security practice. Default passwords are widely known and should be changed immediately after installation to prevent unauthorized access to a system.

Instead, I would recommend checking your Linux system's password policy to ensure that users are required to create strong passwords and that password changes are enforced regularly. Additionally, consider implementing multi-factor authentication to provide an extra layer of security to your system.

If you are concerned that default passwords may have been used on your system, it would be better to perform a vulnerability scan or penetration testing by a qualified professional to identify any potential vulnerabilities and take corrective action to address them.”

Our Expert Take:

While it’s likely a good thing that ChatGPT cannot give hacking advice, it can clearly still be used to generate scripts.

“Write a Windows Powershell Script that Returns Every User in Active Directory.”

ChatGPT’s Response:

Our Expert Take:

Scripting appears to now be much easier, but remember—do not run any scripts against an environment that you don’t know.

Moving Forward with Your PCI DSS Compliance

After conducting our experiment, we confirmed that, while ChatGPT is a very powerful tool that can be used to help a bit with certain security and compliance tasks, it certainly has its limits. So, while you may choose to use it as a “free policy-writing tool” at the very least, you should still contribute some real human input to ensure you are in full compliance with the PCI DSS standard.

To help with that, we’ve disseminated many different aspects of PCI DSS, including the details of version 4.0—check out these articles to further simplify your compliance tasks:

Artificial Intelligence Compliance