News Listing

ChatGPT has started to become an indispensable tool for many people, but, cybercriminals have wasted no time in exploiting its security vulnerabilities.

OpenSSH has released 9.3p2 to fix a vulnerability in various versions of OpenSSH.

Apple has released security updates to fix the vulnerabilities in macOS and Safari.

Apple has released iOS 15.7.8, iOS 16.6, iPadOS 15.7.8 and iPadOS 16.6 to fix the vulnerabilities in various Apple devices.

A new report reveals nearly 70% of financial services and insurance companies have suffered rollout delays due to API security.

As global data privacy compliance increases in scope and complexity, only about half of executives feel “very prepared” to meet regulatory requirements in the United States, United Kingdom and European Union. 

Microsoft released a security update to address multiple vulnerabilities in Microsoft Edge.

The Department of Financial Information System for California (FISCal) has hired Jennifer Benson as the new Chief of the Information Security Office.

A key security challenge facing the online gaming industry is that regulations are lagging behind in key markets, making it a sure bet for hackers.

SUMMARY
The Cybersecurity and Infrastructure Security Agency (CISA) is releasing this Cybersecurity Advisory to warn network defenders about exploitation of CVE-2023-3519, an unauthenticated remote code execution (RCE) vulnerability affecting NetScaler (formerly Citrix) Application Delivery Controller (ADC) and NetScaler Gateway. In June 2023, threat actors exploited this vulnerability as a zero-day to drop a webshell on a critical infrastructure organization’s non-production environment NetScaler ADC appliance. The webshell enabled the actors to perform discovery on the victim’s active directory (AD) and collect and exfiltrate AD data. The actors attempted to move laterally to a domain controller but network-segmentation controls for the appliance blocked movement.
The victim organization identified the compromise and reported the activity to CISA and Citrix. Citrix released a patch for this vulnerability on July 18, 2023.
This advisory provides tactics, techniques, and procedures (TTPs) and detection methods shared with CISA by the victim. CISA encourages critical infrastructure organizations to use the detection guidance included in this advisory for help with determining system compromise. If potential compromise is detected, organizations should apply the incident response recommendations provided in this CSA. If no compromise is detected, organizations should immediately apply patches provided by Citrix.
Download the PDF version of this report:





AA23-201a PDF
(PDF, 409.52 KB
)


TECHNICAL DETAILS
Note: This advisory uses the MITRE ATT&CK for Enterprise framework, version 13. See the MITRE ATT&CK Tactics and Techniques section for a table of the threat actors’ activity mapped to MITRE ATT&CK® tactics and techniques. For assistance with mapping malicious cyber activity to the MITRE ATT&CK framework, see CISA and MITRE ATT&CK’s Best Practices for MITRE ATT&CK Mapping and CISA’s Decider Tool.
Overview
In July 2023, a critical infrastructure organization reported to CISA that threat actors may have exploited a zero-day vulnerability in NetScaler ADC to implant a webshell on their non-production NetScaler ADC appliance. Citrix confirmed that the actors exploited a zero-day vulnerability: CVE-2023-3519. Citrix released a patch on July 18, 2023.[1]
CVE-2023-3519
CVE-2023-3519 is an unauthenticated RCE vulnerability affecting the following versions of NetScaler ADC and NetScaler Gateway:[1]
NetScaler ADC and NetScaler Gateway 13.1 before 13.1-49.13
NetScaler ADC and NetScaler Gateway 13.0 before 13.0-91.13
NetScaler ADC and NetScaler Gateway version 12.1, now end of life
NetScaler ADC 13.1-FIPS before 13.1-37.159
NetScaler ADC 12.1-FIPS before 12.1-55.297
NetScaler ADC 12.1-NDcPP before 12.1-55.297
The affected appliance must be configured as a Gateway (VPN virtual server, ICA Proxy, CVPN, RDP Proxy) or authentication, authorization, and auditing (AAA) virtual server for exploitation.[1]
CISA added CVE-2023-3519 to its Known Exploited Vulnerabilities Catalog on July 19, 2023.
Threat Actor Activity
As part of their initial exploit chain [T1190], the threat actors uploaded a TGZ file [T1105] containing a generic webshell [T1505.003], discovery script [TA0007], and setuid binary [T1548.001] on the ADC appliance and conducted SMB scanning on the subnet [T1046].
The actors used the webshell for AD enumeration [T1016] and to exfiltrate AD data [TA0010]. Specifically, the actors:
Viewed NetScaler configuration files /flash/nsconfig/keys/updated/* and /nsconfig/ns.conf [T1005]. Note: These configuration files contain an encrypted password that can be decrypted by the key stored on the ADC appliance [T1552.001].
Viewed the NetScaler decryption keys (to decrypt the AD credential from the configuration file) [T1552.004].
Used the decrypted AD credential to query the AD via ldapsearch. The actors queried for:
Users (objectClass=user) (objectcategory=person) [T1033]
Computers (objectClass=computer) [T1018]
Groups (objectClass=group) [T1069.002]
Subnets (objectClass=subnet)
Organizational Units (objectClass=organizationalUnit)
Contacts (objectClass=contact)
Partitions (objectClass=partition)
Trusts (objectClass=trustedDomain) [T1482]

Used the following command to encrypt discovery data collected via openssl in “tar ball” [T1560.001]: tar -czvf - /var/tmp/all.txt | openssl des3 -salt -k -out /var/tmp/test.tar.gz. (A “tar ball” is a compressed and zipped file used by threat actors for collection and exfiltration.)
Exfiltrated collected data by uploading as an image file [T1036.008] to a web-accessible path [T1074]: cp /var/tmp/test.tar.gz /netscaler/ns_gui/vpn/medialogininit.png.
The actors’ other discovery activities were unsuccessful due to the critical infrastructure organization’s deployment of their NetScaler ADC appliance in a segmented environment. The actors attempted to:
Execute a subnet-wide curl command to identify what was accessible from within the network as well as potential lateral movement targets.
Verified outbound network connectivity with a ping command (ping -c 1 google.com) [T1016.001].
Executed host commands for a subnet-wide DNS lookup.
The actors also attempted to delete their artifacts [TA0005]. The actors deleted the authorization configuration file (/etc/auth.conf)—likely to prevent configured users (e.g., admin) from logging in remotely (e.g., CLI) [T1531]. To regain access to the ADC appliance, the organization would normally reboot into single use mode, which may have deleted artifacts from the device; however, the victim had an SSH key readily available that allowed them into the appliance without rebooting it.
The actors’ post-exploitation lateral movement attempts were also blocked by network-segmentation controls. The actors implanted a second webshell on the victim that they later removed. This was likely a PHP shell with proxying capability. The actors likely used this to attempt proxying SMB traffic to the DC [T1090.001] (the victim observed SMB connections where the actors attempted to use the previously decrypted AD credential to authenticate with the DC from the ADC via a virtual machine). Firewall and account restrictions (only certain internal accounts could authenticate to the DC) blocked this activity.
MITRE ATT&CK TACTICS AND TECHNIQUES
See Table 1–Table 9 for all referenced threat actor tactics and techniques in this advisory.
Table 1: Cyber Threat Actors ATT&CK Techniques for Initial Access

Technique Title


ID


Use


Exploit Public-Facing Application


T1190


The threat actors exploited CVE-2023-3519 to implant a webshell on the organization’s NetScaler ADC appliance.

 
Table 2: Cyber Threat Actors ATT&CK Techniques for Persistence

Technique Title


ID


Use


Server Software Component: Web Shell


T1505.003


The threat actors implanted a generic webshell on the organization’s NetScaler ADC appliance.

 
Table 3: Cyber Threat Actors ATT&CK Techniques for Privilege Escalation

Technique Title


ID


Use


Abuse Elevation Control Mechanism: Setuid and Setgid


T1548.001


As part of their initial exploit chain uploaded a TGZ file contain a setuid binary on the ADC appliance.

 
Table 4: Cyber Threat Actors ATT&CK Techniques for Defense Evasion

Technique Title


ID


Use


Masquerading: Masquerade File Type


T1036.008


The threat actors exfiltrated data by uploading it as an image file to a web-accessible path.

 
Table 5: Cyber Threat Actors ATT&CK Techniques for Credential Access

Technique Title


ID


Use


Unsecured Credentials: Credentials In Files


T1552.001


The threat actors obtained encrypted passwords from NetScaler ADC configuration files, and the decryption key was stored on the ADC appliance.


Unsecured Credentials: Private Keys


T1552.004


The threat actors obtained decryption keys to decrypt the AD credential obtained from the NetScaler ADC configuration files.

 
Table 6: Cyber Threat Actors ATT&CK Techniques for Discovery

Technique Title


ID


Use


Domain Trust Discovery


T1482


The threat actors queried the AD for trusts.


Permission Groups Discovery: Domain Groups


T1069.002


The threat actors quired the AD for groups.


Remote System Discovery


T1018


The threat actors queried the AD for computers.
The threat actors attempted to execute a subnet-wide curl command to identify what was accessible from within the network as well as potential lateral movement targets. Network-segmentation controls prevented this activity.


System Network Configuration Discovery


T1016


The actors used a webshell for AD enumeration.


System Network Configuration Discovery: Internet Connection Discovery


T1016.001


The threat actors attempted to verify outbound network connectivity with a ping command and executed host commands for a subnet-wide DNS lookup. Network-segmentation controls prevented this activity.


Network Service Discovery


T1046


The threat actors conducted SMB scanning on the organization’s subnet.


Account Discovery: Domain Account


T1087.002


The threat actors queried the AD for users.

 
Table 7: Cyber Threat Actors ATT&CK Techniques for Collection

Technique Title


ID


Use


Archive Collected Data: Archive via Utility


T1560.001


The threat actors encrypted discovery data collected via openssl in “tar ball.”


Data from Local System


T1005


The threat actors viewed NetScaler ADC configuration files flash/nsconfig/keys/updated/* and /nsconfig/ns.conf.


Data Staged


T1074


The threat actors uploaded data as an image file to a web-accessible path: cp /var/tmp/test.tar.gz /netscaler/ns_gui/vpn/medialogininit.png.

 
Table 8: Cyber Threat Actors ATT&CK Techniques for Command and Control

Technique Title


ID


Use


Ingress Tool Transfer


T1105


The threat actors exploited CVE-2023-3519 to upload a TGZ file containing a generic webshell, discovery script, and setuid binary on the ADC appliance.


Proxy: Internal Proxy


T1090.001


The actors likely used a PHP shell with proxying capability to attempt proxying SMB traffic to the DC (the traffic was blocked by a firewall and account restrictions).

 
Table 9: Cyber Threat Actors ATT&CK Techniques for Impact

Technique Title


ID


Use


Account Access Removal


T1531


The threat actors deleted the authorization configuration file (/etc/auth.conf)—likely to prevent configured users from logging in remotely (e.g., CLI).

 
DETECTION METHODS
Run the following victim-created checks on the ADC shell interface to check for signs of compromise:
Check for files newer than the last installation.
Modify the -newermt parameter with the date that corresponds to your last installation:
find /netscaler/ns_gui/ -type f -name *.php -newermt [YYYYMMDD] -exec ls -l {} \;
find /var/vpn/ -type f -newermt [YYYYMMDD] -exec ls -l {} \;
find /var/netscaler/logon/ -type f -newermt [YYYYMMDD] -exec ls -l {} \;
find /var/python/ -type f -newermt [YYYYMMDD] -exec ls -l {} \;

Check http error logs for abnormalities that may be from initial exploit:
zgrep '\.sh' /var/log/httperror.log*
zgrep '\.php' /var/log/httperror.log*

Check shell logs for unusual post-ex commands, for example:
grep '/flash/nsconfig/keys' /var/log/sh.log*

Look for setuid binaries dropped:
find /var -perm -4000 -user root -not -path "/var/nslog/*" -newermt [YYYYMMDD] -exec ls -l {} \;

Review network and firewall logs for subnet-wide scanning of HTTP/HTTPS/SMB (80/443/445) originating from the ADC.
Review DNS logs for unexpected spike in internal network computer name lookup originating from the ADC (this may indicate the threat actor resolving host post-AD enumeration of computer objects).
Review network/firewall logs for unexpected spikes in AD/LDAP/LDAPS traffic originating from the ADC (this may indicate AD/LDAP enumeration).
Review number of connections/sessions from NetScaler ADC per IP address for excessive connection attempts from a single IP (this may indicate the threat actor interacting with the webshell).
Pay attention to larger outbound transfers from the ADC over a short period of session time as it can be indicative of data exfiltration.
Review AD logs for logon activities originating from the ADC IP with the account configured for AD connection. 
If logon restriction is configured for the AD account, check event 4625 where the failure reason is “User not allowed to logon at this computer.”
Review NetScaler ADC internal logs (sh.log*, bash.log*) for traces of potential malicious activity (some example keywords for grep are provided below): 
database.php
ns_gui/vpn
/flash/nsconfig/keys/updated 
LDAPTLS_REQCERT 
ldapsearch 
openssl + salt

Review NetScaler ADC internal access logs (httpaccess-vpn.log*) for 200 successful access of unknown web resources.
INCIDENT RESPONSE
If compromise is detected, organizations should:
Quarantine or take offline potentially affected hosts.
Reimage compromised hosts.
Provision new account credentials.
Collect and review artifacts such as running processes/services, unusual authentications, and recent network connections.
Report the compromise to CISA via CISA’s 24/7 Operations Center (report@cisa.gov or 888-282-0870).
MITIGATIONS
CISA recommends all organizations:
Install the relevant updated version of NetScaler ADC and NetScaler Gateway as soon as possible. See Citrix ADC and Citrix Gateway Security Bulletin for CVE-2023-3519, CVE-2023-3466, CVE-2023-3467 for patch information.
Follow best cybersecurity practices in your production and enterprise environments, including mandating phishing-resistant multifactor authentication (MFA) for all staff and for all services. For additional best practices, see CISA’s Cross-Sector Cybersecurity Performance Goals (CPGs). The CPGs, developed by CISA and the National Institute of Standards and Technology (NIST), are a prioritized subset of information technology (IT) and operational technology (OT) security practices that can meaningfully reduce the likelihood and impact of known cyber risks and common TTPs. Because the CPGs are a subset of best practices, CISA and ACSC also recommend software manufacturers implement a comprehensive information security program based on a recognized framework, such as the NIST Cybersecurity Framework (CSF).
As a longer-term effort, apply robust network-segmentation controls on NetScaler appliances, and other internet-facing devices.
VALIDATE SECURITY CONTROLS
In addition to applying mitigations, CISA recommends exercising, testing, and validating your organization's security program against the threat behaviors mapped to the MITRE ATT&CK for Enterprise framework in this advisory. CISA recommends testing your existing security controls inventory to assess how they perform against the ATT&CK techniques described in this advisory.
To get started:
Select an ATT&CK technique described in this advisory (see Table 1–Table 9).
Align your security technologies against the technique.
Test your technologies against the technique.
Analyze your detection and prevention technologies’ performance.
Repeat the process for all security technologies to obtain a set of comprehensive performance data.
Tune your security program, including people, processes, and technologies, based on the data generated by this process.
CISA recommends continually testing your security program, at scale, in a production environment to ensure optimal performance against the MITRE ATT&CK techniques identified in this advisory.
REFERENCES
[1] Citrix Security Bulletin CTX561482: Citrix ADC and Citrix Gateway Security Bulletin for CVE-2023-3519, CVE-2023-3466, CVE-2023-3467
VERSION HISTORY
July 20, 2023: Initial version.July 24, 2023: Updated affected NetScaler ADC 12.1-FIPS and NetScaler ADC 12.1-NDcPP version numbers. Updated Step 3 in Detection Methods to use zgrep instead of grep.