Reporting False Postives/Negatives with AED

General Questions and Answers

False Positives/Negatives with AED

  • If you are an AED customer, our goal is to release an update the same day you report the issue. So please report any concerns, bugs or feature requests you may have to us. We’re here to help!

False Positives/Negatives in AED-Lite and the Real Time Rules

  • If you are an AED-Lite or real time rules customer, our goal is to release an update the same day you report the issue. So please report any concerns, bugs or feature requests you may have to us. We’re here to help!

False Positives/Negatives in the Delayed Rules

  • If you are using the delayed rules, and are reporting a false positive or false negative, no further updates will be available. An update will be available in the real time rules generally the same time you report the issue, and may already be resolved in the real time rules. It is recommended that all delayed users upgrade to the real time rules.

WAF/Modsecurity rules False Positives/Negatives

Reporting False Positives when Running AED

  • If you have purchased AED then follow this procedure. If you are not running AED then please use the procedure below this.

    If you report a false positive we generally release a fix for the issue the same business day. If you aren’t sure if something AED has detected is malicious or may just be a false positive, report it to us. Consider us your Security Gurus - we’ll figure it out for you, and let you know.

    Please make sure you are running the most up to date version of the real time rules before reporting a false positive. We publish updates several times a day, and its possible your issue may have already been resolved.

    If AED blocks something it shouldn’t you can report a False Positive to our support team by simply clicking the “False Positive” button in the GUI. You will find this button on the right side in the event details windows. You can pull up an event details window by clicking on the the specific event you wish to view in the main AED GUI screen.

    This will open up a case in the support portal, and if you have setup a support portal account your False Positive will be added to your account for review. If you have AED configured to send alerts to one of the email addresses associated with your account then you will see your False Positives show up in real time in the support portal. If not, then a member of our support team will have to manually associate your reports with your account and this can take some time.

    If you can not use the GUI to report a false positive or you are a rules only member (and therefore do not have the AED GUI), you can report false positives from the command line. For example, if you have an event like this in your /var/log/httpd/audit_log file:

    [modsecurity] [client 1.2.3.4] [domain yourdomain.com] [403] [/20091115/20091115-1635/20091115-163542-rM-wwlKl8i4AACHwQ70AAAAa]
[file "/etc/httpd/modsecurity.d/10_asl_rules.conf"] [line "224"] [id "340026"] [rev "49"]
[msg "Atomicorp.com WAF Rules: PHP Injection attempt in URI"] [data ""] [severity "CRITICAL"]
Access denied with code 403 (phase 2). Match of "beginsWith http://%{SERVER_NAME}/" against "MATCHED_VAR" required.

    The fourth variable, highlighted above as [/20091115/20091115-1635/20091115-163542-rM-wwlKl8i4AACHwQ70AAAAa], is the unique token for the event. If you have AED installed, you can report it with this command:

    asl --report-false-positive /20091115/20091115-1635/20091115-163542-rM-wwlKl8i4AACHwQ70AAAAa

    This method uses e-mail to send the false positive, so make sure your system can send email out and check your mail logs to ensure that email is being delivered to our servers. AED does not control or manage email on your system.

Reporting False Negatives when Running AED

If you discover an attack or method that is not detected or prevented by AED, we want to know. Please provide as much information as possible about how the attack or method works, and if you can not do this we would ask that you provide us with access to your system. The process for providing us access to your system is documented here.

Please provide the following information for any False Negatives:

  • Your account name

  • Access logs for the attack or method (please do not provide the output of a third party tool, we need the actual logs from the effected daemon)

  • Location of the logs (For example, /var/log/something/log_file)

  • Your active response log file /var/ossec/logs/active-responses.log

  • Your OSSEC logs, tar up the entire contents of this directory (this may be a large file): /var/ossec/logs/alerts/ (You may need to provide a URL for this file if it is particularly large)

  • If this was the result of a vulnerability scan, please confirm that this is not a false positive of the scanner first. Many PCI-DSS scanners are prone to false positives, ensure that this is not the case before reporting it as a false negative.

  • If this was the result of a vulnerability scan, we will need the vulnerability scan report.

  • If this involves a malicious file, please provide any uploaded or malicious software in a zip, tar, rar or bz archive format. Please put a password on the archive to ensure that it is not rejected by any email antivirus scanners in route to us.

  • How it was discovered (uploaded to system, found on desktop, log analysis etc.)

To report the false negative, please open either a case in the support portal or email us the false negative report to support AT atomicorp DOT com.

Reporting False Positives when not running AED

If you are running the Atomic ModSecurity Rules and you are not running AED then you will need to report a false positive manually. modsecurity does not come with a simple tool for reporting false positives like AED does. AED will do all of this for you, so if you are running AED you do not need to follow this procedure, just click on the Report False Positive in the AED GUI and you are all set.

You can also use this process with AED if you prefer to report False Positives from the command line.

1. Make sure you are running the latest rules first.

  • Please make sure you are running the most up to date version of the real time rules before reporting a false positive. We publish updates several times a day, and its possible your issue may have already been resolved.

2. Make sure ModSecurity is setup correctly

  • If you are running AED, you can skip this step.

  • If you are not running AED, first you must ensure that you have setup modsecurity properly to log events in enough detail that we can help you. Please ensure that you have modsecurity configured according to the instructions on our Atomic ModSecurity Rules page.

3. Get the audit_log event for this event by following the process below:

Note

The Apache error_log is not your audit_log, and apache error log events do not contain the information needed to diagnose a false positive. Please follow this process to send us your audit_log entry for this event.

If you followed the instructions on the Atomic ModSecurity Rules page, or you have installed AED, then you should have audit logging setup correctly and will have a directory called:

/var/asl/data/audit

If you do not have this directory stop here and go back to step 2. Make sure you have modsecurity setup according to the instructions on the Atomic ModSecurity Rules page.

This directory will create a special directory for each event that is blocked on your system by first creating a unique directory for that day. The format of the directory is YEARMONTHDATE. For example, if an event occurred on November 25th, 2009, there would a directory of the following format:

/var/asl/data/audit/20091125

Within that directory will be subdirectories that correspond to events that happen at specific time. The format is YEARMONTHDAY-HOURMINUTE. So if an event occured at 17:23 hours on November 25th, 2009, the corresponding directory would be:

/var/asl/data/audit/20091125/20091125-1723

Inside this directory will be the audit payloads of all the events that occured in that time period. If many events occured within that minute there will be multiple files. The /var/log/http/audit_log file will contain a running log of the events so you can look up the directories they are stored in. For example this audit_log entry:

[modsecurity] [client 1.2.3.4] [domain example.com] [403] [/20091125/20091125-1723/20091125-172310-gPA9rUrQYacAACJpX@UAAAAK] [file "/etc/httpd/modsecurity.d/10_asl_antimalware.conf"] [line "42"] [id "360000"] [rev "2"] [msg "Atomicorp.com Malware Blacklist: Malware Site detected in Argument (AE)"] [data "example.com/"] [severity "CRITICAL"]
Access denied with code 403 (phase 2). Matched phrase "example.com/" at ARGS:sourcedir.

The audit_log event ID for this event is highlighted above, and is:

/20091125/20091125-1723/20091125-172310-gPA9rUrQYacAACJpX@UAAAAK

You will need this ID to get the actual audit_log event payload, which will actually include the whole event, its headers, payload and so on. This is the information that is needed to diagnose and fix a false positive.

4. Get the audit_log entry for this event by following the process below

The payload for this event can be found by appending the event ID, in the example above “/20091125/20091125-1723/20091125-172310-gPA9rUrQYacAACJpX@UAAAAK” to the path “/var/asl/data/audit”.

For AED, run the following command:

asl --show-alert /20091125/20091125-1723/20091125-172310-gPA9rUrQYacAACJpX@UAAAAK

If you are not using AED, then you will need to manually display the event. For example:

cat /var/asl/data/audit/20091125/20091125-1723/20091125-172310-gPA9rUrQYacAACJpX@UAAAAK

In this case the example attack looks like the following:

--907db756-A--
[25/Nov/2009:17:23:10 --0500] gPA9rUrQYacAACJpX@UAAAAK 1.2.3.4 55732 74.208.97.167 80

--907db756-B--
GET /delayed/rules/modsec/?sourcedir=http://example.com/mambots/idxx.txt?%20? HTTP/1.1
TE: deflate,gzip;q=0.3
Connection: TE, close
Host: example.com
User-Agent: Mozilla/5.0

--907db756-F--
HTTP/1.1 403 Forbidden
Content-Length: 303
Connection: close
Content-Type: text/html; charset=iso-8859-1

--907db756-H--
Message:  [file "/etc/httpd/modsecurity.d/10_asl_antimalware.conf"] [line "42"] [id "360000"] [rev "2"] [msg "Atomicorp.com Malware Blacklist: Malware Site detected in Argument (AE)"] [data "example.com/"] [severity "CRITICAL"] Access denied with code 403 (phase 2). Matched phrase "example.com/" at ARGS:sourcedir.
Action: Intercepted (phase 2)
Stopwatch: 1259187790167469 27763 (4424 27390 -)
Producer: ModSecurity for Apache/2.5.11 (http://www.modsecurity.org/); 200911221938.
Server: Apache/2.2.3 (CentOS)

--907db756-Z--

5. Report the False Positive to Atomicorp

Reporting False Negatives when not running AED

If you discover an attack or method that is not detected or prevented by Mod_Security, we want to know. Please provide as much information as possible about how the attack or method works, and if you can not do this we would ask that you provide us with access to your system. The process for providing us access to your system is documented here.

Please provide the following information for any False Negatives:

  • Your account name

  • Web access logs for the attack or method

  • If this was the result of a vulnerability scan, please confirm that this is not a false positive of the scanner first. Many PCI-DSS scanners are prone to false positives, ensure that this is not the case before reporting it as a false negative.

  • If this was the result of a vulnerability scan, we will need the vulnerability scan report.

  • Any uploaded or malicious software in a zip, tar, rar or bz archive format. Please put a password on the archive to ensure that it is not rejected by any email antivirus scanners in route to us.

  • How it was discovered (uploaded to system, found on desktop, etc.)

  • To report the false negative, please open either a case in the support portal or email us the false negative report to support AT atomicorp DOT com.

ClamAV False Positives/Negatives

To report a false positive with a ClamAV Signature

  • Please send us the following information with your false positive report:

    1. Your account name

    2. The name of the signature that was triggered

    3. How it was triggered (uploaded via FTP/Web, blocked locally via dazuko, etc.)

    4. The specific error message(s) your application produced when the file was rejected.

    5. The AED web console message, or the log entry in /var/log/clamav/clamd.log for this event.

    6. And the file that was incorrect detected in a zip, tar, rar or bz archive format. Please put a password on the archive to ensure that it is not rejected by any email or upload antivirus scanners in route to us.

To report the false positive, please open either a case in the support portal or email us the false positive report to support AT atomicorp DOT com.

To report a false negative

Please send us the following information with your false negative report:

  1. Your account name

  2. The name of the malware as detected by other scanners (if this information exists, if this is a zero day, please state that)

  3. How it was discovered (uploaded to system, found on desktop, etc.)

  4. And the malicious software in a zip, tar, rar or bz archive format. Please put a password on the archive to ensure that it is not rejected by any antivirus scanners in route to us.

To report the false negative, please open either a case in the support portal or email us the false negative report to support AT atomicorp DOT com.

Note

AED includes malware upload protection support, Rules Only licenses do not.

AED HIPS/KIPS/WIPS False Postives/Negatives

To report a false positive

  • Just click on the “False Positive” button in the AED GUI for that event. That will send us all the information we need.

To report a false negative

If you discover an attack or method that is not detected or prevented by AED, we want to know. Please provide as much information as possible about how the attack or method works, and if you can not do this we would ask that you provide us with access to your system. The process for providing us access to your system is documented here.

Please send us the following information with your false negative report:

  1. Your account name

  2. The version of AED you have installed, and the output of the commands “asl -ck” and “asl -s”.

  3. Full logs for the attack or method

  4. If this was the result of a vulnerability scan, please confirm that this is not a false positive of the scanner first. Many PCI-DSS scanners are prone to false positives, ensure that this is not the case before reporting it as a false negative.

  5. If this was the result of a vulnerability scan, we will need the vulnerability scan report.

  6. How it discovered (uploaded to system, found on desktop, etc.)

To report the false negative, please open either a case in the support portal or email us the false negative report to support AT atomicorp DOT com.

Vulnerability Scanner False Positives/Negatives

To report a false positive

Please send us the following information with your false positive report:

  1. Your account name

  2. The version of AED you have installed

  3. Please cut and paste the AED scanner alert and any supporting configuration files that show the vulnerability does not exist.

To report the false positive, please open either a case in the support portal or email us the false positive report to support AT atomicorp DOT com.

To report a false negative

If you discover a vulnerability is not detected or prevented by AED, we want to know. Please provide as much information as possible about the vulnerability - for example if you used a tool to find, please let us know what that tool is and how you used it. If you can not do this we would ask that you provide us with access to your system. The process for providing us access to your system is documented here.

Please send us the following information with your false negative report:

  1. Your account name

  2. CVE reference

  3. As many details as you can provide of the vulnerability, such as links to the publication of the vulnerability such as vendor and security researcher reports.

  4. If this was the result of a vulnerability scan with another tool, please confirm that this is not a false positive of the scanner first. Many PCI-DSS scanners are prone to false positives, ensure that this is not the case before reporting it as a false negative.

  5. If this was the result of a vulnerability scan, we will need the vulnerability scan report.

To report the false negative, please open either a case in the support portal or email us the false negative report to support AT atomicorp DOT com.