Log disk utilization is high-fortinet-FortiOS

Log disk utilization is high-fortinet-FortiOS
0

Log disk utilization is high-fortinet-FortiOS

Vendor: fortinet

OS: FortiOS

Description:
Indeni will alert when the log disk utilization on Fortinet devices is high.

Remediation Steps:

  1. Login via https to the Fortinet firewall and then go to the menu “Log & Report” to review the Local Disk utilization pie and Historical Disk Usage graph.
    |2. Login via ssh to the Fortinet firewall and run the FortiOS command “diagnose sys logdisk usage” to review the HD usage and the HD logging space per VDOM.
    |3. If the disk is almost full, transfer the logs or data off the disk to free up space. When a disk is almost full it consumes a lot of resources to find the free space and organize the files. Clean all unused files routinely.
    |4. Remove any debug files after debugging is done.
    |5. If the FortiGate unit has a hard disk, it is enabled by default to store logs. Consider storing logs to Syslog, FortiAnalyzer or FortiCloud instead of memory or hard disk. Logging to local disk will impact overall performance and reduce the lifetime of the unit. Fortinet recommends logging to feature rich FortiCloud or FortiAnalyzer which don’t use much CPU resources.
    |6. Consider enabling the email alert FortiOS feature if the disk usage exceeds 75%. To achieve this login via https to the Fortinet firewall and then go to the menu “Log & Report” to enable the Email Alert Settings. Then choose the “Disk usage exceeds” tab. More details can be found at http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-system-administration-54/Monitoring/Alert%20email.htm
    |7. If the FortiGate unit has only flash memory, disk logging is disabled by default, and it is recommended to keep this default setting. Constant rewrites to flash drives can reduce the lifetime and efficiency of the memory.
    |8. Both logging and WAN Optimization can use hard disk space to save data. On the FortiGate, go to System > Advanced > Disk Settings to switch between Local Log and WAN Optimization. More details can be found at http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-WAN-opt-54/wan_op_intro.htm
    |9. Contact Fortinet Technical support at https://support.fortinet.com/ for further assistance.

How does this work?
This script logs into the FortiGate using SSH and retrieves the local disk information using the output of the “diag sys logdisk usage” command. The ‘diag sys logdisk usage’ command provides detailed information about how much space is currently available to the device.

Why is this important?
This metric is used to identify when a high percentage of the local disk is being utilized. It should be noticed that the FortiGate unit uses only 75 percent of the available disk capacity to avoid a high storage amount, so when there is a high percentage, it refers to the percentage of the 75 percent that is available. For example, 92 percent of the 75 percent is available. Check the link below for more information “FortiGate advanced logging techniques”: http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-logging-reporting-54/config-log-advanced.htm

Without Indeni how would you find this?
An admin would need to log into the Fortinet firewall and manually review the HD utilization. It is possible to use SNMP traps to notify the administrator when disk space usage exceeds a threshold value e.g. 80%. Besides a log message can be generated in case of high disk utilization.

fortios-diagnose-sys-logdisk-usage

name: fortios-diagnose-sys-logdisk-usage
description: FortiGate Diagnose disk utilization
type: monitoring
monitoring_interval: 20 minutes
requires:
    vendor: fortinet
    os.name: FortiOS
    product: firewall
comments:
    fortios-hd-usage:
        why: |
            This metric is used to identify when a high percentage of the local disk is being utilized. It should be
            noticed that the FortiGate unit uses only 75 percent of the available disk capacity
            to avoid a high storage amount, so when there is a high percentage, it refers to the percentage of
            the 75 percent that is available. For example, 92 percent of the 75 percent is available.
            Check the link below for more information "FortiGate advanced logging techniques":
            http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-logging-reporting-54/config-log-advanced.htm
        how: |
            This script logs into the FortiGate using SSH and retrieves the local disk information using the output of
            the "diag sys logdisk usage" command. The 'diag sys logdisk usage' command
            provides detailed information about how much space is currently available to the device.
        can-with-snmp: true
        can-with-syslog: true
    fortios-hd-log-space:
        why: |
            This metric is needed to identify high local disk utilization for the disk partition used for logging.
            Check the link below for more information "FortiGate advanced logging techniques":
            http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-logging-reporting-54/config-log-advanced.htm
        how: |
            This script logs into the FortiGate using SSH and retrieves the local disk information using the output of
            the "diag sys logdisk usage" command. The 'diag sys logdisk usage' command provides detailed information
            about how much space is currently being used for logs.
        can-with-snmp: true
        can-with-syslog: true
    fortios-hd-log-usage-vdom:
        why: |
            This metric is needed to identify high local disk utilization for the disk partition used for logging. The
            logging space usage is provided per VDOM. Check the link below for more information "FortiGate advanced logging
            techniques":
            http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-logging-reporting-54/config-log-advanced.htm
        how: |
            This script logs into the FortiGate using SSH and retrieves the local disk information using the output of the
            "diag sys logdisk usage" command. The 'diag sys logdisk usage' command provides detailed information about how
            much space is currently being used for logs per VDOM.
        can-with-snmp: true
        can-with-syslog: true
steps:
-   run:
        type: SSH
        command: diagnose sys logdisk usage
    parse:
        type: AWK
        file: diagnose_sys_logdisk_usage.parser.1.awk

FortinetLogDiskUsage

// Deprecation warning : Scala template-based rules are deprecated. Please use YAML format rules instead.

package com.indeni.server.rules.library.templatebased.fortinet

import com.indeni.server.rules.RuleContext
import com.indeni.server.rules.library.templates.NumericThresholdOnDoubleMetricWithItemsTemplateRule
case class FortinetLogDiskUsage() extends NumericThresholdOnDoubleMetricWithItemsTemplateRule(
    ruleName = "FortinetLogDiskUsage",
    ruleFriendlyName = "Fortinet Devices: Log disk utilization is high",
    ruleDescription = "Indeni will alert when the log disk utilization on Fortinet devices is high.",
    metricName = "fortios-hd-usage",
    threshold = 80.0,
    applicableMetricTag = "name",
    alertItemsHeader = "Affected Disks",
    alertItemDescriptionFormat = "The current disk log usage is %.0f%%",
    alertDescription = "Log disk utilization is high.",
    baseRemediationText =
       """1. Login via https to the Fortinet firewall and then go to the menu "Log & Report" to review the Local Disk utilization pie and Historical Disk Usage graph.
         |2. Login via ssh to the Fortinet firewall and run the FortiOS command “diagnose sys logdisk usage” to review the HD usage and the HD logging space per VDOM.
         |3. If the disk is almost full, transfer the logs or data off the disk to free up space. When a disk is almost full it consumes a lot of resources to find the free space and organize the files. Clean all unused files routinely.
         |4. Remove any debug files after debugging is done.
         |5. If the FortiGate unit has a hard disk, it is enabled by default to store logs. Consider storing logs to Syslog, FortiAnalyzer or FortiCloud instead of memory or hard disk. Logging to local disk will impact overall performance and reduce the lifetime of the unit. Fortinet recommends logging to feature rich FortiCloud or FortiAnalyzer which don’t use much CPU resources.
         |6. Consider enabling the email alert FortiOS feature if the disk usage exceeds 75%. To achieve this login via https to the Fortinet firewall and then go to the menu “Log & Report” to enable the Email Alert Settings. Then choose the “Disk usage exceeds” tab. More details can be found at http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-system-administration-54/Monitoring/Alert%20email.htm
         |7. If the FortiGate unit has only flash memory, disk logging is disabled by default, and it is recommended to keep this default setting. Constant rewrites to flash drives can reduce the lifetime and efficiency of the memory. 
         |8. Both logging and WAN Optimization can use hard disk space to save data. On the FortiGate, go to System > Advanced > Disk Settings to switch between Local Log and WAN Optimization. More details can be found at http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-WAN-opt-54/wan_op_intro.htm
         |9. Contact Fortinet Technical support at https://support.fortinet.com/ for further assistance.""".stripMargin
    )()