RX packets dropped ratio too high-fortinet-FortiOS

RX packets dropped ratio too high-fortinet-FortiOS
0

RX packets dropped ratio too high-fortinet-FortiOS

Vendor: fortinet

OS: FortiOS

Description:
Indeni tracks the number of packets that had issues and alerts if the ratio is too high.

Remediation Steps:
Packet drops usually occur when the rate of packets received is higher than the device’s ability to handle.

   |1. Run "diag hardware deviceinfo nic <interface>" command to display a list of hardware related error names and values. Review  the next link for more details: http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
   |2. Run command "fnsysctl cat /proc/net/dev" to get a summary of the interface statistics.
   |3. Check for speed and duplex mismatch in the interface settings on both sides of a cable, and check for a damaged cable. Review the next link for more info: http://kb.fortinet.com/kb/documentLink.do?externalID=10653

How does this work?
This script logs into the FortiGate using SSH and retrieves the output of the “fnsysctl ifconfig” FortiOS hidden command. The output includes all the interface related information and statistics.

Why is this important?
Capture the interface Receive Drop counter. Packet loss may have severe impact to the traffic utilization (retransmission for TCP apps) and to the performance of the applications. Check the link below for more information about troubleshooting network interface problems: http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm

Without Indeni how would you find this?
An admin would need to log into the Fortinet firewall and manually review the interface statistics.

fortios-fnsysctl-ifconfig

name: fortios-fnsysctl-ifconfig
description: FortiGate Diagnose Interface
type: monitoring
monitoring_interval: 5 minutes
requires:
    vendor: fortinet
    os.name: FortiOS
    product: firewall
comments:
    network-interface-rx-errors:
        why: |
            Capture the interface Receive Errors counter. Receive errors indicate an issue with duplex/speed matching.
            Check the link below for more information about troubleshooting network interface problems:
            http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
        how: |
            This script logs into the FortiGate using SSH and retrieves the output of the "fnsysctl ifconfig" FortiOS hidden
            command. The output includes all the interface related information and statistics.
        without-indeni: |
            An admin would need to log into the Fortinet firewall and manually review the interface statistics.
        can-with-snmp: true
        can-with-syslog: false
    network-interface-rx-dropped:
        why: |
            Capture the interface Receive Drop counter. Packet loss may have severe impact to the traffic utilization
            (retransmission for TCP apps) and to the performance of the applications. Check the link below for more
            information about troubleshooting network interface problems:
            http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
        how: |
            This script logs into the FortiGate using SSH and retrieves the output of the "fnsysctl ifconfig" FortiOS hidden
            command. The output includes all the interface related information and statistics.
        without-indeni: |
            An admin would need to log into the Fortinet firewall and manually review the interface statistics.
        can-with-snmp: true
        can-with-syslog: false
    network-interface-rx-overruns:
        why: |
            Capture the interface Receive overruns counter. High value of overruns packets may have impact to the traffic
            utilization (retransmission for TCP apps) and to the performance of the applications. In a small number of
            cases, the overrun counter may be incremented because of a software defect. However, in the majority of cases,
            it indicates that the receiving capability of the interface was exceeded. Nothing can be done on the device that
            reports overruns. If possible, the rate that frames are coming should be controlled at the remote end of the
            connection. Check the link below for more information about troubleshooting network interface problems:
            http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
        how: |
            This script logs into the FortiGate using SSH and retrieves the output of the "fnsysctl ifconfig" FortiOS hidden
            command. The output includes all the interface related information and statistics.
        without-indeni: |
            An admin would need to log into the Fortinet firewall and manually review the interface statistics.
        can-with-snmp: true
        can-with-syslog: false
    network-interface-tx-errors:
        why: |
            Capture the interface Transmit Errors counter. Transmit errors indicate an issue with duplex/speed matching.
            Check the link below for more information about troubleshooting network interface problems:
            http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
        how: |
            This script logs into the FortiGate using SSH and retrieves the output of the "fnsysctl ifconfig" FortiOS
            hidden command. The output includes all the interface related information and statistics.
        without-indeni: |
            An admin would need to log into the Fortinet firewall and manually review the interface statistics.
        can-with-snmp: true
        can-with-syslog: false
    network-interface-tx-dropped:
        why: |
            Capture the interface Transmit Drop counter. Packet loss may have severe impact to the traffic utilization
            (retransmission for TCP apps) and to the performance of the applications. Check the link below for more
            information about troubleshooting network interface problems:
            http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
        how: |
            This script logs into the FortiGate using SSH and retrieves the output of the "fnsysctl ifconfig" FortiOS hidden
            command. The output includes all the interface related information and statistics.
        without-indeni: |
            An admin would need to log into the Fortinet firewall and manually review the interface statistics.
        can-with-snmp: true
        can-with-syslog: false
    network-interface-tx-overruns:
        why: |
            Capture the interface Transmit Overruns counter. A high value of overrun packets may have impact on traffic
            utilization, causing a re-transmission of TCP apps and consequently degrade application performance. In a small
            number of cases, the overrun counter may be incremented because of a software defect. However, in the majority
            of cases, it indicates that the receiving capability of the interface was exceeded. Nothing can be done on the
            device that reports overruns. If possible, the rate that frames are coming should be controlled at the remote
            end of the connection. Check the link below for more information about troubleshooting network interface
            problems:
            http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
        how: |
            This script logs into the FortiGate using SSH and retrieves the output of the "fnsysctl ifconfig" FortiOS hidden
            command. The output includes all the interface related information and statistics.
        without-indeni: |
            An admin would need to log into the Fortinet firewall and manually review the interface statistics.
        can-with-snmp: true
        can-with-syslog: false
    network-interface-tx-collisions:
        why: |
            Capture the interface transimit collisions counter and is valid for half duplex mode. High value of collision
            packets may have impact to the traffic utilization (retransmission for TCP apps) and to the performance of the
            applications. Check the link below for more information about troubleshooting network  interface problems:
            http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
        how: |
            This script logs into the FortiGate using SSH and retrieves the output of the "fnsysctl ifconfig" FortiOS hidden
            command. The output includes all the interface related information and statistics.
        without-indeni: |
            An admin would need to log into the Fortinet firewall and manually review the interface statistics.
        can-with-snmp: true
        can-with-syslog: false
    network-interface-interrupts:
        why: |
            Capture the interrupt number information assigned to the NIC of the Fortinet firewall.
        how: |
            This script logs into the Fortigate using SSH and retrieves the output of the "fnsysctl ifconfig" FortiOS hidden
            command. The output includes all the interface related information and statistics.
        without-indeni: |
            An admin would need to log into the Fortinet firewall and manually review the interface statistics.
        can-with-snmp: true
        can-with-syslog: false
    network-interface-rx-bytes:
        why: |
            Capture the interface Received Bytes counter. Knowing the amount of bytes and packets flowing through an
            interface can help estimate an interface's performance and utilization. Check the link below for more
            information about troubleshooting network interface problems:
            http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
        how: |
            This script logs into the FortiGate using SSH and retrieves the output of the "fnsysctl ifconfig" FortiOS hidden
            command. The output includes all the interface related information and statistics.
        without-indeni: |
            An admin would need to log into the Fortinet firewall and manually review the interface statistics.
        can-with-snmp: true
        can-with-syslog: false
    network-interface-tx-bytes:
        why: |
            Capture the interface Transmitted Bytes counter. Knowing the amount of bytes and packets flowing through an
            interface can help estimate an interface's performance and utilization. Check the link below for more
            information about troubleshooting network interface problems:
            http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
        how: |
            This script logs into the FortiGate using SSH and retrieves the output of the "fnsysctl ifconfig" FortiOS hidden
            command. The output includes all the interface related information and statistics.
        without-indeni: |
            An admin would need to log into the Fortinet firewall and manually review the interface statistics.
        can-with-snmp: true
        can-with-syslog: false
    network-interface-rx-packets:
        why: |
            Capture the interface Received Packet counter. Knowing the amount of bytes and packets flowing through an
            interface can help estimate an interface's performance and utilization. Check the link below for more
            information about troubleshooting network interface problems:
            http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
        how: |
            This script logs into the FortiGate using SSH and retrieves the output of the "fnsysctl ifconfig" FortiOS hidden
            command. The output includes all the interface related information and statistics.
        without-indeni: |
            An admin would need to log into the Fortinet firewall and manually review the interface statistics.
        can-with-snmp: true
        can-with-syslog: false
    network-interface-tx-packets:
        why: |
            Capture the interface Transmitted Packet counter. Knowing the amount of bytes and packets flowing through an
            interface can help estimate an interface's performance and utilization. Check the link below for more
            information about troubleshooting network interface problems:
            http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
        how: |
            This script logs into the FortiGate using SSH and retrieves the output of the "fnsysctl ifconfig" FortiOS hidden
            command. The output includes all the interface related information and statistics.
        without-indeni: |
            An admin would need to log into the Fortinet firewall and manually review the interface statistics.
        can-with-snmp: true
        can-with-syslog: false
steps:
-   run:
        type: SSH
        command: fnsysctl ifconfig
    parse:
        type: AWK
        file: show_interfaces_fnsysctl_ifconfig.parser.1.awk

fortios-fnsysctl-ifconfig

name: fortios-fnsysctl-ifconfig
description: FortiGate Diagnose Interface
type: monitoring
monitoring_interval: 5 minutes
requires:
    vendor: fortinet
    os.name: FortiOS
    product: firewall
comments:
    network-interface-rx-errors:
        why: |
            Capture the interface Receive Errors counter. Receive errors indicate an issue with duplex/speed matching.
            Check the link below for more information about troubleshooting network interface problems:
            http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
        how: |
            This script logs into the FortiGate using SSH and retrieves the output of the "fnsysctl ifconfig" FortiOS hidden
            command. The output includes all the interface related information and statistics.
        without-indeni: |
            An admin would need to log into the Fortinet firewall and manually review the interface statistics.
        can-with-snmp: true
        can-with-syslog: false
    network-interface-rx-dropped:
        why: |
            Capture the interface Receive Drop counter. Packet loss may have severe impact to the traffic utilization
            (retransmission for TCP apps) and to the performance of the applications. Check the link below for more
            information about troubleshooting network interface problems:
            http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
        how: |
            This script logs into the FortiGate using SSH and retrieves the output of the "fnsysctl ifconfig" FortiOS hidden
            command. The output includes all the interface related information and statistics.
        without-indeni: |
            An admin would need to log into the Fortinet firewall and manually review the interface statistics.
        can-with-snmp: true
        can-with-syslog: false
    network-interface-rx-overruns:
        why: |
            Capture the interface Receive overruns counter. High value of overruns packets may have impact to the traffic
            utilization (retransmission for TCP apps) and to the performance of the applications. In a small number of
            cases, the overrun counter may be incremented because of a software defect. However, in the majority of cases,
            it indicates that the receiving capability of the interface was exceeded. Nothing can be done on the device that
            reports overruns. If possible, the rate that frames are coming should be controlled at the remote end of the
            connection. Check the link below for more information about troubleshooting network interface problems:
            http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
        how: |
            This script logs into the FortiGate using SSH and retrieves the output of the "fnsysctl ifconfig" FortiOS hidden
            command. The output includes all the interface related information and statistics.
        without-indeni: |
            An admin would need to log into the Fortinet firewall and manually review the interface statistics.
        can-with-snmp: true
        can-with-syslog: false
    network-interface-tx-errors:
        why: |
            Capture the interface Transmit Errors counter. Transmit errors indicate an issue with duplex/speed matching.
            Check the link below for more information about troubleshooting network interface problems:
            http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
        how: |
            This script logs into the FortiGate using SSH and retrieves the output of the "fnsysctl ifconfig" FortiOS
            hidden command. The output includes all the interface related information and statistics.
        without-indeni: |
            An admin would need to log into the Fortinet firewall and manually review the interface statistics.
        can-with-snmp: true
        can-with-syslog: false
    network-interface-tx-dropped:
        why: |
            Capture the interface Transmit Drop counter. Packet loss may have severe impact to the traffic utilization
            (retransmission for TCP apps) and to the performance of the applications. Check the link below for more
            information about troubleshooting network interface problems:
            http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
        how: |
            This script logs into the FortiGate using SSH and retrieves the output of the "fnsysctl ifconfig" FortiOS hidden
            command. The output includes all the interface related information and statistics.
        without-indeni: |
            An admin would need to log into the Fortinet firewall and manually review the interface statistics.
        can-with-snmp: true
        can-with-syslog: false
    network-interface-tx-overruns:
        why: |
            Capture the interface Transmit Overruns counter. A high value of overrun packets may have impact on traffic
            utilization, causing a re-transmission of TCP apps and consequently degrade application performance. In a small
            number of cases, the overrun counter may be incremented because of a software defect. However, in the majority
            of cases, it indicates that the receiving capability of the interface was exceeded. Nothing can be done on the
            device that reports overruns. If possible, the rate that frames are coming should be controlled at the remote
            end of the connection. Check the link below for more information about troubleshooting network interface
            problems:
            http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
        how: |
            This script logs into the FortiGate using SSH and retrieves the output of the "fnsysctl ifconfig" FortiOS hidden
            command. The output includes all the interface related information and statistics.
        without-indeni: |
            An admin would need to log into the Fortinet firewall and manually review the interface statistics.
        can-with-snmp: true
        can-with-syslog: false
    network-interface-tx-collisions:
        why: |
            Capture the interface transimit collisions counter and is valid for half duplex mode. High value of collision
            packets may have impact to the traffic utilization (retransmission for TCP apps) and to the performance of the
            applications. Check the link below for more information about troubleshooting network  interface problems:
            http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
        how: |
            This script logs into the FortiGate using SSH and retrieves the output of the "fnsysctl ifconfig" FortiOS hidden
            command. The output includes all the interface related information and statistics.
        without-indeni: |
            An admin would need to log into the Fortinet firewall and manually review the interface statistics.
        can-with-snmp: true
        can-with-syslog: false
    network-interface-interrupts:
        why: |
            Capture the interrupt number information assigned to the NIC of the Fortinet firewall.
        how: |
            This script logs into the Fortigate using SSH and retrieves the output of the "fnsysctl ifconfig" FortiOS hidden
            command. The output includes all the interface related information and statistics.
        without-indeni: |
            An admin would need to log into the Fortinet firewall and manually review the interface statistics.
        can-with-snmp: true
        can-with-syslog: false
    network-interface-rx-bytes:
        why: |
            Capture the interface Received Bytes counter. Knowing the amount of bytes and packets flowing through an
            interface can help estimate an interface's performance and utilization. Check the link below for more
            information about troubleshooting network interface problems:
            http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
        how: |
            This script logs into the FortiGate using SSH and retrieves the output of the "fnsysctl ifconfig" FortiOS hidden
            command. The output includes all the interface related information and statistics.
        without-indeni: |
            An admin would need to log into the Fortinet firewall and manually review the interface statistics.
        can-with-snmp: true
        can-with-syslog: false
    network-interface-tx-bytes:
        why: |
            Capture the interface Transmitted Bytes counter. Knowing the amount of bytes and packets flowing through an
            interface can help estimate an interface's performance and utilization. Check the link below for more
            information about troubleshooting network interface problems:
            http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
        how: |
            This script logs into the FortiGate using SSH and retrieves the output of the "fnsysctl ifconfig" FortiOS hidden
            command. The output includes all the interface related information and statistics.
        without-indeni: |
            An admin would need to log into the Fortinet firewall and manually review the interface statistics.
        can-with-snmp: true
        can-with-syslog: false
    network-interface-rx-packets:
        why: |
            Capture the interface Received Packet counter. Knowing the amount of bytes and packets flowing through an
            interface can help estimate an interface's performance and utilization. Check the link below for more
            information about troubleshooting network interface problems:
            http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
        how: |
            This script logs into the FortiGate using SSH and retrieves the output of the "fnsysctl ifconfig" FortiOS hidden
            command. The output includes all the interface related information and statistics.
        without-indeni: |
            An admin would need to log into the Fortinet firewall and manually review the interface statistics.
        can-with-snmp: true
        can-with-syslog: false
    network-interface-tx-packets:
        why: |
            Capture the interface Transmitted Packet counter. Knowing the amount of bytes and packets flowing through an
            interface can help estimate an interface's performance and utilization. Check the link below for more
            information about troubleshooting network interface problems:
            http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
        how: |
            This script logs into the FortiGate using SSH and retrieves the output of the "fnsysctl ifconfig" FortiOS hidden
            command. The output includes all the interface related information and statistics.
        without-indeni: |
            An admin would need to log into the Fortinet firewall and manually review the interface statistics.
        can-with-snmp: true
        can-with-syslog: false
steps:
-   run:
        type: SSH
        command: fnsysctl ifconfig
    parse:
        type: AWK
        file: show_interfaces_fnsysctl_ifconfig.parser.1.awk

cross_vendor_rx_drop

// Deprecation warning : Scala template-based rules are deprecated. Please use YAML format rules instead.

package com.indeni.server.rules.library.templatebased.crossvendor

import com.indeni.server.rules.RuleContext
import com.indeni.server.rules.library.templates.NearingCapacityWithItemsTemplateRule
import com.indeni.server.rules.RemediationStepCondition

/**
  *
  */
case class CrossVendorRxDrop() extends NearingCapacityWithItemsTemplateRule(
  ruleName = "cross_vendor_rx_drop",
  ruleFriendlyName = "All Devices: RX packets dropped ratio too high",
  ruleDescription = "Indeni tracks the number of packets that had issues and alerts if the ratio is too high.",
  usageMetricName = "network-interface-rx-dropped",
  limitMetricName = "network-interface-rx-packets",
  applicableMetricTag = "name",
  threshold = 0.5,
  minimumValueToAlert = 100.0, // We don't want to alert if the number of packets is really low
  alertDescription = "Some network interfaces and ports are experiencing a high drop rate. Review the ports below.",
  alertItemDescriptionFormat = "%.0f dropped packets identified out of a total of %.0f received.",
  baseRemediationText = "Packet drops usually occur when the rate of packets received is higher than the device's ability to handle.",
  alertItemsHeader = "Affected Ports")(
  RemediationStepCondition.VENDOR_CISCO ->
    """|
       |1. Run the "show interface" command to review the interface counters and the bitrate. Consider to configure the "load-interval 30" interface sub command to improve the accuracy of the interface measurements. Check for traffic bursts and high traffic utilization.
       |2. Use the "show hardware rate-limit" NX-OS command (if supported) to determine if packets are being dropped because of a rate limit.
       |3. Execute the "show policy-map interface control-plane" NX-OS command to determine if packets are being dropped because of a QoS policy.
       |4. Use the "show hardware internal statistics rates" to determine if packets are being dropped by the hardware.
       |5. Run the "show hardware internal statistics pktflow all" NX-OS command to display per ASIC statistics, including packets into and out of the ASIC. This command helps to identify where packet loss is occurring.""".stripMargin,
  RemediationStepCondition.VENDOR_JUNIPER ->
    """|1. Run the “show interface extensive” command to review the interface statistics.
       |2. Check for packet drops and input/output traffic rate.
       |3. Run the “show class-of-service interface x/x/x detail"  to determine any QoS policy applied to interface which may cause packet drops.
       |4. If the interface is saturated, the number of packets dropped by the is indicated by the input queue of the I/O Manager ASIC. This number increments once for every packet that is dropped by the ASIC's RED mechanism.
       |5. Review the following article on Juniper tech support site: <a target="_blank" href="https://www.juniper.net/documentation/en_US/junos/topics/reference/command-summary/show-interfaces-security.html#jd0e1772">Ethernet Switching and Layer 2 Transparent Mode Feature Guide for Security Devices</a>
       |6. If the problem persists, contact the Juniper Networks Technical Assistance Center (JTAC)""".stripMargin,
  RemediationStepCondition.VENDOR_FORTINET ->
    """
       |1. Run "diag hardware deviceinfo nic <interface>" command to display a list of hardware related error names and values. Review  the next link for more details: http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
       |2. Run command "fnsysctl cat /proc/net/dev" to get a summary of the interface statistics.
       |3. Check for speed and duplex mismatch in the interface settings on both sides of a cable, and check for a damaged cable. Review the next link for more info: http://kb.fortinet.com/kb/documentLink.do?externalID=10653""".stripMargin 
)