TX packets dropped-checkpoint-ipso

error
health-checks
checkpoint
ipso
TX packets dropped-checkpoint-ipso
0

#1

TX packets dropped-checkpoint-ipso

Vendor: checkpoint

OS: ipso

Description:
Indeni tracks the number of packets that had issues and alerts if the ratio is too high.

Remediation Steps:
Packet drops usually occur when the rate of packets transmitted is higher than the device ability to handle.

How does this work?
How many packets sent by the interface is retrieved by running “netstat -idb”.

Why is this important?
It is useful to know how many packets have been transmitted by the interface.

Without Indeni how would you find this?
An administrator could login and manually check interface configuration, or use SNMP.

chkp-ipso-interfaces-novsx

#! META
name: chkp-ipso-interfaces-novsx
description: Get interface information
type: monitoring
monitoring_interval: 1 minute
requires:
    vendor: "checkpoint"
    and:
        -
            os.name: "ipso"
        -
            or:
                -
                    vsx: 
                        neq: "true"
                -
                    mds: "true"

#! COMMENTS
network-interface-state:
    why: |
        Interfaces in the "down" state could result in downtime or reduced redundancy.
    how: |
        The state of the interface is retrieved by running "ifconfig -a".
    without-indeni: |
        An administrator could login and manually check interface status, or use SNMP.
    can-with-snmp: true
    can-with-syslog: false
    vendor-provided-management: |
        This is only accessible from the command line interface, SNMP or WebUI.

network-interface-admin-state:
    why: |
        If the interface is disabled, then it is okay for it to be down. If the interface is enabled however, it should be up.
    how: |
        The state of the interface is retrieved by running "ifconfig -a".
    without-indeni: |
        An administrator could login and manually check interface configuration, or use SNMP.
    can-with-snmp: true
    can-with-syslog: false
    vendor-provided-management: |
        This is only accessible from the command line interface, SNMP or WebUI.

network-interface-speed:
    why: |
        If the interface speed is set to a low value, this could mean auto-negotiation is not working correctly and the interface does not utilize the full bandwidth available.
    how: |
        The speed of the interface is retrieved by running "ifconfig -a".
    without-indeni: |
        An administrator could login and manually check interface configuration, or use SNMP.
    can-with-snmp: true
    can-with-syslog: false
    vendor-provided-management: |
        This is only accessible from the command line interface, SNMP or WebUI.

network-interface-duplex:
    why: |
        If the interface has half-duplex setting, this will reduce throughput, and should be investigated.
    how: |
        The duplex of the interface is retrieved by running "ifconfig -a".
    without-indeni: |
        An administrator could login and manually check interface configuration, or use SNMP.
    can-with-snmp: true
    can-with-syslog: false
    vendor-provided-management: |
        This is only accessible from the command line interface, SNMP or WebUI.

network-interface-ipv4-address:
    why: |
        To be able to search for IP addresses in indeni, this data needs to be stored.
    how: |
        The IP address of the interface is retrieved by running "ifconfig -a".
    without-indeni: |
        An administrator could login and manually check interface configuration, or use SNMP.
    can-with-snmp: true
    can-with-syslog: false
    vendor-provided-management: |
        This is only accessible from the command line interface, SNMP, WebUI or SmartDashboard.

network-interface-ipv4-subnet:
    why: |
        To be able to search for IP addresses in indeni, this data needs to be stored.
    how: |
        The subnet of the interface is retrieved by running "ifconfig -a".
    without-indeni: |
        An administrator could login and manually check interface configuration, or use SNMP.
    can-with-snmp: true
    can-with-syslog: false
    vendor-provided-management: |
        This is only accessible from the command line interface, SNMP, WebUI or SmartDashboard.

network-interface-type:
    why: |
        The type of interface can be useful for administrators.
    how: |
        The type of the interface is retrieved by running "ifconfig -a".
    without-indeni: |
        An administrator could login and manually check interface configuration, or use SNMP.
    can-with-snmp: true
    can-with-syslog: false
    vendor-provided-management: |
        This is only accessible from the command line interface or SNMP.

network-interface-mtu:
    why: |
        The MTU sometimes needs to be adjusted. Storing this gives an administrator an easy way to view the MTU from a large number of devices, as well as identifying incorrectly set MTU.
    how: |
        The MTU of the interface is retrieved by running "ifconfig -a".
    without-indeni: |
        An administrator could login and manually check interface configuration, or use SNMP.
    can-with-snmp: true
    can-with-syslog: false
    vendor-provided-management: |
        This is only accessible from the command line interface, SNMP or WebUI.

network-interface-mac:
    why: |
        To be able to search for MAC addresses in indeni, this data needs to be stored.
    how: |
        The MAC address of the interface is retrieved by running "ifconfig -a".
    without-indeni: |
        An administrator could login and manually check interface configuration, or use SNMP.
    can-with-snmp: true
    can-with-syslog: false
    vendor-provided-management: |
        This is only accessible from the command line interface, SNMP or WebUI.

network-interface-description:
    why: |
        The description is an important way to identify interfaces.
    how: |
        Retrive the information by parsing the IPSO database in /config/active.
    without-indeni: |
        An administrator could login and manually check interface configuration, or use SNMP.
    can-with-snmp: true
    can-with-syslog: false
    vendor-provided-management: |
        This is only accessible from the command line interface, SNMP or WebUI.

network-interface-tx-bytes:
    why: |
        It is useful to know how much data has been transmitted by the interface.
    how: |
        How many bytes sent by the interface is retrieved by running "netstat -idb".
    without-indeni: |
        An administrator could login and manually check interface configuration, or use SNMP.
    can-with-snmp: true
    can-with-syslog: false
    vendor-provided-management: |
        This is only accessible from the command line interface, SNMP or WebUI.

network-interface-rx-bytes:
    why: |
        It is useful to know how much data has been received by the interface.
    how: |
        How many bytes received by the interface is retrieved by running "netstat -idb".
    without-indeni: |
        An administrator could login and manually check interface configuration, or use SNMP.
    can-with-snmp: true
    can-with-syslog: false
    vendor-provided-management: |
        This is only accessible from the command line interface, SNMP or WebUI.

network-interface-tx-packets:
    why: |
        It is useful to know how many packets have been transmitted by the interface.
    how: |
        How many packets sent by the interface is retrieved by running "netstat -idb".
    without-indeni: |
        An administrator could login and manually check interface configuration, or use SNMP.
    can-with-snmp: true
    can-with-syslog: false
    vendor-provided-management: |
        This is only accessible from the command line interface, SNMP or WebUI.

network-interface-rx-packets:
    why: |
        It is useful to know how many packets have been received by the interface.
    how: |
        How many packets received by the interface is retrieved by running "netstat -idb".
    without-indeni: |
        An administrator could login and manually check interface configuration, or use SNMP.
    can-with-snmp: true
    can-with-syslog: false
    vendor-provided-management: |
        This is only accessible from the command line interface, SNMP or WebUI.

network-interface-tx-errors:
    why: |
        Transmit errors on an interface could indicate a problem.
    how: |
        The amount of transmit errors for the interface is retrieved by running "netstat -idb".
    without-indeni: |
        An administrator could login and manually check interface configuration, or use SNMP.
    can-with-snmp: true
    can-with-syslog: false
    vendor-provided-management: |
        This is only accessible from the command line interface, SNMP or WebUI.

network-interface-rx-dropped:
    why: |
        Dropped packets on an interface could indicate a problem and potential traffic loss.
    how: |
        The amount of receive drops for the interface is retrieved by running "netstat -idb".
    without-indeni: |
        An administrator could login and manually check interface configuration, or use SNMP.
    can-with-snmp: true
    can-with-syslog: false
    vendor-provided-management: |
        This is only accessible from the command line interface, SNMP or WebUI.

network-interfaces:
    skip-documentation: true

network-interface-:
    skip-documentation: true

#! REMOTE::SSH
${nice-path} -n 15 ifconfig -a
${nice-path} -n 15 grep "interface" /config/active
${nice-path} -n 15 netstat -idb

#! PARSER::AWK

function ignoreInterface(_name) {
    if (_name ~ /^(loop|soverf|stof|tun).*/) {
        return 1
    } else {
        return 0
    }
}

############
# Script explanation: We should avoid running clish commands due to the excessive logs in /var/log/messages that creates. So ifconfig and parsing /config/active instead.
###########

BEGIN {
    bitMaskToSubnet["32"] = "255.255.255.255"
    bitMaskToSubnet["31"] = "255.255.255.254"
    bitMaskToSubnet["30"] = "255.255.255.252"
    bitMaskToSubnet["29"] = "255.255.255.248"
    bitMaskToSubnet["28"] = "255.255.255.240"
    bitMaskToSubnet["27"] = "255.255.255.224"
    bitMaskToSubnet["26"] = "255.255.255.192"
    bitMaskToSubnet["25"] = "255.255.255.128"
    bitMaskToSubnet["24"] = "255.255.255.0"
    bitMaskToSubnet["23"] = "255.255.254.0"
    bitMaskToSubnet["22"] = "255.255.252.0"
    bitMaskToSubnet["21"] = "255.255.248.0"
    bitMaskToSubnet["20"] = "255.255.240.0"
    bitMaskToSubnet["19"] = "255.255.224.0"
    bitMaskToSubnet["18"] = "255.255.192.0"
    bitMaskToSubnet["17"] = "255.255.128.0"
    bitMaskToSubnet["16"] = "255.255.0.0"
    bitMaskToSubnet["15"] = "255.254.0.0"
    bitMaskToSubnet["14"] = "255.252.0.0"
    bitMaskToSubnet["13"] = "255.248.0.0"
    bitMaskToSubnet["12"] = "255.240.0.0"
    bitMaskToSubnet["11"] = "255.224.0.0"
    bitMaskToSubnet["10"] = "255.192.0.0"
    bitMaskToSubnet["9"] = "255.128.0.0"
    bitMaskToSubnet["8"] = "255.0.0.0"
    bitMaskToSubnet["7"] = "254.0.0.0"
    bitMaskToSubnet["6"] = "252.0.0.0"
    bitMaskToSubnet["5"] = "248.0.0.0"
    bitMaskToSubnet["4"] = "240.0.0.0"
    bitMaskToSubnet["3"] = "224.0.0.0"
    bitMaskToSubnet["2"] = "192.0.0.0"
    bitMaskToSubnet["1"] = "128.0.0.0"
    bitMaskToSubnet["0"] = "0.0.0.0"
}

#loop0c0:  lname loop0c0 flags=57<UP,PHYS_AVAIL,LINK_AVAIL,LOOPBACK,MULTICAST>
#eth1c0:  lname eth1c0 flags=e0<BROADCAST,MULTICAST,AUTOLINK>
/^.+:\s+/ {
    interfaceName = $1
    #eth1c0:
    sub(/:/, "", interfaceName)

    if (!ignoreInterface(interfaceName)) {
        statTags["name"] = interfaceName

        type = interfaceName
        #eth1c0
        sub(/[0-9].*/, "", type)
        writeComplexMetricString("network-interface-type", statTags, type)
        interfaces[interfaceName, "driver"] = type
    } else {
        interfaceName = ""
    }
}

#eth4c0:  lname eth4c0 flags=e4<UP,BROADCAST,MULTICAST,AUTOLINK>
#eth2c0:  lname eth2c0 flags=e7<UP,PHYS_AVAIL,LINK_AVAIL,BROADCAST,MULTICAST,AUTOLINK>
/lname.*flags=/ {
    if (interfaceName != "") {
        if ($0 ~ /PHYS_AVAIL/) {
            linkstate = 1
        } else {
            linkstate = 0
        }
        writeDoubleMetricWithLiveConfig("network-interface-state", statTags, "gauge", "60", linkstate, "Network Interfaces - Up/Down", "state", "name")

        if ($0 ~ /[^A-Z]UP[^A-Z]/) {
            adminState = 1
        } else {
            adminState = 0
        }
        writeDoubleMetricWithLiveConfig("network-interface-admin-state", statTags, "gauge", "60", adminState, "Network Interfaces - Enabled/Disabed", "state", "name")
    }
}

#         ether 00:a0:8e:b2:22:37 speed 10M half duplex
/[0-9a-fA-F]{2}:[0-9a-fA-F]{2}:[0-9a-fA-F]{2}:[0-9a-fA-F]{2}:[0-9a-fA-F]{2}:[0-9a-fA-F]{2}/ {
    if (interfaceName != "") {
        match($0, /[0-9a-fA-F]{2}:[0-9a-fA-F]{2}:[0-9a-fA-F]{2}:[0-9a-fA-F]{2}:[0-9a-fA-F]{2}:[0-9a-fA-F]{2}/)

        macaddress = substr($0, RSTART, RLENGTH)
        writeComplexMetricString("network-interface-mac", statTags, macaddress)
    }
}

#         inet mtu 1500 172.16.20.30/24 broadcast 172.16.20.255
/ inet / {
    if (interfaceName != "") {
        ipAndBitMask = $4
        split(ipAndBitMask, ipArray, "/")
        ip = ipArray[1]
        bitMask = ipArray[2]
        subnet = bitMaskToSubnet[bitMask]

        ipTags["name"] = interfaceName
        ipTags["im.identity-tags"] = "name"
        ipTags["im.dstype.displaytype"] = "string"

        writeComplexMetricStringWithLiveConfig("network-interface-ipv4-address", ipTags, ip, "Network Interfaces - IPv4 Address")
        writeComplexMetricStringWithLiveConfig("network-interface-ipv4-subnet", ipTags, subnet, "Network Interfaces - IPv4 Netmask")
    }
}

#        inet6 mtu 63000 ::1 --> ::1
/^\s.+mtu\s/ {
    if (interfaceName != "") {
        mtu = $0
        #        inet6 mtu 63000 ::1 --> ::1
        sub(/.* mtu /, "", mtu)
        #mtu 63000 ::1 --> ::1
        sub(/ .*/, "", mtu)

        interfaces[interfaceName, "mtu"] = mtu
        writeComplexMetricString("network-interface-mtu", statTags, mtu)
    }
}

#         ether 00:a0:8e:b2:22:37 speed 10M half duplex
/ speed .* duplex/ {
    if (interfaceName != "") {
        speed = $0
        #         ether 00:a0:8e:b2:22:37 speed 10M half duplex
        sub(/.* speed /, "", speed)
        #10M half duplex
        sub(/ .*/, "", speed)

        # speed metric need special tags for alert purpose
        speed_tags["name"] = interfaceName
        speed_tags["alert-item-port-speed"] = interfaceName "-" speed
        writeComplexMetricString("network-interface-speed", speed_tags, speed)

        duplex = $0
        if ($0 ~ /full duplex/) {
            writeComplexMetricString("network-interface-duplex", statTags, "full")
        } else if ($0 ~ /half duplex/) {
            writeComplexMetricString("network-interface-duplex", statTags, "half")
        }
    }
}

#interface:eth2c0:comments my\ comments
/interface:[a-z0-9]+:comments/ {
    if (interfaceName != "") {
        # Get interface name
        split($0, lineParts, ":")
        statTags["name"] = lineParts[2]

        # If the comment is in two words or more, a backslash is inserted. We need to remove it
        commentLine = $0
        gsub(/interface:[a-z0-9]+:comments/, "", commentLine)
        gsub(/(\\)/, "", commentLine)
        writeComplexMetricString("network-interface-description", statTags, trim(commentLine))
    }
}

#Name         Mtu   Network     Address             Ipkts Ierrs     Ibytes    Opkts Oerrs     Obytes  Coll Drop
#eth2         16018 <Link>      0:a0:8e:b2:22:35    39706   440    5481041    28917     0   10158270  1898   0
/^Name\s+Mtu\s+Network\s+/ {
    if (interfaceName != "") {
        inNetstat = "true"
        getColumns(trim($0), "[ \t]+", columns)
    }
}

#eth2         16018 <Link>      0:a0:8e:b2:22:35    39706   440    5481041    28917     0   10158270  1898   0
/[0-9]/ {
    if (inNetstat == "true") {
        name = getColData(trim($0), columns, "Name")
        statTags["name"] = name

        if (!ignoreInterface(name)) {
            writeDoubleMetricWithLiveConfig("network-interface-rx-packets", statTags, "counter", "60", getColData(trim($0), columns, "Ipkts"), "Network Interfaces - RX Packets", "number", "name")

            writeDoubleMetricWithLiveConfig("network-interface-rx-errors", statTags, "counter", "60", getColData(trim($0), columns, "Ierrs"), "Network Interfaces - RX Errors", "number", "name")

            writeDoubleMetricWithLiveConfig("network-interface-rx-bytes", statTags, "counter", "60", getColData(trim($0), columns, "Ibytes"), "Network Interfaces - RX Bytes", "number", "name")

            writeDoubleMetricWithLiveConfig("network-interface-rx-dropped", statTags, "counter", "60", getColData(trim($0), columns, "Drop"), "Network Interfaces - RX Dropped", "number", "name")

            writeDoubleMetricWithLiveConfig("network-interface-tx-packets", statTags, "counter", "60", getColData(trim($0), columns, "Opkts"), "Network Interfaces - TX Packets", "number", "name")

            writeDoubleMetricWithLiveConfig("network-interface-tx-errors", statTags, "counter", "60", getColData(trim($0), columns, "Oerrs"), "Network Interfaces - TX Errors", "number", "name")

            writeDoubleMetricWithLiveConfig("network-interface-tx-bytes", statTags, "counter", "60", getColData(trim($0), columns, "Obytes"), "Network Interfaces - TX Bytes", "number", "name")

            writeDoubleMetricWithLiveConfig("network-interface-tx-collisions", statTags, "counter", "60", getColData(trim($0), columns, "Coll"), "Network Interfaces - TX Collisions", "number", "name")
        }
    }
}


END {
    writeComplexMetricObjectArray("network-interfaces", null, interfaces)
}

cross_vendor_tx_drop

package com.indeni.server.rules.library.templatebased.crossvendor

import com.indeni.server.rules.RuleContext
import com.indeni.server.rules.library.{ConditionalRemediationSteps, NearingCapacityWithItemsTemplateRule}

/**
  *
  */
case class CrossVendorTxDrop(context: RuleContext) extends NearingCapacityWithItemsTemplateRule(context,
  ruleName = "cross_vendor_tx_drop",
  ruleFriendlyName = "All Devices: TX packets dropped",
  ruleDescription = "Indeni tracks the number of packets that had issues and alerts if the ratio is too high.",
  usageMetricName = "network-interface-tx-dropped",
  limitMetricName = "network-interface-tx-packets",
  applicableMetricTag = "name",
  threshold = 0.5,
  minimumValueToAlert = 100.0, // We don't want to alert if the number of packets is really low
  alertDescription = "Some network interfaces and ports are experiencing a high drop rate. Review the ports below.",
  alertItemDescriptionFormat = "%.0f dropped packets identified out of a total of %.0f transmitted.",
  baseRemediationText = "Packet drops usually occur when the rate of packets transmitted is higher than the device ability to handle.",
  alertItemsHeader = "Affected Ports")(
  ConditionalRemediationSteps.OS_NXOS ->
    """|
       |1. Run the "show interface" command to review the interface counters and the bitrate. Consider to configure the "load-interval 30" interface sub command to improve the accuracy of the interface measurements. Check for traffic bursts and high traffic utilization.
       |2. Use the "show hardware rate-limit" NX-OS command (if supported) to determine if packets are being dropped because of a rate limit.
       |3. Execute the "show policy-map interface control-plane" NX-OS command to determine if packets are being dropped because of a QoS policy.
       |4. Use the "show hardware internal statistics rates" to determine if packets are being dropped by the hardware.
       |5. Run the "show hardware internal statistics pktflow all" NX-OS command to display per ASIC statistics, including packets into and out of the ASIC. This command helps to identify where packet loss is occurring.
    """.stripMargin,
  ConditionalRemediationSteps.VENDOR_JUNIPER ->
    """|1. Run the “show interface extensive” command to review the interface statistics.
       |2. Check for packet drops and input/output traffic rate.
       |3. Run the “show class-of-service interface x/x/x detail"  to determine any QoS policy applied to interface which may cause packet drops.
       |4. If the interface is saturated, the number of packets dropped by the is indicated by the input queue of the I/O Manager ASIC. This number increments once for every packet that is dropped by the ASIC's RED mechanism.
       |5. Review the following article on Juniper tech support site: <a target="_blank" href="https://www.juniper.net/documentation/en_US/junos/topics/concept/using-show-commands-for-packet-drops.html">Understanding Dropped Packets and Untransmitted Traffic Using show Commands</a>
       |6. If the problem persists, contact the Juniper Networks Technical Assistance Center (JTAC)""".stripMargin,
  ConditionalRemediationSteps.VENDOR_FORTINET ->
    """
       |1. Run "diag hardware deviceinfo nic <interface>" command to display a list of hardware related error names and values. Review  the next link for more details: http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
       |2. Run command "fnsysctl cat /proc/net/dev" to get a summary of the interface statistics.
       |3. Check for speed and duplex mismatch in the interface settings on both sides of a cable, and check for a damaged cable. Review the next link for more info: http://kb.fortinet.com/kb/documentLink.do?externalID=10653""".stripMargin
)