TX packets dropped-juniper-junos

error
health-checks
junos
juniper
TX packets dropped-juniper-junos
0

#1

TX packets dropped-juniper-junos

Vendor: juniper

OS: junos

Description:
Indeni tracks the number of packets that had issues and alerts if the ratio is too high.

Remediation Steps:
Packet drops usually occur when the rate of packets transmitted is higher than the device ability to handle.
|1. Run the “show interface extensive” command to review the interface statistics.
|2. Check for packet drops and input/output traffic rate.
|3. Run the “show class-of-service interface x/x/x detail" to determine any QoS policy applied to interface which may cause packet drops.
|4. If the interface is saturated, the number of packets dropped by the is indicated by the input queue of the I/O Manager ASIC. This number increments once for every packet that is dropped by the ASIC’s RED mechanism.
|5. Review the following article on Juniper tech support site: Understanding Dropped Packets and Untransmitted Traffic Using show Commands
|6. If the problem persists, contact the Juniper Networks Technical Assistance Center (JTAC)

How does this work?
This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the “show interfaces extensive” command. The output includes all the interface related information and statistics.

Why is this important?
Capture the interface Transmitted Packets counter. Knowing the amount of bytes and packets flowing through an interface can help estimate an interface’s performance and utilization.

Without Indeni how would you find this?
There are many solutions out there for tracking traffic flow on network interfaces and alerting when critical indicators are reached.

junos-show-interfaces-extensive

#! META
name: junos-show-interfaces-extensive
description: JUNOS get interface information (including stats)
type: monitoring
monitoring_interval: 1 minute
requires:
    vendor: juniper
    os.name: junos

#! COMMENTS
network-interface-state:
    why: |
       Capture the interface state. If an interface transitions from up to down an alert would be raised.
    how: |
       This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the "show interfaces extensive" command. The output includes all the interface related information and statistics.
    without-indeni: |
       It is possible to poll this data through SNMP. Interface state transitions will generate a syslog event.
    can-with-snmp: true
    can-with-syslog: true

network-interface-admin-state:
    why: |
       Capture the interface administrative state.
    how: |
       This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the "show interfaces extensive" command. The output includes all the interface related information and statistics.
    without-indeni: |
       It is possible to poll this data through SNMP. Interface state transitions will generate a syslog event.
    can-with-snmp: true
    can-with-syslog: true

network-interface-description:
    skip-documentation: true # We don't have description data

network-interface-speed:
    why: |
       Capture the interface speed in human readable format such as 1G, 10G, etc. indeni will alert if the speed is below 1G.
    how: |
       This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the "show interfaces extensive" command. The output includes all the interface related information and statistics.
    without-indeni: |
       It is possible to poll this data through SNMP.
    can-with-snmp: true
    can-with-syslog: false

network-interface-duplex:
    why: |
       Capture the interface duplex in human readable format such as full or half. In modern network environments it is very uncommon to see half-duplex interfaces, and that should be an indication for a potential exception.
    how: |
       This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the "show interfaces extensive" command. The output includes all the interface related information and statistics.
    without-indeni: |
       It is possible to poll this data through SNMP. If a duplex mismatch is detected on a port syslog messages will be generated.
    can-with-snmp: true
    can-with-syslog: true

network-interface-rx-frame:
    why: |
       Capture the interface Receive Errors (CRC) counter. If this counter increases an alarm will be raised.
    how: |
       This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the "show interfaces extensive" command. The output includes all the interface related information and statistics.
    without-indeni: |
       It is possible to poll this data through SNMP and set specific alert thresholds when the counter increases.
    can-with-snmp: true
    can-with-syslog: false

network-interface-mac:
    why: |
       Capture the interface MAC address.
    how: |
       This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the "show interfaces extensive" command. The output includes all the interface related information and statistics.
    without-indeni: |
       It is possible to poll this data through SNMP for inventory purposes.
    can-with-snmp: true
    can-with-syslog: false

network-interface-type:
    why: |
       Capture the interface type, for example "Ethernet".
    how: |
       This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the "show interfaces extensive" command. The output includes all the interface related information and statistics.
    without-indeni: |
       It is possible to poll this data through SNMP.
    can-with-snmp: true
    can-with-syslog: false

network-interface-rx-bytes:
    why: |
       Capture the interface Received Bytes counter. Knowing the amount of bytes and packets flowing through an interface can help estimate an interface's performance and utilization.
    how: |
       This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the "show interfaces extensive" command. The output includes all the interface related information and statistics.
    without-indeni: |
       There are many solutions out there for tracking traffic flow on network interfaces and alerting when critical indicators are reached.
    can-with-snmp: true
    can-with-syslog: false

network-interface-rx-dropped:
    why: |
       Capture the interface Received Errors counter. Packet loss may impact traffic performance.
    how: |
       This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the "show interfaces extensive" command. The output includes all the interface related information and statistics.
    without-indeni: |
       There are many solutions out there for tracking traffic flow on network interfaces and alerting when critical indicators are reached.
    can-with-snmp: true
    can-with-syslog: false

network-interface-rx-packets:
    why: |
       Capture the interface Received Packets counter. Knowing the amount of bytes and packets flowing through an interface can help estimate an interface's performance and utilization.
    how: |
       This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the "show interfaces extensive" command. The output includes all the interface related information and statistics.
    without-indeni: |
       There are many solutions out there for tracking traffic flow on network interfaces and alerting when critical indicators are reached.
    can-with-snmp: true
    can-with-syslog: false

network-interface-ipv4-address:
    why: |
       Capture the interface IPv4 address. Only relevant for layer 3 interfaces, including Vlan interfaces (SVI).
    how: |
       This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the "show interfaces extensive" command. The output includes all the interface related information and statistics.
    without-indeni: |
       It is possible to poll this data through SNMP.
    can-with-snmp: true
    can-with-syslog: false

network-interface-ipv4-subnet:
    why: |
       Capture the interface IPv4 subnet mask. Only relevant for layer 3 interfaces, including Vlan interfaces (SVI).
    how: |
       This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the "show interfaces extensive" command. The output includes all the interface related information and statistics.
    without-indeni: |
       It is possible to poll this data through SNMP.
    can-with-snmp: true
    can-with-syslog: false

network-interface-tx-carrier:
    why: |
       Capture the interface carrier state change counter. It would increase every time the interface changes state from up to down.
    how: |
       This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the "show interfaces extensive" command. The output includes all the interface related information and statistics.
    without-indeni: |
       There are many solutions out there for tracking traffic flow on network interfaces and alerting when critical indicators are reached.
    can-with-snmp: true
    can-with-syslog: false

network-interface-mtu:
    why: |
       Capture the interface MTU (Maximum Transmit Unit). A low MTU may cause issues.
    how: |
       This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the "show interfaces extensive" command. The output includes all the interface related information and statistics.
    without-indeni: |
       It is possible to poll this data through SNMP and alert if the MTU is too low.
    can-with-snmp: true
    can-with-syslog: false

network-interface-tx-bytes:
    why: |
       Capture the interface Transmitted Bytes counter. Knowing the amount of bytes and packets flowing through an interface can help estimate an interface's performance and utilization.
    how: |
       This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the "show interfaces extensive" command. The output includes all the interface related information and statistics.
    without-indeni: |
       There are many solutions out there for tracking traffic flow on network interfaces and alerting when critical indicators are reached.
    can-with-snmp: true
    can-with-syslog: false

network-interface-tx-errors:
    why: |
       Capture the interface Transmit Errors counter. Transmission errors indicate an issue with duplex/speed matching.
    how: |
       This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the "show interfaces extensive" command. The output includes all the interface related information and statistics.
    without-indeni: |
       There are many solutions out there for tracking traffic flow on network interfaces and alerting when critical indicators are reached.
    can-with-snmp: true
    can-with-syslog: false

network-interface-tx-packets:
    why: |
       Capture the interface Transmitted Packets counter. Knowing the amount of bytes and packets flowing through an interface can help estimate an interface's performance and utilization.
    how: |
       This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the "show interfaces extensive" command. The output includes all the interface related information and statistics.
    without-indeni: |
       There are many solutions out there for tracking traffic flow on network interfaces and alerting when critical indicators are reached.
    can-with-snmp: true
    can-with-syslog: false

network-interface-rx-overruns:
    why: |
       Capture the interface Receive Overrun errors counter.
    how: |
       This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the "show interfaces extensive" command. The output includes all the interface related information and statistics.
    without-indeni: |
       There are many solutions out there for tracking traffic flow on network interfaces and alerting when critical indicators are reached.
    can-with-snmp: true
    can-with-syslog: false


#! REMOTE::SSH
show interfaces extensive | display xml

#! PARSER::XML
_vars:
    root: /rpc-reply//interface-information[1]
_metrics:
    -
        _groups:
            ${root}/physical-interface:
                _tags:
                    "im.name":
                        _constant: "network-interface-state" 
                    "name":
                        _text: "name"
                    "live-config":
                        _constant: "true"
                    "display-name":
                       _constant: "Interfaces - State"
                    "im.dstype.displayType":
                        _constant: "state"            
                    "im.identity-tags":
                        _constant: "name"
                _temp:
                    state:
                        _text: "oper-status"
        _transform:
             _value.double: |
                 {
                     if (temp("state") == "up") {
                        interface_state = "1.0"
                     } else {
                        interface_state = "0.0"
                     }
                     print interface_state
                 }
    -
        _groups:
            ${root}/physical-interface:
                _tags:
                    "im.name":
                        _constant: "network-interface-admin-state" 
                    "name":
                        _text: "name"
                    "live-config":
                        _constant: "true"
                    "display-name":
                       _constant: "Interfaces - Admin State"
                    "im.dstype.displayType":
                        _constant: "state"            
                    "im.identity-tags":
                        _constant: "name"
                _temp:
                    state:
                        _text: "admin-status"
        _transform:
             _value.double: |
                 {
                     if (temp("state") == "up") {
                         admin_state = "1.0"
                     } else {
                         admin_state = "0.0"
                     }
                     print admin_state
                 }
    -
        _groups:
            ${root}/physical-interface:
                _tags:
                    "im.name":
                        _constant: "network-interface-description"
                    "live-config":
                        _constant: "true"
                    "display-name":
                        _constant: "Interfaces"
                    "name":
                        _text: "name"
                    "im.identity-tags":
                        _constant: "name"
                _temp:
                    descriptionPath:
                        _count: description
                    content:
                        _text: description
        _transform:
            _value.complex: 
                value: |
                    {
                        if (temp("descriptionPath") == "0") {  #this line seems not work for complex
                            interface_description = ""
                        }else{
                            interface_description = temp("content")
                        }
                        print interface_description
                    } 
    -
        _groups:
            ${root}/physical-interface[not(contains(speed,'Unlimited') or contains(speed,'Unspecified'))]:
                _tags:
                    "im.name":
                        _constant: "network-interface-speed"
                    "name":
                        _text: "name"
                    "live-config":
                        _constant: "true"
                    "display-name":
                       _constant: "Interfaces - Speed"
                    "im.identity-tags":
                        _constant: "name"
                _temp:
                    speed:
                        _text: speed
        _transform:
            _tags:
                "alert-item-port-speed": |
                    {
                        speed = temp("speed")
                        gsub(/bps/, "", speed)

                        print tag("name") "-" toupper(speed)
                    }
            _value.complex:
                value: |
                    {
                        speed = temp("speed")
                        gsub(/bps/, "", speed)
                        print toupper(speed)
                    }
    -
        _groups:
            ${root}/physical-interface:
                _tags:
                    "im.name":
                        _constant: "network-interface-duplex"
                    "name":
                        _text: "name"
                    "live-config":
                        _constant: "true"
                    "display-name":
                       _constant: "Interfaces - Duplex"
                    "im.identity-tags":
                        _constant: "name"
                _temp:
                    duplex:
                        _text: "link-mode"
        _transform:
            _value.complex:
                value: |
                    {
                        if (tolower(temp("duplex")) ~ /full/) {
                            link_mode = "full"
                        } else {
                            link_mode = "half"
                        }
                        print link_mode
                    }
    -
        _groups:
            ${root}/physical-interface:
                _tags:
                    "im.name":
                        _constant: "network-interface-rx-frame"
                    "dsType":
                        _constant: "counter"
                    "name":
                        _text: "name"
                    "live-config":
                        _constant: "true"
                    "display-name":
                        _constant: "Interfaces - RX Frame Errors"
                    "im.dstype.displayType":
                        _constant: "number"
                    "im.identity-tags":
                        _constant: "name"
                _value.double:
                    _text: "input-error-list/input-errors"
    -
        _groups:
            ${root}/physical-interface[not(contains(current-physical-address,'Unspecified'))]:
                _tags:
                    "im.name":
                        _constant: "network-interface-mac"
                    "name":
                        _text: "name"
                    "live-config":
                        _constant: "true"
                    "display-name":
                        _constant: "Interfaces - MAC Address"
                    "im.identity-tags":
                        _constant: "name"
                _value.complex:
                    value:
                        _text: "current-physical-address"
    -
        _groups:
            ${root}/physical-interface:
                _tags:
                    "im.name":
                        _constant: "network-interface-type"
                    "name":
                        _text: "name"
                    "live-config":
                        _constant: "true"
                    "display-name":
                        _constant: "Interfaces - Type"
                    "im.identity-tags":
                        _constant: "name"
                _value.complex:
                    value:
                        _text: "link-level-type"
    -
        _groups:
            ${root}/physical-interface:
                _tags:
                    "im.name":
                        _constant: "network-interface-rx-bytes"
                    "dsType":
                        _constant: "counter"
                    "name":
                        _text: "name"
                    "live-config":
                        _constant: "true"
                    "display-name":
                        _constant: "Interfaces - RX Bytes"
                    "im.dstype.displayType":
                        _constant: "number"
                    "im.identity-tags":
                        _constant: "name"
                _value.double:
                    _text: "traffic-statistics/input-bytes"
    -
        _groups:
            ${root}/physical-interface:
                _tags:
                    "im.name":
                        _constant: "network-interface-rx-dropped"
                    "dsType":
                        _constant: "counter"
                    "name":
                        _text: "name"
                    "live-config":
                        _constant: "true"
                    "display-name":
                        _constant: "Interfaces - RX Drops"
                    "im.dstype.displayType":
                        _constant: "number"
                    "im.identity-tags":
                        _constant: "name"
                _value.double:
                    _text: "input-error-list/input-drops"
    -
        _groups:
            ${root}/physical-interface:
                _tags:
                    "im.name":
                        _constant: "network-interface-rx-packets"
                    "dsType":
                        _constant: "counter"
                    "name":
                        _text: "name"
                    "live-config":
                        _constant: "true"
                    "display-name":
                        _constant: "Interfaces - RX Packets"
                    "im.dstype.displayType":
                        _constant: "number"
                    "im.identity-tags":
                        _constant: "name"
                _value.double:
                    _text: "traffic-statistics/input-packets"
    -
        _groups:
            ${root}/physical-interface[not(contains(./logical-interface/address-family/interface-address,'Unspecified'))]:
                _tags:
                    "im.name":
                        _constant: "network-interface-ipv4-address"
                    "name":
                        _text: "name"
                    "live-config":
                        _constant: "true"
                    "display-name":
                        _constant: "Interfaces - IPv4 Address"
                    "im.identity-tags":
                        _constant: "name"
                _value.complex:
                    value:
                        _text: "logical-interface/address-family/interface-address/ifa-local"
    -
        _groups:
            ${root}/physical-interface[not(contains(./logical-interface/address-family/interface-address,'Unspecified'))]:
                _tags:
                    "im.name":
                        _constant: "network-interface-ipv4-subnet"
                    "name":
                        _text: "name"
                    "live-config":
                        _constant: "true"
                    "display-name":
                        _constant: "Interfaces - Bitmask"
                    "im.identity-tags":
                        _constant: "name"
                _temp:
                    destination_net:
                        _text: "logical-interface/address-family/interface-address/ifa-destination"
        _transform:
            _value.complex:
                value: |
                    {
                        bitmask = temp("destination_net")
                        sub(/^.*\//, "", bitmask)
                        print bitmask
                    }
    -
        _groups:
            ${root}/physical-interface:
                _tags:
                    "im.name":
                        _constant: "network-interface-tx-carrier"
                    "dsType":
                        _constant: "counter"
                    "name":
                        _text: "name"
                    "live-config":
                        _constant: "true"
                    "display-name":
                        _constant: "Interfaces - TX Carrier Transitions"
                    "im.dstype.displayType":
                        _constant: "number"
                    "im.identity-tags":
                        _constant: "name"
                _value.double:
                    _text: "output-error-list/carrier-transitions"
    -
        _groups:
            ${root}/physical-interface[not(contains(mtu,'Unlimited'))]:
                _tags:
                    "im.name":
                        _constant: "network-interface-mtu"
                    "name":
                        _text: "name"
                    "live-config":
                        _constant: "true"
                    "display-name":
                        _constant: "Interfaces - MTU"
                    "im.identity-tags":
                        _constant: "name"
                _value.complex:
                    value:
                        _text: "mtu"
    -
        _groups:
            ${root}/physical-interface:
                _tags:
                    "im.name":
                        _constant: "network-interface-tx-bytes"
                    "dsType":
                        _constant: "counter"
                    "name":
                        _text: "name"
                    "live-config":
                        _constant: "true"
                    "display-name":
                        _constant: "Interfaces - TX Bytes"
                    "im.dstype.displayType":
                        _constant: "number"
                    "im.identity-tags":
                        _constant: "name"
                _value.double:
                    _text: "traffic-statistics/output-bytes"
    -
        _groups:
            ${root}/physical-interface:
                _tags:
                    "im.name":
                        _constant: "network-interface-tx-errors"
                    "dsType":
                        _constant: "counter"
                    "name":
                        _text: "name"
                    "live-config":
                        _constant: "true"
                    "display-name":
                        _constant: "Interfaces - TX Errors"
                    "im.dstype.displayType":
                        _constant: "number"
                    "im.identity-tags":
                        _constant: "name"
                _value.double:
                    _text: "output-error-list/output-errors"
    -
        _groups:
            ${root}/physical-interface:
                _tags:
                    "im.name":
                        _constant: "network-interface-tx-packets"
                    "dsType":
                        _constant: "counter"
                    "name":
                        _text: "name"
                    "live-config":
                        _constant: "true"
                    "display-name":
                        _constant: "Interfaces - TX Packets"
                    "im.dstype.displayType":
                        _constant: "number"
                    "im.identity-tags":
                        _constant: "name"
                _value.double:
                    _text: "traffic-statistics/output-packets"

cross_vendor_tx_drop

package com.indeni.server.rules.library.templatebased.crossvendor

import com.indeni.server.rules.RuleContext
import com.indeni.server.rules.library.{ConditionalRemediationSteps, NearingCapacityWithItemsTemplateRule}

/**
  *
  */
case class CrossVendorTxDrop(context: RuleContext) extends NearingCapacityWithItemsTemplateRule(context,
  ruleName = "cross_vendor_tx_drop",
  ruleFriendlyName = "All Devices: TX packets dropped",
  ruleDescription = "Indeni tracks the number of packets that had issues and alerts if the ratio is too high.",
  usageMetricName = "network-interface-tx-dropped",
  limitMetricName = "network-interface-tx-packets",
  applicableMetricTag = "name",
  threshold = 0.5,
  minimumValueToAlert = 100.0, // We don't want to alert if the number of packets is really low
  alertDescription = "Some network interfaces and ports are experiencing a high drop rate. Review the ports below.",
  alertItemDescriptionFormat = "%.0f dropped packets identified out of a total of %.0f transmitted.",
  baseRemediationText = "Packet drops usually occur when the rate of packets transmitted is higher than the device ability to handle.",
  alertItemsHeader = "Affected Ports")(
  ConditionalRemediationSteps.OS_NXOS ->
    """|
       |1. Run the "show interface" command to review the interface counters and the bitrate. Consider to configure the "load-interval 30" interface sub command to improve the accuracy of the interface measurements. Check for traffic bursts and high traffic utilization.
       |2. Use the "show hardware rate-limit" NX-OS command (if supported) to determine if packets are being dropped because of a rate limit.
       |3. Execute the "show policy-map interface control-plane" NX-OS command to determine if packets are being dropped because of a QoS policy.
       |4. Use the "show hardware internal statistics rates" to determine if packets are being dropped by the hardware.
       |5. Run the "show hardware internal statistics pktflow all" NX-OS command to display per ASIC statistics, including packets into and out of the ASIC. This command helps to identify where packet loss is occurring.
    """.stripMargin,
  ConditionalRemediationSteps.VENDOR_JUNIPER ->
    """|1. Run the “show interface extensive” command to review the interface statistics.
       |2. Check for packet drops and input/output traffic rate.
       |3. Run the “show class-of-service interface x/x/x detail"  to determine any QoS policy applied to interface which may cause packet drops.
       |4. If the interface is saturated, the number of packets dropped by the is indicated by the input queue of the I/O Manager ASIC. This number increments once for every packet that is dropped by the ASIC's RED mechanism.
       |5. Review the following article on Juniper tech support site: <a target="_blank" href="https://www.juniper.net/documentation/en_US/junos/topics/concept/using-show-commands-for-packet-drops.html">Understanding Dropped Packets and Untransmitted Traffic Using show Commands</a>
       |6. If the problem persists, contact the Juniper Networks Technical Assistance Center (JTAC)""".stripMargin,
  ConditionalRemediationSteps.VENDOR_FORTINET ->
    """
       |1. Run "diag hardware deviceinfo nic <interface>" command to display a list of hardware related error names and values. Review  the next link for more details: http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-toubleshooting-54/troubleshooting_tools.htm
       |2. Run command "fnsysctl cat /proc/net/dev" to get a summary of the interface statistics.
       |3. Check for speed and duplex mismatch in the interface settings on both sides of a cable, and check for a damaged cable. Review the next link for more info: http://kb.fortinet.com/kb/documentLink.do?externalID=10653""".stripMargin
)