Permanent/Monitored VPN tunnel(s) down-fortinet-FortiOS

error
health-checks
fortios
fortinet
Permanent/Monitored VPN tunnel(s) down-fortinet-FortiOS
0

#1

Permanent/Monitored VPN tunnel(s) down-fortinet-FortiOS

Vendor: fortinet

OS: FortiOS

Description:
Some VPN tunnels are set to be permanent, or monitored, to ensure they are always up. indeni will trigger an issue if such VPN tunnels are down.

Remediation Steps:
Review the cause for the tunnels being down.

How does this work?
The script runs the FortiOS command “get ipsec tunnel list” to retrieve IPSec VPN related information.

Why is this important?
The IPSec VPN state can indicate whether the IPSec VPN has been correctly configured and whether the VPN is up or down.

Without Indeni how would you find this?
An administrator can run the FortiOS command “get ipsec tunnel list” via SSH connection to retrieve the same information.

fortios-get-ipsec-tunnel-list

#! META
name: fortios-get-ipsec-tunnel-list
description: FortiGate VPN IPsec status
type: monitoring
monitoring_interval: 5 minutes
requires:
    vendor: fortinet
    os.name: FortiOS
    product: firewall
    vdom_enabled: false
    vdom_root: true

# --------------------------------------------------------------------------------------------------
# Code Notes
#
# The script read the lines of a table and generate a double metric 'vpn-tunnel-state' per line and extract tag-values.
# 1. We ignores the first line (is the table header).
# 2. The value of the metric is '1' if the value of column 'status' is 'up'
# 3. The number of columns IS dynamic. The last column is the 'timeout' but IF the 'STATUS' is 'down' then there is no timeout value.
# 4. The first column-value (the 'NAME') is multi-word (can have spaces). In order to parse it we read the from index-0 to index of the second column which is always an IP.
#
# The script publish the following metrics
#
# [vpn-tunnel-state]           [number (0|1)] and tags 'remote-peer-name' & 'peer-ip'
# --------------------------------------------------------------------------------------------------


#! COMMENTS
vpn-tunnel-state:
    why: |
        The IPSec VPN state can indicate whether the IPSec VPN has been correctly configured and whether the VPN is up
        or down.
    how: |
        The script runs the FortiOS command "get ipsec tunnel list" to retrieve IPSec VPN related information.
    without-indeni: |
        An administrator can run the FortiOS command "get ipsec tunnel list" via SSH connection to retrieve the same
        information.
    can-with-snmp: true
    can-with-syslog: true


#! REMOTE::SSH
get ipsec tunnel list

#! PARSER::AWK
BEGIN{
    # Table index
    table_tunnel_index = 0

    # A flag, to mark the beginning of the table.
    # Set to 1 when we read the table-header
    is_in_table_section = 0
}

# Parse all the needed info ('remote-peer-name', 'peerip' & 'value') and store them in the table
# Sample table lines:
#Remote 10.10.9.1:0 10.0.8.0/255.255.255.0 10.0.9.0/255.255.255.0 up 43333
#test test    1.1.1.1:0        192.168.220.0/255.255.255.0 1.1.1.0/255.255.255.0    down
#ipsec test ATH 3.2.4.5:0        10.10.80.0/255.255.255.0 2.2.2.0/255.255.255.0    down
((NR > 1) && (is_in_table_section == 1)) {

    # Increase index of table
    table_tunnel_index++

    # Find the the index of the first ip in the line (this is actually the second column).
    index_of_ip = match($0, /[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}/)


    # Get the peer name. Is the first part between the 1-index and the start of the first-ip of the sentence
    remote_peer_name = trim(substr($0, 1, index_of_ip-1))

    # Store peer-name (first column)
    table_tunnel[table_tunnel_index, "remote-peer-name"] = remote_peer_name

    # Check if the last column is the 'timeout' (number). Otherwise the last-column is the 'status'
    if ($(NF) ~ /\d+/) {

        # Last column is the 'timeout' (number). So there are total 6 columns in line.
        value_status = $(NF-1)

        # Store peer-ip (second column)
        table_tunnel[table_tunnel_index, "peerip"] = $(NF-4)

    } else {

        # Last column is the 'status'. So there are 5 columns in line.
        value_status = $(NF)

        # Store peer-ip (second column)
        table_tunnel[table_tunnel_index, "peerip"] = $(NF-3)

    }

    # Check if is "up"
    is_status_up = 0
    if (value_status == "up") {
        is_status_up = 1
    }
    table_tunnel[table_tunnel_index, "value"] = is_status_up
}

# Set the flag 'is_in_table_section' to 1, meaning that the table is started, so we are reading table-lines
#NAME REMOTE-GW PROXY-ID-SOURCE PROXY-ID-DESTINATION STATUS TIMEOUT
/^NAME\s/{
    is_in_table_section = 1
}

END {
    # Publishing metrics based on the parsed data
    for (index_of_table = 1; index_of_table < table_tunnel_index + 1; index_of_table++) {

        # Construct tags of "vpn-tunnel-state"
        tags["name"] =  "Gateway " table_tunnel[index_of_table, "peerip"]
        tags["peerip"] =  table_tunnel[index_of_table, "peerip"]
        tags["remote-peer-name"] = table_tunnel[index_of_table, "remote-peer-name"]

        # Publish metric
        writeDoubleMetricWithLiveConfig("vpn-tunnel-state", tags, "gauge", 300, table_tunnel[index_of_table, "value"], "VPN Tunnels - State", "state", "name")
    }

}

cross_vendor_permanent_tunnel_down

package com.indeni.server.rules.library

import com.indeni.ruleengine.expressions.conditions.{And, Equals}
import com.indeni.ruleengine.expressions.core.{StatusTreeExpression, _}
import com.indeni.ruleengine.expressions.data.{SelectTagsExpression, SelectTimeSeriesExpression, TimeSeriesExpression}
import com.indeni.ruleengine.expressions.scope.ScopeValueExpression
import com.indeni.server.common.data.conditions.True
import com.indeni.server.rules._
import com.indeni.server.rules.library.core.PerDeviceRule
import com.indeni.server.sensor.models.managementprocess.alerts.dto.AlertSeverity


case class PermanentOrMonitoredVpnTunnelIsDownRule(context: RuleContext) extends PerDeviceRule with RuleHelper {

  override val metadata: RuleMetadata = RuleMetadata.builder("cross_vendor_permanent_tunnel_down", "Firewall Devices: Permanent/Monitored VPN tunnel(s) down",
    "Some VPN tunnels are set to be permanent, or monitored, to ensure they are always up. indeni will trigger an issue if such VPN tunnels are down.", AlertSeverity.ERROR).build()

  override def expressionTree: StatusTreeExpression = {
    val actualValue = TimeSeriesExpression[Double]("vpn-tunnel-state").last
    val alwaysOn = ScopeValueExpression("always-on").visible().asString().noneable

    StatusTreeExpression(
      // Which objects to pull (normally, devices)
      SelectTagsExpression(context.metaDao, Set(DeviceKey), True),

      // What constitutes an issue
      StatusTreeExpression(

        // The additional tags we care about (we'll be including this in alert data)
        SelectTagsExpression(context.tsDao, Set("peerip", "name", "always-on"), True, Set("remote-peer-name")),

        // The case here is different compared to VpnTunnelIsDownRule, because we only take
        // the tunnels which are always on
        StatusTreeExpression(
          SelectTimeSeriesExpression[Double](context.tsDao, Set("vpn-tunnel-state"), denseOnly = false),
          And(
            Equals(actualValue, ConstantExpression(Some(0.0))),
            Equals(alwaysOn, ConstantExpression(Some("true")))
          )

          // The Alert Item to add for this specific item
        ).withSecondaryInfo(
          scopableStringFormatExpression("${scope(\"name\")} (${scope(\"peerip\")} - ${scope(\":remote-peer-name\")})"),
          scopableStringFormatExpression("This tunnel is down even though it is set to be always up."),
          title = "VPN Tunnels Affected"
        ).asCondition()
      ).withoutInfo().asCondition()

      // Details of the alert itself
    ).withRootInfo(
      getHeadline(),
      ConstantExpression("One or more VPN tunnels which should remain up is now down."),
      ConditionalRemediationSteps("Review the cause for the tunnels being down.",
        ConditionalRemediationSteps.VENDOR_JUNIPER ->
          """|1. Areas to to check for possible root cause:
             | a. is the remote peer up or down?
             | b. verify that Phase I and Phase II configuration match on both ends
             | c. is policy in place to allow traffic?
             | d. NAT issues
             | e. encryption domain
             | f. routes
             | g. firewall logs.
             |2. Consider enabling debugging for the detailed information.             |
             |3. Review this article on Juniper tech support site: <a target="_blank" href="https://kb.juniper.net/InfoCenter/index?page=content&id=KB10100&actp=METADATA">How to troubleshoot a VPN tunnel that is down or not active</a>""".stripMargin
      )
    )
  }
}