Configuration changed on standby member-juniper-junos

Configuration changed on standby member-juniper-junos
0

Configuration changed on standby member-juniper-junos

Vendor: juniper

OS: junos

Description:
Generally, making configuration changes to the standby member of a device is not recommended. indeni will trigger an issue if this happens.

Remediation Steps:
Make the configuration changes to the active member of the cluster.

How does this work?
This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the “show chassis cluster status” command. The output includes the status of all redundancy groups across the cluster.

Why is this important?
Tracking the state of a cluster member is important. If a cluster member which used to be the active member of the cluster no longer is, it may be the result of an issue. In some cases, it is due to maintenance work (and so was anticipated), but in others it may be due to a failure in the firewall or another component in the network.

Without Indeni how would you find this?
The administrator has to run the “show chassis cluster status” on the device to find whether the cluster member is active or not.

junos-show-chassis-cluster-status

name: junos-show-chassis-cluster-status
description: JUNOS collect clustering status
type: monitoring
monitoring_interval: 1 minute
requires:
    vendor: juniper
    os.name: junos
    product: firewall
    high-availability: true
comments:
    cluster-member-active:
        why: |
            Tracking the state of a cluster member is important. If a cluster member which used to be the active member of the cluster no longer is, it may be the result of an issue. In some cases, it is due to maintenance work (and so was anticipated), but in others it may be due to a failure in the firewall or another component in the network.
        how: |
            This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the "show chassis cluster status" command. The output includes the status of all redundancy groups across the cluster.
        without-indeni: "The administrator has to run the \"show chassis cluster status\"\
            \ on the device to find whether the cluster member is active or not. \n"
        can-with-snmp: true
        can-with-syslog: true
    cluster-state:
        why: |
            Tracking the state of a cluster is important. If a cluster which used to be healthy no longer is, it may be the result of an issue. In some cases, it is due to maintenance work (and so was anticipated), but in others it may be due to a failure in the members of the cluster or another component in the network.
        how: |
            This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the "show chassis cluster status" command. The output includes the status of all redundancy groups across the cluster.
        without-indeni: "The administrator has to run the \"show chassis cluster status\"\
            \ on the device to find whether neither of cluster nodes is in primary\
            \ state. \n"
        can-with-snmp: true
        can-with-syslog: true
    cluster-preemption-enabled:
        why: |
            Preemption is a function in clustering which sets a primary member of the cluster to always strive to be the active member. The trouble with this is that if the active member that is set with preemption on has a critical failure and reboots, the cluster will fail over to the secondary and then immediately fail over back to the primary when it completes the reboot. This can result in another crash and the process would happen again and again in a loop.
        how: |
            This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the "show chassis cluster status" command. The output includes the status of all redundancy groups across the cluster.
        without-indeni: "The administrator has to run the \"show chassis cluster status\"\
            \ on the device to find whether preemption is enabled and correctly configured\
            \ if one of the nodes is expected to be always primary node. \n"
        can-with-snmp: false
        can-with-syslog: false
steps:
-   run:
        type: SSH
        file: show-chassis-cluster-status.remote.1.bash
    parse:
        type: AWK
        file: show-chassis-cluster-status.parser.1.awk

junos-show-chassis-cluster-information-configuration-synchronization

name: junos-show-chassis-cluster-information-configuration-synchronization
description: Get chassis cluster configuration synchronization status
type: monitoring
monitoring_interval: 10 minute
requires:
    vendor: juniper
    os.name: junos
    product: firewall
    high-availability: true
comments:
    cluster-config-synced:
        why: "The failure of configuration synchronization will cause misbehaviors\
            \ when the cluster failover occurs. For examples, an interfacer which\
            \ should be enabled is still in disabled state, the latest configuration\
            \ fails to apply to the new active node, and etc. \n"
        how: |
            The script runs the "show chassis cluster information configuration synchronization" command via SSH and retrieves the configuration synchronization status.
        without-indeni: |
            An administrator could log on to the device to manually run this command to get configuration synchronization status.
        can-with-snmp: null
        can-with-syslog: null
        vendor-provided-management: The configuration synchronization status can be
            retrieved via the command line.
steps:
-   run:
        type: SSH
        file: show-chassis-cluster-information-configuration-synchronization.remote.1.bash
    parse:
        type: AWK
        file: show-chassis-cluster-information-configuration-synchronization.parser.1.awk

cross_vendor_config_change_on_standby

package com.indeni.server.rules.library.core
import com.indeni.apidata.time.TimeSpan
import com.indeni.ruleengine.expressions.conditions.{And, Equals, GreaterThanOrEqual}
import com.indeni.ruleengine.expressions.core._
import com.indeni.ruleengine.expressions.data.{SelectTagsExpression, SelectTimeSeriesExpression, TimeSeriesExpression}
import com.indeni.server.common.data.conditions.True
import com.indeni.server.rules.library.{ConditionalRemediationSteps, PerDeviceRule, RuleHelper}
import com.indeni.server.rules.{RuleContext, _}
import com.indeni.server.sensor.models.managementprocess.alerts.dto.AlertSeverity


case class ConfigChangeOnStandbyMemberRule() extends PerDeviceRule with RuleHelper {

  override val metadata: RuleMetadata = RuleMetadata.builder("cross_vendor_config_change_on_standby", "Configuration changed on standby member",
    "Generally, making configuration changes to the standby member of a device is not recommended. indeni will trigger an issue if this happens.",
    AlertSeverity.WARN, categories = Set(RuleCategory.HighAvailability), deviceCategory = DeviceCategory.ClusteredDevices).interval(TimeSpan.fromMinutes(5)).build()

  override def expressionTree(context: RuleContext): StatusTreeExpression = {
    val configUnsavedValue = TimeSeriesExpression[Double]("config-unsaved").last
    val memberStateValue = TimeSeriesExpression[Double]("cluster-member-active").last
    val configSyncValue = TimeSeriesExpression[Double]("cluster-config-synced").last

    StatusTreeExpression(
      // Which objects to pull (normally, devices)
      SelectTagsExpression(context.metaDao, Set(DeviceKey), True),

      StatusTreeExpression(
        // The time-series we check the test condition against:
        SelectTimeSeriesExpression[Double](context.tsDao, Set("config-unsaved", "cluster-member-active", "cluster-config-synced"), denseOnly = false),

        // The condition which, if true, we have an issue. Checked against the time-series we've collected
        And(
          Equals(configUnsavedValue, ConstantExpression(Some(1.0))),
          Equals(memberStateValue, ConstantExpression(Some(0.0))),
          GreaterThanOrEqual(configSyncValue, ConstantExpression(Some(0.0))))
      ).withoutInfo().asCondition()

      // Details of the alert itself
    ).withRootInfo(
      getHeadline(),
      ConstantExpression("The configuration has been changed on this device, but it's not the active member of the cluster. Best practices recommend making changes to the active member of a cluster and then syncing to the standby."),
      ConditionalRemediationSteps("Make the configuration changes to the active member of the cluster.",
        RemediationStepCondition.VENDOR_CISCO ->
          """1. Save the configuration by executing the "copy running startup config" command. Note: Network admin role is required to execute this command.
            |2. Check that there are not unsaved configuration changes by running the “show running-config diff” command to the switches.
            |3. Consider creating snapshots of the configuration by utilizing the Checkpoint and Rollback NX-OS features. The NX-OS checkpoint and rollback feature are extremely useful, and a life saver in some cases, when a new configuration change to a production system has caused unwanted effects or was incorrectly made/planned and we need to immediately return to an original/stable configuration.
            |4. For more information review the following article: <a target="_blank" href="http://www.firewall.cx/cisco-technical-knowledgebase/cisco-data-center/1202-cisco-nexus-checkpoint-rollback-feature.html">Guide to Nexus checkpoint & rollback feature</a>""".stripMargin,
        RemediationStepCondition.VENDOR_JUNIPER ->
          """|1. The chassis cluster synchronization feature automatically synchronizes the configuration from the primary node to the secondary node when the secondary joins the primary as a cluster.
             |2. Review the following article on Juniper tech support site: <a target="_blank" href="https://www.juniper.net/documentation/en_US/junos/topics/concept/chassis-cluster-backup-config-sync.html">Understanding Automatic Chassis Cluster Synchronization Between Primary and Secondary Nodes</a>""".stripMargin
      )
    )
  }
}