Critical process(es) down-checkpoint-all

Critical process(es) down-checkpoint-all
0

Critical process(es) down-checkpoint-all

Vendor: checkpoint

OS: all

Description:
Many devices have critical processes, usually daemons, that must be up for certain functions to work. Indeni will alert if any of these goes down.

Remediation Steps:
Review the cause for the processes being down.

How does this work?
The status of all important processes is retrieved using the built-in Check Point “cpwd_admin list” command. Descriptions are added, based on information from Check Point KB: https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solutionid=sk97638

Why is this important?
The device functionality is dependent on software processes. It is vital for the operation of the device that these processes are running at all times.

Without Indeni how would you find this?
An administrator could login and manually run the command.

chkp-process-state-fw-novsx

name: chkp-process-state-fw-novsx
description: Checking the state of important processes.
type: monitoring
monitoring_interval: 1 minute
requires:
    vendor: checkpoint
    role-firewall: 'true'
    vsx:
        neq: true
comments:
    process-state:
        why: |
            The device functionality is dependent on software processes. It is vital for the operation of the device that these processes are running at all times.
        how: |
            The status of all important processes is retrieved using the built-in Check Point "cpwd_admin list" command. Descriptions are added, based on information from Check Point KB: https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solutionid=sk97638
        without-indeni: |
            An administrator could login and manually run the command.
        can-with-snmp: false
        can-with-syslog: false
        vendor-provided-management: Listing the status of important processes is only
            available from the command line.
steps:
-   run:
        type: SSH
        command: ${nice-path} -n 15 cpstat mg | grep "Active status:";${nice-path}
            -n 15 cpwd_admin list
    parse:
        type: AWK
        file: cpwd-admin-list-m-novsx.parser.1.awk

cross_vendor_critical_process_down_novsx

// Deprecation warning : Scala template-based rules are deprecated. Please use YAML format rules instead.

package com.indeni.server.rules.library.templatebased.crossvendor

import com.indeni.server.common.data.conditions.Equals
import com.indeni.server.rules.RuleContext
import com.indeni.server.rules.library.templates.StateDownTemplateRule
import com.indeni.server.rules.RemediationStepCondition

case class cross_vendor_critical_process_down_novsx() extends StateDownTemplateRule(
  ruleName = "cross_vendor_critical_process_down_novsx",
  ruleFriendlyName = "All Devices: Critical process(es) down",
  ruleDescription = "Many devices have critical processes, usually daemons, that must be up for certain functions to work. Indeni will alert if any of these goes down.",
  metricName = "process-state",
  applicableMetricTag = "process-name",
  descriptionMetricTag = "description",
  alertItemsHeader = "Processes Affected",
  descriptionStringFormat = "${scope(\"description\")}",
  alertDescription = "One or more processes which are critical to the operation of this device, are down.",
  baseRemediationText = "Review the cause for the processes being down.",
  metaCondition = !Equals("vsx", "true"))(
  RemediationStepCondition.VENDOR_CP -> "Check if \"cpstop\" was run.",
  RemediationStepCondition.VENDOR_CISCO ->
    """|
      |1. Use the "show processes cpu" NX-OS command in order to show the CPU usage at the process level.
      |2. Use the "show process cpu detail <pid> " NX-OS command to find out the CPU usage for all threads that belong to a specific process ID (PID).
      |3. Use the "show system internal sysmgr service pid <pid> " NX-OS command in order to display additional details, such as restart time, crash status, and current state, on the process/service by PID.
      |4. Run the "show system internal processes cpu" NX-OS command which is equivalent to the top command in Linux and provides an ongoing look at processor activity in real time""".stripMargin,
  RemediationStepCondition.VENDOR_FORTINET ->
    """
      |1. Login via ssh to the Fortinet firewall and run the FortiOS command "diagnose sys top [refresh_time_sec] [number_of_lines]"
        |>>> to get the Proccess-id, State, CPU & Memory utilization per process. Press <shift-P> to sort by CPU usage or <shift-M> to sort by memory usage.
      |2. Login via ssh to the Fortinet firewall and run the FortiOS command "diagnose sys top-summary '-h' " to get the command options and receive additional
        |>>> info per process. A sample command could be "diagnose sys top-summary '-s mem -i 60 -n 10' ". In case that the value to the FDS (File Descriptors)
        |>>> column keeps constantly increasing, it might indicate a memory leak problem.
      |3. Review the state of each process provided by the above commands. The normal states are S (Sleeping), R (Running) and D (Do not Disturb).
        |>>> The abnormal states are Z (Zombie) and D (Do not Disturb).
      |4. Try to restart the process which has problem by running the command "diag sys kill 11 <process-Id>". The <process-Id> can be found by the aforementioned commands.
      |5. Check the logs for any reasons why the process stops or can't restart.
      |6. If the problem persists, contact Fortinet Technical support at https://support.fortinet.com/ for further assistance.""".stripMargin.replaceAll("\n>>>", "")

)