High memory usage-f5-all
Indeni will alert if the memory utilization of a device is above a high threshold. If the device has multiple memory elements, each will be inspected separately and alert for.
Determine the cause for the high memory usage of the listed elements.
How does this work?
This script uses the F5 REST API to retrieve the current memory usage of each TMM core.
Why is this important?
The various memory components of an F5 unit are important to track to ensure a smooth operation. This particlar metric focuses on the memory usage per TMM core.
Without Indeni how would you find this?
These memory metrics are available through SNMP, TMSH and the web user interface under Statistics/Performance.
name: f5-rest-mgmt-tm-sys-tmm-info description: Determine tmm memory and cpu usage type: monitoring includes_resource_data: true monitoring_interval: 1 minute requires: vendor: f5 product: load-balancer rest-api: 'true' comments: memory-usage: why: | The various memory components of an F5 unit are important to track to ensure a smooth operation. This particlar metric focuses on the memory usage per TMM core. how: | This script uses the F5 REST API to retrieve the current memory usage of each TMM core. without-indeni: | These memory metrics are available through SNMP, TMSH and the web user interface under Statistics/Performance. can-with-snmp: true can-with-syslog: false cpu-usage: why: | Tracking the CPU utilization of all processing cores on an F5 unit is critical to ensure capacity isn't reached. Should capacity be reached on data plane cores, new connections and sessions may fail to be processed. This particular alert focuses on the cpu usage per TMM core. how: | This alert uses the F5 iControl REST API to retrieve the current CPU usage of all TMM cores. without-indeni: | The utilization of the TMM cores is also available through SNMP and TMSH. can-with-snmp: true can-with-syslog: false memory-total-kbytes: skip-documentation: true memory-free-kbytes: skip-documentation: true steps: - run: type: HTTP command: /mgmt/tm/sys/tmm-info?options=kil&$select=fiveSecAvgUsageRatio,memoryTotal,memoryUsed,tmmId parse: type: JSON file: rest-mgmt-tm-sys-tmm-info.parser.1.json.yaml
// Deprecation warning : Scala template-based rules are deprecated. Please use YAML format rules instead. package com.indeni.server.rules.library.templatebased.crossvendor import com.indeni.server.rules.RuleContext import com.indeni.server.rules.library.templates.NearingCapacityWithItemsTemplateRule import com.indeni.server.rules.RemediationStepCondition /** * */ case class cross_vendor_high_memory_usage() extends NearingCapacityWithItemsTemplateRule( ruleName = "cross_vendor_high_memory_usage", ruleFriendlyName = "All Devices: High memory usage", ruleDescription = "Indeni will alert if the memory utilization of a device is above a high threshold. If the device has multiple memory elements, each will be inspected separately and alert for.", usageMetricName = "memory-usage", applicableMetricTag = "name", threshold = 92.0, alertDescription = "Some memory elements are nearing their maximum capacity.", alertItemDescriptionFormat = "Current memory utilization is: %.0f%%", baseRemediationText = "Determine the cause for the high memory usage of the listed elements.", alertItemsHeader = "Memory Elements Affected", itemsToIgnore = Set("^vCMP host - (swap|linux).*".r, "^PA firewall management plane.*".r))( RemediationStepCondition.VENDOR_CP -> """ |Consider reading https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solutionid=sk33781#MEMORY | |Note: In trying to understand this alert, you can use the Linux "free" command to view memory utilization. The output of this command can be confusing. To be sure you correctly understand the output, see: https://serverfault.com/questions/85470/meaning-of-the-buffers-cache-line-in-the-output-of-free | |Also note that Linux has recently changed the "free" output format to make it more intuitive. This change was made in procps 3.3.10 ("procps" is the group of utilities which includes "free"). Use "ps -V" in expert mode to see your version of procps. See also: https://askubuntu.com/questions/770108/what-do-the-changes-in-free-output-from-14-04-to-16-04-mean""".stripMargin, RemediationStepCondition.VENDOR_PANOS -> "Consider opening a support ticket with Palo Alto Networks.", RemediationStepCondition.VENDOR_CISCO -> """| |1. Check from the Indeni the memory utilization history graph for this device and review the pattern. Correlate any change to the pattern with any configuration change |2. The next NX-OS commands output can inform whether the platform memory utilization is normal or un-expected: |• show system resources |• show processes memory. |3. For more information please review the next troubleshooting guide for high memory utilization: http://docwiki.cisco.com/wiki/Cisco_Nexus_7000_Series_NX-OS_Troubleshooting_Guide_--_Troubleshooting_Memory""".stripMargin, RemediationStepCondition.VENDOR_FORTINET -> """ |1. Login via https to the Fortinet firewall and go to menu System > Dashboard > Status. Look at the system resources widget to review the current Memory utilization graph. |2. Login via ssh to the Fortinet firewall and run the FortiOS command "diagnose hardware sysinfo memory" which provides information about current memory usage. |3. Check if the unit is dealing with high traffic volume or with connection pool limits. |4. Check if the Fortinet firewall is in "conserve mode" state by running the FortiOS command "diagnose hardware sysinfo conserve". For more information review the following Fortinet guides: |- http://kb.fortinet.com/kb/viewContent.do?externalId=FD33103 |- http://kb.fortinet.com/kb/viewContent.do?externalId=11076 |5. If the problem persists, contact Fortinet Technical support at https://support.fortinet.com/ for further assistance.""".stripMargin, RemediationStepCondition.VENDOR_BLUECOAT -> """ |1. Login via https to the ProxySG and go to Statistics > System > Resources > Memory use. Review the current Memory utilization graph. |2. Login via ssh to the ProxySG and run the command "show resources" which provides information about current memory usage. |3. Check if the unit is dealing with high traffic volume. |4. Check the ICAP service maximum number of connections. For more information review the following Bluecoat guides: |- https://origin-symwisedownload.symantec.com/resources/webguides/proxysg/certification/sg_firststeps_webguide/Content/Troubleshooting/Malware%20Prevention/troubleshoot_sg_unresponsive.htm |- https://origin-symwisedownload.symantec.com/resources/webguides/contentanalysis/13/system_webguide/Content/Topics/Tasks/Stats_Mem.htm |5. If the problem persists, contact Symantec Technical support at https://support.symantec.com for further assistance. """.stripMargin )