High disk space utilization-juniper-junos
Vendor: juniper
OS: junos
Description:
Many systems store vital configuration and operational data on their " +
"storage devices at different mount points and on different " +
"disk devices. It is important to monitor the usage of the " +
"different storage devices to ensure they do not fill up and " +
"cause issues in the on-going operation of the system.
Remediation Steps:
Determine the cause for the high disk usage of the listed file systems.
|||1. On the devices command line interface execute “show system storage detail” command to review storage utilization.
|2. Run “request system storage disconnect” command to free storage space on the SRX by rotating log files and proposing a list of files for deletion.
|3. Remove the debug files after debug is done.
|4. Consider configuring logs to be sent to remote log servers.
|5. Review the following article on Juniper tech support site: Operational Commands: request system storage disconnect.
How does this work?
This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the “show system storage detail” command. The output includes the device’s storage utilization.
Why is this important?
It is very important to monitor the disk space usage of a system. If the disk space is full it will prevent writing more data to the disk. Compressing and moving data from a disk that is 100% full is time consuming, which is why it is important to take care of any such issue early.
Without Indeni how would you find this?
An administrator could login and manually list the disk space usage. Vendors generally provide tools which provide access to this information.
junos-show-system-storage-detail
name: junos-show-system-storage-detail
description: JUNOS retrieve the storage status
type: monitoring
monitoring_interval: 10 minute
requires:
vendor: juniper
os.name: junos
comments:
disk-usage-percentage:
why: |
It is very important to monitor the disk space usage of a system. If the disk space is full it will prevent writing more data to the disk. Compressing and moving data from a disk that is 100% full is time consuming, which is why it is important to take care of any such issue early.
how: |
This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the "show system storage detail" command. The output includes the device's storage utilization.
can-with-snmp: true
can-with-syslog: false
disk-used-kbytes:
why: |
Used to display how much, in kilobytes, of the partition being used. If the file system gets data that should be written to disk can be lost.
how: |
This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the "show system storage detail" command. The output includes the device's storage utilization.
can-with-snmp: true
can-with-syslog: false
disk-total-kbytes:
why: |
Used to display the total partition size, in kilobytes.
how: |
This script logs into the Juniper JUNOS-based device using SSH and retrieves the output of the "show system storage detail" command. The output includes the device's storage utilization.
can-with-snmp: true
can-with-syslog: false
steps:
- run:
type: SSH
file: show-system-storage-detail.remote.1.bash
parse:
type: AWK
file: show-system-storage-detail.parser.1.awk
high_disk_space_usage
package com.indeni.server.rules.library.core
import com.indeni.ruleengine.expressions.conditions.{And, ConditionHelper, GreaterThanOrEqual}
import com.indeni.ruleengine.expressions.core.{ConstantExpression, StatusTreeExpression}
import com.indeni.ruleengine.expressions.data._
import com.indeni.ruleengine.expressions.math.AverageExpression
import com.indeni.ruleengine.expressions.scope.ScopeValueExpression
import com.indeni.server.common.ParameterValue
import com.indeni.server.common.data.conditions.True
import com.indeni.server.params.ParameterDefinition
import com.indeni.server.rules._
import com.indeni.server.rules.library.{ConditionalRemediationSteps, PerDeviceRule, RuleHelper}
import com.indeni.server.sensor.models.managementprocess.alerts.dto.AlertSeverity
case class HighDiskSpaceUsageRule() extends PerDeviceRule with RuleHelper {
private val excludeDisks = Set("/dev", "/mnt/cdrom", "/proc", "/dev/shm", "/dev/shm", "/junos", "/junos/dev")
val highThresholdParameter: ParameterDefinition = new ParameterDefinition(
"High_Threshold_of_Space_Usage",
"",
"High Threshold of Space Usage",
"What is the threshold for the mount point's disk usage for which once it is crossed " + "an issue will be triggered.",
ParameterDefinition.UIType.DOUBLE,
new ParameterValue((80.0).asInstanceOf[Object])
)
override def metadata: RuleMetadata =
RuleMetadata.builder("high_disk_space_usage",
"High disk space utilization",
"Many systems store vital configuration and operational data on their " +
"storage devices at different mount points and on different " +
"disk devices. It is important to monitor the usage of the " +
"different storage devices to ensure they do not fill up and " +
"cause issues in the on-going operation of the system.",
AlertSeverity.ERROR, categories = Set(RuleCategory.HealthChecks), deviceCategory = DeviceCategory.AllDevices).configParameter(highThresholdParameter).build()
override def expressionTree(context: RuleContext): StatusTreeExpression = {
val diskUsage = AverageExpression(TimeSeriesExpression[Double]("disk-usage-percentage"))
val diskNotExcluded = ScopeValueExpression("file-system").visible().isIn(excludeDisks).not
val diskUsageGreaterThanThreshold =
GreaterThanOrEqual(diskUsage, getParameterDouble(highThresholdParameter))
StatusTreeExpression(
SelectTagsExpression(context.metaDao, Set(DeviceKey), True),
StatusTreeExpression(
SelectTagsExpression(context.tsDao, Set("file-system"), True),
And(
diskNotExcluded,
StatusTreeExpression(
SelectTimeSeriesExpression[Double](context.tsDao, Set("disk-usage-percentage"), denseOnly = false),
diskUsageGreaterThanThreshold
).withSecondaryInfo(
scopableStringFormatExpression("${scope(\"file-system\")}"),
scopableStringFormatExpression("Current disk space utilization is: %.0f%%", diskUsage),
title = "Affected Disks / File Systems"
).asCondition()
),
).withoutInfo().asCondition()
).withRootInfo(
getHeadline(),
ConstantExpression("Some disks or file systems are under high usage."),
ConditionalRemediationSteps("Determine the cause for the high disk usage of the listed file systems.",
RemediationStepCondition.VENDOR_PANOS -> "Review https://live.paloaltonetworks.com/t5/Management-Articles/How-and-When-to-Clear-Disk-Space-on-the-Palo-Alto-Networks/ta-p/55736 and https://live.paloaltonetworks.com/t5/tkb/articleprintpage/tkb-id/FeaturedArticles/article-id/89",
RemediationStepCondition.VENDOR_JUNIPER ->
"""|1. On the devices command line interface execute "show system storage detail" command to review storage utilization.
|2. Run "request system storage disconnect" command to free storage space on the SRX by rotating log files and proposing a list of files for deletion.
|3. Remove the debug files after debug is done.
|4. Consider configuring logs to be sent to remote log servers.
|5. Review the following article on Juniper tech support site: <a target="_blank" href="https://www.juniper.net/documentation/en_US/junos/topics/reference/command-summary/request-system-storage-disconnect.html">Operational Commands: request system storage disconnect</a>.""".stripMargin,
RemediationStepCondition.OS_CISCO_ASA ->
"""Run the “show file system” command in privileged EXEC mode to display the size, bytes available, type of media, flags, and prefix information about the disk file system
|Execute the “show file information <filename>” command to view information about a specific file, including partner application package files
|The contents of this filesystem can be verified with any of these commands:
|dir: Displays the contents of the current directory
|show flash: Displays the contents of the internal flash memory
|show disk0: Displays the contents of the internal flash memory
|show disk1: Displays the contents of the external flash memory
|Run the “delete <filename>” to remove unused files. The “/noconfirm” switch of ‘delete’ command with eliminate the need for interactive confirmation. It is common to have unused files from older ASA or ASDM images which can be safely removed.
|If the flash filesystem is properly accessed by the device and works properly, the device indicates this with a Solid Green on the Flash LED on the front panel of the device. Refer to the next cisco troubleshooting guide in case that you have get flash errors messages:
|https://www.cisco.com/c/en/us/support/docs/security/asa-5500-x-series-next-generation-firewalls/113266-asa-flash-error-ts.html
|Use the “File Transfer…” dialog under “Tools” menu of the ASDM to perform file system manipulations""".stripMargin
)
)
}
}