Self-monitoring Quick Start Guide - SevOne NMS 5.7

Policies

By creating policies, you can receive alerts when the conditions you specify are triggered. Policies let you define thresholds for device groups or object groups. In this section, we're going to create a policy that will alert us if the average available swap memory is less than or equal to 20% for 15 minutes. This policy comes from the SevOne Recommended Policies section (see policy S1_SELFMON_memory_SwapRed).

Perform the following steps to create a policy.

General Settings tab

On the General Settings tab, you can define basic policy settings.

1. From the navigation bar, click Events and select Configuration , then Policy Browser .
   
   

2. Click Create Policy to display the Policy Editor.

   

3. The Enable check box is selected by default. Go ahead and leave it that way. This indicates that the policy is active.

   Disabled policies appear in light text on the Policy Browser.

4. Leave the Technology Type drop-down set to the default, Metric. This indicates that the policy is triggered by non-flow data.

   If you're creating a policy to be triggered by flow data, select Flow here. When you select Flow, additional, flow-related fields will appear (to the right). For more information about these fields, please see SevOne's documentation on creating and editing policies.

5. In the Name field, enter the name of your policy. Let's call this one S1_SELFMON_memory_SwapRed.

6. Leave the Device Group drop-down set to the default, Device Group.

7. To the right of Device Group, use the drop-down to specify a device group to apply the policy to. Under All Device Groups, click + next to Manufacturer and select SevOne.

8. Group Relationship field allows you to associate multiple device or object groups to a policy. Click the drop-down to choose one of the following options.

   • Member of Any - if the device or object is in any of the selected group(s) then the device or object will be used in the policy. It includes devices or objects as an OR operator. For example, devices or objects that are either in Group 1 OR in Group 2 OR in Group 3.

   • Member of All - if the device or object belongs to all the groups then the device or group will be used in the policy. It includes devices or objects as an AND operator. For example, devices or objects that are in Group 1 AND in Group 2 AND in Group 3.

9. Perform the following actions to specify the object type.

   1. Click the Object Type drop-down.

   2. Scroll down to SNMP Poller and click + to expand the section.

   3. Scroll down to Memory and click + to expand the section.

   4. Scroll down until you see Memory (Linux (Net-SNMP)). Click on this to select it as your object type.

      You won't be able to edit this field once you've saved the policy. This also applies to the next field, Subtype.

10. The Subtype drop-down is set to All Subtypes. In this example, there's no need to select an object subtype.

    If you select the check box next to Show Common Subtypes, only the common object subtypes will appear. You specify object subtypes as common using the Object Subtype Manager.

11. Click the Severity drop-down and select Alert(action must be taken immediately) for our example. This is the severity level that displays on the Alerts page when the policy triggers an alert.

    For this policy (S1_SELFMON_memory_SwapRed), we've specified an alert to be triggered if the average available swap memory is less than or equal to 20% . We've designated our severity level as Alert(action must be taken immediately), which indicates urgency. In the Recommended Policies section, there's a similar policy (S1_SELFMON_memory_SwapYellow) that triggers an alert if available swap memory is less than or equal to 50%. For that policy, we recommend specifying the severity level as Warning (warning condition), since the situation is less urgent.

12. Clicking on edit next to Schedule displays a pop-up where you can specify days/dates and times for the policy to run. When a policy is enabled, the alert engine retests it every three minutes. Use the Schedule settings if there are specific times that you do or don't want the policy to be tested. For our example, we're not going to create a schedule. This means that our policy will be retested every three minutes without interruption.

    For more information about configuring Schedule settings, please see SevOne's documentation on creating and editing policies.

13. Next to Email, click edit to display the email configuration pop-up. Perform the following actions to specify who should receive email when the policy triggers an alert.

    

    1. Provide input for one or more of the following sections:

    2. Addresses - manually enter one or more email addresses. Once you've entered an email address, click to add it. Email addresses that you've added will appear in the column on the right.

    3. Users - select one or more users from the Users list and click to add your selection(s).

    4. Roles - click the Roles drop-down and select the check box next to one or more user roles to add as recipients.

    5. Under Mail when the threshold is triggered, select one of the following options:

    6. Just once - to have the email sent only the first time that the policy triggers an alert. This means that no further email will be sent out for the alert. Of course, once the alert is cleared, if the policy triggers an alert again at a future time, an email will be sent out.

    7. One time every - to specify how frequently an email should be sent when the policy triggers an alert. Enter a number in the text field. Then select the drop down and select either minutes, hours, or days.

    8. Click Close.

14. Next to Trap Destinations, click edit to display a pop-up that lets you specify where to send traps from the policy. For our example, let's go with the default settings. Now click Close.

    For more information about trap destinations, please see SevOne's documentation on the topic.

    

15. The Append Condition Message check box is selected by default. Go ahead and leave it this way. On the next two tabs, we'll be setting up trigger conditions and clear conditions. On the Trigger Conditions tab, we'll be able to create a trigger message. We'll also have the option of creating specific messages for each condition of the trigger. By selecting this check box, all of the condition-specific messages we create will be appended to the trigger message. The same concept applies to the Clear Conditions tab.

16. In the Description field, enter a description of the policy. For our example, we'll say Triggers if available swap memory is low .
    
    

17. Continue with the steps below to configure a trigger condition for this policy.

Trigger Conditions tab

This is where you can define the conditions to trigger the policy. You'll also be able to create a trigger message and condition messages.

1. Select the Trigger Conditions tab. In the upper section of the tab, you'll see the object type and device group that you selected on the General Settings tab.

   

2. In the Trigger Message field, enter the message that you want to display for this policy on the Alerts page. For this one, let's enter Available swap memory on $deviceName is getting low. By including the $deviceName variable, we'll be able to see the name of the device that triggered the alert. (Custom message variables are not available for Flow policies.)

   Remember the Append Condition Message check box on the General Settings tab? It was selected by default, which means that when we create custom messages for specific conditions in just a minute, they'll be appended to the trigger message that we just created. (The same thing applies to condition messages for the clear message, which we'll discuss in just a bit.)

3. Under Conditions, click and select Create New. Perform the following actions to configure your new condition.

   

   1. Click the Indicator drop-down and select an indicator to base the condition on. For this condition, our indicator is Available Swap Memory.

   2. The Type drop-down is set to Static by default. Leave this as it is. This will compare the current indicator value to the indicator value that you specify.

   3. Click the Comparison drop-down and select less than equal to.

   4. In the Threshold field, enter the value to trigger the condition. Enter 20 for our example.

   5. Click the drop-down to the right of the Threshold field and select Percent as the unit of measurement for this condition.

   6. In the Duration field, enter the number of minutes for the condition to be met before triggering an alert. Enter 15 for the example. This means that if the amount of available swap memory is 20% or lower for 15 minutes, the condition will trigger an alert. If the available swap memory is 20% or lower but only for 14 minutes, for example, an alert won't be triggered.

   7. The Aggregation drop-down is set to Average as the default data aggregation method. Just leave it as it is.

   8. In the Custom Message field, you can enter a message specific to this condition. For this one, let's sa y Available swap memory 20% or lower for 15 minutes . Since we've specified that we want custom messages appended to the trigger message, this condition message will append to our trigger message. (Custom message variables are not available for Flow policies.)

   9. Now click Save.

4. Our new condition appears under Conditions. Since it's the first condition, it's listed as Condition A.

   

5. In the Rules section, a new rule has been created as a result of the condition we just created. Since it's the first rule, it's listed as Rule 1. This new rule contains one condition: Condition A.

   If you create another condition, it'll be added to this rule using the AND Boolean operator. For example, Rule 1 would now contain Condition A AND Condition B.
   
   If you'd prefer to apply an OR operator, you would first create a second rule (Rule 2). Then delete Condition B from Rule 1. After you've done that, select the check box for Condition B and add it to Rule 2 (click and select Add to Rule 2).

   

6. For Webhooks, HTTP requests can be invoked to the URL location when an alert is triggered. Verb (GET, POST, PUT, PATCH, DELETE) and URL are required to be defined.

   At present, Webhooks can only be configured on Policies . When adding Thresholds , although Webhooks are visible, they are disabled and cannot be configured.

Clear Conditions tab

This is where you can define the conditions to clear the alert. If you don't define a clear condition, then the alerts triggered by the trigger condition will display on the Alerts page until you manually acknowledge them. In the following steps, we're going to define a clear condition to clear the triggered alert (defined on the Trigger Conditions tab) once the available swap memory is higher than 20% for 15 minutes.

1. Select the Clear Conditions tab. You'll notice that it has an uncanny resemblance to the Trigger Conditions tab.

   

2. In the Clear Message field, enter the message to display when the alert has been cleared. For example, you might say something along the lines of $deviceName is reporting an increase in available swap memory.
   

3. Under Conditions, click and select Create New. Perform the following actions to define your new condition.

   

   1. Click the Indicator drop-down and select an indicator to base the condition on. For this condition, we're going to select Available Swap Memory as we did on the Trigger Conditions tab.

   2. For the Type drop-down, let's go with the default, Static, again.

   3. Click the Comparison drop-down and select greater than.

   4. In the Threshold text field, enter 20.

   5. Click the drop-down to the right of the Threshold field and select Percent.

   6. In the Duration field, enter the number of minutes for the condition to be met before the alert can be cleared. Enter 15 for our example. This means that if the amount of available swap memory is higher than 20% for 15 minutes, the alert that was triggered will be cleared. If the available swap memory is higher than 20% but only for 14 minutes , for example, the alert won't be cleared.

   7. The Aggregation drop-down is set to Average as the default data aggregation method. Just leave it as it is.

   8. In the Custom Message field, you can enter a message specific to this condition. For this one, let's say Available swap memory higher than 20% for 15 minutes . Earlier, we specified that we want custom messages appended to the clear message, so this message will append to our clear message. (Custom message variables are not available for Flow thresholds.)

   9. Now click Save.

      

   At present, Webhooks can only be configured on Policies . When adding Thresholds , although Webhooks are visible, they are disabled and cannot be configured.

4. Once you've configured general settings and created trigger and clear conditions, click Save As New at the bottom of the page.

5. Click the Policy Browser button to return to the Policy Browser, where you can see your new policy.

   

Thresholds

While policies apply to entire device groups or object groups, you can use thresholds to receive alerts when conditions are triggered for a specific device. Creating a threshold is much like creating a policy, with just a couple of differences.

In the steps below, we're going to create a threshold that will alert us if the amount of average used disk space for a specified partition of a device is 80% or higher over 15 minutes . This threshold comes from the SevOne Recommended Policies section (see policy S1_SELFMON_Disk_UtilizationYellow).

Perform the following steps to create a threshold.

General Settings tab

On the General Settings tab, you can define basic threshold settings .

1. From the navigation bar, click Events and select Configuration , then Threshold Browser .
   

2. Click New Threshold to display the Threshold Editor.

   

3. The Enable check box is selected by default. Go ahead and leave it that way. This indicates that the threshold is active.

   Disabled thresholds appear in light text on the Threshold Browser.

4. Next to Created by, you'll see Stand Alone. This indicates that the threshold is created independently of any policy.

   For thresholds that were created from a policy, you would see the name of that policy after Created by. For example, if you were to look up a threshold created by our policy from the previous section, you'd see Created by: S1_SELFMON_memory_SwapRed. Clicking S1_SELFMON_memory_SwapRed would take you directly to the policy itself.

5. Leave the Technology Type drop-down set to the default, Metric. This indicates that the threshold is triggered by non-flow data.

   If you're creating a threshold to be triggered by flow data, select Flow here. When you select Flow, additional, flow-related fields will appear (below and to the right). For more information about these fields, please see SevOne's documentation on creating and editing thresholds.

6. In the Name field, enter the name of your threshold. For our example, we're going to call this one IPSLADevice _S1_SELFMON_Disk_UtilizationYellow. Replace IPSLADevice with the name of the device that your threshold applies to .

7. Click the Device Group drop-down and select the device group that the device belongs to. Under All Device Groups, click + next to Manufacturer and select SevOne. If your device belongs to a different device group, select that device group instead.

8. Click the Device drop-down and select the device that this threshold applies to. For our example, we've selected IPSLADevice 1.

9. Click the Severity drop-down to select a severity level to display on the Alerts page when the threshold triggers an alert. Select Warning (warning condition).
   

10. Leave the Schedule field as it is, so that the alert engine will run every three minutes to retest the thresholds.

    For more information about configuring Schedule settings, please see SevOne's documentation on creating and editing thresholds.

11. Next to Email, click edit to display the email configuration pop-up. Perform the following actions to specify who should receive email when the threshold triggers an alert.
    

    1. Provide input for one or more of the following sections:

    2. Addresses - manually enter one or more email addresses. Once you've entered an email address, click to add it. Email addresses that you've added will appear in the column on the right.

    3. Users - select one or more users from the Users list and click to add your selection(s).

    4. Roles - click the Roles drop-down and select the check box next to one or more user roles to add as recipients.

    5. Under Mail when the threshold is triggered, select one of the following options:

    6. Just once - to have the email sent only the first time that the threshold triggers an alert. This means that no further email will be sent out for the alert. Of course, once the alert is cleared, if the theshold triggers an alert again at a future time, an email will be sent out.

    7. One time every - to specify how frequently an email should be sent when the threshold triggers an alert. Enter a number in the text field. Then select the drop down and select either minutes, hours, or days.

    8. Click Close.

12. Next to Trap Destinations, click edit to display a pop-up that lets you specify where to send traps from the threshold. By default, the first two check boxes are selected. Just leave this as it is. Now click Close.

    For more information about trap destinations, please see SevOne's documentation on the topic.

    

13. The Append Condition Message check box is selected by default. Go ahead and leave it selected.

14. In the Description field, enter a description of the policy. For our example, let's say Triggers if IPSLADevice 1 used disk space is high.
    

Trigger Conditions tab

This is where you can define the conditions to trigger the threshold. You'll also be able to create a trigger message and condition messages.

1. Select the Trigger Conditions tab. Next to Device, you'll see the device that you specified on the General Settings tab.

   

2. In the Trigger Message field, enter the message that you want to display for this threshold on the Alerts page. For this one, we'll say Used disk space for IPSLADevice 1 is high . Feel free to modify the message to suit your needs.

3. Under Conditions, click and select Create New. Perform the following actions to define your new condition.

   

   1. Click the Object drop-down and select an object to base the condition on. For this threshold, we need to specify the disk partition that we want to monitor. To find a list of partitions, scroll down to the SNMP Poller section. The partitions should appear near or at the top of the list. For our example, we'll select /boot - /dev/sda2. You may want to select a different partition, depending on your needs.
      

   2. Click the Indicator drop-down and select an indicator to base the condition on. Select Used Disk Space for the example.

   3. Leave the Type drop-down set to the default, Static. This will compare the current indicator value to the indicator value that you define.

   4. Click the Comparison drop-down and select greater than equal to.

   5. In the Threshold field, enter 80 as the value to trigger the condition.

   6. Click the drop-down to the right of the Threshold field and select Percent as the unit of measure for the condition.

   7. In the Duration field, enter 15 as the number of minutes for the condition to be met before triggering an alert. This means that if the used disk space is 80% or higher for 15 minutes , the condition will trigger an alert.

   8. Leave the Aggregation drop-down set to the default data aggregation method, Average.

   9. In the Custom Message field, you can enter a message specific to this condition. For this one, we'll sa y Used disk space for IPSLADevice 1 80% or higher for 15 minutes. Since we've specified that we want custom messages appended to the trigger message, this condition message will append to our trigger message. (Custom message variables are not available for Flow thresholds.)

   10. Now click Save.
       

       At present, Webhooks can only be configured on Policies . When adding Thresholds , although Webhooks are visible, they are disabled and cannot be configured.

Clear Conditions tab

This is where you can define the conditions to clear the alert. If you don't define a clear condition, then the alerts triggered by the trigger condition will display on the Alerts page until you manually acknowledge them. In the following steps, we're going to define a clear condition to clear the triggered alert (defined on the Trigger Conditions tab) onc e the average amount of used disk space is less than 80% for 15 minutes.

1. Select the Clear Conditions tab.

   

2. In the Clear Message field, enter a message to display when the alert has been cleared. For example, you might say something along the lines of IPSLADevice 1 is reporting a decrease in used disk space .
   

3. Under Conditions, click and select Create New. Perform the following actions to define your new condition.
   

   1. Click the Object drop-down and select an object to base the condition on. Scroll down to the SNMP Poller section and select the same disk partition that you specified for the Trigger Conditions tab. For our example, this was /boot - /dev/sda2 .

   2. Click the Indicator drop-down and select Used Disk Space.

   3. Leave the Type drop-down set to the default, Static.

   4. Click the Comparison drop-down and select less than.

   5. In the Threshold field, enter 80.

   6. Click the drop-down to the right of the Threshold field and select Percent.

   7. In the Duration field, enter 15 as the number of minutes for the condition to be met before the alert can be cleared. This means that if the amount of used disk space is lower than 80% for 15 minutes , the alert that was triggered will be cleared.

   8. Leave the Aggregation drop-down set to the default data aggregation method, Average.

   9. In the Custom Message field, you can enter a message specific to this condition. For this one, let's say Used disk space for IPSLADevice 1 below 80% for 15 minutes. (Custom message variables are not available for Flow thresholds.)

   10. Now click Save.
       

       At present, Webhooks can only be configured on Policies . When adding Thresholds , although Webhooks are visible, they are disabled and cannot be configured.

Filters

The Alerts page features filters, which you can use to focus the display results. For example, if you only want to look at Critical alerts, you can easily specify a severity level of Critical on the General tab. If you wanted to narrow your focus a little more, you could also specify a device group–such as the SevOne device group we talked about earlier–on the Device Groups tab. Filters are cumulative. This means that the display results would include only devices that have Critical alerts and belong to the SevOne device group.

Filters are optional. You can maximize the display results by clearing the filters. Following is an overview of the filter tabs. After making any changes on a tab, click Apply Filter to apply your filter settings for that tab.

Click Show Filter at the top of the Alerts page to see the filter tabs.

General tab

The General tab includes the following filters:

• Severity - lets you specify the severity level of the display results. By default, all severity levels are selected.

• Technology type - lets you specify whether the display results are based on flow data only, non-flow data only, or both flow data and non-flow data.

• Message - lets you filter results based on the message text.

• Assigned - lets you filter results based on the person or role that the alert is assigned to . Earlier, when we set up a policy and a threshold, we had the option of specifying someone to receive email when an alert triggers. This is what Assigned refers to.

• Search ID - lets you filter results based on either the Alert ID, the Threshold ID, or the Policy ID.

• Show - lets you choose whether to view active alerts, alerts that have been ignored, or both.

Devices tab

The Devices tab lets you specify one or more devices to filter the display results by. Click the drop-down and select the device(s) that you'd like to add to the filter.

Objects tab

The Objects tab lets you specify one or more objects for a specific device to filter the display results by. Perform the following steps.

1. Click the Device drop-down and select a device.

2. In the Object field, a list of available objects will appear for the selected device. Select the object(s) that you'd like to add to the filter. Then click to add the object(s) to the field on the right. To display alerts for all objects, don't add any objects to the field on the right.

Device Groups tab

The Device Groups tab lets you specify one or more device group(s) to filter the display results by. Click the drop-down and select the check box next to each device group that you'd like to include.

Object Groups tab

The Object Groups tab lets you filter display results based on one or more specific object groups. Select the object group(s) that you'd like to add to the filter. Once you've made your selection, click to add them. Object groups that you've added will appear in the field on the right.

Grouping

Just below the filters area, you'll see the Grouping drop-down. The Grouping option lets you display different levels of granularity. There are four possible Grouping settings, with Alerts being the most granular.

Below is a description of each Grouping setting. Click the Grouping drop-down to select any of the four settings.

Grouping: Alerts

The Alerts grouping setting is set as the default for the Alerts page and displays the following columns:

• ID - the ID number for the Alert. An ID preceded by is unassigned. , on the other hand, indicates that the alert is assigned.

• Device - the name of the device that triggered the alert.

• First - the date and time that the alert was first reported to SevOne NMS.

• Last - the date and time that the alert was last reported.

• Severity - the severity level of the alert.

• Message - the message generated by the threshold or trap.

Alerts - Controls

When you choose the Alerts Grouping, you'll see several controls at the bottom of the page. You can use these to view alert details and manage alerts. To see the controls for a specific alert, select the check box for that alert. The following is an overview of the available controls.

• Device - the device that triggered the alert. Click the device name to display a link to the Device Summary as well as to the report templates for the device.

• Object - the object that triggered the alert. Click the object name to display a link to the Object Summary as well as to the report templates for the object.
  

• Threshold or Trap - the threshold or trap that triggered the alert. Click the threshold name or trap event name to go to the Threshold Editor or Trap Event Editor.
  

• Severity - the severity level of the alert.

• Message - the message generated by the threshold or trap.

• First - the date and time that the alert was first reported.

• Last - the date and time that the alert was last reported.

• Occurrences - the total number of times that the alert triggered.

  The alert engine runs every three minutes. A policy-based threshold requires ten minutes to gather enough data to trigger an alert. This means that it can take up to thirteen minutes for the first alert for a new policy to appear.

• Assigned To - the name of the person or role that the alert is assigned to. By default, alerts are unassigned.

• Log Analytics - a field that appears if your cluster has a Performance Log Appliance (PLA). Click the Check for device logs link to display log data related to the alert.

Alerts - Actions

At the bottom of the page, just below the controls, you'll see four buttons. With the Alerts grouping setting, there are four actions that you can apply to an alert. The following is an overview of those actions. Select the check box next to an alert and do one (or more) of the following:

• Click Acknowledge to acknowledge the alert. Once an alert has been acknowledged, it's permanently moved to the Alert Archive. When you acknowledge an alert, you'll see a pop-up where you can enter an explanation for the acknowledgement.

  

• Click Retest to see if the threshold trigger condition still exists for the alert. The alert engine retests all thresholds every three minutes.

  A couple notes about alerts triggered by thresholds without clear conditions:

  • They'll remain on the Alerts page until they're acknowledged.

  • When you retest them–even if the trigger condition is no longer met–they'll continue to display on the Alerts page, but the last date and time information won't change.

• Click Ignore to ignore the alert. When you ignore an alert, you'll see a pop-up where you can specify how long you would like to ignore the alert. You can also enter a note.

  

• Click Assign to assign the alert to someone. Then click the drop-down next to Assign to select a person or a role.

Grouping: Devices

The Devices grouping setting displays the following columns:

• Device - the name of the device that triggered the alert.

• First - the date and time that the first alert for the device was first reported.

• Last - the date and time that the last alert for the device was last reported.

• Highest Severity - the highest severity level of the alerts for the device.

• Message - information about the total number of alerts triggered by the device along with the highest severity level of the alerts for the device.

To view the alerts for a device, click either the device name or the device message. Clicking either one will apply the Alerts grouping setting to that specific device.

Grouping: Device Groups

The Device Groups grouping setting displays the following columns:

• Device Group - the name of the device group or device type that triggered alerts.

• First - the date and time that the first alert for the device group/device type was first reported.

• Last - the date and time that the last alert for the device group/device type was last reported.

• Highest Severity - the highest severity level of the alerts for the device group/device type.

• Message - information about the total number of alerts triggered by the device group/device type along with the highest severity level of the alerts for the device group/device type.

Click any device group/device type name or message to view the devices belonging to that group/type along with the alerts for those devices. This will apply the Devices grouping setting to that specific device group/device type.

Grouping: Object Groups

The Object Groups grouping setting displays the following columns:

• Object Group - the name of the object group that triggered alerts.

• First - the date and time that the first alert for the object group was first reported.

• Last - the date and time that the last alert for the object group was last reported.

• Highest Severity - the highest severity level of the alerts for the object group.

• Message - information about the total number of alerts triggered by the object group along with the highest severity level of the alerts for the object group.

Click any object group name or message to view the devices associated with that object group along with the alerts for those devices. This will apply the Alerts grouping setting to that specific object group.