HP OpenView System Administration Handbook [Electronic resources] : Network Node Manager, Customer Views, Service Information Portal, HP OpenView Operations نسخه متنی

21.3 CORRECTIVE ACTIONS

OVO corrective actions can be automated and/or manual. Actions can be associated with a message template or they can be standalone applications available from the operator's application desktop. The diagram in Figure 21-3 depicts the process that occurs on the managed node and management server when using automatic actions.

Assume that OVO is configured to monitor the UNIX

sendmail process on a managed node. An OVO agent indicates that the

sendmail process is no longer running on the node by a message in the browser. The OVO administrator has the ability to define a corrective action that will attempt to restart the process without manual intervention. Automatic actions are configured in the message source template.

Consider the following scenario: the operator receives a message that the UNIX

sendmail process is not running on node abc123. The message may look something like this:

Sev SUIAONE  Date  Time  Node Application  MsgGroup Object  Message Text 
Crit  01/01/04  15:03:04  abc123  sendmail   email   process  Sendmail is
 not running

The operator uses the Virtual Terminal application from the application desktop to login the system from which the message originated and executes the command

/sbin/init.d/sendmail start 

Executing this command restarts the

sendmail process (solving the problem). The operator should notify the appropriate template administrator how the problem was resolved. The template administrator may opt to add this command to the message condition that captured the terminated

sendmail process. After modifying the template condition, the template needs to be re-distributed to all managed nodes to which it is assigned.

Recall the meaning of the flags in the message indicator column labeled

SUIAONE ; in particular, the columns

I, A, O and

N . These flags are indicators that the operator can use to determine if there are

actions, instructions (automatic or operator initiated), and/or

notes associated with a message.

An X in the I column indicates that instructions are available to the operator in the Message Details window. An operator cannot add instructions to a message. Only administrators and template administrators can add instructions. The operator should pay attention to this column because instructions indicate that procedures may exist to assist the operator in resolving the problem.

An S in the A column indicates that an automatic action is configured in the template that captured the message and that it ran successfully. An F indicates that the configured automatic action failed. Details for why an action failed can be found in the annotations if the template is configured to save the output. An R in the A column indicates that the action is currently running. The operator can check the status and has the option to re-run the automatic action.

An X in the O column indicates that an operator-initiated action exists for this message. This action requires operator intervention. An S indicates the operator-initiated action ran successfully, an F indicates that it failed, and an R indicates that it is currently running.

An X in the N column indicates that there are annotations (notes) associated with the message. An operator can add as many annotations as necessary to a message. The annotations are useful when the operator needs to refer to a previous message that has been resolved.

The administrator can define automatic actions and operator-initiated actions that will attempt to return a managed node back to its normal state. Operator-initiated actions are convenient for more complicated scenarios that require decision-making by an operator. Sometimes it is necessary for an operator to perform status checks before initiating an action. If the operator needs to perform a series of actions for a specific problem, the administrator may opt to use a combination of instructions, operator-initiated actions, and applications. Applications available in the operator's application can also be used in problem resolution.

21.4 Changing the OVO Environment

As your managed enterprise grows and changes, so must the OVO monitoring. Changes include the managed nodes, the node groups, templates, applications, message groups, and operator workspaces. The administrator is responsible for making most of the modifications. However, as the managed environment grows, administering changes to OVO may become overwhelming for a single individual. For that reason, the OVO administrator can create template administrators that have the ability to create and modify templates.

21.4.1 Administrator Changes

The administrator adds nodes to the node bank, creates node groups, adds nodes to node groups, creates applications, integrates SPIs, and adds and configures operators and template administrators. The administrator should solicit feedback from the operators and template administrators in order to better manage the enterprise.

21.4.2 Template Administrator Changes

The purpose of the template administrator role is to create and maintain message source's templates, commands, and actions in support of monitoring applications. A large managed environment might include many template administrators. Template administrators are typically specialized in different areas, such as operating systems, database applications, or specific SPIs. This allows the configuration of OVO to be distributed among many engineers. As more applications are added to the managed enterprise, more template administrators may be necessary.

21.4.3 Operator Changes

The operator has limited ability to make modifications in OVO. While the ability is limited, the changes and feedback to the OVO administrator made by an operator are essential to a successful OVO implementation. The focal point of the day-to-day activities in OVO is the message browser. The operator receives a message, interprets the message, and takes some action based on the information provided in the message.

The operator has the ability to make modifications to the message in a number of ways. Consider the earlier

sendmail scenario where the operator discovers the fix for a recurring problem. The operator receives a message indicating that the

sendmail process is no longer running. The operator logs into the system and issues a command to restart the process.

If executing a command restarts the

sendmail process (resolving the problem), the operator should notify the appropriate template administrator how the problem was resolved. The template administrator may opt to add this command to the message condition that captured the terminated

sendmail process. After modifying the template condition, the template needs to be re-distributed to all managed nodes to which it is assigned. All three OVO users are involved in this change:

• The operator resolves the recurring problem and notifies the template administrator

• The template administrator updates the message source template with an automatic action

• The administrator re-distributes the message source template to assigned managed nodes