Configuring ITM VIOS Premium Agent for VIO Server Monitoring

Do you use IBM Tivoli Monitoring (ITM) for AIX server monitoring? What about VIO Servers? VIO Server software now comes with the ITM Premium (va) agent pre-installed. It just needs to be configured to communicate with the HMC and the ITM Server. The va agent monitors the availability and health of the VIOS configuration and resources, including the storage and network mappings between the VIOS and its clients.

This article describes how to configure (or reconfigure) the VIOS Premium agent. In our example configuration, we use “test_vios” as the hostname of the example VIO Server, which is connected to HMC “test_hmc” and will be configured to communicate with the “test_itm” ITM server. The VIO Server is running VIOS v 2.2.3.52; the HMC is at v7R7.9.0.2, while the ITM Server is running on Red Hat v5.11.

NOTE: Prior to configuring the agent, in the VIO Servers’ HMC LPAR profiles, check the box for “Allow performance information collection” to allow monitoring of the Available CPU Units in Pool attribute.

Agent Installation

As mentioned previously, the va agent should already be installed on the VIO Server – it gets installed/updated with the VIO Server software.  I haven’t been able to confirm which fileset it is included in, but it appears to be the following fileset:

itm.vios_premium.agent     6.2.6.0    C     F    ITM VIOS Premium Agent

We did have an issue with one of our VIO Servers that had that fileset installed, but for some reason the code for the va agent was missing.  You can list which agents’ code is installed on a server by running the following command (as root):

root@test_vios:/ # /opt/IBM/ITM/bin/cinfo -t

*********** Wed Jan 13 14:17:53 EST 2016 ******************
User: root Groups: staff
Host name : test_vios        Installer Lvl:06.22.04.00
CandleHome: /opt/IBM/ITM
Version Format: VV.RM.FF.II (V: Version; R: Release; M: Modification; F: Fix; I: Interim Fix)
***********************************************************
...Product inventory
PC PRODUCT DESC                              PLAT   VER         BUILD        INSTALL
                                                                               DATE
ax IBM Tivoli Monitoring Shared Libraries    aix523 06.22.04.00 d1083a       20120414
ax IBM Tivoli Monitoring Shared Libraries    aix526 06.22.04.00 d1083a       20120414
gs IBM GSKit Security Interface              aix523 07.40.27.00 d1075a       -
gs IBM GSKit Security Interface              aix526 07.40.27.00 d1075a       -
jr Tivoli Enterprise-supplied JRE            aix523 05.12.01.00 d1075a       -
jr Tivoli Enterprise-supplied JRE            aix526 05.12.01.00 d1075a       -
pk Monitoring Agent for CEC Base             aix523 06.22.02.00 201108071131 20140906
ui Tivoli Enterprise Services User Interface aix523 06.22.04.00 d1083a       20120414
ui Tivoli Enterprise Services User Interface aix526 06.22.04.00 d1083a       20120414
va Monitoring Agent for VIOS Premium         aix523 06.22.02.04 201108071135 20140906

In this case, the va agent is listed, so we are good to go.  In the one case in which the va agent wasn’t listed, we were able to run the following commands to install it (as root):

# cd /usr/lpp/itm.premium/itm_agent
# ./install.sh -q -h /opt/IBM/ITM -p $PWD/silent_install.vios_agent

SSH Key Configuration

First, let’s see if the va agent is already running.  If so, we’ll stop it (as root):

root@test_vios:/ # /opt/IBM/ITM/bin/cinfo -r
*********** Wed Jan 13 14:35:41 EST 2016 ******************
User: root Groups: staff
Host name : test_vios        Installer Lvl:06.22.04.00
CandleHome: /opt/IBM/ITM
***********************************************************
Host           Prod  PID       Owner  Start     ID    ..Status
test_vios  pk    10027124                   None  ...process not running
test_vios  va    13762756  root   10:22:15  None  ...running

root@test_vios:/ # /opt/IBM/ITM/bin/itmcmd agent stop va
Stopping Premium Monitoring Agent for VIOS ...
Product Premium Monitoring Agent for VIOS was stopped gracefully.
Agent stopped...

Note: we could have stopped the va agent from the padmin shell by using command “stopsvc ITM_premium”.

Next, see if a ssh key pair exists for the root userid (as root).  It would be contained in the .ssh sub-directory underneath root’s home directory:

root@test_vios:/ # lsuser -a home root
root home=/

root@test_vios:/ # ls -l /.ssh/*.pub
-rw-r--r--    1 root     staff           228 Sep 26 2014  /.ssh/id_rsa.pub

root@test_vios:/ # cat /.ssh/*.pub
ssh-rsa ADAAB3NzaC1yc2EAAAABIwAAAIEAx5Y2TiJqh744oalnsZ5om5yiXFN64Em31y4Kv1XFTFzzGqzyj2hciaNO0sKtYRaYH2iwiN1xlJ01mq/yzI04+rPhCWW2Q94NJLdbeBmiOub5bQIM5Tsmv05MAvNfCVaKXTmaTDd1SfziJYDE5b6+TBAnZWhZWkRVTOUmdWaTbdc= root@test_vios

In this case, the key already exists, so it doesn’t really need to be recreated.  However, it was created with 1024-bit encryption and we want to use 2048-bit encryption, so we will recreate it using the following commands (which would be the same commands that you would use to create the key from scratch, but you may need to create the .ssh directory first, and you won’t be prompted to overwrite an existing file):

root@test_vios:/ # cd /.ssh

root@test_vios:/.ssh # ls -l
total 16
-rw-r--r--    1 root     staff          1255 Sep 27 2015  authorized_keys2
-rw-------    1 root     staff           887 Sep 26 2014  id_rsa
-rw-r--r--    1 root     staff           228 Sep 26 2014  id_rsa.pub 
-rw-r--r--    1 root     staff           899 Dec 23 2015  known_hosts

root@test_vios:/.ssh # ssh-keygen -b 2048 -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (//.ssh/id_rsa):
//.ssh/id_rsa already exists.
Overwrite (y/n)? y
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in //.ssh/id_rsa.
Your public key has been saved in //.ssh/id_rsa.pub.
The key fingerprint is:
91:ab:78:5e:d4:b4:74:b0:df:e3:2d:74:e5:b3:91:8a root@test_vios
The key's randomart image is:
+--[ RSA 2048]----+
|                 |
|         .       |
|     o .o        |
|    D =  o       |
|     = .S        |
|  . .=.          |
|   o. +oo        |
|  . o.=+*        |
|   ..+*o.o       |
+-----------------+

root@test_vios:/.ssh # ls -l
total 32
-rw-r--r--    1 root     staff          1255 Sep 27 2015  authorized_keys2
-rw-------    1 root     staff          1679 Jan 18 17:19 id_rsa
-rw-r--r--    1 root     staff           400 Jan 18 17:19 id_rsa.pub
-rw-r--r--    1 root     staff           899 Dec 23 2015  known_hosts

root@test_vios:/.ssh # cat /.ssh/id_rsa.pub
ssh-rsa AAAAB3NzaC1yc2EAAAADAWeBAAABAQDG3xnstqdyGOK3mJDcnjtuDRvG5MJm6WfUPios5IhsChVl1gO0dnynuwCEioku7q4dtRBNnphkOY/Pi6vLs32WgoPWNDGYagA6AoLtkFVoT+3kLIJvqltDk5RskUvrZnwRHxwkZmGP77qxJcQU0sYQgY6Ffp+A63JN0CteXsHtitjCstD39aCOtHIEATNxQTXPDk5reDqCEuEwno8+tQwCzmdtKZSm6dD+8MRzTr2E6XVRkBaXimYddsPfRVTM6FC191knoarvjnZyIGU5f1h2zDG+WDp0IsO5WyMRCLFkY0nIOfP3LHYFMDztEvMmV8TlEkG7kn7hw+4ZDJbdZQz9 root@test_vios

Another way to see if the key is created, and to view the key if it exists, is to use the following command in the padmin shell:

padmin@test_vios:/home/padmin $ cfgsvc -key ITM_premium
ssh-rsa AAAAB3NzaC1yc2EAAAADAWeBAAABAQDG3xnstqdyGOK3mJDcnjtuDRvG5MJm6WfUPios5IhsChVl1gO0dnynuwCEioku7q4dtRBNnphkOY/Pi6vLs32WgoPWNDGYagA6AoLtkFVoT+3kLIJvqltDk5RskUvrZnwRHxwkZmGP77qxJcQU0sYQgY6Ffp+A63JN0CteXsHtitjCstD39aCOtHIEATNxQTXPDk5reDqCEuEwno8+tQwCzmdtKZSm6dD+8MRzTr2E6XVRkBaXimYddsPfRVTM6FC191knoarvjnZyIGU5f1h2zDG+WDp0IsO5WyMRCLFkY0nIOfP3LHYFMDztEvMmV8TlEkG7kn7hw+4ZDJbdZQz9 root@test_vios

Now we need to copy this public key into the authorized_keys2 file of the HMC user that we will be using.  In this case, that is hscroot, so we need to login via the HMC command line as hscroot on test_hmc.  Once logged in, run the following “mkauthkeys” command to add the key:

hscroot@test_hmc:~> mkauthkeys --add ‘ssh-rsa AAAAB3NzaC1yc2EAAAADAWeBAAABAQDG3xnstqdyGOK3mJDcnjtuDRvG5MJm6WfUPios5IhsChVl1gO0dnynuwCEioku7q4dtRBNnphkOY/Pi6vLs32WgoPWNDGYagA6AoLtkFVoT+3kLIJvqltDk5RskUvrZnwRHxwkZmGP77qxJcQU0sYQgY6Ffp+A63JN0CteXsHtitjCstD39aCOtHIEATNxQTXPDk5reDqCEuEwno8+tQwCzmdtKZSm6dD+8MRzTr2E6XVRkBaXimYddsPfRVTM6FC191knoarvjnZyIGU5f1h2zDG+WDp0IsO5WyMRCLFkY0nIOfP3LHYFMDztEvMmV8TlEkG7kn7hw+4ZDJbdZQz9 root@test_vios’

We can verify that the key was added to hscroot’s authorized_keys2 file on the HMC by running the following command (as hscroot):

hscroot@test_hmc:~> grep test_vios /home/hscroot/.ssh/authorized_keys2
ssh-rsa AAAAB3NzaC1yc2EAAAADAWeBAAABAQDG3xnstqdyGOK3mJDcnjtuDRvG5MJm6WfUPios5IhsChVl1gO0dnynuwCEioku7q4dtRBNnphkOY/Pi6vLs32WgoPWNDGYagA6AoLtkFVoT+3kLIJvqltDk5RskUvrZnwRHxwkZmGP77qxJcQU0sYQgY6Ffp+A63JN0CteXsHtitjCstD39aCOtHIEATNxQTXPDk5reDqCEuEwno8+tQwCzmdtKZSm6dD+8MRzTr2E6XVRkBaXimYddsPfRVTM6FC191knoarvjnZyIGU5f1h2zDG+WDp0IsO5WyMRCLFkY0nIOfP3LHYFMDztEvMmV8TlEkG7kn7hw+4ZDJbdZQz9 root@test_vios

Now, we complete the connection by running the “ssh” command from test_vios as root:

root@test_vios:/.ssh # ssh hscroot@test_hmc
The authenticity of host 'test_hmc (10.3.33.193)' can't be established.
ECDSA key fingerprint is 16:1a:ba:73:ee:d3:3c:cd:ca:eb:44:ca:3k:e8:1b:b5.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'test_hmc,10.6.192.193' (ECDSA) to the list of known hosts.
Last login: Wed Jan 13 17:16:12 2016 from test_server10
hscroot@test_hmc:~> exit
exit

VIOS Premium Agent Configuration

Now that we have the ssh key connection working correctly, we go back to the padmin shell of the VIO Server to configure the va agent.  Let’s first view the current configuration (as padmin):

padmin@test_vios:/home/padmin $ lssvc ITM_premium
MANAGING_SYSTEM:
HOSTNAME:
RESTART_ON_REBOOT:FALSE
MIRROR:

In this case, it does not have any configuration information, so it has probably never been configured.  The “cfgsvc” command is used to establish the configuration, specifying the ITM server for the “hostname” flag and the HMC for the “managing_system” flag.  I believe it defaults to the hscroot user, so we shouldn’t need to specify that, but we will anyway.  Here is our command (run as padmin):

padmin@test_vios:/home/padmin $ cfgsvc ITM_premium -attr Restart_On_Reboot=TRUE hostname=test_itm managing_system=hscroot@test_hmc
Agent configuration started...
Agent configuration completed...

padmin@test_vios:/home/padmin $ lssvc ITM_premium
MANAGING_SYSTEM:hscroot@test_hmc
HOSTNAME:test_itm
RESTART_ON_REBOOT:TRUE
MIRROR:

Now, it looks correct, so let’s start the service (as padmin):

padmin@test_vios:/home/padmin $ startsvc ITM_premium
Starting Premium Monitoring Agent for VIOS ...
Premium Monitoring Agent for VIOS started

Looks good. Let’s verify as root:

root@test_vios:/ # /opt/IBM/ITM/bin/cinfo -r
*********** Wed Jan 13 17:35:24 EST 2016 ******************
User: root Groups: staff
Host name : test_vios        Installer Lvl:06.22.04.00
CandleHome: /opt/IBM/ITM
***********************************************************
Host           Prod  PID       Owner  Start     ID    ..Status
test_vios  pk    10027124                   None  ...process not running
test_vios  va    8913134   root   17:35:11  None  ...running

root@test_vios:/ # ps -ef | grep kvaagent | grep -v grep
root  8913134        1   0 17:35:11  pts/0  0:00 /opt/IBM/ITM/aix523/va/bin/kvaagent

We are up and running, and our Tivoli Monitoring team confirms that it is reporting properly and is monitoring the VIO Server.

For more details, read the full ITM VIOS Premium Agent User’s Guide.

Leave a Reply

Your email address will not be published. Required fields are marked *