AIX > Administrator > Systems Management

AIX Live Update Best Practices for 2018

AIX Live Update Best Practices

The official IBM AIX Live Update Best Practices are well documented here. Please refer to these when planning a Live Update implementation in your environment. I’ve written about my experiences with Live Update numerous times, including AIX live updates and how to live update your AIX system without rebooting the server

AIX Live Update in a PowerVC Managed Cloud

I’ve been testing and working with Live Updates since 2015. What follows are some of my best practices based on my experiences using this tool in the field. They are not official IBM best practices and should be treated as such. These are merely guidelines, based on anecdotal data I’ve collected whilst using the tool. I hope you find them useful.

1. Always take a mksysb. Always create an alt_disk_copy of rootvg BEFORE you start a Live Update operation. Please ensure you have a way to back out cleanly.

2. Install your TL, SPs and ifixes then run Live Update. As per the picture below (from my LU presentation).

Screen-Shot-2018-06-14-at-12-44-01-PM.png
It may not be well known, but you can use Live Update (geninstall –k) after you’ve installed AIX fixes. You don’t have to include them in the Live Update process. You can patch the OS, and then kick off the LU process when you’re done. The AIX knowledge Center

“Note: You can install any updates and interim fixes by using your preferred methods, and then perform a Live Update operation instead of restarting the system by running the following command:

# geninstall –k”

https://www.ibm.com/support/knowledgecenter/en/ssw_aix_72/com.ibm.aix.install/live_update_geninstall.htm

note.png

Here’s an example of how you could perform a Live Update operation, immediately after you’ve patched your AIX OS. This example is taken from a PowerVC managed environment.

We start with an AIX system running AIX 7.2 TL2 SP1. We would like to update it to SP2 and install ifix IJ04268s2a.180403.AIX72TL02SP02.epkg.Z. We’d like the new SP and ifix to take effect without a reboot, using Live Update.

a) Create clone (backup) of current rootvg.

root@lkutest / # alt_disk_copy -Bd hdisk1
Calling mkszfile to create new /image.data file.
Checking disk sizes.
Creating cloned rootvg volume group and associated logical volumes.
Creating logical volume alt_hd5.
Creating logical volume alt_hd6.
Creating logical volume alt_hd8.
Creating logical volume alt_hd4.
Creating logical volume alt_hd2.
Creating logical volume alt_hd9var.
Creating logical volume alt_hd3.
Creating logical volume alt_hd1.
Creating logical volume alt_hd10opt.
Creating logical volume alt_hd11admin.
Creating logical volume alt_lg_dumplv.
Creating logical volume alt_livedump.
Creating logical volume alt_datalv.
Creating /alt_inst/ file system.
Creating /alt_inst/admin file system.
Creating /alt_inst/data file system.
Creating /alt_inst/home file system.
Creating /alt_inst/opt file system.
Creating /alt_inst/tmp file system.
Creating /alt_inst/usr file system.
Creating /alt_inst/var file system.
Creating /alt_inst/var/adm/ras/livedump file system.
Generating a list of files
for backup and restore into the alternate file system...
Backing-up the rootvg files and restoring them to the alternate file system...
Modifying ODM on cloned disk.
Building boot image on cloned disk.
forced unmount of /alt_inst/var/adm/ras/livedump
forced unmount of /alt_inst/var/adm/ras/livedump
forced unmount of /alt_inst/var
forced unmount of /alt_inst/var
forced unmount of /alt_inst/usr
forced unmount of /alt_inst/usr
forced unmount of /alt_inst/tmp
forced unmount of /alt_inst/tmp
forced unmount of /alt_inst/opt
forced unmount of /alt_inst/opt
forced unmount of /alt_inst/home
forced unmount of /alt_inst/home
forced unmount of /alt_inst/data
forced unmount of /alt_inst/data
forced unmount of /alt_inst/admin
forced unmount of /alt_inst/admin
forced unmount of /alt_inst
forced unmount of /alt_inst
Changing logical volume names in volume group descriptor area.
Fixing LV control blocks...
Fixing file system superblocks...
root@lkutest / #

b) Check current kernel build version. Check OS level prior to installing SP.

root@lkutest / # cat /proc/version
Sep 25 2017
13:39:27
1739A_72F
@(#) _kdb_buildinfo unix_64 Sep 25 2017 13:39:27 1739A_72F

root@lkutest # oslevel -s
7200-02-01-1731

c) Authenticate with the PowerVC management server (or HMC if you’re not using PowerVC).

root@lkutest / # pvcauth -u pvcadmin -p passw0rd -a cgpvc
root@lkutest / # pvcauth -l
Address  : 10.2.55.232
User name: pvcadmin
Project  : ibm-default
Port     : 5000
TTL      : 5:59:58

d) Install SP now. Install bos.rte.install first.

root@lkutest / # mount sydnim:/nim/cg/aix/aix72tl2sp2 /mnt
root@lkutest / # cd /mnt

root@lkutest /mnt # installp -d . bos.rte.install
installp: No action was indicated.
        The -a (apply) flag is being assumed.
+-----------------------------------------------------------------------------+
                    Pre-installation Verification...
+-----------------------------------------------------------------------------+
Verifying selections...done
Verifying requisites...done
Results...

WARNINGS
--------
  Problems described in this section are not likely to be the source of any
  immediate or serious failures, but further actions may be necessary or
  desired.

  Already Installed
  -----------------
  The number of selected filesets that are either already installed
  or effectively installed through superseding filesets is 1.  See
  the summaries at the end of this installation for details.

  NOTE:  Base level filesets may be reinstalled using the "Force"
  option (-F flag), or they may be removed, using the deinstall or
  "Remove Software Products" facility (-u flag), and then reinstalled.

  << End of Warning Section >>

SUCCESSES
---------
  Filesets listed in this section passed pre-installation verification
  and will be installed.

  Mandatory Fileset Updates
  -------------------------
  (being installed automatically due to their importance)
  bos.rte.install 7.2.2.16                    # LPP Install Commands

  << End of Success Section >>

+-----------------------------------------------------------------------------+
                   BUILDDATE Verification ...
+-----------------------------------------------------------------------------+
Verifying build dates...done
FILESET STATISTICS
------------------
    2  Selected to be installed, of which:
        1  Passed pre-installation verification
        1  Already installed (directly or via superseding filesets)
  ----
    1  Total to be installed

+-----------------------------------------------------------------------------+
                         Installing Software...
+-----------------------------------------------------------------------------+

installp: APPLYING software for:
        bos.rte.install 7.2.2.16


. . . . . << Copyright notice for bos >> . . . . . . .
 Licensed Materials - Property of IBM

 5765CD200
   Copyright International Business Machines Corp. 1985, 2018.
   Copyright AT&T 1984, 1985, 1986, 1987, 1988, 1989.
   Copyright Regents of the University of California 1980, 1982, 1983, 1985, 1986, 1987, 1988, 1989.
   Copyright BULL 1993, 2018.
   Copyright Digi International Inc. 1988-1993.
   Copyright Interactive Systems Corporation 1985, 1991.
   Copyright ISQUARE, Inc. 1990.
   Copyright Innovative Security Systems, Inc. 2001-2006.
   Copyright Mentat Inc. 1990, 1991.
   Copyright Open Software Foundation, Inc. 1989, 1994.
   Copyright Sun Microsystems, Inc. 1984, 1985, 1986, 1987, 1988, 1991.

 All rights reserved.
 US Government Users Restricted Rights - Use, duplication or disclosure
 restricted by GSA ADP Schedule Contract with IBM Corp.
. . . . . << End of copyright notice for bos >>. . . .

Successfully updated the Kernel Authorization Table.
Successfully updated the Kernel Role Table.
Successfully updated the Kernel Command Table.
Successfully updated the Kernel Device Table.
Successfully updated the Kernel Object Domain Table.
Successfully updated the Kernel  Domains Table.
Successfully updated the Kernel RBAC log level.
Successfully updated the Kernel Authorization Table.
Successfully updated the Kernel Role Table.
Successfully updated the Kernel Command Table.
Successfully updated the Kernel Device Table.
Successfully updated the Kernel Object Domain Table.
Successfully updated the Kernel  Domains Table.
Successfully updated the Kernel RBAC log level.
Successfully updated the Kernel Authorization Table.
Successfully updated the Kernel Role Table.
Successfully updated the Kernel Command Table.
Successfully updated the Kernel Device Table.
Successfully updated the Kernel Object Domain Table.
Successfully updated the Kernel  Domains Table.
Successfully updated the Kernel RBAC log level.
Successfully updated the Kernel Authorization Table.
Successfully updated the Kernel Role Table.
Successfully updated the Kernel Command Table.
Successfully updated the Kernel Device Table.
Successfully updated the Kernel Object Domain Table.
Successfully updated the Kernel  Domains Table.
Successfully updated the Kernel RBAC log level.
Finished processing all filesets.  (Total time:  3 secs).

+-----------------------------------------------------------------------------+
                                Summaries:
+-----------------------------------------------------------------------------+

Pre-installation Failure/Warning Summary
----------------------------------------
Name                      Level           Pre-installation Failure/Warning
-------------------------------------------------------------------------------
bos.rte.install           7.2.2.1         Already installed


Installation Summary
--------------------
Name                        Level           Part        Event       Result
-------------------------------------------------------------------------------
bos.rte.install             7.2.2.16        USR         APPLY       SUCCESS
bos.rte.install             7.2.2.16        ROOT        APPLY       SUCCESS
root@lkutest /mnt #

e) Install SP2 now using update_all. Confirm that SP2 has been installed. Do NOT reboot.

root@lkutest /mnt # update_all -Y -c -v -d .
........
bos.net.tcp.snmp            7.2.2.15        ROOT        COMMIT      SUCCESS
bos.net.tcp.tcpdump         7.2.2.15        USR         COMMIT      SUCCESS
bos.net.tcp.tcpdump         7.2.2.15        ROOT        COMMIT      SUCCESS
bos.perf.pmaix              7.2.2.15        USR         COMMIT      SUCCESS
bos.perf.pmaix              7.2.2.15        ROOT        COMMIT      SUCCESS
bos.perf.tune               7.2.2.15        USR         COMMIT      SUCCESS
bos.perf.tune               7.2.2.15        ROOT        COMMIT      SUCCESS
bos.sysmgt.nim.client       7.2.2.15        USR         COMMIT      SUCCESS
bos.sysmgt.nim.client       7.2.2.15        ROOT        COMMIT      SUCCESS
mcr.rte                     7.2.2.16        USR         COMMIT      SUCCESS
mcr.rte                     7.2.2.16        ROOT        COMMIT      SUCCESS
rsct.core.utils             3.2.3.1         USR         COMMIT      SUCCESS
rsct.core.utils             3.2.3.1         ROOT        COMMIT      SUCCESS
bos.esagent                 7.2.2.15        USR         COMMIT      SUCCESS
bos.esagent                 7.2.2.15        ROOT        COMMIT      SUCCESS
bos.net.tcp.snmpd           7.2.2.15        USR         COMMIT      SUCCESS
bos.net.tcp.snmpd           7.2.2.15        ROOT        COMMIT      SUCCESS
bos.suma                    7.2.2.15        USR         COMMIT      SUCCESS
bos.suma                    7.2.2.15        ROOT        COMMIT      SUCCESS
rsct.core.rmc               3.2.3.1         USR         COMMIT      SUCCESS
rsct.core.rmc               3.2.3.1         ROOT        COMMIT      SUCCESS
rsct.opt.storagerm          3.2.3.1         USR         COMMIT      SUCCESS
rsct.opt.storagerm          3.2.3.1         ROOT        COMMIT      SUCCESS

installp:  * * *  A T T E N T I O N ! ! !
        Software changes processed during this session require this system
        and any of its diskless/dataless clients to be rebooted in order
        for the changes to be made effective.

    * * *  A T T E N T I O N  * * *
    System boot image has been updated. You must reboot the system
    after install_all_updates completes to properly integrate the updates.

install_all_updates: Checking for recommended maintenance level 7200-02.
install_all_updates: Executing /usr/bin/oslevel -rf, Result = 7200-02
install_all_updates: Verification completed.
install_all_updates: Log file is /var/adm/ras/install_all_updates.log
install_all_updates: Result = SUCCESS
root@lkutest /mnt #

root@lkutest /mnt # oslevel -s
7200-02-02-1810

f) Install the desired ifix now. Do NOT reboot.

root@lkutest /tmp/cg # emgr -e IJ04268s2a.180403.AIX72TL02SP02.epkg.Z
+-------------------------------------------------------------------------+
Efix Manager Initialization
+-------------------------------------------------------------------------+
Initializing log /var/adm/ras/emgr.log ...
Efix package file is: /tmp/cg/IJ04268s2a.180403.AIX72TL02SP02.epkg.Z
MD5 generating command is /usr/bin/csum
MD5 checksum is edb56f88671a3f88f79a40fdadd10d71
Accessing efix metadata ...
Processing efix label "IJ04268s2a" ...
Verifying efix control file ...

+-------------------------------------------------------------------------+
Installp Prerequisite Verification
+-------------------------------------------------------------------------+
Verifying prerequisite file ...
Checking prerequisites ...

Prerequisite Number: 1
   Fileset: bos.cluster.rte
   Minimal Level: 7.2.2.15
   Maximum Level: 7.2.2.15
   Actual Level: 7.2.2.15
   Type: PREREQ
   Requisite Met: yes

All prerequisites have been met.

+-------------------------------------------------------------------------+
Processing APAR reference file
+-------------------------------------------------------------------------+
ATTENTION: Interim fix is enabled for automatic removal by installp.

+-------------------------------------------------------------------------+
Efix Attributes
+-------------------------------------------------------------------------+
LABEL:            IJ04268s2a
PACKAGING DATE:   Tue Apr  3 16:14:26 CDT 2018
ABSTRACT:         CAA clcomd hang fix
PACKAGER VERSION: 7
VUID:             00F850C44C00040316042518
REBOOT REQUIRED:  no
BUILD BOOT IMAGE: no
LU CAPABLE:       yes
PRE-REQUISITES:   yes
SUPERSEDE:        no
PACKAGE LOCKS:    no
E2E PREREQS:      no
FIX TESTED:       no
ALTERNATE PATH:   None
EFIX FILES:       1

Install Scripts:
   PRE_INSTALL:   no
   POST_INSTALL:  yes
   PRE_REMOVE:    no
   POST_REMOVE:   yes

File Number:      1
   LOCATION:      /usr/sbin/clcomd
   FILE TYPE:     Standard (file or executable)
   INSTALLER:     installp
   SIZE:          2144
   ACL:           DEFAULT
   CKSUM:         38581
   PACKAGE:       bos.cluster.rte
   MOUNT INST:    no

+-------------------------------------------------------------------------+
Efix Description
+-------------------------------------------------------------------------+
IJ04268 - hang in clcomd operations

+-------------------------------------------------------------------------+
Efix Lock Management
+-------------------------------------------------------------------------+
Checking locks for file /usr/sbin/clcomd ...

All files have passed lock checks.

+-------------------------------------------------------------------------+
Space Requirements
+-------------------------------------------------------------------------+
Checking space requirements ...

Space statistics (in 512 byte-blocks):
File system: /usr, Free: 2459008, Required: 3696, Deficit: 0.
File system: /tmp, Free: 2318088, Required: 4978, Deficit: 0.

+-------------------------------------------------------------------------+
Efix Installation Setup
+-------------------------------------------------------------------------+
Unpacking efix package file ...
Initializing efix installation ...

+-------------------------------------------------------------------------+
Efix State
+-------------------------------------------------------------------------+
Setting efix state to: INSTALLING

+-------------------------------------------------------------------------+
File Archiving
+-------------------------------------------------------------------------+
Saving all files that will be replaced ...
Save directory is: /usr/emgrdata/efixdata/IJ04268s2a/save
File 1: Saving /usr/sbin/clcomd as EFSAVE1 ...

+-------------------------------------------------------------------------+
Efix File Installation
+-------------------------------------------------------------------------+
Installing all efix files:
Installing efix file #1 (File: /usr/sbin/clcomd) ...

Total number of efix files installed is 1.
All efix files installed successfully.

+-------------------------------------------------------------------------+
Package Locking
+-------------------------------------------------------------------------+
Processing package locking for all files.
File 1: locking installp fileset bos.cluster.rte.

All package locks processed successfully.

+-------------------------------------------------------------------------+
Post-Install Script
+-------------------------------------------------------------------------+
Executing post-install script ...
0513-044 The clcomd Subsystem was requested to stop.
Wait for clcomd to stop.
0513-059 The clcomd Subsystem has been started. Subsystem PID is 21299512.
Return code from post-install script is: 0

+-------------------------------------------------------------------------+
Reboot Processing
+-------------------------------------------------------------------------+
Reboot is not required by this efix package.

+-------------------------------------------------------------------------+
Efix State
+-------------------------------------------------------------------------+
Setting efix state to: STABLE

+-------------------------------------------------------------------------+
Operation Summary
+-------------------------------------------------------------------------+
Log file is /var/adm/ras/emgr.log

EPKG NUMBER       LABEL               OPERATION              RESULT
===========       ==============      =================      ==============
1                 IJ04268s2a          INSTALL                SUCCESS

Return Status = SUCCESS
root@lkutest /tmp/cg #


root@lkutest / # emgr -l

ID  STATE LABEL      INSTALL TIME      UPDATED BY ABSTRACT
=== ===== ========== ================= ========== ======================================
1    S    IJ04268s2a 05/16/18 11:20:57            CAA clcomd hang fix

g) Perform Live Update now.

root@lkutest /tmp/cg # geninstall -k
Validating live update input data.

Computing the estimated time for the liveupdate operation:
-------------------------------------------------------
LPAR: lkutest
Blackout time (in seconds): 12
Total operation time (in seconds): 1344

Checking mirror vg device size:
------------------------------------------
Required device size: 20448 MB
PASSED: The disks specified for the mirrored rootvg resulted in a valid volume group factor.

Checking new root vg device size:
------------------------------------------
Required device size: 20448 MB

Checking temporary storage size for the original LPAR:
------------------------------------------
Required device size: 0 or Undetermined MB

Checking temporary storage size for the surrogate LPAR:
------------------------------------------
Required device size: 0 or Undetermined MB
Checking lpar minimal memory size:
------------------------------------------
Required memory size: 2048 MB

Checking other requirements:
------------------------------------------
PASSED: sufficient space available in /var.
PASSED: sufficient space available in /.
PASSED: no existing altinst_rvgLvup.
PASSED: rootvg is not part of a snapshot.
PASSED: pkcs11 is not installed.
PASSED: DoD/DoDv2 profile is not applied.
PASSED: Advanced Accounting is not on.
PASSED: Virtual Trusted Platform Module is not on.
PASSED: multiple semid lists is not on.
PASSED: sufficient file system space for interim fix(es) is available.
PASSED: The trustchk Trusted Execution Policy is not on.
PASSED: The trustchk Trusted Library Policy is not on.
PASSED: The trustchk TSD_FILES_LOCK policy is not on.
PASSED: the boot disk is set to the current rootvg.
PASSED: the mirrorvg name is available.
PASSED: the rootvg is uniformly mirrored.
PASSED: the rootvg does not have the maximum number of mirror copies.
PASSED: the rootvg does not have stale logical volumes.
PASSED: all of the mounted file systems are of a supported type.
PASSED: this AIX instance is not diskless.
PASSED: no Kerberos configured for NFS mounts.
PASSED: multibos environment not present.
PASSED: Trusted Computing Base not defined.
PASSED: no local tape devices found.
PASSED: live update not executed from console.
PASSED: the execution environment is valid.
PASSED: enough available space for /var to dump Component Trace buffers.
PASSED: enough available space for /var to dump Light weight memory Trace buffers.
PASSED: all devices are virtual devices.
PASSED: No active workload partition found.
PASSED: nfs configuration supported.
PASSED: RSCT services are active.
PASSED: no Kerberos configuration.
PASSED: no virtual log device configured.
PASSED: PowerVC token is valid.
PASSED: PowerVC is at a supported level.
PASSED: User has PowerVC permissions for Live Update processing.
PASSED: Host is not in maintenance mode.
PASSED: PowerVC token expiration date is valid.
PASSED: PowerVC network devices match those present on partition.
PASSED: PowerVC volumes match hdisks present on partition.
PASSED: All rootvg volumes are boot volumes in PowerVC.
PASSED: Capacity check on SRIOV ports of target host.
PASSED: Sufficient processing units available on target host.
PASSED: Sufficient memory available on target host.
PASSED: PowerVC virtual machine health status is 'OK'.
PASSED: the disk configuration is supported.
PASSED: no Generic Routing Encapsulation (GRE) tunnel configured.
PASSED: Firmware level is supported.
PASSED: Consolidated system trace buffers size is within the limit of 64 MB.
PASSED: SMT number is valid.
PASSED: No process attached to vty0.
PASSED: No active ipsec configured.
PASSED: Audit is not enabled in stream mode.
PASSED: No exclusive rsets (sysxrset) found.
INFO: Any system dumps present in the current dump logical volumes will not be available after live update is complete.

Non-interruptable live update operation begins in 10 seconds.


Broadcast message from root@lkutest (pts/0) at 11:21:48 ...

Live AIX update in progress.


Initializing live update on original LPAR.

Validating original LPAR environment.

Beginning live update operation on original LPAR.

Requesting resources required for live update.
........
Notifying applications of impending live update.
....
Creating rootvg for boot of surrogate.
........................................
Starting the surrogate LPAR.
............................................................................................................
Creating mirror of original LPAR's rootvg.
.............................................................................................................
.............................................................................................................
..........................................
Moving workload to surrogate LPAR.
................
        Blackout Time started.

        Blackout Time end.

Workload is running on surrogate LPAR.
............................................................................................
Shutting down the Original LPAR.
........................................The live update operation succeeded.


Broadcast message from root@lkutest (pts/0) at 11:47:00 ...

Live AIX update completed.

h) Verify kernel build version is updated and now active.

root@lkutest / # cat /proc/version
Mar  2 2018
13:02:46
1809C_72H
@(#) _kdb_buildinfo unix_64 Mar  2 2018 13:02:46 1809C_72H

3. If you plan to use the Live Update function on a partition that is managed by PowerVC, you must set the network_allocate_retries property to a minimum value of 10 on the PowerVC. This property must be specified in the /etc/nova/nova.conf file on the PowerVC.

This prevents the following messages from appearing after a Live Update operation:

“Failed to delete a virtual Ethernet adapter on the virtual machine. Check that the RMC is active for the virtual machine. Refer to the logs for more information”

and/or

“Cannot delete a virtual Ethernet adapter on the virtual machine because the HMC operation for delete failed with an HTTP error. Refer to the logs for more information and ensure that the virtual Ethernet adapter is manually removed after you attempt to delete it from the UI”.

PowerVC logs will show error similar to this:

“2018-05-15 11:17:44.998 4643 ERROR powervc_nova.virt.ibmpowervm.hmc.driver [req-00a02a9c-61de-4d19-b809-d5e46ed08b99 80c16094b1b02ff4c70984790330525b64197707fe536f86cbea82d730304161 c3e51c6d67654131bcfaa77b8c4491ba - bf689afa9471460c876fadb2ae838f17 bf689afa9471460c876fadb2ae838f17] Failed to delete a virtual Ethernet adapter on the virtual machine lkutest-548002b8-00000022. Check that the RMC is active for the virtual machine. Refer to the logs for more information.: K2HttpError: HTTP error for GET /rest/api/uom/LogicalPartition/1A088741-D22D-4444-A446-76E58D092761/ClientNetworkAdapter/a5293f04-b636-39db-8e2f-8110921f386f: 404 (Not Found)”

Add this property as a new line as shown in the following example:

network_allocate_retries = 10

You must restart PowerVC for the change to take effect. For example:

# head -2 /etc/nova/nova.conf
[DEFAULT]
network_allocate_retries = 10
….

4. Use the alog command to check the actual blackout period during the Blackout Time, whilst the workload is moved from the Original to the Surrogate partition. For example:

root@lkutest / # alog -t mobte –o
time=051418:11:05:02 pid=18577631930744833 
type=Global dep_mfreq=4572472429Hz dep_sfreq=4572472429Hz 
dep_cont=100 bw=60000000B/s dep_iorbw=571837866B/s dep_iowbw=366226530B/s dep_pipebw=273906256B/s dep_rvgsz=20448MB dep_lulvsz=4542MB 
dep_cpuavail=10000 dep_flags=0x0 arr_mfreq=4572472429Hz arr_sfreq=4572472429Hz arr_cont=100 arr_iorbw=571837866B/s 
arr_iowbw=366226530B/s arr_pipebw=273906256B/s arr_rvgsz=20448MB arr_lulvsz=4542MB arr_cpuavail=10000 
arr_flags=0x0 files=87 procs=44 threads=140 datasz=931088B socks=40 mempg=15077 nmqs=3 nmsgs=0 
nqbs=0 nsems=7 shmsz=0 mmapsz=30677 flags=0x0 stdl_blackout=12.000000s stdl_global=1218.000000s blackout=11.417990s global=1642.088501s

5. If you encounter the message below, when performing a Live Update with PowerVC, ensure that the boot disks are set correctly in PowerVC and try again. You can change the “Boot Set” to Yes for a VM by selecting it in PowerVC and editing its details.
VM-(1).jpg

1430-207 FAILED: Not all rootvg volumes are boot volumes in PowerVC.

6. Please read the Live Update restrictions and limitations resources BEFORE you attempt to use the tool. There are currently many restrictions and limitations that exist with Live Update. Make sure you’re aware of the current constraints and evaluate if they will impact your environment or not.

Live Update Restrictions

LPAR Requirements for Live Update

7. Try Live Updates without applying updates! AIX Live Update is designed to be transparent to the running applications. With AIX 7.2.1, there is an easy way to try it out with your own workload, without having to apply an update.

Simply try geninstall –k –p (preview) to make sure that everything is okay with your configuration, and then use geninstall –k to perform the Live Update operation without applying an update.

8. Ensure that the following filesets are installed on your AIX system. Fileset requisites for bos.liveupdate.rte are:

There is a known issue where the bos.liveupdate.rte update fileset is missing the check for several requisites that are necessary for the live update operation.

IV99459: FIX REQUISITES FOR BOS.LIVEUPDATE.RTE UPDATE FILESET

I've encountered the following error when the rsct.basic fileset is not installed. You will observe the following in the /var/ct/lvupd/ctlvupd.log file:

Wed Nov 15 09:10:06 PST 2017 (12452188) : phase_check_on_original: CHECK phase on ORIGINAL exiting with return code, 5
Wed Nov 15 09:11:12 PST 2017 (11469192) : rsct.basic fileset is not installed.

9. The /usr/sbin/clvupdate utility is an undocumented command that can be used when troubleshooting or resolving issues with AIX Live Update. Please seek advice and guidance from IBM support before using this tool in a Live Update environment. It is included with the bos.liveupdate.rte fileset.

# lslpp -w /usr/sbin/clvupdate
  File                                        Fileset               Type
  ----------------------------------------------------------------------------
  /usr/sbin/clvupdate                         bos.liveupdate.rte    File


Usage:
    Attempt to clean up partition after a failed Live Update operation:
            clvupdate [-e] [-o] [-u]
            clvupdate [-e] [-n] [-u]
            clvupdate -d
            clvupdate -l
            clvupdate -r
            clvupdate -v

Flags:
    -d Remove surrogate boot disks only
    -e Ignore Live Update state
    -l Unlock Live Update lock only
    -n Run LVUP_ERROR phase scripts
    -o Force original shutdown
    -r Reset Live Update state and status only
    -u Ignore Live Update status
    -v Remove volume group from previous Live Update

10. Please refer to the /var/adm/ras/liveupdate/logs/lvupdlog log file for debugging information when troubleshooting a Live Update issue. This log file can provide very useful information if you are trying to determine why Live Update is not behaving as you would expect. The IBM support team will also use this log file to assist them with problem determination. They may ask you to collect additional information. Please refer to the Live Kernel Update Must Gather Technote for the additional steps.

Live Kernel Update Must Gather 

11. In a PowerVC managed environment, please use the clvupdate –v command to remove the lvup_rootvg volume, post a Live Update operation and a reboot.

Removal of original LPAR and cleanup of resources after Live Update. If you wish to remove the Live Update surrogate boot disk (lvup_rootvg, after a reboot), you can use the clvupdate command. This command will contact the PowerVC management server (requires a pvcauth token) and will automatically remove the volume group and disk from the LPAR, as shown below.

; reboot LPAR
root@orion / # uptime  
04:42AM   up 1 mins,  1 user,  load average: 1.08, 0.40, 0.15

root@orion / # lspv
hdisk0          00f94f587d8b0203                    rootvg          active
hdisk1          00f94f584f60b1c1                    lvup_rootvg

root@orion / # pvcauth -u pvcadmin -p maxpeck123 -a cgpvc
root@orion / # pvcauth –l
Address  : 10.1.50.232
User name: pvcadmin
Project  : ibm-default
Port     : 5000
TTL      : 5:59:53

root@orion / # clvupdate –v
The temporary Live Update storage was successfully removed. 
Cleanup action succeeded.

root@orion / # lspv
hdisk0          00f94f587d8b0203                    rootvg          active

A message will appear in the PowerVC user interface indicating that the volume has been detached from the VM and deleted.


messeges-(1).png

Chris Gibson is an AIX and PowerVM specialist located in Melbourne, Australia. He is an IBM Champion for Power Systems, IBM CATE (Power Systems and AIX), and a co-author of several IBM Redbooks publications.



Like what you just read? To receive technical tips and articles directly in your inbox twice per month, sign up for the EXTRA e-newsletter here.


comments powered by Disqus

Advertisement

Advertisement

2018 Solutions Edition

A Comprehensive Online Buyer's Guide to Solutions, Services and Education.

AIX > ADMINISTRATOR > SYSTEMS MANAGEMENT

How to Download Fixes

ADMINSTRATOR > SYSTEMS MANAGEMENT

Understand your options for 12X PCIe I/O drawers

clmgr: A Technical Reference

PowerHA SystemMirror 7.1 introduces a robust CLI utility

IBM Systems Magazine Subscribe Box Read Now Link Subscribe Now Link iPad App Google Play Store
IBMi News Sign Up Today! Past News Letters