Performance tools on HP-UX

I am presenting here some tools that could be useful when facing performance issues on HP-UX. Glance is a very powerful online performance diagnostic tool where you can find almost all you need in order to figure out what could happen in the system, performance related of course. The tool is interactive, like top for example, and you can go trough its features with various shortcuts. Find below some of them:

 a   CPU By Processor
 c   CPU Report
 d   Disk Report
 g   Process List
 i   IO By File System
 k   Diskless  Server  Resource  Utilization  Screen
 l   Network By Interface
 m   Memory Report
 n   NFS By System
 N   Global  NFS  Activity Screen
 t   System Tables Report
 S   Select a NFS  system
 s   Select  a  single  process
 U   IO By HBA Card
 u   IO By Disk
 V   Logical System List
 v   IO By Logical Volume
 w   Swap Space
 F   Open  Files Screen for a process
 M   Memory Regions Screen for a process
 R   Process Resource Summary
 W   Wait  States  Screen  for  a  process _
 b   Scroll page backward
 f   Scroll page forward
 h   Online help
 j   Adjust refresh interval
 o   Adjust  process  threshold
 p   Print toggle
 q   Quit GlancePlus
 r   Refresh the  current  screen
 y   Renice  a  process
 z   Reset statistics  to  zero
 >   Display  next  logical  screen
 <   Display previous screen
 f   Go to the next page
 b    Go to the previous page
 !   Invoke a shell
 ?   Commands Menu

Swapinfo Prints information about device and file system paging space:

gz-hp-ux:/# swapinfo -tam
             Mb      Mb      Mb   PCT  START/      Mb
TYPE      AVAIL    USED    FREE  USED   LIMIT RESERVE  PRI  NAME
dev        8192     158    8019    2%       0       -    1  /dev/vg00/lvol2
dev       90016     152   89861    0%       0       -    1  /dev/vgswap/lvswap
dev        3520     156    3362    4%       0       -    1  /dev/vgswap/lvswap2
dev       46464     157   46299    0%       0       -    1  /dev/vgswap/lvswap3
reserve       -   29827  -29827
memory    75480   42047   33433   56%
total    223672   72497  151147   32%       -       0    -

How to find out which processes are using more memory:

gz-hp-ux:/# top -f top.out -n 10000 -d 1

gz-hp-ux:/# sort -n -k7,7 top.out 

gz-hp-ux:/# sort -n -k7,7 top.out | awk '{if ($7~/M$/) {print $0}}'

Sar command is also very powerful and you can find it in hp-ux. For example, with the options below, you can list a report activity for each lunpath: Report activity for each active lunpath:

gz-hp-ux:/# sar -L 2 2

HP-UX gz-hp-ux B.11.31 U ia64    04/11/12

12:01:30                          lunpath   %busy   avque     r/s     w/s  blks/s  avwait  avserv
                                             %age     num     num     num     num    msec    msec
12:01:32              disk510_lunpath1034    0.50    0.50       4       1    1200    0.01    0.82
                      disk510_lunpath3277    0.50    0.50       3       1     923    0.01    0.99
                      disk511_lunpath3278    1.01    0.50       0       6      31    0.01    2.10
                      disk516_lunpath3283    1.01    0.50       0       3      17    0.01    3.33
                      disk530_lunpath1076    1.01    0.50       2       0      32    0.00    4.21

Average               disk510_lunpath1034    0.40    0.50       4       1    1065    0.01    0.73
Average               disk510_lunpath3277    0.40    0.50       3       1     955    0.01    0.83
Average               disk511_lunpath3278    0.40    0.50       0       2      12    0.01    2.10
Average               disk516_lunpath3283    0.59    0.50       0       2      11    0.01    3.32
Average               disk530_lunpath1076    0.79    0.50       1       0      13    0.00    8.33

Report status of text, process, inode and file tables:

gz-hp-ux:/# sar -v

HP-UX ufp1opb1 B.11.31 U ia64    05/24/12

00:00:01 text-sz  ov  proc-sz  ov  inod-sz  ov  file-sz  ov
00:10:00   N/A   N/A 2270/14000 0  9331/35648 0  54054/2147483647 0
00:20:00   N/A   N/A 2305/14000 0  7026/35648 0  51390/2147483647 0
00:30:01   N/A   N/A 2266/14000 0  7011/35648 0  50658/2147483647 0
00:40:00   N/A   N/A 2352/14000 0  7124/35648 0  54847/2147483647 0
00:50:01   N/A   N/A 2260/14000 0  7030/35648 0  51808/2147483647 0
01:00:01   N/A   N/A 2253/14000 0  7035/35648 0  50806/2147483647 0
01:10:01   N/A   N/A 2368/14000 0  7097/35648 0  52455/2147483647 0
01:20:01   N/A   N/A 2300/14000 0  7025/35648 0  54070/2147483647 0
01:30:04   N/A   N/A 2286/14000 0  6992/35648 0  49722/2147483647 0
01:40:00   N/A   N/A 2224/14000 0  6972/35648 0  50961/2147483647 0
01:50:00   N/A   N/A 2146/14000 0  6923/35648 0  49656/2147483647 0
02:00:00   N/A   N/A 2166/14000 0  6882/35648 0  50018/2147483647 0
02:10:05   N/A   N/A 2169/14000 0  6928/35648 0  46018/2147483647 0
02:20:00   N/A   N/A 2082/14000 0  6848/35648 0  46175/2147483647 0
02:30:01   N/A   N/A 2098/14000 0  6837/35648 0  47282/2147483647 0
02:40:04   N/A   N/A 2071/14000 0  6839/35648 0  45649/2147483647 0
02:50:00   N/A   N/A 2117/14000 0  6838/35648 0  46528/2147483647 0
......

And, of course, you can tune some parameters with kctune command. This is the administrative command for HP-UX kernel tunable parameters. Find below some examples: To see all tunables and their current values:

gz-hp-ux:/# kctune

To see which tunables have new values being held until next boot:

gz-hp-ux:/# kctune -D

To To see the tunables with only non default value

gz-hp-ux:/# kctune -S

To see verbose information about a tunable:

gz-hp-ux:/# kctune -v tunablename

To set a tunable value on the running system:

gz-hp-ux:/# kctune tunable=69

To set a tunable value to be used when the system reboots:

gz-hp-ux:/# kctune -h tunable=69

To increase a tunable's value by 99:

gz-hp-ux:/# kctune tunable+=99