Veritas Volume Manager (VxVM) overview and commands
Veritas Volume Manager (VxVM) sits between the disks and the file systems. It groups disks, builds volumes that can be mirrored or striped, and lets you resize, move and repair storage while applications keep running.
Objects
- VM disk
- A physical disk or LUN under VxVM control, with a private region holding VxVM's configuration.
- Disk group
- A set of disks that belong together and move together between servers. All volumes live in a disk group.
- Subdisk
- A contiguous area on one VM disk.
- Plex
- One copy of the data, made of subdisks (concatenated or striped). A mirrored volume has two or more plexes.
- Volume
- What file systems and databases use:
/dev/vx/dsk/diskgroup/volume(block) and/dev/vx/rdsk/…(raw).
Look around
vxdisk -o alldgs list # every disk, with its disk group (imported or not)
vxdg list # imported disk groups
vxprint -ht -g datadg # full hierarchy: volumes, plexes, subdisks
vxprint -g datadg -v # just the volumes
vxdg -g datadg free # free space per disk
vxassist -g datadg maxsize layout=mirror # largest mirrored volume you could create
Add disks and create a disk group
vxdctl enable # scan for new LUNs
vxdisksetup -i disk_5 # initialise a disk for VxVM
vxdg init datadg datadg01=disk_5
vxdisksetup -i disk_6
vxdg -g datadg adddisk datadg02=disk_6
vxdiskadm offers the same operations through a menu, which is handy when you do them rarely.
Create volumes
vxassist -g datadg make appvol 100g # concatenated
vxassist -g datadg make appvol 100g layout=mirror # two-way mirror
vxassist -g datadg make appvol 100g layout=stripe ncol=4 # striped over 4 disks
vxassist -g datadg make appvol 100g layout=mirror-stripe ncol=2
Then create a file system on it, for example VxFS: mkfs -t vxfs /dev/vx/rdsk/datadg/appvol (see File System).
Resize
Grow the volume and the VxFS file system on it in one step, while it is mounted:
vxresize -g datadg appvol +50g
vxresize -g datadg appvol 120g # to an absolute size (shrinking needs free space in the file system)
Mirrors
vxassist -g datadg mirror appvol # add a mirror to an existing volume
vxassist -g datadg remove mirror appvol # remove one
vxtask list # progress of resynchronisation
Replace a failed disk
- Find the failed disk:
vxdisk listshows it asfailedorerror, andvxprint -htshows plexes asNODEVICEorDISABLED. - Run
vxdiskadmand choose Remove a disk for replacement, then physically replace the disk (or present a new LUN). - In
vxdiskadm, choose Replace a failed or removed disk and select the new device. - VxVM resynchronises the mirrors; follow it with
vxtask list. Runvxrecover -g datadg -bif volumes stay disabled.
With hot relocation enabled (the default), VxVM moves data from a failing disk to spare space automatically and sends a message to root.
Move a disk group to another server
# on the old server: stop the application, unmount, then
vxdg deport datadg
# on the new server:
vxdctl enable
vxdg import datadg
vxvol -g datadg startall
mount -t vxfs /dev/vx/dsk/datadg/appvol /app
In a VCS cluster, the DiskGroup resource does the deport and import for you during failover.
Multipathing with DMP
vxdmpadm listenclosure all # storage arrays
vxdmpadm getsubpaths dmpnodename=disk_5
vxdmpadm getattr enclosure emc0 iopolicy
vxdmpadm iostat show all
DMP spreads I/O over the working paths according to the I/O policy (all active paths on active/active arrays) and fails over automatically when a path breaks.
Root disk
On older Solaris and AIX systems, VxVM was often used to "encapsulate" and mirror the boot disk. On current systems, mirror the operating system with the platform's own tools (ZFS on Solaris, LVM on AIX and Linux) and use VxVM for application data.