Veritas Cluster Server overview: how VCS works

Updated November 2026

Veritas Cluster Server keeps applications running when a server fails. It watches every part of an application, from the network card up to the database, and moves the whole application to another server when something breaks, usually within a minute or two.

Failover, not load balancing

In the most common setup, a failover service group runs on one node at a time. The other node is a hot standby: VCS is running there and watching, but the application is offline. When the active node fails, or an administrator moves the group for maintenance, VCS stops the application on one node (if it still can) and starts it on the other. Clients reconnect to the same virtual IP address.

VCS also supports parallel groups, which run on several nodes at once (used with cluster file systems and Oracle RAC), and clusters of up to dozens of nodes.

Building blocks

Node (system)
A server in the cluster.
Resource
One thing VCS manages: a NIC, an IP address, a disk group, a volume, a mount point, a database instance, a listener, a process.
Resource type and agent
Each resource has a type (IP, Mount, Oracle …) handled by an agent that knows how to bring it online, take it offline and monitor it.
Service group
All resources an application needs, with the order they depend on each other. The group is the unit that fails over.
Dependencies
Rules such as "the mount needs the volume, the volume needs the disk group". VCS brings resources online from the bottom up and offline from the top down.

A typical database group, from bottom to top:

          Oracle  ── Listener
            │          │
          Mount       IP
            │          │
          Volume      NIC
            │
        DiskGroup

Communication: LLT and GAB

Split brain and I/O fencing

If all heartbeat links fail while both nodes are still running, each might think the other is dead and start the application, and both would write to the same disks. That is split brain, and it corrupts data. I/O fencing (vxfen) prevents it: coordination points (three small SAN disks with SCSI-3 persistent reservations, or coordination point servers) decide which side of a split cluster survives; the losing nodes are stopped from writing and panic. Production clusters with shared storage should always run with fencing enabled.

What happens at startup

  1. LLT starts and finds the other nodes over the private links.
  2. GAB waits until the number of nodes in /etc/gabtab is present, then seeds the cluster. If a node boots alone (the others are down), it waits; an administrator can seed it manually (see debugging).
  3. Fencing starts and registers with the coordination points.
  4. HAD starts, loads the configuration (from the running cluster if another node is up, otherwise from main.cf) and probes every resource on every node.
  5. Service groups with an AutoStartList come online on the first available node in that list, bringing up resources in dependency order. If a critical resource fails to come online, the group faults on that node and VCS tries the next.

Files and logs

PathContents
/etc/VRTSvcs/conf/config/main.cfThe cluster configuration: nodes, groups, resources, dependencies
/etc/VRTSvcs/conf/config/types.cfDefinitions of the resource types
/etc/llttab, /etc/llthostsLLT links and node IDs
/etc/gabtabHow GAB is started
/etc/vxfenmode, /etc/vxfendgI/O fencing configuration
/var/VRTSvcs/log/engine_A.logThe main log: read this first
/var/VRTSvcs/log/<Type>_A.logOne log per agent: Oracle_A.log, Mount_A.log, IP_A.log …

Use ls -ltr /var/VRTSvcs/log to find the logs that changed most recently.

Changing the configuration

Change the configuration with commands while the cluster is running; VCS distributes the change to every node.

haconf -makerw                                   # open the configuration for writing
hagrp -modify oragrp SystemList node1 0 node2 1
hares -modify ora_mnt MountPoint /u02
haconf -dump -makero                             # save to main.cf on all nodes and close

Only edit main.cf by hand when the whole cluster is stopped:

hastop -all -force                # stop VCS on all nodes, leave applications running
cd /etc/VRTSvcs/conf/config
cp main.cf main.cf.$(date +%F)
vi main.cf
hacf -verify .                    # check syntax before starting again
hastart                           # on this node first, then on the others
Be very careful

A mistake in the configuration can take the application down or stop it from failing over when you need it. Keep a copy of every working main.cf, change one thing at a time, and run a failover test after any change (see testing failover).