Oracle ASM Failure Groups: Mirroring, Rebalance Power, and Verification

The Problem: Unprotected ASM Disk Groups and Silent Rebuilds

\n

Oracle Automatic Storage Management (ASM) stores database files across disks in disk groups. If a disk group uses external redundancy and a disk fails, ASM has no mirrored copy to fall back on. Even with normal or high redundancy, an uncontrolled rebalance can saturate I/O and slow production workloads while ASM rebuilds redundancy. The useful takeaway is to define failure groups before you need them, choose a rebalance power that matches your maintenance window, and verify the operation with dynamic performance views rather than assuming the disk group healed.

\n\n

Desired Outcome

\n

Create or modify an ASM disk group with explicit failure groups so that ASM mirrors extents across separate failure sets. After a disk is dropped or fails, confirm that ASM rebalances to restore redundancy and that you can monitor progress and throttle the rebuild if necessary.

\n\n

Prerequisites

\n

\n
• Oracle Grid Infrastructure and an ASM instance running on the node
\n
• SQL*Plus access as SYSASM, or OS membership in the OSASM/OSDBA group
\n
• Raw disks or LUNs visible to ASM with consistent size characteristics within a failure group
\n
• Oracle Database 12c Release 2 or later; examples assume Oracle 19c ASM behavior
\n

\n\n

Step 1: Inspect Existing Disk Groups and Failure Groups

\n

Connect to SQL*Plus on a node running the ASM instance as a user with SYSASM privileges:

\n

sqlplus / as sysasm

\n

Run:

\n

SELECT name, type, sector_size, block_size, allocation_unit_size, state\nFROM   v$asm_diskgroup;\n\nSELECT dg.name AS disk_group, fd.failgroup, fd.name AS disk_name, fd.path, fd.header_status, fd.mode_status\nFROM   v$asm_diskgroup dg\nJOIN   v$asm_disk fd ON fd.group_number = dg.group_number\nORDER BY dg.name, fd.failgroup;

\n

Expected check: Look for type values of EXTERNAL, NORMAL, or HIGH. External redundancy means no mirroring; normal means two-way mirroring; high means three-way. Each disk should show MEMBER in header_status and ONLINE in mode_status.

\n

Risk: Do not drop disks from a production disk group until you confirm that redundancy can survive the loss.

\n\n

Step 2: Create a Disk Group with Failure Groups

\n

A failure group is a set of disks that share a common failure point, such as the same disk controller or storage enclosure. ASM mirrors extents to disks in different failure groups.

\n

Example as SYSASM:

\n

CREATE DISKGROUP DATA NORMAL REDUNDANCY\n  FAILGROUP controller1 DISK '/dev/oracleasm/disks/disk01' NAME disk01,\n                  DISK '/dev/oracleasm/disks/disk02' NAME disk02\n  FAILGROUP controller2 DISK '/dev/oracleasm/disks/disk03' NAME disk03,\n                  DISK '/dev/oracleasm/disks/disk04' NAME disk04\n  ATTRIBUTE 'compatible.asm' = '19.0.0.0',\n            'compatible.rdbms' = '19.0.0.0',\n            'au_size' = '4M';

\n

Where to run: SQL*Plus connected to the ASM instance as SYSASM.

\n

Permissions: SYSASM or a user in the OSASM group.

\n

Risk: Creating a disk group initializes the disks. Ensure the paths are unused and correct; mistyping a path can affect data on an unintended device.

\n

Expected check: Query V$ASM_DISKGROUP and confirm type = NORMAL and state = MOUNTED.

\n\n

Step 3: Set Rebalance Power and Understand the Trade-Off

\n

When a disk is dropped or fails, ASM rebalances extents to restore redundancy. The REBALANCE POWER value controls how aggressively ASM rebuilds. Higher power finishes faster but consumes more I/O and CPU; lower power throttles the rebuild to protect production workloads.

\n

Change power on an active rebalance:

\n

ALTER DISKGROUP DATA REBALANCE POWER 4;

\n

Where to run: SQL*Plus connected to ASM as SYSASM.

\n

Valid values: Typically 0 to 11, where 0 pauses automatic rebalance and higher numbers increase parallelism. Default depends on ASM version and disk group attributes.

\n

Risk: Setting power too high during peak hours can increase latency for database sessions reading from the disk group.

\n\n

Step 4: Drop a Disk and Monitor Rebalance Progress

\n

To test or remove a disk:

\n

ALTER DISKGROUP DATA DROP DISK disk02;

\n

Monitor the operation:

\n

SELECT operation, state, power, actual_power, sofar, totalwork,\n       ROUND(sofar/totalwork*100, 2) AS pct_complete,\n       start_time, est_minutes\nFROM   v$asm_operation;

\n

Expected check: state should move from WAITING or RUNNING to COMPLETED. pct_complete should increase over time. If est_minutes is large and the system is busy, reduce power with the ALTER DISKGROUP command above.

\n

Verify redundancy restoration:

\n

SELECT name, type, state\nFROM   v$asm_diskgroup\nWHERE  name = 'DATA';

\n

Also confirm the dropped disk no longer appears as MEMBER in V$ASM_DISK, or appears with header_status = FORMER.

\n\n

Expected Checks and Verification

\n

\n
1. Redundancy level: V$ASM_DISKGROUP.TYPE matches your protection requirement (NORMAL or HIGH for mirroring).
\n
2. Failure group spread: Each disk in V$ASM_DISK belongs to a failure group, and mirrored extents reside in different groups.
\n
3. Rebalance completion: V$ASM_OPERATION.STATE reaches COMPLETED after a disk drop.
\n
4. Disk status: Replaced or new disks show HEADER_STATUS = MEMBER and MODE_STATUS = ONLINE.
\n

\n\n

Recovery Options

\n

If a disk failure leaves the disk group unable to restore redundancy because too many disks are missing:

\n

\n
• Replace the failed hardware and add a new disk to the same failure group:
\n

\n

ALTER DISKGROUP DATA ADD DISK '/dev/oracleasm/disks/disk05' NAME disk05 FAILGROUP controller1;

\n

ASM automatically rebalances to redistribute extents. Monitor with V$ASM_OPERATION.

\n

If rebalance is harming performance, lower power:

\n

ALTER DISKGROUP DATA REBALANCE POWER 1;

\n

To pause entirely (not recommended for long periods because redundancy remains below target):

\n

ALTER DISKGROUP DATA REBALANCE POWER 0;

\n

Rollback note: Dropping a disk is a state-changing operation. There is no direct undo for DROP DISK; recovery requires adding replacement storage and letting ASM rebalance. Do not drop multiple disks simultaneously from a NORMAL redundancy group unless you are certain at least one copy of every extent remains online.

\n\n

Limitations

\n

\n
• Failure groups cannot span disk groups: A failure group belongs to exactly one disk group.
\n
• External redundancy offers no mirroring: Use it only when the underlying storage array provides RAID protection.
\n
• Rebalance power is not a hard I/O cap: It influences parallelism and throttle behavior but does not guarantee a specific throughput limit.
\n
• Disk group compatibility: Some features require matching COMPATIBLE.ASM and COMPATIBLE.RDBMS versions across instances in a cluster.
\n

\n\n

Quick Verification Checklist

\n

✓ V$ASM_DISKGROUP shows TYPE = NORMAL or HIGH and STATE = MOUNTED\n✓ V$ASM_DISK shows each disk assigned to a FAILGROUP\n✓ After DROP DISK, V$ASM_OPERATION shows STATE progressing to COMPLETED\n✓ Replacement disk shows HEADER_STATUS = MEMBER and MODE_STATUS = ONLINE\n✓ REBALANCE POWER adjusted before peak production hours if needed