ASMFD : How to Migrate ASM Diskgroups from ASMLIB to ASMFD (ASM Filter Driver) on Oracle Grid Infrastructure (RAC) (Doc ID 2172754.1)

 In this Document

Goal
Solution
References

APPLIES TO:

Oracle Database - Enterprise Edition - Version 12.1.0.2 to 12.1.0.2 [Release 12.1]
Oracle Database Cloud Schema Service - Version N/A and later
Oracle Database Exadata Express Cloud Service - Version N/A and later
Oracle Database Cloud Service - Version N/A and later
Gen 1 Exadata Cloud at Customer (Oracle Exadata Database Cloud Machine) - Version N/A and later
Linux x86-64

GOAL

What is Oracle ASM Filter Driver (Oracle ASMFD) ?

  • Oracle ASM Filter Driver (Oracle ASMFD) is a kernel module that resides in the I/O path of the Oracle ASM disks. Oracle ASM uses the filter driver to validate write I/O requests to Oracle ASM disks.
  • The Oracle ASMFD simplifies the configuration and management of disk devices by eliminating the need to rebind disk devices used with Oracle ASM each time the system is restarted.
  • The Oracle ASM Filter Driver rejects any I/O requests that are invalid. This action eliminates accidental overwrites of Oracle ASM disks that would cause corruption in the disks and files within the disk group. For example, the Oracle ASM Filter Driver filters out all non-Oracle I/Os which could cause accidental overwrites.

Oracle ASM Filter Driver (Oracle ASMFD) is installed with an Oracle Grid Infrastructure installation. If you have an existing Oracle ASM library driver (Oracle ASMLIB) configuration, then this document will guide you to migrate the existing ASM diskgroup disks which are stamped with ASMLib to ASMFD (ASM Filter Driver).

SOLUTION

Assumptions

Following assumptions are made before we proceed with the next steps 

1.  Please check the existing environment is certified or not. Check below document

ASMFD (ASM Filter Driver) Support on OS Platforms (Certification Matrix).(Doc ID 2034681.1)

2.  It is recommended to apply latest PSU for successful migration.

3.  Take the backup of the database before proceeding this activity.

Step by Step Procedure

1.  As a root user run below command to generate the which ASM Disk is associated with the OS disk. In a multipath configuration, please use the multipath name

[root@hostname1 ~]# /etc/init.d/oracleasm listdisks | xargs /etc/init.d/oracleasm querydisk -p

Disk "ACFSD01" is a valid ASM disk
/dev/xvdg1: LABEL="ACFSD01" TYPE="oracleasm"
Disk "ACFSD02" is a valid ASM disk
/dev/xvdg2: LABEL="ACFSD02" TYPE="oracleasm"
Disk "DATA01" is a valid ASM disk
/dev/xvde1: LABEL="DATA01" TYPE="oracleasm"
Disk "DATA02" is a valid ASM disk
/dev/xvdf1: LABEL="DATA02" TYPE="oracleasm"
Disk "OCRVD01" is a valid ASM disk
/dev/xvdc1: LABEL="OCRVD01" TYPE="oracleasm"
Disk "OCRVD02" is a valid ASM disk
/dev/xvdd1: LABEL="OCRVD02" TYPE="oracleasm"

2.  As a Grid Infrastructure (GI) software owner, get the present ASM diskstring value

grid@hostname1 ~]$ asmcmd dsget
parameter:/dev/oracleasm/disks/*
profile:/dev/oracleasm/disks/*

In case the ASMLIB disk discovery string is NOT /dev/oracleasm/disks/* but "ORCL:*".

grid@hostname1 ~]$ asmcmd dsget
parameter:ORCL:*
profile:ORCL:*

3.  As a GI owner update the Oracle ASM disk discovery string to enable Oracle ASMFD to discover devices in the future.

[grid@hostname1 ~]$ asmcmd dsset '/dev/oracleasm/disks/*','AFD:*'

[grid@hostname1 ~]$ asmcmd dsget
parameter:/dev/oracleasm/disks/*, AFD:*
profile:/dev/oracleasm/disks/*,AFD:*

In case the ASMLIB disk discovery string is NOT /dev/oracleasm/disks/* but "ORCL:*".

[grid@hostname1 ~]$ asmcmd dsset 'ORCL:*','AFD:*'

[grid@hostname1 ~]$ asmcmd dsget
parameter:ORCL:*, AFD:*
profile:ORCL:*,AFD:*

4.  As a root user stop the cluster accessing the shared ASMLib disks

[root@hostname1 ~]# $ORACLE_HOME/bin/crsctl stop crs

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'hostname1'
CRS-2673: Attempting to stop 'ora.crsd' on 'hostname1'
....

CRS-2677: Stop of 'ora.gipcd' on 'hostname1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'hostname1' has completed
CRS-4133: Oracle High Availability Services has been stopped. 

Note : Repeat the above steps on all the nodes

5.  Check and Stop the oracleasm driver and acfsload as root user

[root@hostname1 ~]# oracleasm status
Checking if ASM is loaded: yes
Checking if /dev/oracleasm is mounted: yes

[root@hostname1 ~]# oracleasm exit
Unmounting ASMlib driver filesystem: /dev/oracleasm
Unloading module "oracleasm": oracleasm

[root@hostname1 ~]# ls -ltr /dev/oracleasm/
total 0

[root@hostname1 ~]# lsmod | grep acfs
oracleacfs 3498241 2
oracleoks 503994 2 oracleacfs,oracleadvm

[root@hostname1 ~]# $ORACLE_HOME/bin/acfsload stop 

[root@hostname1 ~]# lsmod | grep acfs
[root@hostname1 ~]# 

Note: It is recommended to remove the ASMLib related RPMs to avoid any conflict.

 [root@hostname1 trace]# rpm -qa | grep asm

oracleasm-support-2.1.8-3.el7.x86_64
[root@hostname1 trace]# rpm -e oracleasm-support-2.1.8-3.el7.x86_64
warning: /etc/sysconfig/oracleasm saved as /etc/sysconfig/oracleasm.rpmsave

6.  As root, configure Oracle ASMFD to filter at the node level:

[root@hostname1 ~]# $ORACLE_HOME/bin/asmcmd afd_configure
Connected to an idle instance.
AFD-627: AFD distribution files found.
AFD-636: Installing requested AFD software.
AFD-637: Loading installed AFD drivers.
AFD-9321: Creating udev for AFD.
AFD-9323: Creating module dependencies - this may take some time.
AFD-9154: Loading 'oracleafd.ko' driver.
AFD-649: Verifying AFD devices.
AFD-9156: Detecting control device '/dev/oracleafd/admin'.
AFD-638: AFD installation correctness verified.
Modifying resource dependencies - this may take some time.

7.  As the Oracle Grid Infrastructure / root owner verify the status of Oracle ASMFD:

[root@hostname1 ~]# $ORACLE_HOME/bin/asmcmd afd_state
Connected to an idle instance.
ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'DISABLED' on host 'hostname1'           <<<< Filtering is disabled

8.  As a root user set the AFD diskstring and enable the Filtering

[root@hostname1 ~]# vi /etc/oracleafd.conf
afd_diskstring='/dev/xvd*1'                      <<<<<<<<<<<< set the afd_diskstring

[root@hostname1 ~]# $ORACLE_HOME/bin/asmcmd afd_filter -e
Connected to an idle instance.

[root@hostname1 ~]# $ORACLE_HOME/bin/asmcmd afd_state
Connected to an idle instance.
ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'ENABLED' on host 'hostname1'               <<<< Filtering is enabled

 

9.  As a root user, with the help of the data gathered in step 1, migrate (label) all the ASMLib disks 

$ORACLE_HOME/bin/asmcmd afd_label <OLD ASMLib Disk Label Name> <Disk Path> --migrate

e.g

[root@hostname1 ~]# $ORACLE_HOME/bin/asmcmd afd_label ACFSD01 /dev/xvdg1 --migrate
Connected to an idle instance.
[root@hostname1 ~]# $ORACLE_HOME/bin/asmcmd afd_label ACFSD02 /dev/xvdg2 --migrate
Connected to an idle instance.
[root@hostname1 ~]# $ORACLE_HOME/bin/asmcmd afd_label DATA01 /dev/xvde1 --migrate
Connected to an idle instance.
[root@hostname1 ~]# $ORACLE_HOME/bin/asmcmd afd_label DATA02 /dev/xvdf1 --migrate
Connected to an idle instance.
[root@hostname1 ~]# $ORACLE_HOME/bin/asmcmd afd_label OCRVD01 /dev/xvdc1 --migrate
Connected to an idle instance.
[root@hostname1 ~]# $ORACLE_HOME/bin/asmcmd afd_label OCRVD02 /dev/xvdd1 --migrate
Connected to an idle instance.

10.  Validate that all the disks have been migrated to the ASMFD.

[root@hostname1 ~]# $ORACLE_HOME/bin/asmcmd afd_lsdsk
Connected to an idle instance.
---------------------------------------------------- 
Label             Filtering              Path
==================================================== 
ACFSD01           ENABLED               /dev/xvdg1
ACFSD02           ENABLED               /dev/xvdg2
DATA01            ENABLED               /dev/xvde1
DATA02            ENABLED               /dev/xvdf1
OCRVD01           ENABLED               /dev/xvdc1
OCRVD02           ENABLED               /dev/xvdd1

[root@hostname1 ~]# ls -ltr /dev/oracleafd/disks/
total 24
-rwxrwx---. 1 grid asmadmin 11 Aug 14 05:59 ACFSD01
-rwxrwx---. 1 grid asmadmin 11 Aug 14 05:59 ACFSD02
-rwxrwx---. 1 grid asmadmin 11 Aug 14 05:59 DATA01
-rwxrwx---. 1 grid asmadmin 11 Aug 14 05:59 DATA02
-rwxrwx---. 1 grid asmadmin 11 Aug 14 05:59 OCRVD01
-rwxrwx---. 1 grid asmadmin 11 Aug 14 05:59 OCRVD02

11.  Edit the afd_diskstring with below content. (Ensure that you put the correct afd_diskstring, otherwise ASMFD disks will not get mounted in the mount point i.e in "/dev/oracleafd/disks/" folder)

[root@hostname1 ~]# vi /etc/afd.conf
#afd_diskstring='/dev/oracleasm/disks/*'
afd_diskstring='/dev/xvd*'
afd_filtering=enable 
Tips:- This value can be derived from the path value displayed while running the "asmcmd afd_lsdsk" 

12.  Repeat above steps on all the nodes of the cluster.

13.  Start the CRS on all the nodes  ( before starting the CRS, start the ACFS module)

[root@hostname1 ~]# $ORACLE_HOME/bin/acfsload start

ACFS-9391: Checking for existing ADVM/ACFS installation.
ACFS-9392: Validating ADVM/ACFS installation files for operating system.
ACFS-9393: Verifying ASM Administrator setup.
ACFS-9308: Loading installed ADVM/ACFS drivers.
ACFS-9154: Loading 'oracleoks.ko' driver.
ACFS-9154: Loading 'oracleadvm.ko' driver.
ACFS-9154: Loading 'oracleacfs.ko' driver.
ACFS-9327: Verifying ADVM/ACFS devices.
ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'.
ACFS-9156: Detecting control device '/dev/ofsctl'.
ACFS-9322: completed

[root@hostname1 disks]# lsmod | grep acfs
oracleacfs 3498241 2
oracleoks 503994 2 oracleacfs,oracleadvm

[root@hostname1 ~]# $ORACLE_HOME/bin/crsctl start crs
CRS-4123: Oracle High Availability Services has been started.

[root@hostname2 ~]# $ORACLE_HOME/bin/crsctl start crs
CRS-4123: Oracle High Availability Services has been started.

[root@hostname1 ~]# crsctl check cluster -all
**************************************************************
hostname1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
hostname2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************

14. As the Disks are migrated to ASMFD, remove ASMLIB disk path from disk discovery string. Execute the following command as a Grid Software owner from any one of the node :-

[grid@hostname1 ~]$ asmcmd dsget
parameter:/dev/oracleasm/disks/*, AFD:*
profile:/dev/oracleasm/disks/*,AFD:*
[grid@hostname1 ~]$ asmcmd dsset 'AFD:*'
[grid@hostname1 ~]$ asmcmd dsget
parameter:AFD:*
profile:AFD:*

15.  As a Grid Software owner validate that all the diskgroups are mounted properly

[grid@hostname1 ~]$ asmcmd lsdg
State    Type  Rebal   Sector Block  AU       Total_MB    Free_MB     Req_mir_free_MB Usable_file_MB  Offline_disks  Voting_files   Name
MOUNTED EXTERN N       512    4096  1048576   5119          2969          0               2969                 0       N            ACFSDG/
MOUNTED EXTERN N       512    4096  1048576   10238         8237          0               8237                 0       N            DATA/
MOUNTED EXTERN N       512    4096  1048576   10238         5763          0               5763                 0       Y            OCRVD/
[grid@hostname1 ~]$ asmcmd lsdsk -G DATA
Path
AFD:DATA01
AFD:DATA02
[grid@hostname1 ~]$ asmcmd lsdsk -G OCRVD
Path
AFD:OCRVD01
AFD:OCRVD02
[grid@hostname1 ~]$ asmcmd lsdsk -G ACFSDG
Path
AFD:ACFSD01
AFD:ACFSD02

16.  Lastly validate all the disks are using ASMFD Library :- (Login into ASM instance as sysasm) 

set linesize 300
col PATH for a20
col LIBRARY for a45
col NAME for a15
select inst_id,group_number grp_num,name,state,header_status header,mount_status mount,path, library
from gv$asm_disk order by inst_id,group_number,name;

17.  Check whether the ASMFD disk is accessible even by the root user also.

[root@hostname1 ~]# kfed read /dev/oracleafd/disks/DATA01
KFED-00303: unable to open file '/dev/oracleafd/disks/DATA01'

18. Now the migration is completed and all the disks are using ASM Filter Driver. 

Comments

Post a Comment

Popular posts from this blog

How To Purge Optimizer Statistics Advisor Old Records From 12.2 Onwards (Doc ID 2660128.1)

  In this Document Goal Solution References APPLIES TO: Oracle Database - Enterprise Edition - Version 12.2.0.1 and later Oracle Cloud Infrastructure - Database Service - Version N/A and later Information in this document applies to any platform. GOAL Goal of this document is to provide the method to purge the old records of Optimizer Statistics Advisor Task namely AUTO_STATS_ADVISOR_TASK that consumes huge SYSAUX space. Huge no.of old records retained in WRI$_ADV_OBJECTS for AUTO_STATS_ADVISOR_TASK or INDIVIDUAL_STATS_ADVISOR_TASK incurs heavy SYSAUX space. AUTO_STATS_ADVISOR_TASK is meant for Automatic Statistics Advisor task while INDIVIDUAL_STATS_ADVISOR_TASK is for Manual Statistics Advisor task.  DBA_ADVISOR_PARAMETERS displays all advisor task parameters and their current values in the database. There is a parameter name called EXECUTION_DAYS_TO_EXPIRE. This parameter is set in no.of days. Executions older than the value(no.of days) set would be purged automaticall...
Read more

Oracle session snapper

-- use dbms_session.sleep instead of dbms_lock on oracle 18c or higher -- last column  is sid  -- scipt_name   stat_to_collect   secods_of_snap  how_many_snap   sessions_to_snap -- @snapper all 100 1 lgdr@* -- @snapper all 100 1 sqlid@* -- @snapper all 190 1 all -- @snapper all 190 1 584 -- @snapper all 30 3 584 -- @snapper all 5 1 869 -- @snapper all 30 1 584 -- @snapper ash 5 1 144 -- @snapper ash=sql_id+event+wait_class+blocking_session+p2+p3 5 1 144 -- @snapper ash 5 1 156 -- @snapper ash 190  1 all -- @snapper ash 5 1 all -- @snapper ash=sid+event+wait_class 5 1 all -- @snapper ash=sid+event+wait_class,ash1=plsql_object_id+plsql_subprogram_id+sql_id 5 1 all -- @snapper ash=sid+event+wait_class,ash1=plsql_object_id+plsql_subprogram_id+sql_id,ash2=program+module+action 5 1 all -- @snapper ash=event+wait_class,stats,gather=ts,tinclude=CPU,sinclude=parse 5 1 all -- @snapper ash=sid+event+wait_class,ash1=sid+sqlid+module,stats,gather...
Read more

Oracle Materialized View In-Depth and Materialized View refresh issues in 19c

   We    had  situation demanding  tuning of  MV  refresh and  options  like  TUNE_MVIEW  and Atomic_Refresh was explored . Hence  Documenting materialized view sample commands  and views in this  blog .     After migration if we are  facing slowness  in  Materialized View refresh , we can explore to re-create  Materialized View   Since  main topic for this blog   was MV issues  in 19c ,  posting that topic   first in line below  We also   have Real-Time Materialized Views  to explore .  To Note :  1) We can  create Mv  on ON PREBUILT TABLE  2) We can create Index on Mv  3) We can  gather stats on mv  just like other table .  4) Mv can be REFRESH_MODE ON COMMIT / REFRESH ON DEMAND. 5) Mv  REFRESH_METHOD can be  Complete / FAST or FORCE . A FORCE attempts a FAST...
Read more