Perdeu todos os Voting Disks?
Calma, não há motivo para desespero. Na demonstração abaixo, eu tenho apenas um Voting Disk para o Oracle RAC, em um compartilhamento NFS. Ou seja, uma péssima idéia: lembre-se que se eu perder 50% + 1 dos meus Voting Disks, o Cluster fica indisponível.
Veja que ao apenas remove-lo do diretório, o Cluster não gera erro algum (talvez porque o Voting Disk estava em Lock pelos Nodes, via NFS ou SO, e não foi realmente removido). Só ao reiniciar o Cluster que ele sente a falta do Voting Disk.
Mas não há problema, veja que basta reiniciar um dos Nodes em modo Exclusivo, e adicionar um novo Voting Disk, que pode ser até com o mesmo nome do anterior.
[root@nerv01 ~]# /u01/app/11.2.0/grid/bin/crsctl check cluster -all
**************************************************************
nerv01:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
nerv02:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
[root@nerv01 ~]# /u01/app/11.2.0/grid/bin/crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
ONLINE ONLINE nerv01
ONLINE ONLINE nerv02
ora.asm
OFFLINE OFFLINE nerv01
OFFLINE OFFLINE nerv02
ora.eons
ONLINE ONLINE nerv01
ONLINE ONLINE nerv02
ora.gsd
OFFLINE OFFLINE nerv01
OFFLINE OFFLINE nerv02
ora.net1.network
ONLINE ONLINE nerv01
ONLINE ONLINE nerv02
ora.ons
ONLINE ONLINE nerv01
ONLINE ONLINE nerv02
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE nerv01
ora.nerv01.vip
1 ONLINE ONLINE nerv01
ora.nerv02.vip
1 ONLINE ONLINE nerv02
ora.oc4j
1 OFFLINE OFFLINE
ora.orcl.db
1 ONLINE ONLINE nerv01 Open
2 ONLINE ONLINE nerv02 Open
ora.scan1.vip
1 ONLINE ONLINE nerv01
[root@nerv01 ~]# /u01/app/11.2.0/grid/bin/crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE a4c0f24cf62e4f89bf6b7e3eabf3cf93 (/u01/shared_config/rac01/vdsk) []
Located 1 voting disk(s).
[root@nerv01 ~]# /u01/app/11.2.0/grid/bin/ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 2468
Available space (kbytes) : 259652
ID : 1167354248
Device/File Name : /u01/shared_config/rac01/ocr
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded
[root@nerv01 ~]# mv /u01/shared_config/rac01/vdsk /root
[root@nerv01 ~]# /u01/app/11.2.0/grid/bin/crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE a4c0f24cf62e4f89bf6b7e3eabf3cf93 (/u01/shared_config/rac01/vdsk) []
Located 1 voting disk(s).
[root@nerv01 ~]# ls -l /u01/shared_config/rac01/vdsk
ls: /u01/shared_config/rac01/vdsk: No such file or directory
[root@nerv01 ~]# /u01/app/11.2.0/grid/bin/crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE a4c0f24cf62e4f89bf6b7e3eabf3cf93 (/u01/shared_config/rac01/vdsk) []
Located 1 voting disk(s).
[root@nerv01 ~]# /u01/app/11.2.0/grid/bin/crsctl check cluster -all
**************************************************************
nerv01:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
nerv02:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
[root@nerv01 ~]# /u01/app/11.2.0/grid/bin/crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
ONLINE ONLINE nerv01
ONLINE ONLINE nerv02
ora.asm
OFFLINE OFFLINE nerv01
OFFLINE OFFLINE nerv02
ora.eons
ONLINE ONLINE nerv01
ONLINE ONLINE nerv02
ora.gsd
OFFLINE OFFLINE nerv01
OFFLINE OFFLINE nerv02
ora.net1.network
ONLINE ONLINE nerv01
ONLINE ONLINE nerv02
ora.ons
ONLINE ONLINE nerv01
ONLINE ONLINE nerv02
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE nerv01
ora.nerv01.vip
1 ONLINE ONLINE nerv01
ora.nerv02.vip
1 ONLINE ONLINE nerv02
ora.oc4j
1 OFFLINE OFFLINE
ora.orcl.db
1 ONLINE ONLINE nerv01 Open
2 ONLINE ONLINE nerv02 Open
ora.scan1.vip
1 ONLINE ONLINE nerv01
[root@nerv01 ~]# /u01/app/11.2.0/grid/bin/crsctl stop cluster
CRS-2673: Attempting to stop 'ora.crsd' on 'nerv01'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'nerv01'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'nerv01'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'nerv01'
CRS-2673: Attempting to stop 'ora.orcl.db' on 'nerv01'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'nerv01' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'nerv01'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'nerv01' succeeded
CRS-2673: Attempting to stop 'ora.nerv01.vip' on 'nerv01'
CRS-2677: Stop of 'ora.scan1.vip' on 'nerv01' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'nerv02'
CRS-2677: Stop of 'ora.nerv01.vip' on 'nerv01' succeeded
CRS-2672: Attempting to start 'ora.nerv01.vip' on 'nerv02'
CRS-2676: Start of 'ora.scan1.vip' on 'nerv02' succeeded
CRS-2676: Start of 'ora.nerv01.vip' on 'nerv02' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'nerv02'
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'nerv02' succeeded
CRS-2677: Stop of 'ora.orcl.db' on 'nerv01' succeeded
CRS-2673: Attempting to stop 'ora.eons' on 'nerv01'
CRS-2673: Attempting to stop 'ora.ons' on 'nerv01'
CRS-2677: Stop of 'ora.ons' on 'nerv01' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'nerv01'
CRS-2677: Stop of 'ora.net1.network' on 'nerv01' succeeded
CRS-2677: Stop of 'ora.eons' on 'nerv01' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'nerv01' has completed
CRS-2677: Stop of 'ora.crsd' on 'nerv01' succeeded
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'nerv01'
CRS-2673: Attempting to stop 'ora.ctssd' on 'nerv01'
CRS-2673: Attempting to stop 'ora.evmd' on 'nerv01'
CRS-2677: Stop of 'ora.cssdmonitor' on 'nerv01' succeeded
CRS-2677: Stop of 'ora.evmd' on 'nerv01' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'nerv01' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'nerv01'
CRS-2677: Stop of 'ora.cssd' on 'nerv01' succeeded
CRS-2673: Attempting to stop 'ora.diskmon' on 'nerv01'
CRS-2677: Stop of 'ora.diskmon' on 'nerv01' succeeded
[root@nerv01 ~]# ssh nerv02
The authenticity of host 'nerv02 (192.168.0.2)' can't be established.
RSA key fingerprint is 3f:a7:e2:9f:30:9b:8b:01:3f:87:3f:ee:5e:8d:91:91.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'nerv02' (RSA) to the list of known hosts.
root@nerv02's password:
Last login: Mon Oct 4 13:25:49 2010 from 192.168.0.201
[root@nerv02 ~]# /u01/app/11.2.0/grid/bin/crsctl stop cluster
CRS-2673: Attempting to stop 'ora.crsd' on 'nerv02'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'nerv02'
CRS-2673: Attempting to stop 'ora.nerv01.vip' on 'nerv02'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'nerv02'
CRS-2673: Attempting to stop 'ora.orcl.db' on 'nerv02'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'nerv02'
CRS-2677: Stop of 'ora.nerv01.vip' on 'nerv02' succeeded
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'nerv02' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'nerv02'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'nerv02' succeeded
CRS-2673: Attempting to stop 'ora.nerv02.vip' on 'nerv02'
CRS-2677: Stop of 'ora.nerv02.vip' on 'nerv02' succeeded
CRS-2677: Stop of 'ora.scan1.vip' on 'nerv02' succeeded
CRS-2677: Stop of 'ora.orcl.db' on 'nerv02' succeeded
CRS-2673: Attempting to stop 'ora.eons' on 'nerv02'
CRS-2673: Attempting to stop 'ora.ons' on 'nerv02'
CRS-2677: Stop of 'ora.ons' on 'nerv02' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'nerv02'
CRS-2677: Stop of 'ora.net1.network' on 'nerv02' succeeded
CRS-2677: Stop of 'ora.eons' on 'nerv02' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'nerv02' has completed
CRS-2677: Stop of 'ora.crsd' on 'nerv02' succeeded
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'nerv02'
CRS-2673: Attempting to stop 'ora.ctssd' on 'nerv02'
CRS-2673: Attempting to stop 'ora.evmd' on 'nerv02'
CRS-2677: Stop of 'ora.cssdmonitor' on 'nerv02' succeeded
CRS-2677: Stop of 'ora.evmd' on 'nerv02' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'nerv02' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'nerv02'
CRS-2677: Stop of 'ora.cssd' on 'nerv02' succeeded
CRS-2673: Attempting to stop 'ora.diskmon' on 'nerv02'
CRS-2677: Stop of 'ora.diskmon' on 'nerv02' succeeded
[root@nerv02 ~]# exit
logout
Connection to nerv02 closed.
[root@nerv01 ~]# /u01/app/11.2.0/grid/bin/crsctl start cluster
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'nerv01'
CRS-2676: Start of 'ora.cssdmonitor' on 'nerv01' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'nerv01'
CRS-2672: Attempting to start 'ora.diskmon' on 'nerv01'
CRS-2674: Start of 'ora.diskmon' on 'nerv01' failed
CRS-2679: Attempting to clean 'ora.diskmon' on 'nerv01'
CRS-2681: Clean of 'ora.diskmon' on 'nerv01' succeeded
CRS-2674: Start of 'ora.cssd' on 'nerv01' failed
CRS-2679: Attempting to clean 'ora.cssd' on 'nerv01'
CRS-2681: Clean of 'ora.cssd' on 'nerv01' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'nerv01'
CRS-2672: Attempting to start 'ora.diskmon' on 'nerv01'
CRS-2674: Start of 'ora.diskmon' on 'nerv01' failed
CRS-2679: Attempting to clean 'ora.diskmon' on 'nerv01'
CRS-2681: Clean of 'ora.diskmon' on 'nerv01' succeeded
CRS-2674: Start of 'ora.cssd' on 'nerv01' failed
CRS-2679: Attempting to clean 'ora.cssd' on 'nerv01'
CRS-2681: Clean of 'ora.cssd' on 'nerv01' succeeded
CRS-4000: Command Start failed, or completed with errors.
[root@nerv01 ~]# /u01/app/11.2.0/grid/bin/crsctl stop cluster
CRS-2796: The command may not proceed when Cluster Ready Services is not running
CRS-4000: Command Stop failed, or completed with errors.
[root@nerv01 ~]# /u01/app/11.2.0/grid/bin/crsctl disable crs
CRS-4621: Oracle High Availability Services autostart is disabled.
[root@nerv01 ~]# reboot
Broadcast message from root (pts/0) (Fri Oct 8 06:56:40 2010):
The system is going down for reboot NOW!
[root@nerv01 ~]# Connection to nerv01 closed by remote host.
Connection to nerv01 closed.
[root@nerv00 ~]# ssh oracle@nerv01
oracle@nerv01's password:
Last login: Fri Oct 8 06:54:21 2010 from nas1.localdomain
[oracle@nerv01 ~]$ su -
Password:
[root@nerv01 ~]# /u01/app/11.2.0/grid/bin/crsctl start crs -excl
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.gipcd' on 'nerv01'
CRS-2672: Attempting to start 'ora.mdnsd' on 'nerv01'
CRS-2676: Start of 'ora.mdnsd' on 'nerv01' succeeded
CRS-2676: Start of 'ora.gipcd' on 'nerv01' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'nerv01'
CRS-2676: Start of 'ora.gpnpd' on 'nerv01' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'nerv01'
CRS-2676: Start of 'ora.cssdmonitor' on 'nerv01' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'nerv01'
CRS-2679: Attempting to clean 'ora.diskmon' on 'nerv01'
CRS-2681: Clean of 'ora.diskmon' on 'nerv01' succeeded
CRS-2672: Attempting to start 'ora.diskmon' on 'nerv01'
CRS-2676: Start of 'ora.diskmon' on 'nerv01' succeeded
CRS-2676: Start of 'ora.cssd' on 'nerv01' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'nerv01'
CRS-2676: Start of 'ora.ctssd' on 'nerv01' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'nerv01'
CRS-2676: Start of 'ora.crsd' on 'nerv01' succeeded
[root@nerv01 ~]# /u01/app/11.2.0/grid/bin/crsctl query css votedisk
Located 0 voting disk(s).
[root@nerv01 ~]# /u01/app/11.2.0/grid/bin/crsctl add css votedisk /u01/shared_config/rac01/vdsk
Now formatting voting disk: /u01/shared_config/rac01/vdsk.
CRS-4603: Successful addition of voting disk /u01/shared_config/rac01/vdsk.
[root@nerv01 ~]# /u01/app/11.2.0/grid/bin/crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'nerv01'
CRS-2673: Attempting to stop 'ora.crsd' on 'nerv01'
CRS-2677: Stop of 'ora.crsd' on 'nerv01' succeeded
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'nerv01'
CRS-2673: Attempting to stop 'ora.ctssd' on 'nerv01'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'nerv01'
CRS-2677: Stop of 'ora.cssdmonitor' on 'nerv01' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'nerv01' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'nerv01' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'nerv01'
CRS-2677: Stop of 'ora.cssd' on 'nerv01' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'nerv01'
CRS-2673: Attempting to stop 'ora.diskmon' on 'nerv01'
CRS-2677: Stop of 'ora.gpnpd' on 'nerv01' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'nerv01'
CRS-2677: Stop of 'ora.gipcd' on 'nerv01' succeeded
CRS-2677: Stop of 'ora.diskmon' on 'nerv01' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'nerv01' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@nerv01 ~]# /u01/app/11.2.0/grid/bin/crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root@nerv01 ~]# /u01/app/11.2.0/grid/bin/crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 698fd2a16db04f04bf45ac75feeba342 (/u01/shared_config/rac01/vdsk) []
Located 1 voting disk(s).
[root@nerv01 ~]# /u01/app/11.2.0/grid/bin/crsctl check cluster
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
[root@nerv01 ~]# /u01/app/11.2.0/grid/bin/crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
ONLINE ONLINE nerv01
ora.asm
OFFLINE OFFLINE nerv01
ora.eons
ONLINE ONLINE nerv01
ora.gsd
OFFLINE OFFLINE nerv01
ora.net1.network
ONLINE ONLINE nerv01
ora.ons
ONLINE ONLINE nerv01
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE nerv01
ora.nerv01.vip
1 ONLINE ONLINE nerv01
ora.nerv02.vip
1 ONLINE INTERMEDIATE nerv01 FAILED OVER
ora.oc4j
1 OFFLINE OFFLINE
ora.orcl.db
1 ONLINE ONLINE nerv01 Open
2 ONLINE OFFLINE
ora.scan1.vip
1 ONLINE ONLINE nerv01
[root@nerv01 ~]#