stop and start RAC 10g/11g

Simple Formula Lister>instance>nodeapps>cluster



######################STOP###################################

## Stop listener

## Stop instance
srvctl stop instance -i orcl1 -d orcl -o immidiate
srvctl stop instance -i orcl2 -d orcl -o immidiate


## Stop Nodeapps
srvctl stop nodeapps -n rac1
srvctl stop nodeapps -n rac2

## Stop cluster
crsctl stop crs     (Node1 and Node2)



#####################START################

## Start cluster on both node
crsctl start crs     (on both node)


## Start nodeapps
srvctl start nodeapps -n rac1
srvctl start nodeapps -n rac2

##Start database instance
srvctl start instance -i orcl1 -d orcl
srvctl start instance -i orcl2 -d orcl

##Start listener


################RAC Userfull Commands##########

## ps -ef|grep smon

## ps -ef|grep ohasd

## ps -ef|grep d.bin

## crs_stat -t

## crsctl check crs

## srvctl status nodeapps -n rac1

## srvctl status instance -i orcl1 -d orcl

ORA-15001: DISKGROUP “DATA” DOES NOT EXIST OR IS NOT MOUNTED

Problem:

ORA-00210: cannot open the specified control file
ORA-00202: control file: '+DATA/kbazerdr/controlfile/current.256.825098041'
ORA-17503: ksfdopn:2 Failed to open file +DATA/kbazerdr/controlfile/current.256.825098041
ORA-15001: diskgroup "DATA" does not exist or is not mounted
ORA-15077: could not locate ASM instance serving a required diskgroup
ORA-205 signalled during: ALTER DATABASE MOUNT STANDBY DATABASE...

Analyze:

1) check if CRS is running on the node

$ crsctl check crs
CSS appears healthy
CRS appears healthy
EVM appears healthy

2) check if ASM instance is running

sys@+asm> select status from v$instance;
STATUS
————
STARTED

3) check diskgroup status

bash-3.2$ ps -afe | grep smon
 oracle 1788 1 0 Sep 24 ? 15:48 asm_smon_+ASM
 oracle 2807 2799 0 20:33:44 pts/4 0:00 grep smon
bash-3.2$ . oraenv
ORACLE_SID = [KBAZER] ? +ASM
The Oracle base remains unchanged with value /u01/app/oracle
bash-3.2$ asmcmd
ASMCMD> ls
ASMCMD> 
ASMCMD> 
ASMCMD> exit
bash-3.2$ 
bash-3.2$ sqlplus / as sysasm
SQL*Plus: Release 11.2.0.3.0 Production on Tue Mar 31 20:35:11 2015
Copyright (c) 1982, 2011, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Automatic Storage Management option
SQL> 
SQL> 
SQL> select status from v$instance;

STATUS
------------
STARTED

SQL> select name, state from v$asm_diskgroup;

NAME STATE
------------------------------ -----------
DATA DISMOUNTED
ARCH DISMOUNTED

SQL> show parameter asm_diskgroups;

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
asm_diskgroups string ARCH

Solution:

1) manually mount the diskgroups (sqlplus / as sysasm)

SQL> alter diskgroup DATA mount;
Diskgroup altered.

SQL> alter diskgroup ARCH mount;
Diskgroup altered.

SQL> select name, state from v$asm_diskgroup;

NAME STATE
------------------------------ -----------
DATA MOUNTED
ARCH MOUNTED

SQL> show parameter asm_diskgroups;

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
asm_diskgroups string ARCH, DATA

srvctl stop asm failed on 11gR2

when shutting down ASM using "srvctl stop asm -n linix101", I got this error. 

PRCR-1014 : Failed to stop resource ora.asm
PRCR-1065 : Failed to stop resource ora.asm
CRS-2673: Attempting to stop 'ora.asm' on 'linux101'
ORA-15097: cannot SHUTDOWN ASM instance with connected client
CRS-2675: Stop of 'ora.asm' on 'linux101' failed


The reason you need to stop crs therefore stopping ASM is your OCR is located on ASM disks. Thus ASM can't be shutdown while cluster is running. 

See metalink 984663.1.

this error may occur if my ocr and voting disk were residing on ASM then it will not allow me to stop the asm and I should use (Check below links)

www.dba-village.com/village/dvp_forum.OpenThread?ThreadIdA=43116

But I have verified my ocr file and voting disk are residing on ASM.

[root@RAC1 bin]# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :    8381156
         Used space (kbytes)      :       6060
         Available space (kbytes) :    8375096
         ID                       : 1141966925
         Device/File Name         : /dev/raw/raw1
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

         Cluster registry integrity check succeeded

         Logical corruption check succeeded


[root@RAC1 bin]# ./crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   03db5164ffb96f85ff95896b184eb362 (/dev/raw/raw2) []
Located 1 voting disk(s).



From 11g R2 onwards, ASM disk groups are now registered as resources within Grid Infrastructure and the dependency has been redefined, means the dependency is already set in resource definition. This dependency is created with one or more ASM disk groups.

For ex.
-------
[oracle@london1 ~]$ crsctl status resource ora.prod.db -p
NAME=ora.prod.db
TYPE=ora.database.type
ACL=owner:oracle:rwx,pgrp:oinstall:rwx,other::r--
[...]
SPFILE=+DATA/prod/spfileprod.ora
START_DEPENDENCIES=hard(ora.DATA.dg) [...] pullup(ora.DATA.dg)
STOP_DEPENDENCIES=hard(intermediate:ora.asm,shutdown:ora.DATA.dg)

You can see that a start dependency has been created for the disk group DATA, which will also be started if needed as part of the database start. Similarly, a stop dependency exists for the same disk group. The hard dependency at "ora.asm" shows that the database must be shutdown before stopping the ASM.

Note that not only database, but other resources may also have dependency on the ASM, like ACFS. So if you use "force" option to execute the stop command on ASM, still some components won't stop, due to too many dependencies on other resources.

So, to stop ASM in 11gR2, "crsctl stop crs" or "crsctl stop cluster" is the only option to stop the ASM, since it first stop all the dependent resources the stop the ASM.

So, I think in my case, there must be some resources which depends on ASM, like my database files would be on ASM.