11gR2 RAC ASM启动揭秘

11gR2 RAC中ocr和votedisk终于可以存放在ASM中了, 这避免了10g中仍需要为这2个RAC的关键点划分裸设备的窘境,  随之 11gR2 中ASM的spfile也可以存放到ASM diskgroup中以实现多节点ASM的共享管理了。

 

这听上去似乎有些不可思议,照常理来说 ASM实例启动并mount diskgroup后才能够访问diskgroup上的文件, 但是ASM实例只有获得ASM spfile后才能够启动实例,这2者形成了死循环。

 

有同学在T.askmac.cn上提问关于ASM启动的疑问

 

hello maclean,

查看spfile位置
ASMCMD> spget
+CRSDG/rac/asmparameterfile/registry.253.787925627
就有个疑问,ASM 也算是一种ORACLE instance,自动的系统参数文件在自己的diskgroup,我的问题是它是如何启动从自身未启动的磁盘组读的参数文件?
thanks.!

 

我们来解释这个问题:

从11.2开始Oracle Cluterware标示voting disk files的方法较之前的版本11.1或10.2有所区别,11.2之前voting disk file的位置存放在OCR中, 但是因为从11.2开始ocr和votedisk可以存放在ASM了 , 所以自11.2始voting disk file通过GPNP profile中的CSS voting file discovery string来定位。

CSS voting disk file的discovery string将指向ASM,所以它要使用ASM discovery string的值。  如以下的例子使用udev绑定设备名作为ASM使用的LUN, 这些udev获得的设备形式如/dev/rasm-disk* , 我们利用gpnptool get命令获得gpnp profile:

 

 

[grid@maclean1 trace]$ gpnptool get



Warning: some command line parameters were defaulted. Resulting command line:
/g01/grid/app/11.2.0/grid/bin/gpnptool.bin get -o-

<?xml version="1.0" encoding="UTF-8"?><gpnp:GPnP-Profile Version="1.0" xmlns="http://www.grid-pnp.org/2005/11/gpnp-profile" xmlns:gpnp="http://www.grid-pnp.org/2005/11/gpnp-profile" 
xmlns:orcl="http://www.oracle.com/gpnp/2005/11/gpnp-profile" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xsi:schemaLocation="http://www.grid-pnp.org/2005/11/gpnp-profile gpnp-profile.xsd" 
ProfileSequence="9" ClusterUId="452185be9cd14ff4ffdc7688ec5439bf" 
ClusterName="maclean-cluster" PALocation=""><gpnp:Network-Profile><gpnp:HostNetwork id="gen" 
HostName="*"><gpnp:Network id="net1" IP="192.168.1.0" Adapter="eth0" 
Use="public"/><gpnp:Network id="net2" IP="172.168.1.0" Adapter="eth1" 
Use="cluster_interconnect"/></gpnp:HostNetwork></gpnp:Network-Profile><
orcl:CSS-Profile id="css" DiscoveryString="+asm" LeaseDuration="400"/><orcl:ASM-Profile id="asm" DiscoveryString="/dev/rasm*" SPFile="+SYSTEMDG/maclean-cluster/asmparameterfile/registry.253.788682933"/><
ds:Signature xmlns:ds="http://www.w3.org/2000/09/xmldsig#"><ds:SignedInfo><ds:CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/><ds:SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"/><ds:Reference URI=""><ds:Transforms><ds:Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/><ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"> <InclusiveNamespaces 
xmlns="http://www.w3.org/2001/10/xml-exc-c14n#" 
PrefixList="gpnp orcl xsi"/></ds:Transform></ds:Transforms><
ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/><ds:DigestValue>L1SLg10AqGEauCQ4ne9quucITZA=</ds:DigestValue><
/ds:Reference></ds:SignedInfo><ds:SignatureValue>rTyZm9vfcQCMuian6isnAThUmsV4xPoK2fteMc1l0GIvRvHncMwLQzPM/QrXCGGTCEvgvXzUPEKzmdX2oy5vLcztN60UHr6AJtA2JYYodmrsFwEyVBQ1D6wH+HQiOe2SG9UzdQnNtWSbjD4jfZkeQWyMPfWdKm071Ek0Rfb4nxE=</ds:SignatureValue></ds:Signature></gpnp:GPnP-Profile>
Success.

 

 

 

其中重要的2条记录:

 

<orcl:CSS-Profile id=”css” DiscoveryString=”+asm” LeaseDuration=”400″/>
==》css voting disk指向+ASM
<orcl:ASM-Profile id=”asm” DiscoveryString=”/dev/rasm*” SPFile=”+SYSTEMDG/maclean-cluster/asmparameterfile/registry.253.788682933″/>
==》该记录表达了ASM的DiscoveryString=”/dev/rasm*”,即ASM实例启动时会去寻找的设备路径,SPFILE记录了ASM Parameter FILE的ALIAS

 

但是请注意虽然GPNP记录了ASM Parameter FILE的ALIAS,但这不代表ASM直接能访问到该SPFILE,在实际Diskgroup被Mount之前光知道一个ASM ALIAS是没有用的。

我们来看一下+SYSTEMDG/maclean-cluster/asmparameterfile/registry.253.788682933这个SPFILE在ASM中所处的位置:

 

 

[grid@maclean1 wallets]$ sqlplus  / as sysasm

SQL*Plus: Release 11.2.0.3.0 Production on Tue Jul 17 05:45:35 2012

Copyright (c) 1982, 2011, Oracle.  All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> 
set linesize 140 pagesize 1400
col "FILE NAME" format a40
set head on
select NAME         "FILE NAME",
           AU_KFFXP     "AU NUMBER",
           NUMBER_KFFXP "FILE NUMBER",
           DISK_KFFXP   "DISK NUMBER"
      from x$kffxp, v$asm_alias
     where GROUP_KFFXP = GROUP_NUMBER
       and NUMBER_KFFXP = FILE_NUMBER
       and name in ('REGISTRY.253.788682933')
   order by  DISK_KFFXP,AU_KFFXP;

FILE NAME                                 AU NUMBER FILE NUMBER DISK NUMBER
---------------------------------------- ---------- ----------- -----------
REGISTRY.253.788682933                           39         253           1
REGISTRY.253.788682933                           35         253           3
REGISTRY.253.788682933                           35         253           4


SQL> col path for a50
SQL> select disk_number,path from v$asm_disk where disk_number in (1,3,4) and GROUP_NUMBER=3;

DISK_NUMBER PATH
----------- --------------------------------------------------
          3 /dev/rasm-diske
          4 /dev/rasm-diskf
          1 /dev/rasm-diskc

 

 

可以看到该ASM SPFILE共有三份镜像(redundancy=high),分别保留在 /dev/rasm-diskc的AU=39和/dev/rasm-diske AU=35、/dev/rasm-diskf AU=35。

我们利用kfed命令分别检查这三个ASM DISK的header:

 

[grid@maclean1 wallets]$ kfed read /dev/rasm-diske|grep spfile
kfdhdb.spfile:                       35 ; 0x0f4: 0x00000023

[grid@maclean1 wallets]$ kfed read /dev/rasm-diskc|grep spfile 
kfdhdb.spfile:                       39 ; 0x0f4: 0x00000027

[grid@maclean1 wallets]$ kfed read /dev/rasm-diskf|grep spfile 
kfdhdb.spfile:                       35 ; 0x0f4: 0x00000023

 

 

可以看到ASM disk header的kfdhdb.spfile指向ASM SPFILE在这个DISK上的AU NUMBER即其位置, ASM实例在启动时只需要通过GPNP PROFILE中的 DiscoveryString找到合适的设备路径,并读取其ASM disk header即可以找到kfdhdb.spfile这个位置属性,从而在没有MOUNT DISKGROUP的情况下读取ASM SPFILE,并成功启动ASM, 这也就解决了鸡生蛋、蛋生鸡的难题。

Comments

  1. in Oracle RAC environment information is stored in GPNP profile and there is GPNPD process that maintains and manages this profile information. In Oracle Restart, as we can see OHASD process executes this role, and because there is no PGNP profile it stores information in OLR file. So, what next? Our process starts to scan all disks to identify ASM headers, after identifying them, it identifies which disks belongs to which diskgroups by information from header. There is many other metadata in the ASM diskheader that it reads, including pointers to spfile and votig disk file, it is kfdhdb.spfile, kfdhdb.spfflg (first block and number of blocks) and kfdhdb.vfstart, kfdhdb.vfend (begin block and end block). It is possible to read disk header using kfed utility from $ORACLE_HOME/bin directory.

  2. slow soul says

    关于博文有点疑问:

    FILE NAME AU NUMBER FILE NUMBER DISK NUMBER
    —————————————- ———- ———– ———–
    REGISTRY.253.788682933 39 253 1
    REGISTRY.253.788682933 35 253 3
    REGISTRY.253.788682933 35 253 4
    这些都是从基表和视图中查出来,asm 未启动前,又怎么知道 disk number 呢?

  3. 每个ASM disk header有kfdhdb.spfile属性,这个属性指向spfile文件位置,这样就不需要asm instance在open状态读取file元数据结构。直接读取asm disk就行了。

  4. KF3_FIELD(kfdhdb, ub4, spfile_kfdhdb) /* starting AU for spfile */

沪ICP备14014813号-2

沪公网安备 31010802001379号