solaris10如何使用裸设备？急！

Question

小机M8000，solaris10系统，emc存储，详细如下：

bash-3.00# uname -a
SunOS racdb2 5.10 Generic_142900-12 sun4u sparc SUNW,SPARC-Enterprise
bash-3.00# cat /etc/release
                      Solaris 10 10/09 s10s_u8wos_08a SPARC
           Copyright 2009 Sun Microsystems, Inc.  All Rights Reserved.
                        Use is subject to license terms.
                           Assembled 16 September 2009
bash-3.00#

安装oracle10G RAC 的 crs时候，执行root.sh时候报错：

]# ./root.sh
WARNING: directory '/app/oracle/product/10.2.0' is not owned by root
WARNING: directory '/app/oracle/product' is not owned by root
WARNING: directory '/app/oracle' is not owned by root
WARNING: directory '/app' is not owned by root
Checking to see if Oracle CRS stack is already configured

Setting the permissions on OCR backup directory
Setting up NS directories
PROT-1: Failed to initialize ocrconfig
Failed to upgrade Oracle Cluster Registry configuration

经网上搜索有人说是由于裸设备的原因，我的裸设备划分具体如下：

#format
…
…
128. emcpower0b <EMC-SYMMETRIX-5773 cyl 57136 alt 2 hd 30 sec 128>
          /pseudo/emcp@0
…
…
Specify disk (enter its number):  128
selecting emcpower0b
[disk formatted]

FORMAT MENU:
        disk       - select a disk
        type       - select (define) a disk type
        partition  - select (define) a partition table
        current    - describe the current disk
        format     - format and analyze the disk
        repair     - repair a defective sector
        label      - write label to the disk
        analyze    - surface analysis
        defect     - defect list management
        backup     - search for backup labels
        verify     - read and display labels
        save       - save new disk/partition definitions
        inquiry    - show vendor, product and revision
        volname    - set 8-character volume name
        !<cmd>     - execute <cmd>, then return
        quit
format> p

PARTITION MENU:
        0      - change `0' partition
        1      - change `1' partition
        2      - change `2' partition
        3      - change `3' partition
        4      - change `4' partition
        5      - change `5' partition
        6      - change `6' partition
        7      - change `7' partition
        select - select a predefined table
        modify - modify a predefined partition table
        name   - name the current table
        print  - display the current table
        label  - write partition map and label to the disk
        !<cmd> - execute <cmd>, then return
        quit
partition> p
Current partition table (original):
Total disk cylinders available: 57136 + 2 (reserved cylinders)

Part      Tag    Flag     Cylinders         Size            Blocks
  0       root    wm       0 -  5461       10.00GB    (5462/0/0)   20974080
  1       swap    wu       0                0         (0/0/0)             0
  2     backup    wu       0 - 57135      104.62GB    (57136/0/0) 219402240
  3 unassigned    wm    5462 - 10923       10.00GB    (5462/0/0)   20974080
  4 unassigned    wm   10924 - 16385       10.00GB    (5462/0/0)   20974080
  5 unassigned    wm   16386 - 21847       10.00GB    (5462/0/0)   20974080
  6        usr    wm   21848 - 51673       54.61GB    (29826/0/0) 114531840
  7 unassigned    wm   51674 - 57135       10.00GB    (5462/0/0)   20974080

partition> q

bash-3.00# ls -l /dev/rdsk/emcpower0*
lrwxrwxrwx   1 root     root          33 Sep 13 12:23 /dev/rdsk/emcpower0a -> ../../devices/pseudo/emcp@0:a,raw
lrwxrwxrwx   1 root     root          33 Sep 13 12:23 /dev/rdsk/emcpower0b -> ../../devices/pseudo/emcp@0:b,raw
lrwxrwxrwx   1 root     root          33 Sep 13 12:23 /dev/rdsk/emcpower0c -> ../../devices/pseudo/emcp@0:c,raw
lrwxrwxrwx   1 root     root          33 Sep 13 12:23 /dev/rdsk/emcpower0d -> ../../devices/pseudo/emcp@0:d,raw
lrwxrwxrwx   1 root     root          33 Sep 13 12:23 /dev/rdsk/emcpower0e -> ../../devices/pseudo/emcp@0:e,raw
lrwxrwxrwx   1 root     root          33 Sep 13 12:23 /dev/rdsk/emcpower0f -> ../../devices/pseudo/emcp@0:f,raw
lrwxrwxrwx   1 root     root          33 Sep 13 12:23 /dev/rdsk/emcpower0g -> ../../devices/pseudo/emcp@0:g,raw
lrwxrwxrwx   1 root     root          33 Sep 13 12:23 /dev/rdsk/emcpower0h -> ../../devices/pseudo/emcp@0:h,raw
bash-3.00#

bash-3.00# ls -l /devices/pseudo/emcp@0*
brw-rw----   1 oracle   dba      332,  0 Sep 17 15:09 /devices/pseudo/emcp@0:a,blk
crw-r-----   1 root     oinstall 332,  0 Sep 17 15:08 /devices/pseudo/emcp@0:a,raw
brw-rw----   1 oracle   dba      332,  1 Sep 17 15:09 /devices/pseudo/emcp@0:b,blk
crw-rw----   1 oracle   dba      332,  1 Sep 17 15:08 /devices/pseudo/emcp@0:b,raw
brw-rw----   1 oracle   dba      332,  2 Sep 17 15:09 /devices/pseudo/emcp@0:c,blk
crw-rw----   1 oracle   dba      332,  2 Sep 17 15:08 /devices/pseudo/emcp@0:c,raw
brw-rw----   1 oracle   dba      332,  3 Sep 17 15:09 /devices/pseudo/emcp@0:d,blk
crw-r-----   1 root     oinstall 332,  3 Sep 17 15:08 /devices/pseudo/emcp@0:d,raw
brw-rw----   1 oracle   dba      332,  4 Sep 17 15:09 /devices/pseudo/emcp@0:e,blk
crw-r--r--   1 oracle   oinstall 332,  4 Sep 17 15:08 /devices/pseudo/emcp@0:e,raw
brw-rw----   1 oracle   dba      332,  5 Sep 17 15:09 /devices/pseudo/emcp@0:f,blk
crw-r--r--   1 oracle   oinstall 332,  5 Sep 17 15:08 /devices/pseudo/emcp@0:f,raw
brw-rw----   1 oracle   dba      332,  6 Sep 17 15:09 /devices/pseudo/emcp@0:g,blk
crw-rw----   1 oracle   dba      332,  6 Sep 17 15:08 /devices/pseudo/emcp@0:g,raw
brw-rw----   1 oracle   dba      332,  7 Sep 17 15:09 /devices/pseudo/emcp@0:h,blk
crw-r--r--   1 oracle   oinstall 332,  7 Sep 17 15:08 /devices/pseudo/emcp@0:h,raw

/devices/pseudo/emcp@0:
total 0
bash-3.00#

说明一下，因为solaris下每块磁盘最多有8个分区，我将128号磁盘划分了这样几个空间：

Part      Tag    Flag     Cylinders         Size            Blocks
  0       root    wm       0 -  5461       10.00GB    (5462/0/0)   20974080
  1       swap    wu       0                0         (0/0/0)             0
  2     backup    wu       0 - 57135      104.62GB    (57136/0/0) 219402240
  3 unassigned    wm    5462 - 10923       10.00GB    (5462/0/0)   20974080
  4 unassigned    wm   10924 - 16385       10.00GB    (5462/0/0)   20974080
  5 unassigned    wm   16386 - 21847       10.00GB    (5462/0/0)   20974080
  6        usr    wm   21848 - 51673       54.61GB    (29826/0/0) 114531840
  7 unassigned    wm   51674 - 57135       10.00GB    (5462/0/0)   20974080
part编号为0、3、4、5、7五个分别划分10G，用作oracle rac的crs和vot使用，其对应在系统中的物理设备应该就是：/dev/rdsk/emcpower0a、/dev/rdsk/emcpower0d、/dev/rdsk/emcpower0e、/dev/rdsk/emcpower0f、/dev/rdsk/emcpower0h,所以我在安装过程中直接使用的是：
crs：  /dev/rdsk/emcpower0a
crs ： /dev/rdsk/emcpower0d
vot：/dev/rdsk/emcpower0e
vot：/dev/rdsk/emcpower0f
vot：/dev/rdsk/emcpower0h
这样的裸设备对应方式，这种配置正确么？

solaris系统刚刚接触，还请大家帮忙！多谢！

Answer 1

还有安装rac的时候需要注意的
1 安装oracle CRS指定OCR和Voting disk时需要两边的设备文件名称相同。使用ASM也同样需要文件名相同，而且ASM管理磁盘时不能链接在/dev/rdsk/目录下，会导致ASM不能正常探测识别lun。
2 在Solaris系统下，磁盘的第一个block即0 block保存这磁盘标识，只有ufs才能用到block 0而不丢失磁盘标识，其他程序写block 0时会导致磁盘丢失。因此，使用磁盘时不要使用磁盘的全部，而是划分一个分区使用柱面 3开始的后面部分，不包括block 0。手工划分0分区starting cyl 3开始的后面所有容量

在需要重新安装的情况下，先卸载crs，贴一个卸载的文档。
Doc ID: 239998.1

PURPOSE
-------

The purpose of this document is to help DBA's and support analysts understand how
to clean up a failed CRS (Cluster Ready Services) install for 10g RAC.

SCOPE & APPLICATION
-------------------

DBA's and Support Analysts

10g RAC: How to Clean Up After a Failed CRS Install
---------------------------------------------------

Not cleaning up a failed CRS install can cause problems like node reboots.
Follow these steps to clean up a failed CRS install:

1. Run the rootdelete.sh script then the rootdeinstall.sh script from the
$ORA_CRS_HOME/install directory on any nodes you are removing CRS from. Running
these scripts should be sufficent to clean up your CRS install. Rootdelete.sh
accepts options like nosharedvar/sharedvar, and nosharedhome/sharedhome. Make
yourself familiar with these options by reading the Oracle Clusterware and
Oracle Real Application Clusters Administration and Deployment Guide.
If you have any problems with these scripts please open a service request.

If for some reason you have to manually remove the install due to problems
with the scripts, continue to step 2:

2. Stop the Nodeapps on all nodes:

srvctl stop nodeapps -n <node name>

3. Prevent CRS from starting when the node boots. To do this issue the following
as root:

Sun:

rm /etc/init.d/init.cssd
rm /etc/init.d/init.crs
rm /etc/init.d/init.crsd
rm /etc/init.d/init.evmd
rm /etc/rc3.d/K96init.crs
rm /etc/rc3.d/S96init.crs
       rm -Rf /var/opt/oracle/scls_scr
       rm -Rf /var/opt/oracle/oprocd
rm /etc/inittab.crs
cp /etc/inittab.orig /etc/inittab

Linux:

       rm /etc/oracle/*
rm -f /etc/init.d/init.cssd
rm -f /etc/init.d/init.crs
rm -f /etc/init.d/init.crsd
rm -f /etc/init.d/init.evmd
rm -f /etc/rc2.d/K96init.crs
rm -f /etc/rc2.d/S96init.crs
rm -f /etc/rc3.d/K96init.crs
rm -f /etc/rc3.d/S96init.crs
rm -f /etc/rc5.d/K96init.crs
rm -f /etc/rc5.d/S96init.crs
       rm -Rf /etc/oracle/scls_scr
rm -f /etc/inittab.crs
cp /etc/inittab.orig /etc/inittab

HP-UX:

rm /sbin/init.d/init.cssd
rm /sbin/init.d/init.crs
rm /sbin/init.d/init.crsd
rm /sbin/init.d/init.evmd
       rm /sbin/rc2.d/K960init.crs
       rm /sbin/rc2.d/K001init.crs
rm /sbin/rc3.d/K960init.crs
rm /sbin/rc3.d/S960init.crs
       rm -Rf /var/opt/oracle/scls_scr
       rm -Rf /var/opt/oracle/oprocd
rm /etc/inittab.crs
cp /etc/inittab.orig /etc/inittab

HP Tru64:

rm /sbin/init.d/init.cssd
rm /sbin/init.d/init.crs
rm /sbin/init.d/init.crsd
rm /sbin/init.d/init.evmd
rm /sbin/rc3.d/K96init.crs
rm /sbin/rc3.d/S96init.crs
       rm -Rf /var/opt/oracle/scls_scr
       rm -Rf /var/opt/oracle/oprocd
rm /etc/inittab.crs
cp /etc/inittab.orig /etc/inittab

IBM AIX:

rm /etc/init.cssd
rm /etc/init.crs
rm /etc/init.crsd
rm /etc/init.evmd
rm /etc/rc.d/rc2.d/K96init.crs
rm /etc/rc.d/rc2.d/S96init.crs
       rm -Rf /etc/oracle/scls_scr
       rm -Rf /etc/oracle/oprocd
rm /etc/inittab.crs
cp /etc/inittab.orig /etc/inittab

4. If they are not already down, kill off EVM, CRS, and CSS processes or reboot
   the node:

ps -ef | grep crs
kill <crs pid>
ps -ef | grep evm
kill <evm pid>
ps -ef | grep css
kill <css pid>

   Do not kill any OS processes, for example icssvr_daemon process !

5. If there is no other Oracle software running (like listeners, DB's, etc...),
   you can remove the files in /var/tmp/.oracle or /tmp/.oracle. Example:

        rm -f /var/tmp/.oracle/*

        or

        rm -f /tmp/.oracle/*

6. Remove the ocr.loc
   Usually the ocr.loc can be found at /etc/oracle

7. De-install the CRS home in the Oracle Universal Installer

8. Remove the CRS install location:

rm -Rf <CRS Install Location>/*

9. Clean out the OCR and Voting Files with dd commands. Example:

       dd if=/dev/zero of=/dev/rdsk/V1064_vote_01_20m.dbf bs=1M count=256
       dd if=/dev/zero of=/dev/rdsk/ocrV1064_100m.ora bs=1M count=256

   See the Clusterware Installation Guide for sizing requirements...

   If you placed the OCR and voting disk on a shared filesystem, remove them.

   If you are removing the RDBMS installation, also clean out any ASM disks if
   they have already been used.

10. It is good practice to reboot the node before starting the next install.

11.If you would like to re-install CRS, follow the steps in the RAC Installation manual.

Answer 2

bash-3.00# ls -l /devices/pseudo/emcp@0*
brw-rw----   1 oracle   dba      332,  0 Sep 17 15:09 /devices/pseudo/emcp@0:a,blk
crw-r-----   1 root     oinstall 332,  0 Sep 17 15:08 /devices/pseudo/emcp@0:a,raw
brw-rw----   1 oracle   dba      332,  1 Sep 17 15:09 /devices/pseudo/emcp@0:b,blk
crw-rw----   1 oracle   dba      332,  1 Sep 17 15:08 /devices/pseudo/emcp@0:b,raw
brw-rw----   1 oracle   dba      332,  2 Sep 17 15:09 /devices/pseudo/emcp@0:c,blk
crw-rw----   1 oracle   dba      332,  2 Sep 17 15:08 /devices/pseudo/emcp@0:c,raw
brw-rw----   1 oracle   dba      332,  3 Sep 17 15:09 /devices/pseudo/emcp@0:d,blk
crw-r-----   1 root     oinstall 332,  3 Sep 17 15:08 /devices/pseudo/emcp@0:d,raw
brw-rw----   1 oracle   dba      332,  4 Sep 17 15:09 /devices/pseudo/emcp@0:e,blk
crw-r--r--   1 oracle   oinstall 332,  4 Sep 17 15:08 /devices/pseudo/emcp@0:e,raw
brw-rw----   1 oracle   dba      332,  5 Sep 17 15:09 /devices/pseudo/emcp@0:f,blk
crw-r--r--   1 oracle   oinstall 332,  5 Sep 17 15:08 /devices/pseudo/emcp@0:f,raw
brw-rw----   1 oracle   dba      332,  6 Sep 17 15:09 /devices/pseudo/emcp@0:g,blk
crw-rw----   1 oracle   dba      332,  6 Sep 17 15:08 /devices/pseudo/emcp@0:g,raw
brw-rw----   1 oracle   dba      332,  7 Sep 17 15:09 /devices/pseudo/emcp@0:h,blk
crw-r--r--   1 oracle   oinstall 332,  7 Sep 17 15:08 /devices/pseudo/emcp@0:h,raw

你这里不是很多都已经是oracle dba的属主和属组了吗
刚好你建立crs要用的两个没有更改，所以报错。
更改这个目录下的
chown oracle:dba /devices/pseudo/emcp@0*

Answer 3

裸设备属主改的不对。
byuq 发表于 2010-09-18 23:44

    不知道为什么，我更改/dev/rdsk/emcpower0a的属性，

chown rootinstall /dev/rdsk/emcpower0a
chown rootinstall /dev/rdsk/emcpower0d
chown rootinstall /dev/rdsk/emcpower0e
chown rootinstall /dev/rdsk/emcpower0f
chown rootinstall /dev/rdsk/emcpower0h
chmod 660 /dev/rdsk/emcpower0a
chmod 660 /dev/rdsk/emcpower0d
chmod 660 /dev/rdsk/emcpower0e
chmod 660 /dev/rdsk/emcpower0f
chmod 660 /dev/rdsk/emcpower0h

但是好像对于/dev/rdsk/emcpower0*这些设备并不起作用，应该是这些东西也是系统的一个软连接吧，软连接的属主没有改变，但是他们连接到的具体的设备属性确实改变了，由于这些软连接是由root创建，所以他们的属主不管怎么更改都是root，

而且我尝试用用oracle用户，在/opt/oracle/lncrs、/opt/oracle/lnvot下面创建了一些软连接：

ln -s -f /devices/pseudo/emcp@0:a,raw crs
ln -s -f /devices/pseudo/emcp@0:d,raw crs1

ln -s -f /devices/pseudo/emcp@0:e,raw vot
ln -s -f /devices/pseudo/emcp@0:f,raw vot1
ln -s -f /devices/pseudo/emcp@0:h,raw vot2

这些软连接的属主为oracle和install，但是问题依旧，安装rac执行root.sh时候仍旧出现同样问题：
Setting the permissions on OCR backup directory
Setting up NS directories
PROT-1: Failed to initialize ocrconfig
Failed to upgrade Oracle Cluster Registry configuration

Answer 4

裸设备属主改的不对。