今天安装oracle rac 11.2.0.4，操作系统是oracle linux 7u9，磁盘采用的是iSCSI共享挂载。
重启的时候出现一个奇怪的现象，crs群集无法正常启动。

[grid@rac1 ~]$ crsctl stat res -t
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Status failed, or completed with errors.

查看/u01/app/11.2.0/grid/log/rac1/alertrac1.log日志，发现无法识别到votedisk

[cssd(3348)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/rac1/cssd/ocssd.log
2022-05-01 12:51:13.898:
[cssd(3348)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/rac1/cssd/ocssd.log
2022-05-01 12:51:28.914:
[cssd(3348)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/rac1/cssd/ocssd.log

检查oracleasm configure，已经设置开机启动扫描磁盘

[root@rac1 ~]# oracleasm configure
ORACLEASM_ENABLED=true
ORACLEASM_UID=grid
ORACLEASM_GID=asmadmin
ORACLEASM_SCANBOOT=true
ORACLEASM_SCANORDER=""
ORACLEASM_SCANEXCLUDE=""
ORACLEASM_SCAN_DIRECTORIES=""
ORACLEASM_USE_LOGICAL_BLOCK_SIZE="false"

而且手动扫描磁盘是可以扫到磁盘的，说明配置是没啥问题的

[root@rac1 ~]# oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
Instantiating disk "OCR_VOTE1"
Instantiating disk "OCR_VOTE2"
Instantiating disk "OCR_VOTE3"

那么问题出在哪呢？

solution

经过排查可能是oracleasm.service的服务比iscsid.service的服务更早启动，导致没有扫描到iscsi磁盘，所以修改一下配置让oracleasm在iscsid后面启动。

[grid@rac1 ~]$ cat /usr/lib/systemd/system/oracleasm.service
[Unit]
Description=Load oracleasm Modules
Requires=iscsid.service # add
After=iscsi.service iscsid.service # add

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/sbin/oracleasm.init start_sysctl
ExecStop=/usr/sbin/oracleasm.init stop_sysctl
ExecReload=/usr/sbin/oracleasm.init restart_sysctl

[Install]
WantedBy=multi-user.target

重启后，问题解决。