这篇文章主要讲解了“怎么修kingsql中RAC节点故障”，文中的讲解内容简单清晰，易于学习与理解，下面请大家跟着小编的思路慢慢深入，一起来研究和学习“怎么修kingsql中RAC节点故障”吧！
很久之前安装的虚拟机，今天开机之后，rac2和rac3集群服务无法启动
| [grid@rac2 cssd]$ crs_stat -t CRS-0184: Cannot communicate with the CRS daemon. |
| [grid@rac3 cssd]$ crs_stat -t CRS-0184: Cannot communicate with the CRS daemon. |
| [grid@rac1 ~]$ srvctl status database -d kingsql Instance kingsql1 is running on node rac1 Instance kingsql2 is not running on node rac2 Instance kingsql3 is not running on node rac3 |
rac2的cssd日志发现问题
| 2016-04-19 17:00:47.205: [ CSSD][3666859776]clssnmvDHBValidateNCopy: node 1, rac1, has a disk HB, but no network HB, DHB has rcfg 356447081, wrtcnt, 294277, LATS 4294697710, lastSeqNo 294274, uniqueness 1461034467, timestamp 1461056437/21174804 2016-04-19 17:00:47.205: [ CSSD][3666859776]clssnmvDHBValidateNCopy: node 3, rac3, has a disk HB, but no network HB, DHB has rcfg 356447080, wrtcnt, 41563, LATS 4294697710, lastSeqNo 41560, uniqueness 1463643843, timestamp 1463643914/18290394 |
| [grid@rac1 ~]$ crs_stop -all CRS-2500: Cannot stop resource 'ora.gsd' as it is not running Attempting to stop `ora.oc4j` on member `rac1` Attempting to stop `ora.CRS.dg` on member `rac1` Attempting to stop `ora.DATA.dg` on member `rac1` Attempting to stop `ora.kingsql.db` on member `rac1` Attempting to stop `ora.ons` on member `rac1` CRS-2789: Cannot stop resource 'ora.gsd' as it is not running on server 'rac1' Stop of `ora.ons` on member `rac1` succeeded. Attempting to stop `ora.cvu` on member `rac1` Attempting to stop `ora.rac2.vip` on member `rac1` Attempting to stop `ora.rac3.vip` on member `rac1` Attempting to stop `ora.LISTENER_SCAN1.lsnr` on member `rac1` Stop of `ora.rac3.vip` on member `rac1` succeeded. Stop of `ora.rac2.vip` on member `rac1` succeeded. Stop of `ora.LISTENER_SCAN1.lsnr` on member `rac1` succeeded. Attempting to stop `ora.scan1.vip` on member `rac1` Stop of `ora.scan1.vip` on member `rac1` succeeded. Stop of `ora.oc4j` on member `rac1` succeeded. Stop of `ora.cvu` on member `rac1` succeeded. Attempting to stop `ora.net1.network` on member `rac1` Stop of `ora.net1.network` on member `rac1` succeeded. Stop of `ora.kingsql.db` on member `rac1` succeeded. Stop of `ora.DATA.dg` on member `rac1` succeeded. Stop of `ora.CRS.dg` on member `rac1` succeeded. Attempting to stop `ora.asm` on member `rac1` Stop of `ora.asm` on member `rac1` succeeded. Attempting to stop `ora.asm` on member `rac1` --卡在这不动了，于是执行crsctl stop crs CRS-0184: Cannot communicate with the CRS daemon. |
| [root@rac1 ~]# /u01/app/11.2.3/grid/bin/crsctl stop crs CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac1' CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1' CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1' CRS-2673: Attempting to stop 'ora.evmd' on 'rac1' CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac1' CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac1' succeeded CRS-2677: Stop of 'ora.mdnsd' on 'rac1' succeeded CRS-2677: Stop of 'ora.evmd' on 'rac1' succeeded CRS-2677: Stop of 'ora.ctssd' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'rac1' CRS-2677: Stop of 'ora.cssd' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.crf' on 'rac1' CRS-2677: Stop of 'ora.crf' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1' CRS-2677: Stop of 'ora.gipcd' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1' CRS-2677: Stop of 'ora.gpnpd' on 'rac1' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac1' has completed CRS-4133: Oracle High Availability Services has been stopped. |
| [root@rac2 ~]# /u01/app/11.2.3/grid/bin/crsctl stop crs CRS-2796: The command may not proceed when Cluster Ready Services is not running CRS-4687: Shutdown command has completed with errors. CRS-4000: Command Stop failed, or completed with errors. |
| [root@rac3 ~]# /u01/app/11.2.3/grid/bin/crsctl stop crs CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac3' CRS-2673: Attempting to stop 'ora.crsd' on 'rac3' CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac3' CRS-2673: Attempting to stop 'ora.CRS.dg' on 'rac3' CRS-2673: Attempting to stop 'ora.DATA.dg' on 'rac3' CRS-2677: Stop of 'ora.DATA.dg' on 'rac3' succeeded CRS-2677: Stop of 'ora.CRS.dg' on 'rac3' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'rac3' CRS-2677: Stop of 'ora.asm' on 'rac3' succeeded CRS-2673: Attempting to stop 'ora.ons' on 'rac3' CRS-2677: Stop of 'ora.ons' on 'rac3' succeeded CRS-2673: Attempting to stop 'ora.net1.network' on 'rac3' CRS-2677: Stop of 'ora.net1.network' on 'rac3' succeeded CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac3' has completed CRS-2677: Stop of 'ora.crsd' on 'rac3' succeeded CRS-2673: Attempting to stop 'ora.ctssd' on 'rac3' CRS-2673: Attempting to stop 'ora.evmd' on 'rac3' CRS-2673: Attempting to stop 'ora.asm' on 'rac3' CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac3' CRS-2677: Stop of 'ora.evmd' on 'rac3' succeeded CRS-2677: Stop of 'ora.mdnsd' on 'rac3' succeeded CRS-2677: Stop of 'ora.ctssd' on 'rac3' succeeded CRS-2677: Stop of 'ora.asm' on 'rac3' succeeded CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac3' CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac3' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'rac3' CRS-2677: Stop of 'ora.cssd' on 'rac3' succeeded CRS-2673: Attempting to stop 'ora.crf' on 'rac3' CRS-2677: Stop of 'ora.crf' on 'rac3' succeeded CRS-2673: Attempting to stop 'ora.gipcd' on 'rac3' CRS-2677: Stop of 'ora.gipcd' on 'rac3' succeeded CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac3' CRS-2677: Stop of 'ora.gpnpd' on 'rac3' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac3' has completed CRS-4133: Oracle High Availability Services has been stopped. |
启动
| [root@rac1 ~]# /u01/app/11.2.3/grid/bin/crsctl start crs CRS-4123: Oracle High Availability Services has been started. |
| [root@rac2 ~]# /u01/app/11.2.3/grid/bin/crsctl start crs CRS-4640: Oracle High Availability Services is already active CRS-4000: Command Start failed, or completed with errors. --还是有问题 |
| [root@rac3 ~]# /u01/app/11.2.3/grid/bin/crsctl start crs CRS-4123: Oracle High Availability Services has been started. |
| [grid@rac1 ~]$ crs_start -all [grid@rac1 ~]$ crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora.CRS.dg ora....up.type ONLINE ONLINE rac1 ora.DATA.dg ora....up.type ONLINE ONLINE rac1 ora....ER.lsnr ora....er.type ONLINE ONLINE rac1 ora....N1.lsnr ora....er.type ONLINE ONLINE rac3 ora.asm ora.asm.type ONLINE ONLINE rac1 ora.cvu ora.cvu.type ONLINE ONLINE rac3 ora.gsd ora.gsd.type OFFLINE OFFLINE ora.kingsql.db ora....se.type ONLINE ONLINE rac3 ora....network ora....rk.type ONLINE ONLINE rac1 ora.oc4j ora.oc4j.type ONLINE ONLINE rac3 ora.ons ora.ons.type ONLINE ONLINE rac1 ora....SM1.asm application ONLINE ONLINE rac1 ora....C1.lsnr application ONLINE ONLINE rac1 ora.rac1.gsd application OFFLINE OFFLINE ora.rac1.ons application ONLINE ONLINE rac1 ora.rac1.vip ora....t1.type ONLINE ONLINE rac1 ora.rac2.vip ora....t1.type ONLINE ONLINE rac1 ora....SM3.asm application ONLINE ONLINE rac3 ora....C3.lsnr application ONLINE ONLINE rac3 ora.rac3.gsd application OFFLINE OFFLINE ora.rac3.ons application ONLINE ONLINE rac3 ora.rac3.vip ora....t1.type ONLINE ONLINE rac3 ora.scan1.vip ora....ip.type ONLINE ONLINE rac3 |
重启rac2操作系统
| [root@rac2 ~]# reboot [root@rac2 ~]# Broadcast message from root@rac2 (/dev/pts/0) at 17:40 ... The system is going down for reboot NOW! |
| [grid@rac1 ~]$ crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora.CRS.dg ora....up.type ONLINE ONLINE rac1 ora.DATA.dg ora....up.type ONLINE ONLINE rac1 ora....ER.lsnr ora....er.type ONLINE ONLINE rac1 ora....N1.lsnr ora....er.type ONLINE ONLINE rac3 ora.asm ora.asm.type ONLINE ONLINE rac1 ora.cvu ora.cvu.type ONLINE ONLINE rac3 ora.gsd ora.gsd.type OFFLINE OFFLINE ora.kingsql.db ora....se.type ONLINE ONLINE rac3 ora....network ora....rk.type ONLINE ONLINE rac1 ora.oc4j ora.oc4j.type ONLINE ONLINE rac3 ora.ons ora.ons.type ONLINE ONLINE rac1 ora....SM1.asm application ONLINE ONLINE rac1 ora....C1.lsnr application ONLINE ONLINE rac1 ora.rac1.gsd application OFFLINE OFFLINE ora.rac1.ons application ONLINE ONLINE rac1 ora.rac1.vip ora....t1.type ONLINE ONLINE rac1 ora....SM2.asm application ONLINE ONLINE rac2 ora....C2.lsnr application ONLINE ONLINE rac2 ora.rac2.gsd application OFFLINE OFFLINE ora.rac2.ons application ONLINE ONLINE rac2 ora.rac2.vip ora....t1.type ONLINE ONLINE rac2 ora....SM3.asm application ONLINE ONLINE rac3 ora....C3.lsnr application ONLINE ONLINE rac3 ora.rac3.gsd application OFFLINE OFFLINE ora.rac3.ons application ONLINE ONLINE rac3 ora.rac3.vip ora....t1.type ONLINE ONLINE rac3 ora.scan1.vip ora....ip.type ONLINE ONLINE rac3 |
| [grid@rac1 ~]$ crsctl status res -t -------------------------------------------------------------------------------- NAME TARGET STATE SERVER STATE_DETAILS -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.CRS.dg ONLINE ONLINE rac1 ONLINE ONLINE rac2 ONLINE ONLINE rac3 ora.DATA.dg ONLINE ONLINE rac1 ONLINE ONLINE rac2 ONLINE ONLINE rac3 ora.LISTENER.lsnr ONLINE ONLINE rac1 ONLINE ONLINE rac2 ONLINE ONLINE rac3 ora.asm ONLINE ONLINE rac1 Started ONLINE ONLINE rac2 Started ONLINE ONLINE rac3 Started ora.gsd OFFLINE OFFLINE rac1 OFFLINE OFFLINE rac2 OFFLINE OFFLINE rac3 ora.net1.network ONLINE ONLINE rac1 ONLINE ONLINE rac2 ONLINE ONLINE rac3 ora.ons ONLINE ONLINE rac1 ONLINE ONLINE rac2 ONLINE ONLINE rac3 -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE rac3 ora.cvu 1 ONLINE ONLINE rac3 ora.kingsql.db 1 ONLINE ONLINE rac3 Open 2 ONLINE ONLINE rac2 Open 3 ONLINE ONLINE rac1 Open ora.oc4j 1 ONLINE ONLINE rac3 ora.rac1.vip 1 ONLINE ONLINE rac1 ora.rac2.vip 1 ONLINE ONLINE rac2 ora.rac3.vip 1 ONLINE ONLINE rac3 ora.scan1.vip 1 ONLINE ONLINE rac3 |
感谢各位的阅读，以上就是“怎么修kingsql中RAC节点故障”的内容了，经过本文的学习后，相信大家对怎么修kingsql中RAC节点故障这一问题有了更深刻的体会，具体使用情况还需要大家实践验证。这里是创新互联，小编将为大家推送更多相关知识点的文章，欢迎关注！