The solution is to bring up HAIP on all nodes. Note: some instances is using HAIP while others are not, so they can not talk to each other
KEY SOFTWARE ORACLE OLR CANNOT BE OPENED.ERROR 2 SOFTWARE
No firewall is allowed on private network (cluster_interconnect) between nodes including software firewall like iptables, ipmon etcĬluster communication is configured to use the following interface(s) for this instance In this case, an USB network device gets IP 169.254.95.118 from DHCP server which disrupted HAIP routing, and solution is to black list the device in udev from being activated automatically. Link local IP must not be used by any other network on cluster nodes.
![key software oracle olr cannot be opened.error 2 key software oracle olr cannot be opened.error 2](https://s3.manualzz.com/store/data/028297118_1-2f139b2d2bdd46efe18edd37aa0cc22f.png)
Note: it’s usb0 in this case, but it can be any other adapter which uses link local Usb0 Link encap:Ethernet HWaddr E6:1F:13:AD:EE:D3 15:59:01: Checking the status of ora.asm 15:59:00: Executing cmd: /ispiris-qa/app/11.2.0.3/bin/crsctl status resource ora.asm -init 15:59:00: Executing cmd: /ispiris-qa/app/11.2.0.3/bin/crsctl check resource ora.asm -init > CRS-4000: Command Start failed, or completed with errors. > CRS-2681: Clean of ‘ora.asm’ on ‘racnode1’ succeeded > CRS-2679: Attempting to clean ‘ora.asm’ on ‘racnode1’ > CRS-2674: Start of ‘ora.asm’ on ‘racnode1’ failed For details refer to “(:CLSN00107:)” in “/ispiris-qa/app/11.2.0.3/log/racnode1/agent/ohasd/oraagent_grid/oraagent_grid.log”.
![key software oracle olr cannot be opened.error 2 key software oracle olr cannot be opened.error 2](https://blog.dbi-services.com/wp-insides/uploads/sites/2/2020/10/Picture8.png)
> ORA-03113: end-of-file on communication channel > CRS-5017: The resource action “ora.asm start” encountered the following error: > CRS-2672: Attempting to start ‘ora.asm’ on ‘racnode1’ 15:56:48: Executing cmd: /ispiris-qa/app/11.2.0.3/bin/crsctl start resource ora.asm -init $GRID_HOME/cfgtoollogs/crsconfig/rootcrs_. For details refer to “(:CLSN00107:)” in “/ocw/grid/log/racnode1/agent/ohasd/oraagent_grid/oraagent_grid.log”.ĬRS-2674: Start of ‘ora.asm’ on ‘racnode1’ failedįailed to start ASM at /ispiris-qa/app/11.2.0.3/crs/install/crsconfig_lib.pm line 1272 If the issue happens while running root script (root.sh or rootupgrade.sh) as part of Grid Infrastructure installation/upgrade process, the following symptoms will present:ĬRS-2672: Attempting to start ‘ora.asm’ on ‘racnode1’ĬRS-5017: The resource action “ora.asm start” encountered the following error: Reconfiguration started (old inc 64, new inc 66) The instance eviction reason is 0x20000000 LMON received an instance eviction notification from instance 1 Remote instance kill is issued with system inc 64 Please check instance 2’s alert log and LMON trace file for more details. LMON (ospid: 15986) tries to kill the instance 2 in 37 seconds. LMON (ospid: 15986) detects hung instances during IMR reconfiguration Kjzdattdlm: Can not attach to DLM (LMON up=, DB mounted=) Kjzdattdlm: Can not attach to DLM (LMON up=, DB mounted=). ORA-29740: evicted by instance number 1, group incarnation 10ĭiag trace from non-first ASM (+ASMn_diag_. If ASM on non-first node was running previously, likely the following will be in alert.log when it failed originally:
![key software oracle olr cannot be opened.error 2 key software oracle olr cannot be opened.error 2](https://www.red-gate.com/simple-talk/wp-content/uploads/imported/1722-2cef17a8-c42c-4ade-8306-ef85f2a27678.png)
Note: ASM instance terminates shortly after “lmon registered with NM” PMON (ospid: 19095): terminating the instance due to error 481ĭumping diagnostic data in directory=, requested by (instance=2, osid=19095 (PMON)), summary=. System State dumped to trace file /g01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_diag_c
![key software oracle olr cannot be opened.error 2 key software oracle olr cannot be opened.error 2](https://us.v-cdn.net/6032257/uploads/jive_attachments/9/3/5/93573954egami.png)
System state dump requested by (instance=2, osid=19095 (PMON)), summary=. Lmon registered with NM – instance number 2 (internal mem no 1) This note lists common causes of ASM start up failure with the following error on non-first node (second or others): Information in this document applies to any platform. Oracle Server – Enterprise Edition – Version 11.2.0.1 and later For details refer to "(:CLSN00107:)" in "/g01/app/oracle/product/11.2.0/grid/log/agent/ohasd/oraagent_grid/oraagent_grid.log".ĬRS-2674: Start of 'ora.asm' on 'racnode1' failedįailed to start ASM at /software/app/11.2.0.3/crs/install/crsconfig_lib.pm line 1272ĪSM on Non First Node (Second or Other Node) Fails to Come up With: PMON (ospid: nnnn): terminating the instance due to error 481Ĭase1: link local IP (.x) is being used by other adapter/networkĬase2: firewall exists between nodes on private network (iptables etc)Ĭase3: HAIP is up on some nodes but not on allĬase4: HAIP is up on all nodes but some do not have route info ORA-03113: end-of-file on communication channel CRS-2672: Attempting to start 'ora.asm' on 'ptdb1'ĬRS-5017: The resource action "ora.asm start" encountered the following error: