shareplex 错误处理一例 Cop: Address already in use tcp server bind addr port - exiting

今天遇到一个问题，shareplex启动之后，sp_ctrl进入不了。查看进程，只有几个，很不完整，查看event_log， 发现如下提示：

Cop: Address already in use tcp server bind addr 1.1.1.1 port 11111 - exiting [module cop]

看了文档，kill当前sp_开头的进程，重启shareplex，即可。

文档URL，自己备用参考：

https://support.software.dell.com/kb/sol98270

Knowledge Article 98270

Title

Error starting cop tcp server bind addr 192.168.1.212 port 40481 - exiting

Description

When trying to start cop with a command like sp_cop -uPORT & an error like the following is thrown "tcp server bind addr 192.168.1.212 port 40481 - exiting"
See example below

[korcl@oel5u7x64 bin]$ sp_ctrl
*******************************************************

* SharePlex for Oracle Command Utility

* © 2010 Quest Software, Inc.

* ALL RIGHTS RESERVED.

* Protected by U.S. Patents: 7,461,103 and 7,065,538

*******************************************************
sp_ctrl (oel5u7x64:8606)> shutdown

[korcl@oel5u7x64 bin]$ ps -ef | grep sp_

korcl 9280 9219 0 21:21 pts/1 00:00:00 grep sp_

[korcl@oel5u7x64 bin]$ pwd

/splex/proddir/bin

[korcl@oel5u7x64 bin]$ /splex/proddir/bin/sp_cop -u8606 &

[1] 9299

[korcl@oel5u7x64 bin]$

*******************************************************

* SharePlex for Oracle Startup

* © 2010 Quest Software, Inc.

* ALL RIGHTS RESERVED.

* Protected by U.S. Patents: 7,461,103 and 7,065,538

* Version: 7.6.2.59-m64-oracle110

* VarDir : /splex/vardir

* Port : 8606

*******************************************************
[korcl@oel5u7x64 bin]$ /splex/proddir/bin/sp_cop -u8606 &

[2] 9312

[korcl@oel5u7x64 bin]$

*******************************************************

* SharePlex for Oracle Startup

* © 2010 Quest Software, Inc.

* ALL RIGHTS RESERVED.

* Protected by U.S. Patents: 7,461,103 and 7,065,538

* Version: 7.6.2.59-m64-oracle110

* VarDir : /splex/vardir

* Port : 8606

*******************************************************

tcp server bind addr 192.168.1.212 port 40481 - exiting
[korcl@oel5u7x64 bin]$

[2]+ Exit 1 /splex/proddir/bin/sp_cop -u8606

[korcl@oel5u7x64 bin]$

Cause

This is usually caused by one of two errors, either the cop is already running and you are trying to restart it, or the cop has hung/stuck memory segments and or

from a previous cop.

Resolution

Check to see if the cop or any children are running.

unix> sp_ctrl

If you can get into sp_ctrl then cop was already running

unix> ps -ef | grep sp_

Are any processes left over that need to killed

example below

ps -ef | grep sp_

orcl 20721 1 0 Oct09 ? 00:00:20 /splex/proddir/.app-modules/sp_cop -u8606

orcl 21552 20721 5 Oct09 ? 02:06:17 /splex/proddir/.app-modules/sp_ocap -u8606

Kill these processes

kill -9 20721

kill -9 21552

now try and start cop in the background
If the issue is stuck/hung memory segments or semaphores the follow this

go to the $SP_SYS_VARDIR/rim and

[korcl@oel5u7x64 rim]$ od -x shmaddr.loc

0000000 800c 0049 2000 a274 7f29 0000 50b7 0017

0000020 2000 0002 0004 000a

0000030

[korcl@oel5u7x64 rim]$ od -x shstinfo.ipc

0000000 000b 0049 c000 a181 7f29 0000 50b7 0007

0000020 0000 0080 8003 0009

0000030

Take the 8th field to find the key and
[korcl@oel5u7x64 rim]$ ipcs -a | grep 50b7

0x000750b7 4784139 korcl 644 8388608 6

0x001750b7 4816908 korcl 644 139264 6

0x003750b7 4849677 korcl 644 8388608 2

0x004750b7 4882446 korcl 644 33554432 2

0x005750b7 4915215 korcl 644 8388608 3

0x000750b7 622595 korcl 666 12

0x001750b7 655364 korcl 666 26

[korcl@oel5u7x64 rim]$

You will want to run the ipcs -a without the grep to see which are memory segments and which are semaphores.

[korcl@oel5u7x64 rim]$ ipcs -a
------ Shared Memory Segments --------

key shmid owner perms bytes nattch status

0x00000000 3637248 root 644 80 2

0x00000000 3670017 root 644 16384 2

0x00000000 3702786 root 644 280 2

0x00000000 3833861 korcl 640 4096 0

0x00000000 3866630 korcl 640 4096 0

0x3b1671d0 3899399 korcl 640 4096 0

0x00000000 3964936 korcl 640 4096 0

0x00000000 3997705 korcl 640 4096 0

0xf000e0d4 4030474 korcl 640 4096 0

0x000750b7 4784139 korcl 644 8388608 6

0x001750b7 4816908 korcl 644 139264 6

0x003750b7 4849677 korcl 644 8388608 2

0x004750b7 4882446 korcl 644 33554432 2

0x005750b7 4915215 korcl 644 8388608 3
------ Semaphore Arrays --------

key semid owner perms nsems

0x6f5dd7b8 131073 korcl 640 124

0xf274e4d0 262146 korcl 640 154

0x000750b7 622595 korcl 666 12

0x001750b7 655364 korcl 666 26
------ Message Queues --------

key msqid owner perms used-bytes messages

now kill them using the ipcrm command with the flag -m for memory segments and -s for semaphores

ipcrm -s 622595

ipcrm -s 655364
ipcrm -m 4784139

ipcrm -m 4816908

ipcrm -m 4849677

ipcrm -m 4882446

ipcrm -m 4915215