No space left on device: mod_rewrite: could not create rewrite_log_lock

When you see the error;

No space left on device: mod_rewrite: could not create rewrite_log_lock

in your Apache logs or when Apache fails to start, try the following;

ipcs -s

—— Semaphore Arrays ——–

key semid owner perms nsems

0×00000000 327680 nobody 600 1

0×00000000 1540097 nobody 600 1

0×00000000 6225922 nobody 600 1

0×00000000 9175043 nobody 600 1

0×00000000 9306116 nobody 600 1

0×00000000 10027013 nobody 600 1

If you see more than 5 of them, you may need to clear them with the following command;

ipcs -s | perl -ane '/^0x00000000/ && `ipcrm -s $F[1]`'

案例1:

Sometimes Apache refuses to start. When
this happens sweat instantly appears on your forehead because without
apache running none of your website's are available. Here are some
things you can try based on problems we've come across. Hitting the
reset switch isn't the solution and not something you should do when
you come across a problem like this.

Starting Apache

The correct way to start, stop or restart apache is by using the apachectl program. Such as:-

apachectl graceful

apachectl stop

apachectl restart

Checking Apache configuration

Apachectl can also be used to check the apache configuration, such as:-

apachectl configtest


This will return warnings and errors.

Check the Apache error logs

Take a look at the error logs (usually "/var/log/httpd/") and see if you can find what's causing the problem.

(98)Address already in use: make_sock: could not bind to address 0.0.0.0:443

no listening sockets available, shutting down

This is caused by one or more
processes running on the 443 (secure socket) port. To fix this problem
first find the process ID's that are running on port 443:-

fuser 443/tcp


This will return results which look something like:-

443/tcp: xxxx yyyy zzzz <- processes using 443


Where xxxx yyyy & zzzz are numbers for the process ID's. Now kill the processes with:-

Kill -9 xxxx yyyy & zzzz

Apache will not start. Error log contains:-

[emerg] (28)No space left on device: Couldn't create accept lock

or

[crit] (28)No space left on device: mod_rewrite: could not create rewrite_log_lock Configuration Failed

Checking your disk shows that you have plenty of space. The problem is
that apache didn't shut down properly, and it's left myriads of
semaphore-arrays left, owned by my apache-user. Run:-

www.ixdba.net

ipcs -s | grep nobody

Removing these semaphores immediately should solve the problem and allow apache to start.

ipcs -s | grep nobody | perl -e 'while (<STDIN>) { @a=split(//s+/); print `ipcrm sem $a[1]`}'

Other errors

If you encounter an error that isn't
listed here, put it in to google and give it a search. Someone else
will have come across the problem before. Remember to remove any
details from the search that are specific to your machine, such as your
IP.

案例2:

apache claims 'no space on device'


keywords : apache ipcs semaphore no space on device lock accept

tested with apach

When dealing with mem-leaks in my mod_perl-apps I ran into a curious
apache-problem. After a while apache could not be started but failed
with strange errors like:

[emerg] (28)No space left on device: Couldn't create accept lock

or

[crit] (28)No space left on device: mod_rewrite: could not create rewrite_log_lock Configuration Failed

There was definitely enough space on the device where the locks are
stored (default /usr/local/apache2/logs/). I tried to explicetely
different Lockfiles using the LockFile-directive but this did not help.
I also tried a non-default AcceptMutex (flock) which then solved the
acceptlock-issue and ended in the rewrite_log_lock-issue.

Only reboot of the system helped out of my crisis.

Solution: There were myriads of semaphore-arrays left, owned by my apache-user.

ipcs -s | grep nobody

Removing this semaphores immediately solved the problem.

ipcs -s | grep nobody | perl -e 'while (<STDIN>) { @a=split(//s+/); print `ipcrm sem $a[1]`}'

peter at 2003-04-11 (updated 2003-04-11)

serial : 224

If you found any nonsense in this entry or want to see
important improvements, I would appreatiate to receive your comments at
knowledge@goldfisch.at


disclaimer :
all
these entries are part of my very private knowledgebase that I created
while solving problems. Many solutions are taken from other webpages or
from usenet. There is no warranty for this entries of course. Some of
the articles are even stupid and one day you might even find the name
of my prefered pizza-service in here, cause I always forget about it.
Remember : This is my
knowledgebase. If you need professional support and are willing to pay for it just email me at pilsl@goldfisch.at

For enlightment take a look at http://leblogsportif.sportnation.at
案例3:

在一台DELL 1U Server(code:SIMBA)机器上 使用daemontool来守护httpd(Apache)服务, 但最近启动apachectl start时无报错信息

，但不能启动apache.

这让人好生疑惑, 重装了守护进程，问题依旧.

在查看apache error日志时发现了如下info:

No space left on device: mod_rewrite: could not create rewrite_log_lock Configuration Failed

google后知晓信号量占满， 查看ipcs发现塞满了daemon(apache启动uid)

ipcs -s | grep daemon | awk '{ print $2 }' | xargs ipcrm sem

ps: reboot the system is ok ,too.