Today to my great surprise I notices that my Apache web server randomly returns 403 Forbidden HTTP error code to search engine robots. Yandex Search Engine reported that on my favorite website 263 pages are OK and 210 pages has 403 status and some other site has 394/139 pages respectively.
In Apache error logs I found the following messages:
Permission denied: /home/<site-name>/.htaccess pcfg_openfile: unable to check htaccess file, ensure it is readable
as far as I see they appears whenever this 403 error is reported.
After doing extensive Google search I realized that this problem is caused by apache2-mpm-itk module bug described there: http://lists.err.no/pipermail/mpm-itk/2011-March/000394.html
apache2.2-mpm-itk 2.2.17-01, released 2011-03-21:
Fix an issue where users can sometimes get spurious 403s on persistent connections, if the .htaccess files are not world readable.
My current version of apache2-mpm-itk is 2.2.14-5ubuntu8.9. Found it out by this command:
dpkg -s apache2-mpm-itk
My current Ubuntu version is 10.04.4 LTS (Codename – lucid). Commands to find out Ubuntu version:
cat /etc/issue.net lsb_release -a cat /etc/lsb-release
New version apache2-mpm-itk is shipped only with new Ubuntu distribution, so the problem could be solved by doing dist-upgrade or by making .htaccess files world readable.
To upgrade Ubuntu Desktop from 10.04.4 to 12.04 I pressed Alt+F2 and typed in update-manager –d.
Steps to upgrade a Server:
# Install update-manager-core if it is not already installed: sudo apt-get install update-manager-core # Launch the upgrade tool: sudo do-release-upgrade –d # Follow the on-screen instructions.
And probably edit /etc/update-manager/release-upgrades and set Prompt=normal.
Finally I upgraded my Ubuntu Server to “Ubuntu 12.04 LTS” codename “precise” with apache2-mpm-itk version “2.2.22-1ubuntu1”. Upgrade installed fine, the only problem that I met was that proftpd stopped working saying:
ubuntu proftpd[24621]: mod_tls/2.4.3: compiled using OpenSSL version ‘OpenSSL 1. 0.0e 6 Sep 2011’ headers, but linked to OpenSSL version ‘OpenSSL 1.0.1 14 Mar 20 12’ library
ubuntu proftpd[24621]: Fatal: LoadModule: error loading module ‘mod_vroot.c’: Op eration not permitted on line 68 of ‘/etc/proftpd/modules.conf’
As a temporarily workaround I removed mod_tls.c from /etc/proftpd/modules.conf:
# # This file is used to manage DSO modules and features. # # This is the directory where DSO modules reside ModulePath /usr/lib/proftpd # Allow only user root to load and unload modules, but allow everyone # to see which modules have been loaded ModuleControlsACLs insmod,rmmod allow user root ModuleControlsACLs lsmod allow user * LoadModule mod_ctrls_admin.c #LoadModule mod_tls.c ...
Now I am still getting warnings in Apache error log:
Permission denied: Couldn’t read /home/<site-name>/.htaccess closing connection.
As far as I see now mpm-itk module closes connection when it does setuid(). See mod-itk 2.2.17: Wrong 403 error messages.