I've been getting some setrlimit "Security Events" reported by logcheck the past few weeks from a Debian 4.0 (Etch) virtual host running on a Debian 5.0 (Lenny) host. I was busy when they started and didn't drill down to see if it started with a security update in one system or the other.
Jun 15 10:09:01 green pam_limits: setrlimit limit #11 to soft=-1, hard=-1 failed: Operation not permitted; uid=0 euid=0 Jun 15 10:09:01 green pam_limits: setrlimit limit #12 to soft=-1, hard=-1 failed: Operation not permitted; uid=0 euid=0
Today I did some searching and found a few related posts ranging from bugs in sshd's limit setting to Linux-Vserver specific cases.
One suggestion from the Linux-Vserver list was to just comment out the use of pam_limit in the pam configuration files. I didn't want to throw the baby out with the bath water so I decided to take their other suggestion and modify the /etc/security/limits.conf file.
First I had to find out what limit #11 and #12 were on my system. I'm running a 686 kernel so I looked in include/asm-generic/resource.h of the matching kernel source. There I found that 11 was RLIMIT_SIGPENDING and 12 was RLIMIT_MSGQUEUE. Before getting there I had grepped around and noticed that the alpha and mips architectures swap a few values around so you may need to look at more than the generic header.
Next I used the ulimit command to list the values of pending signals (-i) and POSIX message queues (-q) applied to a default user session in the host system. I modified the /etc/security/limits.conf to set the hard and soft limits for these two resources to match. This stopped the setrlimit messages in /var/log/auth.log for new ssh logins, but they remained when cron attempts to run as root even after restarting cron. The same happens if I allow root ssh logins and login as root.