I was troubleshooting a problem with some other vendor’s software tonight on a Red Hat Enterprise Linux 5.3 system. We were able to reproduce the problem in the lab, which was a huge boost to production, and insight, but we hit a wall when we got this error:
GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-42.el5)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
Reading symbols from /opt/vendor/redacted/process...(no debugging symbols found)...done.
Attaching to program: /opt/vendor/redacted/process, process 16492
ptrace: Operation not permitted.
The weird thing is that we were running gdb as root, and it was 2.6.18. In the latest Ubuntu versions, a security hardening option has been added to the kernel to limit gdb (profiling, particularly, which gdb requires) to only being run on child processes. Since this was Red Hat Enterprise Linux 5.3, it didn’t have this option.
Well, it turns out that explaining it to #gdb on Freenode pointed us to the solution: a parent process had been attached to via “strace -f”. Since only one profiling process can run on any program at one time, the parent process’s “strace”, by following all forks with the “-f”, blocked out our “gdb” from attaching to the child. Simply adding 1 line:
pkill strace
gdb /opt/vendor/redacted/process `pgrep process`
to our debugging solved the mysterious “ptrace: Operation not permitted.” error which was showing me no results in web searches. FYI: this absolutely will block gcore in the same way.