kernel.org status: hints on how to check your machine for intrusion

Greg KH
2011-09-30T23:59:39Z

[-- Attachment #1: Type: text/plain, Size: 4816 bytes --]

The compromise of kernel.org and related machines has made it clear that
some developers, at least, have had their systems penetrated. As we
seek to secure our infrastructure, it is imperative that nobody falls
victim to the belief that it cannot happen to them. We all need to
check our systems for intrusions. Here are some helpful hints as
proposed by a number of developers on how to check to see if your Linux
machine might be infected with something:

0. One way to be sure that your system is not compromised is to simply
do a clean install; we can all benefit from a new start sometimes.
Before reinstalling any systems, though, consider following the steps
below to learn if your system has been hit or not.

1. Install the chkrootkit package from your distro repository and see if it
reports anything. If your distro doesn't have the chkroot package,
download it from:
http://www.chkrootkit.org/

Another tool is the ossec-rootcheck tool which can be found at:
http://www.ossec.net/main/rootcheck

And another one is the rkhunter program:
http://www.rootkit.nl/projects/rootkit_hunter.html
[Note, this tool has the tendancy to give false-positives on some
Debian boxes, please read /usr/share/doc/rkhunter/README.Debian.gz if
you run this on a Debian machine]

2. Verify that your package signatures match what your package manager thinks
they are.

To do this on a rpm-based system, run the following command:
rpm --verify --all
Please read the rpm man page for information on how to interpret the
output of this command.

To do this on a Debian based system, run the following bash snippet:
dpkg -l \*|while read s n rest; do if [ "$s" == "ii" ]; then echo $n;
fi; done > ~/tmp.txt
for f in `cat ~/tmp.txt`; do debsums -s -a $f; done

If you have a source-based system (Gentoo, LFS, etc.) you presumably
know what you are doing already.

3. Verify that your packages are really signed with the distro's keys.

Here's a bash snippet that can do this on a rpm based system to
verify that the packages are signed with any key, not necessarily
your distro's key. That exercise is left for the reader:

for package in `rpm -qa`; do
sig=`rpm -q --qf '%{SIGPGP:pgpsig}\n' $package`
if [ -z "$sig" ] ; then
# check if there is a GPG key, not a PGP one
sig=`rpm -q --qf '%{SIGGPG:pgpsig}\n' $package`
if [ -z "$sig" ] ; then
echo "$package does not have a signature!!!"
fi
fi
done

Unfortunately there is no known way of verifying this on Debian-based
systems.

4. To replace a package that you find suspect, uninstall it and install
it anew from your distro. For example, if you want to reinstall the
ssh daemon, you would do:
$ /etc/init.d/sshd stop
rpm -e openssh
zypper install openssh # for openSUSE based systems
yum install openssh # for Fedora based systems

Ideally do this from a live cdrom boot, using the 'rpm --root' option
to point rpm at the correct location.

5. From a liveCD environment, look for traces such as:
a. Rogue startup scripts in /etc/rc*.d and equivalent directories.
b. Strange directories in /usr/share that do not belong to a package.
This can be checked on an rpm system with the following bash snippet:
for file in `find /usr/share/`; do
package=`rpm -qf -- ${file} | grep "is not owned"`
if [ -n "$package" ] ; then
echo "weird file ${file}, please check this out"
fi
done

6. Look for mysterious log messages, such as:
a. Unexpected logins in wtmp and /var/log/secure*, quite possibly
from legitimate users from unexpected hosts.
b. Any program trying to touch /dev/mem.
c. References to strange (non-text) ssh version strings in
/var/log/secure*. These do not necessarily indicate *successful*
breakins, but they indicate *attempted* breakins which means your
system or IP address has been targeted.

7. If any of the above steps show possible signs of compromise, you
should investigate further and identify the actual cause. If it
becomes clear that the system has indeed been compromised, you should
certainly reinstall the system from the beginning, and change your
credentials on all machines that this machine would have had access
to, or which you connected to through this machine. You will need
to check your other systems carefully, and you should almost
certainly notify the administrators of other systems to which you
have access.

Finally, please note that these hints are not guaranteed to turn up
signs of a compromised systems. There are a lot of attackers out there;
some of them are rather more sophisticated than others. You should
always be on the alert for any sort of unexpected behavior from the
systems you work with.

thanks,

greg k-h

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]