记录一次linux线上服务器被黑事件（转）

[b]1.原因：[/b]本来在家正常休息了，我们放在上海托管机房的线上服务器突然蹦了远程不了，服务启动不了，然后让上海机房重启了一次，还是直接挂了，一直到我远程上才行。 2.现象：远程服务器发现出现这类信息Hi, please view:
http://pastie.org/pastes/10800563/text?key=hzzm4hk4ihwx1jfxzfizzq for further information in regards to your files!
Hi, please view:
http://pastie.org/pastes/10800563/text?key=hzzm4hk4ihwx1jfxzfizzq for further information in regards to your files!
Hi, please view:
http://pastie.org/pastes/10800563/text?key=hzzm4hk4ihwx1jfxzfizzq for further information in regards to your files!
Hi, please view:
http://pastie.org/pastes/10800563/text?key=hzzm4hk4ihwx1jfxzfizzq for further information in regards to your files!
Hi, please view:
http://pastie.org/pastes/10800563/text?key=hzzm4hk4ihwx1jfxzfizzq for further information in regards to your files!
Hi, please view:
http://pastie.org/pastes/10800563/text?key=hzzm4hk4ihwx1jfxzfizzq for further information in regards to your files!
登录信息然后***去了国外网站查看

Greetings,Your server has been hacked and your files have been deleted.Before they were deleted, we backed them up to a server we control.You must send a total of 3 BTC to the address: 1B1oU6EdREYffif3**********Failure to do so will result in your files being deleted after 5 days. We may also leak your files.You can e-mail onewayout@sigaint.org for support. We will not give any files before a payment has been made.Goodbye!发现被黑。3.开始排查： 首先检查日志，以前做过安全运维，所以写过类似于检查命令和工具，开始一一排查。#查看是否为管理员增加或者修改
find / -type f -perm 4000#显示文件中查看是否存在系统以外的文件rpm -Vf /bin/lsrpm -Vf /usr/sbin/sshdrpm -Vf /sbin/ifconfigrpm -Vf /usr/sbin/lsof#检查系统是否有elf文件被替换#在web目录下运行grep -r "getRuntime" ./#查看是否有***find . -type f -name "*.jsp" | xargs grep -i "getRuntime"#运行的时候被连接或者被任何程序调用find . -type f -name "*.jsp" | xargs grep -i "getHostAddress"#返回ip地址字符串find . -type f -name "*.jsp" | xargs grep -i "wscript.shell"#创建WshShell对象可以运行程序、操作注册表、创建快捷方式、访问系统文件夹、管理环境变量find . -type f -name "*.jsp" | xargs grep -i "gethostbyname"#gethostbyname()返回对应于给定主机名的包含主机名字和地址信息的hostent结构指针find . -type f -name "*.jsp" | xargs grep -i "bash"#调用系统命令提权find . -type f -name "*.jsp" | xargs grep -i "jspspy"#Jsp***默认名字find . -type f -name "*.jsp" | xargs grep -i "getParameter"fgrep - R "admin_index.jsp" 20120702.log > log.txt#检查是否有非授权访问管理日志#要进中间件所在日志目录运行命令fgrep - R "and1=1"*.log>log.txtfgrep - R "select "*.log>log.txtfgrep - R "union "*.log>log.txtfgrep - R "../../"*.log >log.txt
fgrep - R "Runtime"*.log >log.txtfgrep - R "passwd"*.log >log.txt#查看是否出现对应的记录fgrep - R "uname -a"*.log>log.txtfgrep - R "id"*.log>log.txtfgrep - R "ifconifg"*.log>log.txtfgrep - R "ls -l"*.log>log.txt#查看是否有shell***
#以root权限执行cat /var/log/secure#查看是否存在非授权的管理信息tail -n 10 /var/log/securelast cat /var/log/wtmpcat /var/log/sulog#查看是否有非授权的su命令cat /var/log/cron#查看计划任务是否正常tail -n 100 ~./bash_history | more查看临时目录是否存在***者***时留下的残余文件ls -la /tmpls -la /var/tmp#如果存在.c .py .sh为后缀的文件或者2进制elf文件。

Apr 17 03:14:56 localhost sshd[11499]: warning: /etc/hosts.deny, line 14: missing ":" separatorApr 17 03:15:01 localhost sshd[11499]: Address 46.214.146.198 maps to 46-214-146-198.next-gen.ro, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT!Apr 17 03:15:01 localhost sshd[11499]: Invalid user ubnt from 46.214.146.198Apr 17 03:15:01 localhost sshd[11500]: input_userauth_request: invalid user ubntApr 17 03:15:01 localhost sshd[11499]: pam_unix(sshd:auth): check pass; user unknownApr 17 03:15:01 localhost sshd[11499]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=46.214.146.198 Apr 17 03:15:01 localhost sshd[11499]: pam_succeed_if(sshd:auth): error retrieving information about user ubntApr 17 03:15:03 localhost sshd[11499]: Failed password for invalid user ubnt from 46.214.146.198 port 34989 ssh2Apr 17 03:15:03 localhost sshd[11500]: Connection closed by 46.214.146.198


就是他了，查看历史记录
日志发现 Invalid user ubnt from 46.214.146.198历史记录和相关访问日志已经被删除，痕迹清除。发现检查没有异常

打开 vi /etc/motd 发现
查找不出后门也找不到相关命令，感觉思路受损，晕头转向。最后查找下单天的web访问日志和相关ip访问
发现一条命令让我好奇，

GET /cgi-bin/center.cgi?id=

20

HTTP/

1.1

并且有点异常

发现像目前最流行的bash shell 漏洞，测试一下，果然存在漏洞。env x='() { :;}; echo vulnerable' bash -c "echo this is a test"[root@mall ~]# env x='() { :;}; echo vulnerable' bash -c "echo this is a test"vulnerablethis is a test4.修复升级命令：yum -y install yum-downloadonlyyum -y install bash-4.1.2-33.el6_7.1.x86_64.rpm [root@mall tmp]# yum -y install bash-4.1.2-33.el6_7.1.x86_64.rpm 已加载插件：fastestmirror, security设置安装进程诊断 bash-4.1.2-33.el6_7.1.x86_64.rpm: bash-4.1.2-33.el6_7.1.x86_64bash-4.1.2-33.el6_7.1.x86_64.rpm 将作为 bash-4.1.2-15.el6_4.x86_64 的更新Loading mirror speeds from cached hostfile * base: ftp.sjtu.edu.cn * extras: mirrors.skyshe.cn * updates: ftp.sjtu.edu.cn解决依赖关系--> 执行事务检查---> Package bash.x86_64 0:4.1.2-15.el6_4 will be 升级---> Package bash.x86_64 0:4.1.2-33.el6_7.1 will be an update--> 完成依赖关系计算依赖关系解决======================================================================================================================================================================================总文件大小：3.0 M下载软件包：运行 rpm_check_debug 执行事务测试事务测试成功执行事务 正在升级 : bash-4.1.2-33.el6_7.1.x86_64 清理 : bash-4.1.2-15.el6_4.x86_64 Verifying : bash-4.1.2-33.el6_7.1.x86_64 Verifying : bash-4.1.2-15.el6_4.x86_64 更新完毕: bash.x86_64 0:4.1.2-33.el6_7.1 完毕！
重新测试[root@mall tmp]# env x='() { :;}; echo vulnerable' bash -c "echo this is a test"this is a test5.完成后做了如下措施：1.修改了系统账号密码。2.修改了sshd端口为2220。3.修改nginx用户nologin。4.发现系统服务器存在bash严重漏洞 破壳漏洞(Shellshock)并修复。5.更新完成后后面没有发现被***或者服务器自动宕机现象。6.漏洞被利用过程：我发送GET请求–>目标服务器cgi路径目标服务器解析这个get请求，碰到UserAgent后面的参数，Bash解释器就执行了后面的命令7.Shellshock介绍Shellshock，又称Bashdoor，是在Unix中广泛使用的Bash shell中的一个安全漏洞，首次于2014年9月24日公开。许多互联网守护进程，如网页服务器，使用bash来处理某些命令，从而允许***者在易受***的Bash版本上执行任意代码。这可使***者在未授权的情况下访问计算机系统。8.相关文章：http://zone.wooyun.org/content/15392http://www.freebuf.com/articles/system/50707.htmlhttp://bobao.360.cn/news/detail/408.html
http://hacker-falcon.blog.163.com/blog/static/23979900320148294531576/ ps：原文链接http://chenhao6.blog.51cto.com/6228054/1768695