{"id":45249,"date":"2024-03-24T21:27:28","date_gmt":"2024-03-24T21:27:28","guid":{"rendered":"https:\/\/www.devopsschool.com\/blog\/?p=45249"},"modified":"2024-09-09T04:49:06","modified_gmt":"2024-09-09T04:49:06","slug":"devsecops-scanning-remote-hosts-on-the-internet-for-security-vulnerabilities","status":"publish","type":"post","link":"https:\/\/www.devopsschool.com\/blog\/devsecops-scanning-remote-hosts-on-the-internet-for-security-vulnerabilities\/","title":{"rendered":"DevSecOps: Your AWS Abuse Report &#8211; scanning remote hosts on the internet for security vulnerabilities"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">Incidents<\/h2>\n\n\n\n<p>It has been implicated in activity which resembles scanning remote hosts on the internet for security vulnerabilities. Activity of this nature is forbidden in the AWS Acceptable Use Policy (<a href=\"https:\/\/aws.amazon.com\/aup\/\" target=\"_blank\" rel=\"noreferrer noopener\">https:\/\/aws.amazon.com\/aup\/<\/a>). We&#8217;ve included the original report below for your review.<\/p>\n\n\n\n<p>This is a notification of unauthorized uses of systems or networks.<\/p>\n\n\n\n<p>We have observed IP addresses from your network probing my servers for TCP open ports. Due to their dubious behavior, they are suspected to be compromised botnet computers.<\/p>\n\n\n\n<p>If you regularly collect IP traffic information of your network, you will see the IPs listed connected to various TCP ports of my server at the time logged, and I suspect that they also connected to TCP ports of many other IPs.<\/p>\n\n\n\n<p>If a Linux system was at the attacker&#8217;s IP, you might want to use the command &#8220;netstat -ntp&#8221; to list its active network connections. If there is still some suspicious connection, find out what PID\/program\/user ID they belong to as you might find something to help you solve this problem.<\/p>\n\n\n\n<p>In addition to the above, kindly notify the victims (owners of those botnet computers) as this will assist them in taking the appropriate action to clean their computers. Once this action is completed, not only will it prevent severe incidents such as data leakage and DDos but, it will also stand off botnets from taking up your network bandwidth.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Solution<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Step &#8211; 1 &#8211; Investigate the Process<\/h3>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1012\" height=\"421\" src=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2024\/03\/image-787.png\" alt=\"\" class=\"wp-image-47081\" srcset=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2024\/03\/image-787.png 1012w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2024\/03\/image-787-300x125.png 300w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2024\/03\/image-787-768x319.png 768w\" sizes=\"auto, (max-width: 1012px) 100vw, 1012px\" \/><\/figure>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n<pre class=\"wp-block-code\"><span><code class=\"hljs\">$ netstat -ntp\n$ ps -p 207082 -o command=\n$ ls -l \/proc\/207082\/exe\n<\/code><\/span><\/pre>\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"903\" height=\"410\" src=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2024\/03\/image-751.png\" alt=\"\" class=\"wp-image-45250\" srcset=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2024\/03\/image-751.png 903w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2024\/03\/image-751-300x136.png 300w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2024\/03\/image-751-768x349.png 768w\" sizes=\"auto, (max-width: 903px) 100vw, 903px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Step 2 &#8211; Check for Service Units or Cron Jobs<\/h2>\n\n\n\n<p>The process might be started by a systemd service or a cron job:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Systemd service:<\/strong> Check for custom systemd service files in <code>\/etc\/systemd\/system\/<\/code> or <code>\/lib\/systemd\/system\/<\/code> and use <code>systemctl<\/code> to list all services to see if any custom or suspicious service is running.<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"893\" height=\"516\" src=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2024\/03\/image-752.png\" alt=\"\" class=\"wp-image-45251\" srcset=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2024\/03\/image-752.png 893w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2024\/03\/image-752-300x173.png 300w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2024\/03\/image-752-768x444.png 768w\" sizes=\"auto, (max-width: 893px) 100vw, 893px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Step 3 &#8211; Stop and Disable the Process<\/h2>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n<pre class=\"wp-block-code\"><span><code class=\"hljs\">If the process is managed by a systemd service, disable and stop it:\n\n\n$ sudo systemctl stop servicename.service\n$ sudo systemctl disable servicename.service\n\nIf it's a cron job, remove or comment out the line in the crontab.<\/code><\/span><\/pre>\n\n\n<h2 class=\"wp-block-heading\">Step 4 &#8211; Remove Executable and Clean Up<\/h2>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n<pre class=\"wp-block-code\"><span><code class=\"hljs\">After identifying the executable, consider carefully removing it if it's confirmed to be malicious or not needed:\n\n$ sudo rm \/path\/to\/executable\n\nBe cautious, as removing system files or legitimate processes can harm your system.<\/code><\/span><\/pre>\n\n\n<h2 class=\"wp-block-heading\">5. Further Security Measures<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Rootkit Check:<\/strong> Since this process appears suspicious, it&#8217;s a good idea to run a tool like <code>rkhunter<\/code> or <code>chkrootkit<\/code> to check for rootkits.<\/li>\n\n\n\n<li><strong>Malware Scan:<\/strong> Use a Linux-based antivirus tool to scan your system for malware.<\/li>\n\n\n\n<li><strong>Audit Logs:<\/strong> Check <code>\/var\/log\/auth.log<\/code>, <code>\/var\/log\/syslog<\/code>, or relevant system logs for any unusual activity, especially around the times the process was started.<\/li>\n\n\n\n<li><strong>System Updates:<\/strong> Ensure your system and all applications are up-to-date with the latest security patches.<\/li>\n\n\n\n<li><strong>Firewall Review:<\/strong> Verify your <code>iptables<\/code> rules and ensure no unwanted rules are allowing traffic through.<\/li>\n\n\n\n<li><strong>Network Monitoring:<\/strong> Monitor outbound and inbound connections for further suspicious activities.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">CULPRIT#1 <\/h2>\n\n\n\n<p>I see the process was getting created by daemon user and through crontab<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"797\" height=\"131\" src=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2024\/03\/image-753.png\" alt=\"\" class=\"wp-image-45254\" srcset=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2024\/03\/image-753.png 797w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2024\/03\/image-753-300x49.png 300w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2024\/03\/image-753-768x126.png 768w\" sizes=\"auto, (max-width: 797px) 100vw, 797px\" \/><\/figure>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-1\" data-shcb-language-name=\"JavaScript\" data-shcb-language-slug=\"javascript\"><span><code class=\"hljs language-javascript\">file \/<span class=\"hljs-keyword\">var<\/span>\/tmp\/bash18\nfile \/<span class=\"hljs-keyword\">var<\/span>\/tmp\/sh18\nfile \/<span class=\"hljs-keyword\">var<\/span>\/tmp\/init18\n\nfile \/<span class=\"hljs-keyword\">var<\/span>\/bash18\nfile \/<span class=\"hljs-keyword\">var<\/span>\/sh18\nfile \/<span class=\"hljs-keyword\">var<\/span>\/init18\n\nfile \/<span class=\"hljs-keyword\">var<\/span>\/lock\/bash18\nfile \/<span class=\"hljs-keyword\">var<\/span>\/lock\/sh18\nfile \/<span class=\"hljs-keyword\">var<\/span>\/lock\/init18<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-1\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">JavaScript<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">javascript<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Investigation Further -&gt; How this code was added in Crontab?<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">I am wondering how these executable such as bash18, sh18 and init18 is set in user daemon crontab using XAMPP. Any clue?<\/h3>\n\n\n\n<p>To search through your JavaScript (<code>.js<\/code>) and PHP (<code>.php<\/code>) codebase for patterns that might indicate code responsible for adding cron jobs, you can use the <code>grep<\/code> command in Linux.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-2\" data-shcb-language-name=\"PHP\" data-shcb-language-slug=\"php\"><span><code class=\"hljs language-php\">Basic grep Search\nThis searches all PHP <span class=\"hljs-keyword\">and<\/span> JS files <span class=\"hljs-keyword\">for<\/span> the word <span class=\"hljs-string\">\"crontab\"<\/span>:\n\n$ grep -r --<span class=\"hljs-keyword\">include<\/span>=\\*.{php,js} <span class=\"hljs-string\">\"crontab\"<\/span> .\n\nSearch <span class=\"hljs-keyword\">for<\/span> Shell Execution Functions\n\nMany malicious scripts <span class=\"hljs-keyword\">use<\/span> <span class=\"hljs-title\">PHP<\/span>'<span class=\"hljs-title\">s<\/span> <span class=\"hljs-title\">shell<\/span> <span class=\"hljs-title\">execution<\/span> <span class=\"hljs-title\">capabilities<\/span> (<span class=\"hljs-title\">exec<\/span>, <span class=\"hljs-title\">shell_exec<\/span>, <span class=\"hljs-title\">system<\/span>, <span class=\"hljs-title\">passthru<\/span>, <span class=\"hljs-title\">and<\/span> <span class=\"hljs-title\">backticks<\/span>) <span class=\"hljs-title\">to<\/span> <span class=\"hljs-title\">run<\/span> <span class=\"hljs-title\">system<\/span> <span class=\"hljs-title\">commands<\/span>.\n\n\n$ <span class=\"hljs-title\">grep<\/span> -<span class=\"hljs-title\">r<\/span> --<span class=\"hljs-title\">include<\/span>=\\*.<span class=\"hljs-title\">php<\/span> \"\\(<span class=\"hljs-title\">exec<\/span>\\|<span class=\"hljs-title\">shell_exec<\/span>\\|<span class=\"hljs-title\">system<\/span>\\|<span class=\"hljs-title\">passthru<\/span>\\|`\\)\" .\n\n<span class=\"hljs-title\">This<\/span> <span class=\"hljs-title\">command<\/span> <span class=\"hljs-title\">will<\/span> <span class=\"hljs-title\">search<\/span> <span class=\"hljs-title\">for<\/span> <span class=\"hljs-title\">any<\/span> <span class=\"hljs-title\">of<\/span> <span class=\"hljs-title\">the<\/span> <span class=\"hljs-title\">mentioned<\/span> <span class=\"hljs-title\">functions<\/span> <span class=\"hljs-title\">in<\/span> <span class=\"hljs-title\">PHP<\/span> <span class=\"hljs-title\">files<\/span>. <span class=\"hljs-title\">Note<\/span> <span class=\"hljs-title\">the<\/span> <span class=\"hljs-title\">use<\/span> <span class=\"hljs-title\">of<\/span> <span class=\"hljs-title\">backslashes<\/span> <span class=\"hljs-title\">to<\/span> <span class=\"hljs-title\">escape<\/span> <span class=\"hljs-title\">certain<\/span> <span class=\"hljs-title\">characters<\/span> <span class=\"hljs-title\">and<\/span> <span class=\"hljs-title\">the<\/span> <span class=\"hljs-title\">use<\/span> <span class=\"hljs-title\">of<\/span> \\| <span class=\"hljs-title\">to<\/span> <span class=\"hljs-title\">indicate<\/span> <span class=\"hljs-title\">an<\/span> \"<span class=\"hljs-title\">OR<\/span>\" <span class=\"hljs-title\">condition<\/span>.\n\n<span class=\"hljs-title\">Searching<\/span> <span class=\"hljs-title\">for<\/span> <span class=\"hljs-title\">Suspicious<\/span> <span class=\"hljs-title\">Base64<\/span> <span class=\"hljs-title\">Encodings<\/span>\n\n<span class=\"hljs-title\">Malicious<\/span> <span class=\"hljs-title\">code<\/span> <span class=\"hljs-title\">is<\/span> <span class=\"hljs-title\">often<\/span> <span class=\"hljs-title\">encoded<\/span> <span class=\"hljs-title\">in<\/span> <span class=\"hljs-title\">Base64<\/span> <span class=\"hljs-title\">to<\/span> <span class=\"hljs-title\">obfuscate<\/span> <span class=\"hljs-title\">its<\/span> <span class=\"hljs-title\">presence<\/span>. <span class=\"hljs-title\">Searching<\/span> <span class=\"hljs-title\">for<\/span> <span class=\"hljs-title\">Base64<\/span> <span class=\"hljs-title\">encoded<\/span> <span class=\"hljs-title\">strings<\/span> <span class=\"hljs-title\">can<\/span> <span class=\"hljs-title\">sometimes<\/span> <span class=\"hljs-title\">uncover<\/span> <span class=\"hljs-title\">hidden<\/span> <span class=\"hljs-title\">malicious<\/span> <span class=\"hljs-title\">code<\/span>.\n\n\n$ <span class=\"hljs-title\">grep<\/span> -<span class=\"hljs-title\">r<\/span> --<span class=\"hljs-title\">include<\/span>=\\*.{<span class=\"hljs-title\">php<\/span>,<span class=\"hljs-title\">js<\/span>} \"<span class=\"hljs-title\">base64_decode<\/span>\" .<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-2\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">PHP<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">php<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<h2 class=\"wp-block-heading\"><strong>List Crontabs for All Users<\/strong><\/h2>\n\n\n\n<p>To view the crontab entries for all users, you can check the crontab files stored in <code>\/var\/spool\/cron\/crontabs\/<\/code> (for most Linux distributions). Use the following commands:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-3\" data-shcb-language-name=\"JavaScript\" data-shcb-language-slug=\"javascript\"><span><code class=\"hljs language-javascript\"><span class=\"hljs-keyword\">for<\/span> user <span class=\"hljs-keyword\">in<\/span> $(cut -f1 -d: <span class=\"hljs-regexp\">\/etc\/<\/span>passwd); <span class=\"hljs-keyword\">do<\/span>\n  echo <span class=\"hljs-string\">\"Crontab for user: $user\"<\/span>;\n  sudo crontab -l -u $user <span class=\"hljs-number\">2<\/span>&gt;<span class=\"hljs-regexp\">\/dev\/<\/span><span class=\"hljs-literal\">null<\/span>;\ndone<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-3\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">JavaScript<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">javascript<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>","protected":false},"excerpt":{"rendered":"<p>Incidents It has been implicated in activity which resembles scanning remote hosts on the internet for security vulnerabilities. Activity of this nature is forbidden in the AWS Acceptable Use Policy (https:\/\/aws.amazon.com\/aup\/). We&#8217;ve included the original report below for your review. This is a notification of unauthorized uses of systems or networks. We have observed IP&#8230;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","_joinchat":[],"footnotes":""},"categories":[2],"tags":[],"class_list":["post-45249","post","type-post","status-publish","format-standard","hentry","category-uncategorised"],"_links":{"self":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/45249","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/comments?post=45249"}],"version-history":[{"count":10,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/45249\/revisions"}],"predecessor-version":[{"id":47082,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/45249\/revisions\/47082"}],"wp:attachment":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/media?parent=45249"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/categories?post=45249"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/tags?post=45249"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}