{"id":414,"date":"2022-12-15T16:54:31","date_gmt":"2022-12-15T14:54:31","guid":{"rendered":"http:\/\/codestage.com\/?p=414"},"modified":"2022-12-15T17:08:41","modified_gmt":"2022-12-15T15:08:41","slug":"successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation","status":"publish","type":"post","link":"https:\/\/codestage.com\/en\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/","title":{"rendered":"Successfully Tracking Down the Individuals Behind a Costly Scraping Operation"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">As a company, we have always been committed to protecting our clients&#8217; interests and ensuring that their businesses are not harmed by unscrupulous individuals or organizations. Recently, we were approached by one of the biggest online publications in Romania who was experiencing significant financial losses as a result of a scraping operation that was costing them thousands of dollars every month.<\/p>\n\n\n\n<div style=\"height:35px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">Scraping, or the unauthorized use of a website&#8217;s content, can have serious consequences for businesses. Not only does it cause financial losses, but it can also damage a company&#8217;s reputation because of the assimilation with plagiarized content.<\/p>\n\n\n\n<div style=\"height:35px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">After conducting a thorough investigation, we were able to track down the individuals behind the scraping. In this blog post, we will share the challenges and the steps we took to identify the perpetrators.<\/p>\n\n\n\n<div style=\"height:35px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">The first step in our investigation was to gather as much information as possible about the scraping operation. We worked closely with our client to gather detailed logs of the scraping activity, which allowed us to identify patterns and anomalies in the data. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In total, we had around 30M records of logs that matches certain initial criteria in a time frame of 3 months.<\/p>\n\n\n\n<div style=\"height:35px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">Once we had a clear picture of the scraping operation, we developed a suite of tools designed to extract suspicious activity based on the patterns we set from the big dataset of logs. Fast forward, as a result of our tools, we had a list of 10 possible scrapers.<\/p>\n\n\n\n<div style=\"height:35px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">Then we investigated each one, until we found a website that was hosted on one of the suspicious IPs. One of the biggest challenges was to bypass Cloudflare, as that IP was part of Cloudflare&#8217;s network. <\/p>\n\n\n\n<div style=\"height:35px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">Leveraging one of WordPress&#8217; weak points, we managed to get the real IP of that website. Conducted an investigation upon that website&#8217;s connection and discovered a whole network of news websites across the world.<\/p>\n\n\n\n<div style=\"height:35px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">Even the companies behind this operation. Even the individuals behind the companies.<\/p>\n\n\n\n<div style=\"height:35px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">We also wrote a comprehensive report for the legal action that our client intended on this activity and finally worked with them to put measures in place to prevent future scraping operations from occurring.<\/p>\n\n\n\n<div style=\"height:35px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">In the end, our efforts were successful, and the client was able to cut future losses in revenue and protect their business from further damage. This case is a clear example of the importance of taking a proactive approach to combating scraping and other forms of online theft.<\/p>\n\n\n\n<div style=\"height:35px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">At Codestage, we are committed to helping our clients defend themselves against these kinds of attacks and ensure that their businesses are not harmed by unscrupulous individuals. If you are experiencing losses due to scraping or other forms of online theft, we encourage you to contact us for assistance.<\/p>\n\n\n\n<div style=\"height:35px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n","protected":false},"excerpt":{"rendered":"<p>As a company, we have always been committed to protecting our clients&#8217; interests and ensuring that their businesses are not harmed by unscrupulous individuals or organizations. Recently, we were approached by one of the biggest online publications in Romania who was experiencing significant financial losses as a result of a scraping operation that was costing&hellip;<\/p>\n","protected":false},"author":1,"featured_media":415,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[],"class_list":["post-414","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-case-studies"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v22.3 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Successfully Tracking Down the Individuals Behind a Costly Scraping Operation - Codestage<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/codestage.com\/en\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Successfully Tracking Down the Individuals Behind a Costly Scraping Operation - Codestage\" \/>\n<meta property=\"og:description\" content=\"As a company, we have always been committed to protecting our clients&#8217; interests and ensuring that their businesses are not harmed by unscrupulous individuals or organizations. Recently, we were approached by one of the biggest online publications in Romania who was experiencing significant financial losses as a result of a scraping operation that was costing&hellip;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/codestage.com\/en\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/\" \/>\n<meta property=\"og:site_name\" content=\"Codestage\" \/>\n<meta property=\"article:published_time\" content=\"2022-12-15T14:54:31+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-12-15T15:08:41+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/codestage.com\/wp-content\/uploads\/2022\/12\/kevin-ku-w7ZyuGYNpRQ-unsplash-scaled.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"2560\" \/>\n\t<meta property=\"og:image:height\" content=\"1919\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"cds\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"cds\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/\"},\"author\":{\"name\":\"cds\",\"@id\":\"https:\/\/codestage.com\/#\/schema\/person\/bc17aedb776a5e9585009a060779f1ad\"},\"headline\":\"Successfully Tracking Down the Individuals Behind a Costly Scraping Operation\",\"datePublished\":\"2022-12-15T14:54:31+00:00\",\"dateModified\":\"2022-12-15T15:08:41+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/\"},\"wordCount\":470,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/codestage.com\/#organization\"},\"image\":{\"@id\":\"https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/codestage.com\/wp-content\/uploads\/2022\/12\/kevin-ku-w7ZyuGYNpRQ-unsplash-scaled.jpg\",\"articleSection\":[\"Case studies\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/\",\"url\":\"https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/\",\"name\":\"Successfully Tracking Down the Individuals Behind a Costly Scraping Operation - Codestage\",\"isPartOf\":{\"@id\":\"https:\/\/codestage.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/codestage.com\/wp-content\/uploads\/2022\/12\/kevin-ku-w7ZyuGYNpRQ-unsplash-scaled.jpg\",\"datePublished\":\"2022-12-15T14:54:31+00:00\",\"dateModified\":\"2022-12-15T15:08:41+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/#primaryimage\",\"url\":\"https:\/\/codestage.com\/wp-content\/uploads\/2022\/12\/kevin-ku-w7ZyuGYNpRQ-unsplash-scaled.jpg\",\"contentUrl\":\"https:\/\/codestage.com\/wp-content\/uploads\/2022\/12\/kevin-ku-w7ZyuGYNpRQ-unsplash-scaled.jpg\",\"width\":2560,\"height\":1919},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/codestage.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Successfully Tracking Down the Individuals Behind a Costly Scraping Operation\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/codestage.com\/#website\",\"url\":\"https:\/\/codestage.com\/\",\"name\":\"Codestage\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/codestage.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/codestage.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/codestage.com\/#organization\",\"name\":\"Codestage\",\"url\":\"https:\/\/codestage.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/codestage.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/codestage.com\/wp-content\/uploads\/2022\/12\/Codestage-inline.png\",\"contentUrl\":\"https:\/\/codestage.com\/wp-content\/uploads\/2022\/12\/Codestage-inline.png\",\"width\":1011,\"height\":199,\"caption\":\"Codestage\"},\"image\":{\"@id\":\"https:\/\/codestage.com\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/codestage.com\/#\/schema\/person\/bc17aedb776a5e9585009a060779f1ad\",\"name\":\"cds\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/codestage.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/07d90f1813b422d4d347d3d2ec15008ee7fcaa9e99b4758e9665493c49edb581?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/07d90f1813b422d4d347d3d2ec15008ee7fcaa9e99b4758e9665493c49edb581?s=96&d=mm&r=g\",\"caption\":\"cds\"},\"sameAs\":[\"http:\/\/localhost\/codestage-wp\"],\"url\":\"https:\/\/codestage.com\/en\/author\/cds\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Successfully Tracking Down the Individuals Behind a Costly Scraping Operation - Codestage","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/codestage.com\/en\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/","og_locale":"en_US","og_type":"article","og_title":"Successfully Tracking Down the Individuals Behind a Costly Scraping Operation - Codestage","og_description":"As a company, we have always been committed to protecting our clients&#8217; interests and ensuring that their businesses are not harmed by unscrupulous individuals or organizations. Recently, we were approached by one of the biggest online publications in Romania who was experiencing significant financial losses as a result of a scraping operation that was costing&hellip;","og_url":"https:\/\/codestage.com\/en\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/","og_site_name":"Codestage","article_published_time":"2022-12-15T14:54:31+00:00","article_modified_time":"2022-12-15T15:08:41+00:00","og_image":[{"width":2560,"height":1919,"url":"https:\/\/codestage.com\/wp-content\/uploads\/2022\/12\/kevin-ku-w7ZyuGYNpRQ-unsplash-scaled.jpg","type":"image\/jpeg"}],"author":"cds","twitter_card":"summary_large_image","twitter_misc":{"Written by":"cds","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/#article","isPartOf":{"@id":"https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/"},"author":{"name":"cds","@id":"https:\/\/codestage.com\/#\/schema\/person\/bc17aedb776a5e9585009a060779f1ad"},"headline":"Successfully Tracking Down the Individuals Behind a Costly Scraping Operation","datePublished":"2022-12-15T14:54:31+00:00","dateModified":"2022-12-15T15:08:41+00:00","mainEntityOfPage":{"@id":"https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/"},"wordCount":470,"commentCount":0,"publisher":{"@id":"https:\/\/codestage.com\/#organization"},"image":{"@id":"https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/#primaryimage"},"thumbnailUrl":"https:\/\/codestage.com\/wp-content\/uploads\/2022\/12\/kevin-ku-w7ZyuGYNpRQ-unsplash-scaled.jpg","articleSection":["Case studies"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/","url":"https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/","name":"Successfully Tracking Down the Individuals Behind a Costly Scraping Operation - Codestage","isPartOf":{"@id":"https:\/\/codestage.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/#primaryimage"},"image":{"@id":"https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/#primaryimage"},"thumbnailUrl":"https:\/\/codestage.com\/wp-content\/uploads\/2022\/12\/kevin-ku-w7ZyuGYNpRQ-unsplash-scaled.jpg","datePublished":"2022-12-15T14:54:31+00:00","dateModified":"2022-12-15T15:08:41+00:00","breadcrumb":{"@id":"https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/#primaryimage","url":"https:\/\/codestage.com\/wp-content\/uploads\/2022\/12\/kevin-ku-w7ZyuGYNpRQ-unsplash-scaled.jpg","contentUrl":"https:\/\/codestage.com\/wp-content\/uploads\/2022\/12\/kevin-ku-w7ZyuGYNpRQ-unsplash-scaled.jpg","width":2560,"height":1919},{"@type":"BreadcrumbList","@id":"https:\/\/codestage.com\/successfully-tracking-down-the-individuals-behind-a-costly-scraping-operation\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/codestage.com\/"},{"@type":"ListItem","position":2,"name":"Successfully Tracking Down the Individuals Behind a Costly Scraping Operation"}]},{"@type":"WebSite","@id":"https:\/\/codestage.com\/#website","url":"https:\/\/codestage.com\/","name":"Codestage","description":"","publisher":{"@id":"https:\/\/codestage.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/codestage.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/codestage.com\/#organization","name":"Codestage","url":"https:\/\/codestage.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/codestage.com\/#\/schema\/logo\/image\/","url":"https:\/\/codestage.com\/wp-content\/uploads\/2022\/12\/Codestage-inline.png","contentUrl":"https:\/\/codestage.com\/wp-content\/uploads\/2022\/12\/Codestage-inline.png","width":1011,"height":199,"caption":"Codestage"},"image":{"@id":"https:\/\/codestage.com\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/codestage.com\/#\/schema\/person\/bc17aedb776a5e9585009a060779f1ad","name":"cds","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/codestage.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/07d90f1813b422d4d347d3d2ec15008ee7fcaa9e99b4758e9665493c49edb581?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/07d90f1813b422d4d347d3d2ec15008ee7fcaa9e99b4758e9665493c49edb581?s=96&d=mm&r=g","caption":"cds"},"sameAs":["http:\/\/localhost\/codestage-wp"],"url":"https:\/\/codestage.com\/en\/author\/cds\/"}]}},"_links":{"self":[{"href":"https:\/\/codestage.com\/en\/wp-json\/wp\/v2\/posts\/414","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/codestage.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/codestage.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/codestage.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/codestage.com\/en\/wp-json\/wp\/v2\/comments?post=414"}],"version-history":[{"count":6,"href":"https:\/\/codestage.com\/en\/wp-json\/wp\/v2\/posts\/414\/revisions"}],"predecessor-version":[{"id":422,"href":"https:\/\/codestage.com\/en\/wp-json\/wp\/v2\/posts\/414\/revisions\/422"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/codestage.com\/en\/wp-json\/wp\/v2\/media\/415"}],"wp:attachment":[{"href":"https:\/\/codestage.com\/en\/wp-json\/wp\/v2\/media?parent=414"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/codestage.com\/en\/wp-json\/wp\/v2\/categories?post=414"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/codestage.com\/en\/wp-json\/wp\/v2\/tags?post=414"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}