Successfully Tracking Down the Individuals Behind a Costly Scraping Operation

As a company, we have always been committed to protecting our clients’ interests and ensuring that their businesses are not harmed by unscrupulous individuals or organizations. Recently, we were approached by one of the biggest online publications in Romania who was experiencing significant financial losses as a result of a scraping operation that was costing them thousands of dollars every month.

Scraping, or the unauthorized use of a website’s content, can have serious consequences for businesses. Not only does it cause financial losses, but it can also damage a company’s reputation because of the assimilation with plagiarized content.

After conducting a thorough investigation, we were able to track down the individuals behind the scraping. In this blog post, we will share the challenges and the steps we took to identify the perpetrators.

The first step in our investigation was to gather as much information as possible about the scraping operation. We worked closely with our client to gather detailed logs of the scraping activity, which allowed us to identify patterns and anomalies in the data.

In total, we had around 30M records of logs that matches certain initial criteria in a time frame of 3 months.

Once we had a clear picture of the scraping operation, we developed a suite of tools designed to extract suspicious activity based on the patterns we set from the big dataset of logs. Fast forward, as a result of our tools, we had a list of 10 possible scrapers.

Then we investigated each one, until we found a website that was hosted on one of the suspicious IPs. One of the biggest challenges was to bypass Cloudflare, as that IP was part of Cloudflare’s network.

Leveraging one of WordPress’ weak points, we managed to get the real IP of that website. Conducted an investigation upon that website’s connection and discovered a whole network of news websites across the world.

Even the companies behind this operation. Even the individuals behind the companies.

We also wrote a comprehensive report for the legal action that our client intended on this activity and finally worked with them to put measures in place to prevent future scraping operations from occurring.

In the end, our efforts were successful, and the client was able to cut future losses in revenue and protect their business from further damage. This case is a clear example of the importance of taking a proactive approach to combating scraping and other forms of online theft.

At Codestage, we are committed to helping our clients defend themselves against these kinds of attacks and ensure that their businesses are not harmed by unscrupulous individuals. If you are experiencing losses due to scraping or other forms of online theft, we encourage you to contact us for assistance.