Scraping Rule and Laws

Rules of Scraping

The first rule of scraping the web is do not harm the website. The second rule of web crawling is do NOT harm the website

Laws of Scraping

Disclaimer: This is not legal advice. Talk to your lawyer for that!

Here is what common sense tells us:

1. Obey Robots.txt
2. Follow Terms of service (also known as terms of use and terms and conditions