Automating page scraping with Selenium and Chrome
Sometimes you are interested in information on some page, be it on some social media, or on a news site, or a blog. But you do not have an automated way via API to access this data. I.e. you want to stock the public FB posts on the girl that did not accept your friend request. Or you want to monitor a site for product availabily or price range. Or you might want to get notified if the university web site pushes a new update. Be warned that it might be agains the respective sites policy to make such automated scannings, or you might risk suspending your account or legal proceedings. But lately I have the need to monitor a web site for auto parts. I was interested in the availability of a certain part for my car. I wanted to get notified when there is something new. The vendor does not have API or RSS feeds, but claims that the site is updated the moment there is a new part available and the moment when an old part is sold. Fortunatelly the filter for figuring out the car, model, yea...