Hi! I'm trying to parse an online betting web page and retrieve the available competitions and matches along with the odds offered. The problem I have is that as you scroll the page, javascript code is executed and competitions are dynamically populated. Using the method you described in your tutorial (PyQt4 and Beautiful Soup) doesn't work since the on_page_load method is called without the relevant javascript having been executed. I am using Windows 10, Python 3.6, PyQt4 and Beautiful Soup.
Is there another approach that I am missing? Are there other tools I could/should use in order to accomplish my objective?
P.S. Currently I am successfully using Selenium and Chrome to access the web page and scroll it in order to trigger the javascript to execute but parsing the content is taking forever. P.P.S. The web page I am trying to scrape is https://www. superbet. ro/ pariuri-sportive
Thank you!
You must be logged in to post. Please login or register an account.