Python and RequestsHTML Web Scraping Dynamic Content from JavaScript applications

YOUR LINK HERE:

http://youtube.com/watch?v=-PmNcIX9En4

In this video, we'll learn how to scrape content that is NOT present in initial page loads, but instead is loaded dynamically by JavaScript. • This is a common problem with scraping the modern web: the initial response contains minimal HTML and a SPA-based JavaScript app (React, Vue, Angular, etc). The data that we want to scrape on the page is therefore not present, but is rendered later via API calls from the SPA application. • We will look at how we can use requests-html to solve this issue in Python when scraping such sites. We'll also look at using this with BeautifulSoup in order to find data on the page. • This video makes use of the following sample website (a React application): • https://react-amazon-bestsellers-book... • 📌 𝗖𝗵𝗮𝗽𝘁𝗲𝗿𝘀: • 00:00 Intro • 02:15 Sending GET request using Python requests library • 04:00 Finding objects with BeautifulSoup • 05:15 Installing requests-html • 06:38 Executing JavaScript on page using requests-html • ☕️ 𝗕𝘂𝘆 𝗺𝗲 𝗮 𝗰𝗼𝗳𝗳𝗲𝗲: • To support the channel and encourage new videos, please consider buying me a coffee here: • https://ko-fi.com/bugbytes • 𝗦𝗼𝗰𝗶𝗮𝗹 𝗠𝗲𝗱𝗶𝗮: • 📖 Blog: https://bugbytes.io/posts/ • 👾 Github: https://github.com/bugbytes-io/ • 🐦 Twitter: / bugbytesio • 📚 𝗙𝘂𝗿𝘁𝗵𝗲𝗿 𝗿𝗲𝗮𝗱𝗶𝗻𝗴 𝗮𝗻𝗱 𝗶𝗻𝗳𝗼𝗿𝗺𝗮𝘁𝗶𝗼𝗻: • requests-html: https://pypi.org/project/requests-html/ • BeautifulSoup: https://beautiful-soup-4.readthedocs.... • requests: https://pypi.org/project/requests/ • Sample website: https://react-amazon-bestsellers-book... • #python #webscraping #datascience

#############################

New on site