Download file from website using web scraping

Question

I want to download a file from the website by web scraping. Can anyone explain how to do this in R with an example?

Answer 1

Hi,

Follow the below steps:

1. Use rvest package to get the href link to download the file.

2. Use download.file(URL,"file.ext") to download the file and store it.

For example, I want to download the CSV file from this 

• Fetch the href link using html_attr

paths_allowed("https://file-examples.com/index.php/text-files-and-archives-download/")
page = read_html("https://file-examples.com/index.php/text-files-and-archives-download/")
links = page %>% html_nodes(".file-link a") %>% html_attr("href")

• Use the href link within download.file() to download teh file.

download.file(url=links[1],destfile = "dummy.csv")

• The downloaded file will be saved in your working directory.

Hope it helps!