0

Where can I learn python scrapping ?

I wanna convert data from pages to csv

7th Nov 2019, 1:39 PM
Abhishek Shukla
Abhishek Shukla - avatar
2 Answers
+ 2
Web scraping is quite common these days. There are a lot of books about this topic, I mean web scraping with Python. But in fact scraping can be quite simple. It is no rocket science and you just need some common sense. The difficulty depends on the web page. If it is just a simple page without any login that contains only generated html, you can do just with regular expressions without any parser or library. It is quick and safe if you know how to do it. You must watch out for any changes in page structure that could break your app. For example if you read data from an html table, it is a good idea to check that the table header is the same etc. If you read more pages in sequence, it is smart to remember pages that you were not able to load and read them again in the end. If it takes some time to load, you can even interrupt the application when your success rate goes down and wait for some time etc. The situation is more complicated if you have a web app that is built with javascript threads from serialized data loaded via separate ajax requests. Then you may need some library that emulates browser which is no big deal either.
7th Nov 2019, 3:13 PM
Michal Straka
Michal Straka - avatar
8th Nov 2019, 5:34 AM
David Ashton
David Ashton - avatar