+ 2

Would you rather build a web crawler with PHP or Python? And why?

well the title is explanatory, so what are the reasons that will influence your choice? python has scrappy, PHP has simple html_dom parse which you can include to your bot script file!

28th Mar 2018, 11:46 AM
Strange
Strange - avatar
7 Respostas
+ 4
It depends a bit one the nuances. PHP is a language mainly used to develop websites (to present them), whereas Python is used for both programming websites and programming other applications. Thus, it depends on how application-like the scraping will be or how much it will be included into the (presentational) website. Python is well-suited for scraping tasks as it has a lot of third party libraries (Scrapy, BeautifulSoup, lxml, request, ...) and also built in libraries for requesting URLs. With PHP you will have a harder life requesting other websites. As said, it is mainly focused on building an own website. Also it of course depends on your skill level in each language. I'd split the application into two parts anyway: presentation logic (your own website) scraping logic (scraping other websites) Both will be connected through a database. The scraping logic collects information and stores it into the database. The presentation logic takes information from the database and displays it to the user as HTML. For the scraping part, you then want to prefer Python over PHP. And for the presentation part, it actually does not matter and is a matter of preference or knowledge.
28th Mar 2018, 12:55 PM
Baraa AB
Baraa AB - avatar
+ 4
Welcome bro!! Good luck 💙
28th Mar 2018, 1:29 PM
Baraa AB
Baraa AB - avatar
28th Mar 2018, 1:39 PM
Bill Zelenko
Bill Zelenko - avatar
+ 1
thanks man
28th Mar 2018, 1:29 PM
Strange
Strange - avatar
0
Great @Baraa thanks. but is there a decent way to connect to MySQL database and query it with Python? cause your suggestion is actually my plan but I know less of how flexible I can use python to scrap then send the info straight to a database! php can handle the website back end and other functionalities l!
28th Mar 2018, 1:02 PM
Strange
Strange - avatar
0
thanks bill
28th Mar 2018, 1:39 PM
Strange
Strange - avatar