When to use Selenium and Beautiful Soup for scrapping in python (Usecase Specific)

Question

Selenium and Beautiful Soup python libraries are for web scraping, but which one to choose in different scenarios. In other words, which is better in performance, benefits over each other ??

Answer 1

Selenium provides browser automation, while bs4 (beautifulsoup) is just a parser (but a very good one!). You can think of combining the two, if what you are looking for are ultimately HTML or XML elements (that's what bs4 is good at) and need to navigate through repetitively similar websites (that's what Selenium does well).

But if you need a fully-fledged, spider-like webcrawler, there are other, more powerful frameworks for that - including scrapy:
https://docs.scrapy.org