Please help. How to: Create my first bot to fetch counts of medical literature w/ python, r, or another data science lang

Question

https://marcobonzanini.com/2015/01/12/searching-pubmed-with-JUMP_LINK__&&__python__&&__JUMP_LINK/

I want to know exhaustively how many unique books and articles exist on a particular topic and whether within the results how many use certain words. how would I begin to understand how to do something like this using python or a web scraping bot? I am familiar with urllib module in theory, but haven't really seen how it really practically works. curious what the community might have to say.

yes, I have googled, it's just a lot to sift through and hope someone here could leave me a few bread crumb trails to the jackpot.

thanks!

Answer 1

I normally use beautifulsoup for web scraping which is another python module like urllib but it is very popular and so has lots if tutorials and documentation and is easy to use

https://www.crummy.com/software/BeautifulSoup/bs4/doc/

Answer 2

yeah its free just install it in python using Pip

Answer 3

cool, thanks! 🤔😊

Answer 4

thanks! I'll check it out. I vaguely remember seeing that before. it's free right?