Is this possible

Question

So i was just curious if it's possible to write a code that would look at data on a website that needs to be inputted and then maybe output on something like a google doc whats been done and what hasn't if it is i figured python would be the language to do it in

Answer 1

I hear that Scrapy is the module for scraping data.
-- not linking because limited experience: your search selection may be better than mine

Google publishes an API for Sheets (not apparent for Docs?) so as long as you can send the right network requests the language doesn't matter as much, as long as you're okay with Sheets.
-- want the link?

The 'requests' library for Python would help here (not available on SoloLearn, I had to use sockets).
-- not linking my code because why be complex when you could just use requests

Answer 2

This reads from / and updates / a Google Sheet using the 'requests' import:
https://code.sololearn.com/c37XERvwb2Cz/?ref=app

The spreadsheet it's accessing (Google page template, drop in spreadsheet, save -- zero coding):
https://sites.google.com/site/sllkspersist/

Notes:
The Sheet contains a smallish macro to add some flexibility (like an auto timestamp); it communicates using "json".
"JavaScript Object Notation" is so common "there's a function for that" (no hand-tooling required)

To see some json, you can look at the output of the SoloLearn version:
https://code.sololearn.com/cJTcw0o5WgaZ/?ref=app
but please ignore the code (customized and fragile, for SoloLearn).

Answer 3

Just to clarify: I don't see a Docs programmer interface, just Sheets.
Also, "best practices" say not to use spreadsheets as databases ... but lots of people do it anyway.

If you're already manually entering data in both places then there's a better way than scraping (in fact, just skip that for now). I'm going to go get some facts straight about Google Forms, static sites, etc. think and come back. Hopefully others have ideas too :)

Answer 4

basically the google doc is supposed to be a check sheet bassicly an x in a column saying datas been input on the main site in that section but people are having trouble keeping up with the check sheet because its a lot so i wanted to automate that whole thing so there's no room for error

Answer 5

Okay, Google Forms doesn't look quite right here; I was hoping it would support mass data entry but it's geared towards personal surveys.

Some questions:
A) The web page -- are you typing directly into it, like this (my point is, the data is in the page ... not from a database):
https://code.sololearn.com/Wjt00NvvIH5H/?ref=app

B) Do you know what webserver software you have (or maybe you just have a hosting provider)?
C) If you're using a database already somewhere -- any idea what you might have?
D) Can you / are you willing to use a commercial service like Amazon Web Services or Google Cloud?
(You could try AWS for free, 1 year, and get a full stack environment -- Google has deals too)

I don't want to complicate this for you -- I'm just trying to assess what you have so we (and anyone who might jump in) can look at the language more realistically. Python might be able to make request to update a Google record, but so can many existing webservers / it may be less disruptive.

Answer 6

Whether or not you admin the destination site, you might be able to craft a POST to the site from Python, take the server response, then turn right around and record a success in Sheets.

Answer 7

well i work with kids and we have to input tons of data on a site like test scores and whatnot and we've been manually keeping track on a google doc so my goal was to write a code that would check all the data slots and then check on the google doc whether its been done or not im an amateur at coding to say the least but if there's a way to make it work then it'll be an awesome learning experience would scrappy help do that?

Answer 8

also would python be the optimal language for a job like this

Answer 9

also thank you for taking the time to help me out

Answer 10

So the site we use is a third party site where we have tons of sections to input like test scores permission slip info and other things I'm like 100% amature at this moment this isn't even my job i just thought it would be a cool project so i don't have any of those resources you mentioned however i could try a few of the free ones out

Answer 11

also it's close to the code you wrote except really i just wanted to do like a check mark like say we did test 1 but haven't done test 2 on little bob here ill look at the spreadsheet and see he has a check mark on test 1 but not test 2 so i know i need to give it to him