Get context from an external Webpage

Question

I want visulise data from an external webpage(not my own) for example: www.example.com. i need just the contet of 1 div container there but dont know how?

Answer 1

jack here the scrape using your code and exec function with regex

https://code.sololearn.com/WDFKf0oDY1Km/?ref=app

Answer 2

jack To be clear... This is only possible for code not running within a modern browser.

That is... unless the code is being applied to a website on the same origin or there is a cross origin resource sharing (CORS) policy on the target domain for the current domain.

Answer 3

jack
My apologies. Ajax doesn't work in a way to extract data from another web pages rather it's useful to extract data from an api.
You can't get website data just using jQuery get or ajax method coz CORS won't let you do that.

Your try seems well coz we must use cross-domain tools such as anyorigin, allorigins or any other similar to scrape a website.

Answer 4

Use regex to further extract the data

Answer 5

jack You can implement this via any number of server side solutions.

You can search for examples on any of the following:
(with a few options that come to mind)

"HTTP Request PHP"
- cURL
- fopen
- file_get_contents

"HTTP Request .NET"
- WebRequest

"HTTP Request NodeJS"
- request()
- https.get()
- SuperAgent

You get the point.

Note... The response body is a string that will need to be parsed. You can either do this using various string search methods or load via HTML Parsing Library.

You'll need to search for available parsers for your specific language.

Answer 6

jack
Possible? Absolutely!

However, depending on the authentication strategy used on a given website, it can get quite involved.

Most commercial websites implement thorough integration tests that would include the automatation for logging in and doing everything else you would want to simulate a user's experience on a website.

Answer 7

I don't think you can access the exact visual data from a website but sure you can extract data from a website or a link by using jQuery Ajax.

Answer 8

i dont need visuals just a part of a divcontainer

Answer 9

If you just need written data then I think you should go with Ajax.

Answer 10

@jack There is notthing to extract from these elements.
What do you intend to do?

Answer 11

There is no hidden input. Checked from devtools

Answer 12

No, not possible, don't waste your time.

Answer 13

And thanks for your great overview

Answer 14

its possible with a little trick but not for the url i want it. here is what i got: https://code.sololearn.com/WyYP43VP3nRZ/?ref=app

Answer 15

jack Show the external website link and the div container you want, I try to scrape it if possible.

Answer 16

Cool thanks Calvin

Answer 17

Can you show me an example

Answer 18

Thx

Answer 19

Cool wait could take a bit I'm at the dinner;)

Answer 20

The domain called:"www.swcombine.com"
Need there :
<input name="log" type="hidden"..
<input name="hand" id="hand"....
<input name="pass" id="pass"....