Bryan5
Posts: 1
Joined: Thu Jul 15, 2021 5:12 pm

How can I import data from a website to my program, or file

Thu Jul 15, 2021 5:47 pm

Hi all,

I want to import data from a web site (this is refresh every day), but I need only part of it, then read the necessary data from there. I tried to block it (array), but it did not work, and I've been looking for a lot of solutions, but I did not find it really good. How can I solve this?

Thanks so much if anyone solves my problem.

knute
Posts: 702
Joined: Thu Oct 23, 2014 12:14 am
Location: Texas
Contact: Website

Re: How can I import data from a website to my program, or file

Fri Jul 16, 2021 4:51 pm

How about an example URL. It is really easy to get the contents of a web page or separate pieces like images.

pidd
Posts: 2336
Joined: Fri May 29, 2020 8:29 pm
Location: Wirral, UK
Contact: Website

Re: How can I import data from a website to my program, or file

Fri Jul 16, 2021 5:19 pm

Use wget or similar to grab the whole html web page then filter it down to what you want with awk.

I use php but its the same principle, the html tags make it easy to find/get the parts you want.

User avatar
neilgl
Posts: 3360
Joined: Sun Jan 26, 2014 8:36 pm
Location: Near The National Museum of Computing

Re: How can I import data from a website to my program, or file

Fri Jul 16, 2021 8:46 pm

Have a look at BeautifulSoup with python.

Heater
Posts: 18365
Joined: Tue Jul 17, 2012 3:02 pm

Re: How can I import data from a website to my program, or file

Sat Jul 17, 2021 12:13 pm

If you really must use Java a quick google search for something like "scanning web site java" or "crawling web page java" will find you many ways to do it.

For example: https://www.scrapingbee.com/blog/introd ... with-java/
Memory in C++ is a leaky abstraction .

Katzaa
Posts: 1
Joined: Sat Jul 31, 2021 4:28 pm

Re: How can I import data from a website to my program, or file

Sat Jul 31, 2021 4:35 pm

I think there are plenty, just a quick search on Google should do the trick. I've always been of the opinion that it's better to see for yourself what works best for your personal needs.

Return to “Java”