Python weather script


15 posts
by canibalimao » Mon Dec 17, 2012 11:59 am
Hi guys,

I'm starting now using python and I want to create a weather script. Then I'll put that script working with a 16x2 LCD, but that will be easy.
Moving on... I picked up a python program already made by raspberrypi-spy that used to work with Google API. Since google closed that API support I'd seached for another one and found Foreca.
I've managed to put it working with the script, but the problem is that the script is only made to show the temperature and I also want to show the weather condition (Overcast, Clear, Rain, etc) and I can't put this working...

So, the script I've picked up is here. Is the weather_lcd.py file.

And my code is this one:

Code: Select all
 # Getting weather stats
import urllib2
import re

weatherURL1 = "http://aaltohtml5name.foreca.com/aaltohtml5-oct12a/data.php?l=102735943&products=cc"

 
response1 = urllib2.urlopen(weatherURL1)
html1 = response1.read()

 
  #print html
r_temp = re.compile("tf=\"[0-9]{1,3}\"")
r_cond = re.compile("sT=\"[A-Z]{1,20}\"")

 
temp_search = r_temp.search(html1)
cond_search = r_cond.search(html1)
 
if temp_search:
    temp1 = temp_search.group(0)
    temp1 = re.sub("\D", "", temp1)
    print temp1+"ºC"
else:
    print "no temp found"

if cond_search:
    cond = cond_search.group(0)
    cond = re.sub("\S", "", cond)
    print cond
else:
    print "no cond found"


Could anyone help me?

By the way (only one more thing), after this I'll want to have it showing a 3 day forecast from here but I also don't know how to search the info from different lines :(


Sorry for the very very long post. I only want to give you the most info I can.
Posts: 528
Joined: Tue Aug 07, 2012 8:48 pm
Location: Portugal
by gordon77 » Mon Dec 17, 2012 2:11 pm
Here's a simple program that reads the data you want if it helps.

I am not sure what you mean by reading more lines, it only seems to bring one back from the site.

# Getting weather stats
import urllib2
import re

weatherURL1 = "http://aaltohtml5name.foreca.com/aaltohtml5-oct12a/data.php?l=102735943&products=cc"

response1 = urllib2.urlopen(weatherURL1)
html1 = response1.read()

print html1

a = " "
x = 0
while a != " t=" and x < 250:
x +=1
a = html1[x:x+3]
t = html1[x+4:x+6]

if x < 250:
print t+"C"
else :
print "no temp found"

a = " "
z = 0
b = " "
v = " "
while a != " sT=" and z < 250:
z +=1
a = html1[z:z+4]
x = z
if z < 250:
while b != "/" :
x +=1
b = html1[x+4:x+5]
v = v+b
c = len(v)
s = v[1:(c-3)]
if z < 250:
print s
else :
print "no cond found"
Posts: 691
Joined: Sun Aug 05, 2012 3:12 pm
by canibalimao » Mon Dec 17, 2012 3:15 pm
Thanks for the help. I want to keep it reading like the script I postd to keep it "simple", but I think that's not possible using those functions.

What I want with "reading more lines" is to read this: http://aaltohtml5name.foreca.com/aaltoh ... s=cc,daily

I want to read the first, the third and the fourth lines (weather now, tomorow and day after tomorow) but I don't know how.
Posts: 528
Joined: Tue Aug 07, 2012 8:48 pm
Location: Portugal
by gordon77 » Wed Dec 19, 2012 9:24 am
Hopefully this fixes it.

Gordon 77

Code: Select all
 # Getting weather stats
import urllib2
import re

weatherURL1 = "http://aaltohtml5name.foreca.com/aaltohtml5-oct12a/data.php?l=102735943&products=cc"

 
response1 = urllib2.urlopen(weatherURL1)
html1 = response1.read()

 
print html1
r_temp = re.compile("tf=\"[0-9]{1,3}\"")
r_cond = re.compile("sT=\"[A-z]{1,20}\"")
 
temp_search = r_temp.search(html1)
cond_search = r_cond.search(html1)
 
if temp_search:
    temp1 = temp_search.group(0)
    temp1 = re.sub("\D", "", temp1)
    print temp1+"ºC"
else:
    print "no temp found"

if cond_search:
    cond = cond_search.group(0)
    cond = re.sub("\W", "", cond)[2:len(cond)]
    print cond
else:
    print "no cond found"
Posts: 691
Joined: Sun Aug 05, 2012 3:12 pm
by canibalimao » Wed Dec 19, 2012 9:37 am
Thank you very much :D It worked really well!

Now I'll just try to read all the lines from here :mrgreen:

Thanks again!
Posts: 528
Joined: Tue Aug 07, 2012 8:48 pm
Location: Portugal
by gordon77 » Wed Dec 19, 2012 9:50 am
I can't get re.search etc to extract the lines (not enough experience!).

If its any use this bit of code will extract them into an array arr[(w,1)] where you can read them from, just change the value of w.

Code: Select all
# Getting weather stats
import urllib2

weatherURL1 = "http://aaltohtml5name.foreca.com/aaltohtml5-oct12a/data.php?l=102735943&products=cc,daily"
response1 = urllib2.urlopen(weatherURL1)
html1 = response1.read()
 
#print html1
w = 1
arr = {}
a = ""
x=0
while a != "</w":
    while a != "<fc" and a != "</w":
        a = html1[x:x+3]
        x +=1
    x +=1
    arr[(w,1)] = a
    while a != " />" and a != "</w":
        a = html1[x:x+3]
        arr[(w,1)] = arr[(w,1)] + a[1:2]
        x +=1
    arr[(w,1)] = arr[(w,1)] + ">"
    print arr[(w,1)]
    w +=1
Posts: 691
Joined: Sun Aug 05, 2012 3:12 pm
by canibalimao » Wed Dec 19, 2012 10:03 am
If I have more experience I can search in that array and extract right the bytes I want.
I want the "tn =" and the "tx =" value present in that URL's 3rd line but that's not easy...

I will keep the search and If I found something I'll post it here.

Thank you very muck, really :D
Posts: 528
Joined: Tue Aug 07, 2012 8:48 pm
Location: Portugal
by gordon77 » Wed Dec 19, 2012 10:29 am
This will extract the first 3 line values of tx and tn

Code: Select all
# Getting weather stats
import urllib2
import re

weatherURL1 = "http://aaltohtml5name.foreca.com/aaltohtml5-oct12a/data.php?l=102735943&products=cc,daily"
response1 = urllib2.urlopen(weatherURL1)
html1 = response1.read()
 
#print html1
w = 1
arr = {}
a = ""
x=0
while a != "</w":
    while a != "<fc" and a != "</w":
        a = html1[x:x+3]
        x +=1
    x +=1
    arr[(w,1)] = a
    while a != " />" and a != "</w":
        a = html1[x:x+3]
        arr[(w,1)] = arr[(w,1)] + a[1:2]
        x +=1
    arr[(w,1)] = arr[(w,1)] + ">"
    # print arr[(w,1)]
    w +=1

y=1
while y < 4:
    html1 = arr[(y,1)]
    #print html1
    r_temp1 = re.compile("tx=\"[0-9]{1,3}\"")
    r_temp2 = re.compile("tn=\"[0-9]{1,3}\"")
 
    temp1_search = r_temp1.search(html1)
    temp2_search = r_temp2.search(html1)
 
    if temp1_search:
        temp1 = temp1_search.group(0)
        temp1 = re.sub("\D", "", temp1)
        print temp1+"C"
    else:
        print "no temp1 found"
       
    if temp2_search:
        temp2 = temp2_search.group(0)
        temp2 = re.sub("\D", "", temp2)
        print temp2+"C"
    else:
        print "no temp2 found"

    y +=1
Posts: 691
Joined: Sun Aug 05, 2012 3:12 pm
by gordon77 » Wed Dec 19, 2012 11:33 am
I found a problem in that if I try to extract the "sT=" section I can't get it to extract letters and spaces.
This should solve that.
You may want to change the first code I sent today as it may only work becasue today is 'Cloudy' and not more than 1 word

Code: Select all
# Getting weather stats
import urllib2
import re

weatherURL1 = "http://aaltohtml5name.foreca.com/aaltohtml5-oct12a/data.php?l=102735943&products=cc,daily"
response1 = urllib2.urlopen(weatherURL1)
html1 = response1.read()
 
#print html1
w = 1
arr = {}
a = ""
x=0
while a != "</w":
    while a != "<fc" and a != "</w":
        a = html1[x:x+3]
        x +=1
    x +=1
    arr[(w,1)] = a
    while a != " />" and a != "</w":
        a = html1[x:x+3]
        arr[(w,1)] = arr[(w,1)] + a[1:2]
        x +=1
    arr[(w,1)] = arr[(w,1)] + ">"
    # print arr[(w,1)]
    w +=1

y=1
while y < 4:
    html1 = arr[(y,1)]
    print html1
    r_temp1 = re.compile("tx=\"[0-9]{1,3}\"")
    r_temp2 = re.compile("tn=\"[0-9]{1,3}\"")
    r_cond1 = re.compile("sT=\"[\D]{1,20}\"")

    print r_temp1
    print r_temp2
    print r_cond1
   
    temp1_search = r_temp1.search(html1)
    temp2_search = r_temp2.search(html1)
    cond1_search = r_cond1.search(html1)


    print temp1_search
    print temp2_search
    print cond1_search

    if temp1_search:
        temp1 = temp1_search.group(0)
        temp1 = re.sub("\D", "", temp1)
        print temp1+"C"
    else:
        print "no temp1 found"
       
    if temp2_search:
        temp2 = temp2_search.group(0)
        temp2 = re.sub("\D", "", temp2)
        print temp2+"C"
    else:
        print "no temp2 found"

    if cond1_search:
        cond1 = cond1_search.group(0)
        cond1 = cond1[4:len(cond1)-1]
        print cond1
    else:
        print "no cond found"

    y +=1
Posts: 691
Joined: Sun Aug 05, 2012 3:12 pm
by gordon77 » Wed Dec 19, 2012 11:38 am
Sorry ignore the code in the last reply, it still had issues, and lots of print statements from my testing !
Code: Select all
# Getting weather stats
import urllib2
import re

weatherURL1 = "http://aaltohtml5name.foreca.com/aaltohtml5-oct12a/data.php?l=102735943&products=cc,daily"
response1 = urllib2.urlopen(weatherURL1)
html1 = response1.read()
 
#print html1
w = 1
arr = {}
a = ""
x=0
while a != "</w":
    while a != "<fc" and a != "</w":
        a = html1[x:x+3]
        x +=1
    x +=1
    arr[(w,1)] = a
    while a != " />" and a != "</w":
        a = html1[x:x+3]
        arr[(w,1)] = arr[(w,1)] + a[1:2]
        x +=1
    arr[(w,1)] = arr[(w,1)] + ">"
    # print arr[(w,1)]
    w +=1

y=1
while y < 4:
    html1 = arr[(y,1)]
    r_temp1 = re.compile("tx=\"[0-9]{1,3}\"")
    r_temp2 = re.compile("tn=\"[0-9]{1,3}\"")
    r_cond1 = re.compile("sT=\"[\D\W]{1,40}\"")
     
    temp1_search = r_temp1.search(html1)
    temp2_search = r_temp2.search(html1)
    cond1_search = r_cond1.search(html1)

    if temp1_search:
        temp1 = temp1_search.group(0)
        temp1 = re.sub("\D", "", temp1)
        print temp1+"C"
    else:
        print "no temp1 found"
       
    if temp2_search:
        temp2 = temp2_search.group(0)
        temp2 = re.sub("\D", "", temp2)
        print temp2+"C"
    else:
        print "no temp2 found"

    if cond1_search:
        cond1 = cond1_search.group(0)
        cond1 = cond1[4:len(cond1)-8]
        print cond1
    else:
        print "no cond found"

    y +=1
Posts: 691
Joined: Sun Aug 05, 2012 3:12 pm
by SN » Wed Dec 19, 2012 11:48 am
Ummm - guys that URL returns a little slab of XML - rather than chop up the xml into string elements and try and pick stuff out of it, is there not an common Python library which will process XML into an object structure?

have a look at http://stackoverflow.com/questions/3106 ... ython?rq=1 for some ideas - there is a specific weather data example using lxml.etree half way down the page

Alternatively, this is a trivial problem to resolve with awk...
Steve N – binatone mk4->intellivision->zx81->spectrum->cbm64->cpc6128->520stfm->pc->raspi ?
User avatar
Posts: 1008
Joined: Mon Feb 13, 2012 8:06 pm
Location: Romiley, UK
by gordon77 » Wed Dec 19, 2012 12:14 pm
Thanks for the info.

Wow! I have a lot to learn in Python, the options seem endless.
Posts: 691
Joined: Sun Aug 05, 2012 3:12 pm
by gordon77 » Wed Dec 19, 2012 12:17 pm
updated version of first code, hopefully it can now cope with more than 1 word descriptions of 'sT'

Gordon77

Code: Select all
# Getting weather stats
import urllib2
import re

weatherURL1 = "http://aaltohtml5name.foreca.com/aaltohtml5-oct12a/data.php?l=102735943&products=cc"

 
response1 = urllib2.urlopen(weatherURL1)
html1 = response1.read()
 
print html1
r_temp = re.compile("tf=\"[0-9]{1,3}\"")
r_cond = re.compile("sT=\"[\D\W]{1,40}\"")
 
temp_search = r_temp.search(html1)
cond_search = r_cond.search(html1)
 
if temp_search:
    temp1 = temp_search.group(0)
    temp1 = re.sub("\D", "", temp1)
    print temp1+"ºC"
else:
    print "no temp found"

if cond_search:
    cond = cond_search.group(0)
    cond = cond[4:len(cond)-1]
    print cond
else:
    print "no cond found"

Posts: 691
Joined: Sun Aug 05, 2012 3:12 pm
by canibalimao » Wed Dec 19, 2012 1:13 pm
SN wrote:Ummm - guys that URL returns a little slab of XML - rather than chop up the xml into string elements and try and pick stuff out of it, is there not an common Python library which will process XML into an object structure?

have a look at http://stackoverflow.com/questions/3106 ... ython?rq=1 for some ideas - there is a specific weather data example using lxml.etree half way down the page

Alternatively, this is a trivial problem to resolve with awk...


Really? So I was doing fool of myself trying to use that type of scrypt... Thank you very much :D
I'll try that now and try to import multiple lines information :)

gordon77 thank you so much! You were a great help!
Posts: 528
Joined: Tue Aug 07, 2012 8:48 pm
Location: Portugal
by canibalimao » Wed Dec 19, 2012 1:57 pm
gordon77 it is working really well! Now I'd made a translation to Portuguese and it's ready to be send to my LCD screen :D

Thank you very much :D


Btw, about the lxml library that is really usefull, but just in case I juss want to read the weather info for today. For the other days it takes a bit more code, I think, because the next lines have the same code as the first one...
But I think I'll put it working that way too. Just need a bit patience :mrgreen:

EDIT: I can't import lxml.etree. Python gives me an error:

Code: Select all
Traceback (most recent call last):
  File "C:/Downloads/Raspberry Pi/weather_xml.py", line 5, in <module>
    import lxml.etree
ImportError: No module named lxml.etree
Posts: 528
Joined: Tue Aug 07, 2012 8:48 pm
Location: Portugal