xml parsing Python reads file incorrectly -

-

i trying parse xml file online , obtain data need file. code displayed below:

import urllib2 xml.dom.minidom import parse import pandas pd import time  page = urllib2.urlopen('http://www.wrh.noaa.gov/mesowest/getobextxml.php?sid=kbfi&num=360') page_content = page.read() open('kbfi.xml', 'w') fid:     fid.write(page_content)  data = []  xml = parse('kbfi.xml') percp = 0 station in xml.getelementsbytagname('station'): ob in xml.getelementsbytagname('ob'):     # convert time sting time_struct ignoring last 4 chars ' pdt'     ob_time = time.strptime(ob.getattribute('time')[:-4],'%d %b %i:%m %p')     variable in xml.getelementsbytagname('variable'):         if variable.getattribute('var') == 'pcp1h':             percp = true             # unindent if want variables             if variable.getattribute('value') == 't':                 data.append([ob_time.tm_mday,                              ob_time.tm_hour,                              ob_time.tm_min,                              0])             elif variable.getattribute('value') >= 0:                 data.append((ob_time.tm_mday,                             ob_time.tm_hour,                             ob_time.tm_min,                             variable.getattribute('value')))     if not percp:         # if pcp1h wasn't found add 0         data.append([ob_time.tm_mday,                     ob_time.tm_hour,                     ob_time.tm_min,                     0]) print data

unfortunately cannot post image of xml file, version of saved current directory if script run.

i code check existence of 'variable' pcph1 , print 'value' if exists (only 1 entry per 'ob'). if doesn't exist or provides value of 't', print '0' particular hour. output (the script provided can run see output) contains incorrect values , there 6 entries per hour instead of one. wrong code?

main issue in code , in each for loop, getting elements using -

xml.getelementsbytagname('ob')

this starts search xml element, in case in root element, same in case of xml.getelementsbytagname('variable') , starts search @ root element, each time getting elements tag variable , why getting 6 entries per hour, instead of 1 (since there 6 of them in complete xml).

you should instead using - 

ob.getelementsbytagname('variable')

and ob element using -

station.getelementsbytagname('ob')

so check inside particular element iterating on (not complete xml document).

also, side issue , doing -

elif variable.getattribute('value') >= 0:

if not wrong, getattribute() returns string, check true, irrespective of actual value is. in xml, see value has strings numbers, not sure want condition (though not main issue, main issue 1 described above).

example code changes -

import urllib2 xml.dom.minidom import parse import pandas pd import time  page = urllib2.urlopen('http://www.wrh.noaa.gov/mesowest/getobextxml.php?  sid=kbfi&num=360') page_content = page.read() open('kbfi.xml', 'w') fid:     fid.write(page_content.decode())  data = []  xml = parse('kbfi.xml') percp = 0 station in xml.getelementsbytagname('station'):     ob in station.getelementsbytagname('ob'):         # convert time sting time_struct ignoring last 4 chars ' pdt'         ob_time = time.strptime(ob.getattribute('time')[:-4],'%d %b %i:%m %p')         variable in ob.getelementsbytagname('variable'):             if variable.getattribute('var') == 'pcp1h':                 percp = true                 # unindent if want variables                 if variable.getattribute('value') == 't':                     data.append([ob_time.tm_mday,                                  ob_time.tm_hour,                                  ob_time.tm_min,                                  0])                 elif variable.getattribute('value') >= 0:                     data.append((ob_time.tm_mday,                                 ob_time.tm_hour,                                 ob_time.tm_min,                                 variable.getattribute('value')))         if not percp:             # if pcp1h wasn't found add 0             data.append([ob_time.tm_mday,                         ob_time.tm_hour,                         ob_time.tm_min,                         0]) print data

Comments

Post a Comment

Popular posts from this blog

Fail to load namespace Spring Security http://www.springframework.org/security/tags -

-
i have problem namespace of spring security on xhtml page. don't understand why can't load uri, , have following issue: whitelabel error page application has no explicit mapping /error, seeing fallback. tue jul 21 16:58:12 utc 2015 there unexpected error (type=not found, status=404). please need help! possible uri has been changed? best regards you using incorrect namespace url. correct url http://www.springframework.org/schema/security the error page seeing spring source website since not recognize url. try visiting http://www.springsource.org/security/tags in browser update : spring security tags used in jsp page correct tag lib declaration is: <%@ taglib prefix="sec" uri="http://www.springframework.org/security/tags" %>
Read more

sql - MySQL query optimization using coalesce -

-
can explain me why second query faster first? (first 1 takes around 40 seconds, second 1 less 0.1) select releases.feature_code, releases.platform_id, version, release_date, general_available, uri, filesize, display_name, feature_name.full_name, if ((select exists(select 1 release_exceptions releases.feature_code = release_exceptions.feature_code , releases.platform_id = release_exceptions.platform_id , releases.version = release_exceptions.version) = 1), 1, 0) release_exceptions releases left outer join platform using(platform_id) left outer join feature_name using(feature_code) order version desc; ~~~ query 2 ------------------ ~~~ select feature_code, version, release_date, general_available, platform_id, uri, filesize, display_name, feature_name.full_name, coalesce(release_exceptions, 0) release_exceptions mus.releases left outer join ( select distinct feature_code, version, platform_id, 1 release_exceptions mus.release_exceptions ) release_exceptions using ...
Read more

unity3d - Unity local avoidance in user created world -

-
i'm using unity3d networked multiplayer online game have large complex 3d terrain scene forest, trees, cliff, hills, mountains, bounders, etc. players can build structures sort of minecraft, , put them anywhere in scene, or move them around anytime. aside human controlled players, there automated ai players , objects animals roaming around scene following path. the problem how make these automated ai players , animals, able navigate around static , dynamic player created structures, because path follow can blocked player created structures, or other players , other ai objects, cliffs etc. have find away around them or on track if tumble down off high cliff example. so made navmesh , used navagents, takes care of static, non moving objects, how make ai players navigate around each other , dynamic structures created players can number in hundreds? i thought of adding navmeshobstacle everything, result in being attached hundreds of objects since user created structures b...
Read more