如何使用Python固定交互式网页

发布于 2025-02-02 12:52:26 字数 578 浏览 1 评论 0 原文

我想知道以下网站: http://chonos.ifop.cl/flow/

网页在右侧有一个地图,当您单击HighCharts图中左时间序列上显示的每个点时,我想迭代地提取这些系列,但我仍然不能。到目前为止,这是我的代码:

from io import BytesIO
import gzip
site_url='http://chonos.ifop.cl/flow/'
r = urllib.request.urlopen(site_url)
site_content = r.read()
s = BeautifulSoup(site_content, 'html.parser')
print(s.prettify()[:100])
s.find_all('td')
s.find_all('table')
s.findAll('table',attrs={'class':'uk-table uk-table-small uk-table-striped'})

I would like to know to webscrape the following website: http://chonos.ifop.cl/flow/

The web page has a map on the right, when you click on each point it shows on the left time series in Highcharts graphs, I would like to extract these series iteratively but I still can't. Here is my code up to now:

from io import BytesIO
import gzip
site_url='http://chonos.ifop.cl/flow/'
r = urllib.request.urlopen(site_url)
site_content = r.read()
s = BeautifulSoup(site_content, 'html.parser')
print(s.prettify()[:100])
s.find_all('td')
s.find_all('table')
s.findAll('table',attrs={'class':'uk-table uk-table-small uk-table-striped'})

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

南城追梦 2025-02-09 12:52:26

当我在firefox/chrome中使用 devtools (tab:网络)以查看所有浏览器在我单击地图时浏览器发送到服务器的请求,然后我在下面看到这样的url,一些JSON数据,有名称系列

您可以单击此链接以直接在浏览器中查看JSON数据


我也可以在代码

import requests

url = 'http://chonos.ifop.cl/flow/mapclick'

params = {
    'REQUEST': 'GetFeatureInfo',
    'SERVICE': 'WMS',
    'SRS': 'EPSG:4326',
    'STYLES': '',
    'TRANSPARENT': 'true',
    'VERSION': '1.1.1',
    'FORMAT': 'image.png',
    'BBOX': '-84.48486328125,-50.16282433381728,-59.54589843750001,-45.75219336063107',
    'HEIGHT': '300',
    'WIDTH': '1135',
    'LAYERS': 'aguadulce:outlet_points',
    'QUERY_LAYERS': 'aguadulce:outlet_points',
    'INFO_FORMAT': 'text.html',
    'LAT': '-46.528634695271684',
    'LON': '-71.41113281250001',
    'X': '595',
    'Y': '51',
}

response = requests.get(url, params=params)

data = response.json()

for item in data['series']['sim']:
    print(item)

结果中使用此链接:

[283996800000, 985.352]
[284083200000, 1115.734]
[284169600000, 1099.139]
[284256000000, 1146.895]
[284342400000, 1127.501]
[284428800000, 1146.251]
[284515200000, 1048.681]
[284601600000, 939.899]
[284688000000, 941.33]
[284774400000, 905.143]

...

在链接中,我看到 lat = lon = - 因此,如果您会更改纬度您应该获取其他位置的数据。


编辑:

如所说的@Modammed-当您单击特殊位置时,它会从链接中加载数据,例如

https://chonos.ifop.cl/flow/stnclick?index=50

,您可以以与以前的链接相同的方式使用此链接。

如果您更改索引,则获得不同的位置。

import requests
    
url = 'http://chonos.ifop.cl/flow/stnclick'

params = {
    'index': 0
}

for number in range(10):
    params['index'] = number
    
    response = requests.get(url, params=params)

    data = response.json()

    print('---', data['name'], '---')
    
    
    #for item in data['series']['sim'][:5]: # show first 5 values
    for item in data['series']['sim']:      # show all values 
        print(item)

结果(每个位置的前5个值):

--- Rio Caleta En Tierra Del Fuego ---
[283996800000, 4.41]
[284083200000, 4.27]
[284169600000, 4.13]
[284256000000, 4.0]
[284342400000, 3.95]

--- Rio La Plata Antes Junta Rio Hueyusca ---
[283996800000, 4.43]
[284083200000, 4.15]
[284169600000, 3.88]
[284256000000, 3.63]
[284342400000, 3.39]

--- Rio Hueyusca En Camarones ---
[283996800000, 12.46]
[284083200000, 11.71]
[284169600000, 11.0]
[284256000000, 10.33]
[284342400000, 9.7]

--- Rio Negro En Las Lomas ---
[283996800000, 9.97]
[284083200000, 8.98]
[284169600000, 8.08]
[284256000000, 7.3]
[284342400000, 6.61]

--- Rio Maullin En Las Quemas ---
[283996800000, 35.37]
[284083200000, 33.34]
[284169600000, 31.53]
[284256000000, 29.8]
[284342400000, 28.47]

When I use DevTools in Firefox/Chrome (tab: Network) to see all requests which browser sends to server when I click on map then I see url like this belowe which gives some JSON data and there is name series.

You can click this link to see JSON data directly in browser

http://chonos.ifop.cl/flow/mapclick?&REQUEST=GetFeatureInfo&SERVICE=WMS&SRS=EPSG%3A4326&STYLES=&TRANSPARENT=true&VERSION=1.1.1&FORMAT=image%2Fpng&BBOX=-84.48486328125%2C-50.16282433381728%2C-59.54589843750001%2C-45.75219336063107&HEIGHT=300&WIDTH=1135&LAYERS=aguadulce%3Aoutlet_points&QUERY_LAYERS=aguadulce%3Aoutlet_points&INFO_FORMAT=text%2Fhtml&LAT=-46.528634695271684&LON=-71.41113281250001&X=595&Y=51


And I can use this link also in code

import requests

url = 'http://chonos.ifop.cl/flow/mapclick'

params = {
    'REQUEST': 'GetFeatureInfo',
    'SERVICE': 'WMS',
    'SRS': 'EPSG:4326',
    'STYLES': '',
    'TRANSPARENT': 'true',
    'VERSION': '1.1.1',
    'FORMAT': 'image.png',
    'BBOX': '-84.48486328125,-50.16282433381728,-59.54589843750001,-45.75219336063107',
    'HEIGHT': '300',
    'WIDTH': '1135',
    'LAYERS': 'aguadulce:outlet_points',
    'QUERY_LAYERS': 'aguadulce:outlet_points',
    'INFO_FORMAT': 'text.html',
    'LAT': '-46.528634695271684',
    'LON': '-71.41113281250001',
    'X': '595',
    'Y': '51',
}

response = requests.get(url, params=params)

data = response.json()

for item in data['series']['sim']:
    print(item)

Result:

[283996800000, 985.352]
[284083200000, 1115.734]
[284169600000, 1099.139]
[284256000000, 1146.895]
[284342400000, 1127.501]
[284428800000, 1146.251]
[284515200000, 1048.681]
[284601600000, 939.899]
[284688000000, 941.33]
[284774400000, 905.143]

...

In link I see LAT=, LON= - so if you will change latitude, longtitude` then you should get data for other locations.


EDIT:

As said @Modammed - when you click on special locations then it loads data from links like

https://chonos.ifop.cl/flow/stnclick?index=50

and you can use this link in the same way as previous link.

If you change index then you get different location.

import requests
    
url = 'http://chonos.ifop.cl/flow/stnclick'

params = {
    'index': 0
}

for number in range(10):
    params['index'] = number
    
    response = requests.get(url, params=params)

    data = response.json()

    print('---', data['name'], '---')
    
    
    #for item in data['series']['sim'][:5]: # show first 5 values
    for item in data['series']['sim']:      # show all values 
        print(item)

Result (first 5 values for every location):

--- Rio Caleta En Tierra Del Fuego ---
[283996800000, 4.41]
[284083200000, 4.27]
[284169600000, 4.13]
[284256000000, 4.0]
[284342400000, 3.95]

--- Rio La Plata Antes Junta Rio Hueyusca ---
[283996800000, 4.43]
[284083200000, 4.15]
[284169600000, 3.88]
[284256000000, 3.63]
[284342400000, 3.39]

--- Rio Hueyusca En Camarones ---
[283996800000, 12.46]
[284083200000, 11.71]
[284169600000, 11.0]
[284256000000, 10.33]
[284342400000, 9.7]

--- Rio Negro En Las Lomas ---
[283996800000, 9.97]
[284083200000, 8.98]
[284169600000, 8.08]
[284256000000, 7.3]
[284342400000, 6.61]

--- Rio Maullin En Las Quemas ---
[283996800000, 35.37]
[284083200000, 33.34]
[284169600000, 31.53]
[284256000000, 29.8]
[284342400000, 28.47]
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文