如何在web中提取纬度和经度

Question

我在第一列https://www.peakbagger.com/peak.aspx?pid=10882下的https://www.peakbagger.com/list.aspx?lid=5651中提取了一些信息

from urllib.request import urlopen
from bs4 import BeautifulSoup
import pandas as pd

url = 'https://www.peakbagger.com/peak.aspx?pid=10882'
html = urlopen(url)
soup = BeautifulSoup(html, 'html.parser')

a= soup.select("td")
a

我只想从我得到的输出中检索纬度和经度，即35.360638138.727347

它在

E<br/>35.360638, 138.727347 

还有，除了逐一检索外，还有没有更好的方法从/www.peakbagger.com/list.aspx？lid=5651的链接检索所有纬度和经度

谢谢

Answer 1

正如Brutus所提到的，它非常具体，如果您不习惯使用etree，这可能是一种替代方法

• find()<td>带字符串Latitude/Longitude (WGS84)
• findNext()它的下一个{}
• 抓住它的内容
• 替换，并用空格将其拆分
• 通过将结果切片到前两个元素，您将得到带有lat和long的列表

data = soup.find('td', string='Latitude/Longitude (WGS84)')\
            .findNext('td')\
            .contents[2]\
            .replace(',','')\
            .split()[:2]

data

编辑

你有一个URL列表并在上面循环-为了考虑到网站而不是被禁止，在页面上运行会有一些延迟（time.sleep()）

import time
import requests
from bs4 import BeautifulSoup
urls = ['https://www.peakbagger.com/peak.aspx?pid=10882',
 'https://www.peakbagger.com/peak.aspx?pid=10866',
 'https://www.peakbagger.com/peak.aspx?pid=10840',
 'https://www.peakbagger.com/peak.aspx?pid=10868',
 'https://www.peakbagger.com/peak.aspx?pid=10832']

data = {}

for url in urls:
    
    r = requests.get(url)
    soup = BeautifulSoup(r.content, 'lxml')
    

    ll= soup.find('td', string='Latitude/Longitude (WGS84)')\
                .findNext('td')\
                .contents[2]\
                .replace(',','')\
                .split()[:2]
    
    data[soup.select_one('h1').get_text()]={
        'url':url,
        'lat':ll[0],
        'long':ll[1]
    }
 
    time.sleep(3)

data

输出

{'Fuji-san, Japan': {'url': 'https://www.peakbagger.com/peak.aspx?pid=10882',
  'lat': '35.360638',
  'long': '138.727347'},
 'Kita-dake, Japan': {'url': 'https://www.peakbagger.com/peak.aspx?pid=10866',
  'lat': '35.674537',
  'long': '138.238833'},
 'Hotaka-dake, Japan': {'url': 'https://www.peakbagger.com/peak.aspx?pid=10840',
  'lat': '36.289203',
  'long': '137.647986'},
 'Aino-dake, Japan': {'url': 'https://www.peakbagger.com/peak.aspx?pid=10868',
  'lat': '35.646037',
  'long': '138.228292'},
 'Yariga-take, Japan': {'url': 'https://www.peakbagger.com/peak.aspx?pid=10832',
  'lat': '36.34198',
  'long': '137.647625'}}

Answer 2

这个答案对你的问题非常具体。这里的问题是br标签出现在td标签中。etree模块（lxml的一部分）允许您访问标记后面的文本（也称为itstail）。此代码将打印显示为所需输出的值

import requests
from lxml import etree

with requests.Session() as session:
    r = session.get('https://www.peakbagger.com/peak.aspx?pid=10882')
    r.raise_for_status()
    tree = etree.HTML(r.text)
    print(' '.join(tree.xpath('//table[@class="gray"][1]/*//br')[1].tail.split()[:2]))