如何在python中将数据从分页API转换为数据帧

Question

我需要从RESTAPI获取一些数据来创建web地图。我想先看看数据，这就是为什么我尝试将json转换为dataframe。当它只是API中的一个页面时，我的代码（如下）运行良好，但当我遍历所有页面并将结果存储在列表中时，它会给我一个错误。我做错了什么？还有别的办法吗

import geopandas as gpd
import requests
import pandas as pd 
import folium
import json 

api_url = "https://pmap.minregion.gov.ua/index.php?r=api/document&access-token=[my token]&limit=500"

response = requests.get(api_url).json()
api_dataset = response

#looping through and putting data to the list api_dataset
for page in range (1, 62):
    response = requests.get(api_url + f"&page={page}").json()
    api_dataset.extend(response)

from pandas.io.json import json_normalize 

pmap_data = json_normalize(api_dataset) #transformation to dataframe

这里我收到了错误：“AttributeError:'str'对象没有属性'items'”。但同样地，在API的一页中，这种方法工作得非常完美

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-120-eb5bf90e478d> in <module>
----> 1 pmap_data = json_normalize(api_dataset)
      2 pmap_data.shape

E:\Programming\Anaconda\lib\site-packages\pandas\io\json\normalize.py in json_normalize(data, record_path, meta, meta_prefix, record_prefix, errors, sep)
    208             # TODO: handle record value which are lists, at least error
    209             #       reasonably
--> 210             data = nested_to_record(data, sep=sep)
    211         return DataFrame(data)
    212     elif not isinstance(record_path, list):

E:\Programming\Anaconda\lib\site-packages\pandas\io\json\normalize.py in nested_to_record(ds, prefix, sep, level)
     71 
     72         new_d = copy.deepcopy(d)
---> 73         for k, v in d.items():
     74             # each key gets renamed with prefix
     75             if not isinstance(k, compat.string_types):

AttributeError: 'str' object has no attribute 'items'

Answer 1

问题解决了。API的响应列表中包含一些字符串以及DICT。使用以下代码检查并删除它们：

all(isinstance(x, dict) for x in api_dataset) #check if there only dicts

for x in api_dataset:
    if type(x) == str: 
         api_dataset.remove(x)

len(api_dataset)