大型嵌套JSON到Pandas数据帧的Python转换

2024-09-28 22:21:38 发布

您现在位置:Python中文网/ 问答频道 /正文

我有一个大的嵌套json对象,我想读入pandas数据帧。json对象内部有许多嵌套的json对。基本分层是:

  • API调用元数据(我真的不关心这个)
  • 调查响应元数据(我希望这些信息包含在最终输出中)
  • 调查的页数和页码是什么
  • 问题(问题和相关回答)

示例如下:

{
    "per_page": 50,
    "total": 4,
    "data": [
        {
            "total_time": 5276,
            "href": "https://somelink.com",
            "custom_variables": {},
            "ip_address": "XXX.XXX.XX.XX",
            "id": "1111111",
            "logic_path": {},
            "date_modified": "2018-08-17T19:57:43+00:00",
            "response_status": "completed",
            "custom_value": "",
            "analyze_url": "https://somelink.com/respondent_id=1111111",
            "pages": [
                {
                    "id": "38638937",
                    "questions": []
                },
                {
                    "id": "38638938",
                    "questions": [
                        {
                            "id": "124810659",
                            "answers": [
                                {
                                    "text": "some_answer_text"
                                }
                            ]
                        },
                        {
                            "id": "124810660",
                            "answers": [
                                {
                                    "text": "some_other_answer_text"
                                }
                            ]
                        }
                    ]
                },
                {
                    "id": "38638944",
                    "questions": [
                        {
                            "id": "124810656",
                            "answers": [
                                {
                                    "col_id": "905794209",
                                    "choice_id": "905794459",
                                    "row_id": "905794204"
                                },
                                {
                                    "col_id": "905794210",
                                    "choice_id": "905794463",
                                    "row_id": "905794204"
                                },
                                {
                                    "col_id": "905794209",
                                    "choice_id": "905794459",
                                    "row_id": "905794205"
                                },
                                {
                                    "col_id": "905794210",
                                    "choice_id": "905794464",
                                    "row_id": "905794205"
                                }
                            ]
                        }
                    ]   
                }
                .
                .
                .
                .
                .
                .
                .
                .
            ],
            "page_path": [],
            "recipient_id": "4107168056",
            "collector_id": "216279750",
            "date_created": "2018-09-05T15:28:38+00:00",
            "survey_id": "222222222",
            "collection_mode": "default",
            "edit_url": "https://www.somelink.com/somerefnumber",
            "metadata": {
                "contact": {
                    "email": {
                        "type": "string",
                        "value": "name@somememail.com"
                    }
                }
            }
        },
        {
            "total_time": 6978,
            "href": "https://somelink.com",
            "custom_variables": {},
            "ip_address": "XXX.XXX.XX.XX",
            "id": "4444444",
            "logic_path": {},
            "date_modified": "2018-08-15T19:16:43+00:00",
            "response_status": "completed",
            "custom_value": "",
            "analyze_url": "https://somelink.com/respondent_id=4444444",
            "pages": [
                    .
                    .
                    .

            ]
        }

    ],
    "page": 1,
    "links": {
        "self": "https://api.somelink.com/22222222/responses/bulk?page=1&per_page=50"
    }
}

一个页面中可以有任意数量的回答、页面和问题。在

我的问题是: 如何将上面的json放入如下所示的pandas datafame中: Formatted Pandas dataframe (as excel for reference)

我试过使用json_normalize,但我相信我犯了一些错误。在

^{pr2}$

作为附加信息,这是来自SurveyMonkey API。SurveyMonkey允许您export survey results to csv via the web interface,但我想使用API重新创建标准响应报告,并最终生成自定义报告/执行其他操作。在

我用的是Python3.6。我假设我需要做一些预处理来整理数据,但我不知道如何做。谢谢任何帮助!在


Tags: 数据texthttpscomapiidjsoncustom