在使用selenium和Beautifulsoup执行javascript之后对web进行爬网

Question

我想在执行javascript“Click”事件后爬网

网络如下所示

function initPage() {

initCorpInfo();

var Tree = Ext.tree;

var treeRoot = new Tree.TreeNode({
    text: "total",
    id: "root",
    href: "javascript: viewDoc('20150515001896', '4671059', null, null, null, 'dart3.xsd')"
});

    treeNode2 = new Tree.TreeNode({
        text: "4. financial statement",
        id: "17",
        cls: "text",
        listeners: {
            click: function() {viewDoc('20150515001896', '4671059', '17', '1015699', '132786',


    });
}

function viewDoc(rcpNo, dcmNo, eleId, offset, length, dtd) {

currentDocValues.rcpNo = rcpNo;
currentDocValues.dcmNo = dcmNo;
currentDocValues.eleId = eleId;
currentDocValues.offset = offset;
currentDocValues.length = length;
currentDocValues.dtd = dtd;
var params = "";
params += "?rcpNo=" + rcpNo;
params += "&dcmNo=" + dcmNo;
if (eleId != null)
    params += "&eleId=" + eleId;
if (offset != null)
    params += "&offset=" + offset;
if (length != null)
    params += "&length=" + length;
params += "&dtd=" + dtd;
document.getElementById("ifrm").src = "/report/viewer.do" + params;

}

查看源：http://dart.fss.or.kr/dsaf001/main.do?rcpNo=20150515001896 （单击左栏中的4

我可以使用selenium和beautifulsoup执行“click:function（）{viewDoc（'2015051001896'，'4671059'，'17'，'1015699'，'132786'，”吗？在

我应该使用scrapy而不是Beautifulsoup来接近javascript的函数吗？在

Answer 1

就这样简单地解决了。在

from selenium import webdriver
from bs4 import BeautifulSoup
import time

browser=webdriver.Firefox()
browser.get("http://dart.fss.or.kr/dsaf001/main.do?rcpNo=20150515001896")

soup=BeautifulSoup(browser.page_source) 
browser.execute_script("viewDoc('20150515001896', '4671059', '17', '1015699', '132786', 'dart3.xsd');")