使用scrapy python从javascript到python获取数据

Question

如何使用scrapy python从javascript内容中获取数据？javascript是这样的

<script type="text/javascript">
  var ad_reply_url = "http://www2.mudah.my/ar/send/0?ca=3_s&id=49825097&l=0";
  var mcvl = "";
  var images = [
     'http://img.rnudah.com/images/13/133608119523265.jpg', 
     'http://img.rnudah.com/images/13/135608116569903.jpg', 
     'http://img.rnudah.com/images/13/137608113616541.jpg', 
     'http://img.rnudah.com/images/13/139608119186498.jpg'
  ];
 var thumbnails = [
    'http://img.rnudah.com/thumbs/13/133608119523265.jpg',
    'http://img.rnudah.com/thumbs/13/135608116569903.jpg',
    'http://img.rnudah.com/thumbs/13/137608113616541.jpg',
    'http://img.rnudah.com/thumbs/13/139608119186498.jpg'
 ];</script>

所以，我想要的是。我想要var图像的数据，然后像这样打印这些数据

['http://img.rnudah.com/images/13/133608119523265.jpg','http://img.rnudah.com/images/13/135608116569903.jpg', 'http://img.rnudah.com/images/13/137608113616541.jpg','http://img.rnudah.com/images/13/139608119186498.jpg' ];

有人能帮我吗？谢谢。你知道吗

Answer 1

我没有使用Scrapy Python，只是使用普通Python。 不过，这很简单：

代码示例：

import ast
import re

page_source = '''
<script type="text/javascript">
  var ad_reply_url = "http://www2.mudah.my/ar/send/0?ca=3_s&id=49825097&l=0";
  var mcvl = "";
  var images = [
     'http://img.rnudah.com/images/13/133608119523265.jpg',
     'http://img.rnudah.com/images/13/135608116569903.jpg',
     'http://img.rnudah.com/images/13/137608113616541.jpg',
     'http://img.rnudah.com/images/13/139608119186498.jpg'
  ];
 var thumbnails = [
    'http://img.rnudah.com/thumbs/13/133608119523265.jpg',
    'http://img.rnudah.com/thumbs/13/135608116569903.jpg',
    'http://img.rnudah.com/thumbs/13/137608113616541.jpg',
    'http://img.rnudah.com/thumbs/13/139608119186498.jpg'
 ];</script>
'''

variables = re.findall('(?si)var(.*?);', page_source)

var_collection = {}
for var in variables:
    var = var.strip()
    var_key = var.split(' = ')[0]
    var_value = ast.literal_eval(var.split(' = ')[1])
    var_collection.update({var_key: var_value})

print(var_collection['images'])

输出：

['http://img.rnudah.com/images/13/133608119523265.jpg', 'http://img.rnudah.com/images/13/135608116569903.jpg', 'http://img.rnudah.com/images/13/137608113616541.jpg', 'http://img.rnudah.com/images/13/139608119186498.jpg']

相关： https://stackoverflow.com/a/18108644/295246