我想保存的链接刮保存在我的数据库,以便在后续运行,它只刮和附加新的链接到列表。
这是我下面的代码,但在一天结束时,我的数据库是空的。我能做些什么改变来克服这个问题?提前谢谢
from django.template.loader import get_template
from django.shortcuts import render_to_response
from bs4 import BeautifulSoup
import urllib2, sys
import urlparse
import re
from listing.models import jobLinks
#this function extract the links
def businessghana():
site = "http://www.businessghana.com/portal/jobs"
hdr = {'User-Agent' : 'Mozilla/5.0'}
req = urllib2.Request(site, headers=hdr)
jobpass = urllib2.urlopen(req)
soup = BeautifulSoup(jobpass)
for tag in soup.find_all('a', href = True):
tag['href'] = urlparse.urljoin('http://www.businessghana.com/portal/', tag['href'])
return map(str, soup.find_all('a', href = re.compile('.getJobInfo')))
# result from businssghana() saved to a variable to make them iterable as a list
all_links = businessghana()
#this function should be saving the links to the database unless the link already exist
def save_new_links(all_links):
current_links = jobLinks.objects.all()
for i in all_links:
if i not in current_links:
jobLinks.objects.create(url=i)
# I called the above function here hoping that it will save to database
save_new_links(all_links)
# return my httpResponse with this function
def display_links(request):
name = all_links()
return render_to_response('jobs.html', {'name' : name})
我的django模型.py看起来像这样:
^{pr2}$
你的代码有一些问题。首先,
jobLinks
模型没有url
字段。您应该在create()
语句中使用links=i
。其次,检查字符串url是否在QuerySet中。永远不会这样,因为QuerySet的元素是模型类的对象,在您的例子中是jobLinks
对象。相反,你可以这样做:我还建议在将来,你可以用单数词来描述你的模型。例如,在本例中,我将调用模型}。在
jobLink
和字段{相关问题 更多 >
编程相关推荐