使用Python抓取和爬行Quora

from selenium import webdriver options = webdriver.ChromeOptions() options.add_argument('--headless') options.add_argument('--no-sandbox') options.add_argument('--disable-dev-shm-usage') # open it, go to a website, and get results wd = webdriver.Chrome(options=options) import time URL = "https://www.quora.com/What-was-your-prom-like" #URL = "https://www.quora.com/profile/%E0%AE%AA%E0%AE%BE%E0%AE%B2%E0%AE%BE%E0%AE%9C%E0%AE%BF-%E0%AE%B5%E0%AE%BF%E0%AE%B8%E0%AF%8D%E0%AE%B5%E0%AE%A8%E0%AE%BE%E0%AE%A4%E0%AE%A9%E0%AF%8D-Balaji-Viswanathan" wd.get(URL) PAUSE_TIME = 2 lh = wd.execute_script("return document.body.scrollHeight") while True: wd.execute_script("window.scrollTo(0, document.body.scrollHeight);") time.sleep(PAUSE_TIME) nh = wd.execute_script("return document.body.scrollHeight") if nh == lh: break lh = nh spans = wd.find_elements_by_css_selector('span.q-box.qu-userSelect--text') for span in spans: print(span.text) print('-' * 70 + '\n')

What was your prom like? ---------------------------------------------------------------------- Prom. That experience of mythical proportions, whipped up into a foam and placed on a pedestal. “Oh, you simply must go! You’ll regret it when you’re older…” I didn’t want to go. I was adamant! And yet, on prom night, I found myself stuffed into a suit (for the last time, I promised myself) and sitting at a table. It started okay; I found some friends and we talked and walked around. One of my friends had a good laugh when I couldn’t find my table. The food wasn’t amazing, best described as downright bland, but I always expect food at parties to be lacklustre at best. Continue Reading ---------------------------------------------------------------------- ---------------------------------------------------------------------- Are you expected to have sex with your prom date after the prom? ---------------------------------------------------------------------- Did you have sex after prom? ---------------------------------------------------------------------- My daughter has been crying because no one asked her to the prom. What should I do? ---------------------------------------------------------------------- What is prom like? ---------------------------------------------------------------------- What did you wear to your prom? ---------------------------------------------------------------------- My Prom was tonight Continue Reading ---------------------------------------------------------------------- ---------------------------------------------------------------------- I ended up with three dates by mistake… whoops Names have been omitted to protect the innocent Senior year, I asked out a very dear friend of mine to prom. She told me maybe which later turned into a no. She decided she was going with her friend from her band class. Continue Reading ---------------------------------------------------------------------- ---------------------------------------------------------------------- My date's existence offended the sensibilities of a drunk guy. My date, who I'll call Nancy, had graduated from high school. We had been dating for months, so I invited her to my senior prom. Nancy was reluctant to go. She felt like she would be conspicuous and out of place. I convinced her, though. It was important to me. So we went. We'd been there maybe ten minutes when a drunk student yelled at Nancy. “Are you a dyke?” His speech was loud, belligerent, and slurred. Then, he turned and left. Thinking that I should talk to him, I called for him to wait. Wiser than me, Nancy pulled me back. “We sh Continue Reading ---------------------------------------------------------------------- ---------------------------------------------------------------------- What do people usually do after prom? ---------------------------------------------------------------------- What do teenagers usually do after prom? ---------------------------------------------------------------------- Does college have a prom? ---------------------------------------------------------------------- What age should a teen be able to come home from prom? ---------------------------------------------------------------------- What is a high school prom like? ---------------------------------------------------------------------- My best friend decided that we were all going to the prom. "We" being the collection of druggies, artists and outcasts that comprised the ragged clique that clung together likes ants crossing a river during the hell that was high school. Grace was a bit different. She was a proud, stoned outcast and valedictorian. She vexed authority and we loved her for it. If she said prom, we said sure. We asked why. she said, " Its ironic don't you see? We'll take acid and the very act of tripping will stand as protest to this ridiculous rite of passage that is America's paltry offering to its disenfranchise Continue Reading ---------------------------------------------------------------------- ---------------------------------------------------------------------- I hated my prom. I had gone because my parents guilted me into it. “I always wanted to go but was too poor. I wish I had the chance.” -Mom So I went. I don't even like dresses. I prefer my usual shorts/jeans and a tank top. I don't wear much make up. My sister slathered me in it. I don't have a dancing bone in my body. I felt uncomfortable being dragged to dance. Continue Reading ---------------------------------------------------------------------- ---------------------------------------------------------------------- The problem with the British version of a prom is, it's shit! Granted I can only talk for my own school and my own prom, but I'm almost certain it's the same right across the U.K. There's no real build up, hardly any money is spent, there's no committee organising it, there's no pomp or pizazz, it's no right of passage as it is in the US and A. Those that want to lose their virginity already have. Nobody is saving themselves for prom night. You're either one of the "cool people" that have already "done it" or you're one of the other kinds of people that haven't "done it", and feel like you'll ne Continue Reading ---------------------------------------------------------------------- ---------------------------------------------------------------------- Prom Sonnet My parents said I had to go to prom. A venture I knew would end badly. I said alright but I had my qualms. But I still bought a suit that was classy. I paid a couple hundred for both tickets. When the day came I went to go get her. Her friend was there trying to solicit. Just like that I felt like a chauffeur. We arrive at prom and she didn’t want to dance. I had no problem so we stuck to talking. I went to the bathroom and she saw her chance. I returned to see her and the quarterback grinding. I said nothing, but there was anger in my eyes. I was a fool not to see through her guise. ---------------------------------------------------------------------- Uh. We had “most likely” awards. You know which one. It was at Ritz-Carlton so I was confused why the food was just okay (actually I think the desserts were pretty good.) Some people slow danced. We had an after-party rave. It was alright, not too bad. At least I got to hang out with my friends. ---------------------------------------------------------------------- Nothing dramatic, unless you count me eating my dinner in a rush and almost throwing up in the bathroom. Oh, and the guy that I like was eyeing me from across the room. But past that, it was a very fun night for my best friends and me. We danced our butts off, to say the least. Everyone was dancing (some more inappropriate than others), the teachers joined in, and no one was fighting. Before prom, my friends and I took pictures to commemorate the occasion. There were a few wild after parties, but I didn’t go. Instead, I went to Whataburger with my best friend before heading home. Just a regular ni Continue Reading ---------------------------------------------------------------------- ---------------------------------------------------------------------- I don’t know; I wasn’t there. Yes, I made it through the entire four years of high school without attending a single adolescent mating ritual (also known as a school dance), and I didn’t come away as a social pariah (at least not for that). The thought of hanging out with a bunch of my peers who were overly-concerned about getting drunk and/or getting laid didn’t appeal to me in the least. I was too busy geeking out about Gene Puerling ’s use of vocal harmony, or the latest Manhattan Transfer project. These interests still intrigue me 35 years later, but I couldn’t tell you who went with whom to Continue Reading ---------------------------------------------------------------------- ----------------------------------------------------------------------

1条回答

网友

1楼 · 发布于 2024-09-28 19:06:19

我只能回答问题3，因为我在我的网络刮板上做过类似的事情

soup = BeautifulSoup(r.content, "html.parser")
links = []
for link in soup.find_all('a', href=True):
    if(link['href'].startswith("https://")):
        links.append(link['href'])

我正在使用请求，所以我不确定它将如何与您的代码一起工作，但是如果您使用了BeautifulSoup，您应该能够使用我代码的某些部分

相关问题更多 >

编程相关推荐

热门问题

热门文章