Web-scraping with beautifulsoup in different siblings

Question

I've tried different ways to scrape Answer1 and Answer2 from a website through BeautifulSoup, urllib and Selenium, but without success. Here's the simplified version:

<div class="div1">
  <p class="p1"></p>
  <p class="p2">
    <span>Question1</span>
    <strong>Answer1</strong>
    <br>
    <span>Question2</span>
    <strong>Answer2</strong>
    <br>

In selenium, I try to find Question1, then go to its parent and scrape Answer1. Below is the code I use, although it's not correct.

browser.find_elements_by_xpath("//span[contains(text(), 'Question1')]/parent::p/following::strong")

I believe bs is more efficient than selenium in this case. How would you do this in bs? Thanks!

Answer 1

The tags are not closed in your HTML. Check the code below...i've closed the <p> and <div> tags:

from bs4 import BeautifulSoup as BS
html = """
<div class="div1">
  <p class="p1"></p>
  <p class="p2">
    <span>Question1</span>
    <strong>Answer1</strong>
    <br>
    <span>Question2</span>
    <strong>Answer2</strong>
    <br>
    </p>
</div>
"""
soup = BS(html,'lxml')
QA = {x.text:y.text for x,y in zip(soup.select('span'),soup.select('strong'))}
print(QA)