ValueError arrays must all be same length ValueError arrays must all be same length

Question

from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup
import pandas as pd
import xlsxwriter

my_url = "https://www.ebay.com/sch/i.html?_from=R40&_trksid=p2495737.m570.l1313.TR11.TRC2.A0.H1.XSamsung+Galaxy+.TRS1&_nkw=Samsung+Galaxy+&_sacat=0"

uClient = uReq(my_url)
page_html = uClient.read()
uClient.close()

page_soup = soup(page_html, "html.parser")

containers = page_soup.findAll("div", {"class": "s-item__info clearfix"})

product = []
price = []
condition = []

# filename = "ebay2021.csv"
# f = open(filename, "w", encoding='utf-8')
# headers = "product_name, product_price, product_condition\n"
# f.write(headers)

for container in containers:
    brand = container.findAll('h3', {'class': 's-item__title'})
    product_name = brand[0].text

    price = container.findAll('span', {'class': 's-item__price'})
    product_price = price[0].text
    #
    condition = container.findAll("span", {"class": "SECONDARY_INFO"})
    product_condition = condition[0].text

    product.append(product_name)
    price.append(product_price)
    condition.append(product_condition)

    df = pd.DataFrame({'Product Name': product, 'Price': price, 'Rating': condition})
    df.to_csv('products.csv', index=False, encoding='utf-8')

Comments

Hey, You can do this to avoid the error:

a = {'Product Name': product, 'Price': price, 'Rating': condition}
df = pd.DataFrame.from_dict(a, orient='index')
df.transpose()

---

Yes, this worked! make sure to add the last line otherwise it will not store data into CSV.

Thanks :-)

a = {

    "Price": prices,

    "Property Type" : types,

    "Infromation": descriptions,

    "Location " : addresses,

    "Features" : features

}

df = pd.DataFrame.from_dict(a, orient='index')

df.transpose()

df.to_csv('website3.csv', index=False,header=True, encoding='utf-8')

Answer 1

Hi, 

You need to specify index for your DataFrame. Without an index, Pandas will not you to convert your dictionary to DataFrame. You can modify your code as given below.

a = {'Product Name': product, 'Price': price, 'Rating': condition}
df = pd.DataFrame.from_dict(a, orient='index')
df.transpose()
df.to_csv('website3.csv', index=False,header=True, encoding='utf-8')