当前位置： 文江博客话题详情

从包括隐藏的网站（包括隐藏的网站）中提取所有URL

发布于 2025-01-21 20:08:26 字数 636 浏览 1 评论 0原文

我想从网页中提取所有URL，包括出现在“隐藏”按钮后面的网页（请参阅图像）。

”，因此，按下蓝色'button''

：也可以在搜索中包含多个页面（请参阅图像）

< img src =“ https://i.sstatic.net/i1bwy.png” alt =“第二问题示例”>

谢谢！

我设法从页面中提取了URL，但这不包括隐藏的URL。

req = Request('https://www.sainsburys.co.uk/gol-ui/SearchResults/vegan') #example 

html_page = urlopen(req)

soup = BeautifulSoup(html_page, "lxml")

df = pandas.DataFrame()

links = []
for link in soup.findAll('a'):
    links.append(link.get('href'))

set_df = set(links)
df['Urls'] = list(set_df)
df = df.sort_values("Urls")
原文

I want to extract all Urls from a webpage including the ones appearing behind an 'hidden' button (see image).

so when pressing the blue 'button'

Also: would it be possible to include multiple pages in a search (see image)

2nd problem example

Thank you!

I managed to extract the urls from the pages, but this does not include the hidden ones.

req = Request('https://www.sainsburys.co.uk/gol-ui/SearchResults/vegan') #example 

html_page = urlopen(req)

soup = BeautifulSoup(html_page, "lxml")

df = pandas.DataFrame()

links = []
for link in soup.findAll('a'):
    links.append(link.get('href'))

set_df = set(links)
df['Urls'] = list(set_df)
df = df.sort_values("Urls")

如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论， 你可以免费 注册 一个本站的账号。

评论（1）

黎夕旧梦 2025-01-28 20:08:26

使用API​​。然后以有效载荷参数迭代页面：

import requests
import pandas as pd

url = 'https://www.sainsburys.co.uk/groceries-api/gol-services/product/v1/product'
payload ={
'filter[keyword]': 'vegan',
'include[PRODUCT_AD]': 'citrus',
'page_number': '1',
'page_size': '60',
'sort_order': 'FAVOURITES_FIRST'
 }


jsonData = requests.get(url, params=payload).json()
products = jsonData['products']

df = pd.DataFrame(products)

输出：

print(df.head(5).to_string())
  product_uid favourite_uid                                                          eans product_type                                                                    name                                                                                                                                    image image_zoom                                                                                                                          image_thumbnail                                                                                                                    image_thumbnail_small                                                                                                   full_url                                               unit_price                        retail_price  is_available                                                                                                                                                                                                                                                                                                                                                                                         promotions associations  is_alcoholic  is_spotlight  is_intolerant  is_mhra badges                                                                                                                       labels  zone department                                                                                 reviews breadcrumbs                                                                                                       assets description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 important_information attachments                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              categories               attributes     display_icons                                                  pdp_deep_link
0     2289977          None  [8719200006843, 0000050241569, 8719200175297, 5000118078002]        BASIC                                         Flora Lighter Vegan Spread 500g  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/97/8719200175297/8719200175297_L.jpeg       None  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/97/8719200175297/8719200175297_M.jpeg  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/97/8719200175297/8719200175297_S.jpeg                     https://www.sainsburys.co.uk/shop/gb/groceries/product/details/flora-light-spread-500g     {'price': 4.4, 'measure': 'kg', 'measure_amount': 1}   {'price': 2.2, 'measure': 'unit'}          True                                                                                                                                                                                                                                                                                                                                                                                                 []           []         False         False          False    False     []  [{'label_uid': 'Chilled', 'text': 'Chilled', 'alt_text': 'Chilled', 'color': '#005096', 'link_opens_in_new_window': False}]  None       None  {'is_enabled': True, 'product_uid': '2289977', 'total': 113, 'average_rating': 4.1504}          []  {'plp_image': 'https://assets.sainsburys-groceries.co.uk/gol/2289977/image.jpg', 'images': [], 'video': []}          []  [The above details have been prepared to help you select suitable products. Products and their ingredients are liable to change., <strong>You should always read the label before consuming or using the product and never rely solely on the information presented here., If you require specific advice on any Sainsbury's branded product, please contact our Customer Careline on 0800 636262. For all other products, please contact the manufacturer, whose details will appear on the packaging or label. Sainsbury's is therefore unable to accept liability for any incorrect information., You should also note that the picture / images show only our serving suggestions of how to serve or prepare your food – all accessories and additional items and/or ingredients pictured with the product you are purchasing are not included., This information is supplied for your personal use only. It may not be reproduced in any way without the prior consent of Sainsbury's Supermarkets Ltd and due acknowledgement]          []                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       [{'id': '267410', 'name': 'All butter & spreads'}, {'id': '267413', 'name': 'Buttery taste & sunflower spreads'}, {'id': '387882', 'name': 'Buttery taste & sunflower spreads'}, {'id': '387879', 'name': 'All butter & spreads'}, {'id': '428879', 'name': 'Buttery taste & sunflower spreads'}, {'id': '428884', 'name': 'Dairy free spreads'}, {'id': '428875', 'name': 'All butter & spreads'}, {'id': '428940', 'name': 'All dairy free'}, {'id': '462852', 'name': 'All vegan'}, {'id': '461862', 'name': 'Food'}, {'id': '461871', 'name': 'Vegetarian & plant based'}, {'id': '534859', 'name': 'Vegan'}]     {'brand': ['Flora']}  [Chilled, Vegan]  /shop/ProductDisplay?storeId=10151&langId=44&productId=110206
1     6728602          None                 [8000920580806, 8714100635698, 0000012384433]        BASIC  Magnum Vegan Mini Classic Chocolate & Almond Ice Cream Sticks x6 55 ml  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/98/8714100635698/8714100635698_L.jpeg       None  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/98/8714100635698/8714100635698_M.jpeg  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/98/8714100635698/8714100635698_S.jpeg  https://www.sainsburys.co.uk/shop/gb/groceries/product/details/magnum-mini--almond-white-ice-cream-6x60ml  {'price': 0.91, 'measure': 'ml', 'measure_amount': 100}     {'price': 3, 'measure': 'unit'}          True  [{'promotion_uid': '10513357', 'icon': 'https://www.sainsburys.co.uk/wcsstore/Sainsburys/Promotion assets/Promotion icons/SO_Fixed_Price_S_Icon.gif', 'link': '/shop/PromotionDisplayView?promotionId=10513357&productId=137364&langId=44&storeId=10151', 'strap_line': 'Only £3.00: Save £1.00', 'start_date': '2022-04-12T23:00:00Z', 'end_date': '2022-04-25T23:00:00Z', 'original_price': 4}]           []         False         False          False    False     []     [{'label_uid': 'Frozen', 'text': 'Frozen', 'alt_text': 'Frozen', 'color': '#005096', 'link_opens_in_new_window': False}]  None       None  {'is_enabled': True, 'product_uid': '6728602', 'total': 140, 'average_rating': 4.7214}          []  {'plp_image': 'https://assets.sainsburys-groceries.co.uk/gol/6728602/image.jpg', 'images': [], 'video': []}          []  [The above details have been prepared to help you select suitable products. Products and their ingredients are liable to change., <strong>You should always read the label before consuming or using the product and never rely solely on the information presented here., If you require specific advice on any Sainsbury's branded product, please contact our Customer Careline on 0800 636262. For all other products, please contact the manufacturer, whose details will appear on the packaging or label. Sainsbury's is therefore unable to accept liability for any incorrect information., You should also note that the picture / images show only our serving suggestions of how to serve or prepare your food – all accessories and additional items and/or ingredients pictured with the product you are purchasing are not included., This information is supplied for your personal use only. It may not be reproduced in any way without the prior consent of Sainsbury's Supermarkets Ltd and due acknowledgement]          []                                                                                                                                                                                                                                                                                                                                                     [{'id': '478363', 'name': 'Ben & Jerry's, Carte D'Or, Magnum & Wall's'}, {'id': '474596', 'name': 'Frozen essentials'}, {'id': '248078', 'name': 'Summer'}, {'id': '267389', 'name': 'Ice cream shop'}, {'id': '269815', 'name': 'All ice creams'}, {'id': '269819', 'name': 'All'}, {'id': '269820', 'name': 'Ice cream'}, {'id': '279839', 'name': 'Special offers'}, {'id': '385374', 'name': 'Quick tea time'}, {'id': '410376', 'name': 'Special offers'}, {'id': '410372', 'name': 'Special offers'}, {'id': '419866', 'name': 'Magnum'}, {'id': '419879', 'name': 'Handheld ice cream & lollies'}, {'id': '432368', 'name': 'Special offers'}, {'id': '431864', 'name': 'Summer is coming'}, {'id': '474586', 'name': 'All Summer'}, {'id': '465367', 'name': 'Frozen takeaway'}, {'id': '465352', 'name': 'Nectar prices'}]    {'brand': ['Magnum']}          [Frozen]  /shop/ProductDisplay?storeId=10151&langId=44&productId=137364
2     2001722          None                                [8719200088986, 0000050241293]        BASIC                                           Stork Vegan Baking Block 250g  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/86/8719200088986/8719200088986_L.jpeg       None  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/86/8719200088986/8719200088986_M.jpeg  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/86/8719200088986/8719200088986_S.jpeg               https://www.sainsburys.co.uk/shop/gb/groceries/product/details/stork-vegan-baking-block-250g       {'price': 5, 'measure': 'kg', 'measure_amount': 1}  {'price': 1.25, 'measure': 'unit'}          True                                                                                                                                                                                                                                                                                                                                                                                                 []           []         False         False          False    False     []  [{'label_uid': 'Chilled', 'text': 'Chilled', 'alt_text': 'Chilled', 'color': '#005096', 'link_opens_in_new_window': False}]  None       None       {'is_enabled': True, 'product_uid': '2001722', 'total': 5, 'average_rating': 4.2}          []  {'plp_image': 'https://assets.sainsburys-groceries.co.uk/gol/2001722/image.jpg', 'images': [], 'video': []}          []  [The above details have been prepared to help you select suitable products. Products and their ingredients are liable to change., <strong>You should always read the label before consuming or using the product and never rely solely on the information presented here., If you require specific advice on any Sainsbury's branded product, please contact our Customer Careline on 0800 636262. For all other products, please contact the manufacturer, whose details will appear on the packaging or label. Sainsbury's is therefore unable to accept liability for any incorrect information., You should also note that the picture / images show only our serving suggestions of how to serve or prepare your food – all accessories and additional items and/or ingredients pictured with the product you are purchasing are not included., This information is supplied for your personal use only. It may not be reproduced in any way without the prior consent of Sainsbury's Supermarkets Ltd and due acknowledgement]          []                                                                                                                                                                                                                                                                                                                                                                                                           [{'id': '474592', 'name': 'Dairy & chilled essentials'}, {'id': '267417', 'name': 'Cooking & baking'}, {'id': '267410', 'name': 'All butter & spreads'}, {'id': '267412', 'name': 'Block butter'}, {'id': '267413', 'name': 'Buttery taste & sunflower spreads'}, {'id': '387881', 'name': 'Block butter'}, {'id': '387882', 'name': 'Buttery taste & sunflower spreads'}, {'id': '387883', 'name': 'Cooking & baking'}, {'id': '387879', 'name': 'All butter & spreads'}, {'id': '428879', 'name': 'Buttery taste & sunflower spreads'}, {'id': '428880', 'name': 'Cooking & baking butter & spreads'}, {'id': '428875', 'name': 'All butter & spreads'}, {'id': '428877', 'name': 'Block butter'}, {'id': '462852', 'name': 'All vegan'}, {'id': '461862', 'name': 'Food'}]     {'brand': ['Stork']}         [Chilled]   /shop/ProductDisplay?storeId=10151&langId=44&productId=80536
3     7267574          None                                               [5013683305442]        BASIC                                          Cauldron Vegan Tofu Block 396g  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/42/5013683305442/5013683305442_L.jpeg       None  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/42/5013683305442/5013683305442_M.jpeg  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/42/5013683305442/5013683305442_S.jpeg                 https://www.sainsburys.co.uk/shop/gb/groceries/product/details/cauldron-original-tofu-396g    {'price': 5.05, 'measure': 'kg', 'measure_amount': 1}     {'price': 2, 'measure': 'unit'}          True                                                                                                                                                                                                                                                                                                                                                                                                 []           []         False         False          False    False     []  [{'label_uid': 'Chilled', 'text': 'Chilled', 'alt_text': 'Chilled', 'color': '#005096', 'link_opens_in_new_window': False}]  None       None      {'is_enabled': True, 'product_uid': '7267574', 'total': 36, 'average_rating': 4.5}          []  {'plp_image': 'https://assets.sainsburys-groceries.co.uk/gol/7267574/image.jpg', 'images': [], 'video': []}          []  [The above details have been prepared to help you select suitable products. Products and their ingredients are liable to change., <strong>You should always read the label before consuming or using the product and never rely solely on the information presented here., If you require specific advice on any Sainsbury's branded product, please contact our Customer Careline on 0800 636262. For all other products, please contact the manufacturer, whose details will appear on the packaging or label. Sainsbury's is therefore unable to accept liability for any incorrect information., You should also note that the picture / images show only our serving suggestions of how to serve or prepare your food – all accessories and additional items and/or ingredients pictured with the product you are purchasing are not included., This information is supplied for your personal use only. It may not be reproduced in any way without the prior consent of Sainsbury's Supermarkets Ltd and due acknowledgement]          []                                                                                                                                                                                                                                                                                                                                                                                                            [{'id': '267524', 'name': 'All pies, pasties & quiche'}, {'id': '274776', 'name': 'Special offers'}, {'id': '275319', 'name': 'All vegetarian & meatfree'}, {'id': '283832', 'name': 'Tofu'}, {'id': '371855', 'name': 'Quorn'}, {'id': '400867', 'name': 'Vegetarian'}, {'id': '400868', 'name': 'Vegan'}, {'id': '410376', 'name': 'Special offers'}, {'id': '428954', 'name': 'Pies'}, {'id': '438352', 'name': '5x Nectar points'}, {'id': '428991', 'name': 'Top rated'}, {'id': '448380', 'name': 'Simple meals'}, {'id': '461871', 'name': 'Vegetarian & plant based'}, {'id': '462852', 'name': 'All vegan'}, {'id': '461862', 'name': 'Food'}, {'id': '534857', 'name': 'All meat free'}, {'id': '534859', 'name': 'Vegan'}, {'id': '489876', 'name': 'Meat free'}]  {'brand': ['Cauldron']}         [Chilled]  /shop/ProductDisplay?storeId=10151&langId=44&productId=140978
4     4108351          None                                               [5010219501705]        BASIC                             Pure Dairy Free Vegan Sunflower Spread 500g  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/05/5010219501705/5010219501705_L.jpeg       None  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/05/5010219501705/5010219501705_M.jpeg  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/05/5010219501705/5010219501705_S.jpeg       https://www.sainsburys.co.uk/shop/gb/groceries/product/details/pure-dairy-free-sunflower-spread-500g     {'price': 3.6, 'measure': 'kg', 'measure_amount': 1}   {'price': 1.8, 'measure': 'unit'}          True                                                                                                                                                                                                                                                                                                                                                                                                 []           []         False         False          False    False     []  [{'label_uid': 'Chilled', 'text': 'Chilled', 'alt_text': 'Chilled', 'color': '#005096', 'link_opens_in_new_window': False}]  None       None   {'is_enabled': True, 'product_uid': '4108351', 'total': 75, 'average_rating': 3.3733}          []  {'plp_image': 'https://assets.sainsburys-groceries.co.uk/gol/4108351/image.jpg', 'images': [], 'video': []}          []  [The above details have been prepared to help you select suitable products. Products and their ingredients are liable to change., <strong>You should always read the label before consuming or using the product and never rely solely on the information presented here., If you require specific advice on any Sainsbury's branded product, please contact our Customer Careline on 0800 636262. For all other products, please contact the manufacturer, whose details will appear on the packaging or label. Sainsbury's is therefore unable to accept liability for any incorrect information., You should also note that the picture / images show only our serving suggestions of how to serve or prepare your food – all accessories and additional items and/or ingredients pictured with the product you are purchasing are not included., This information is supplied for your personal use only. It may not be reproduced in any way without the prior consent of Sainsbury's Supermarkets Ltd and due acknowledgement]          []  [{'id': '267416', 'name': 'Dairy free spreads'}, {'id': '267410', 'name': 'All butter & spreads'}, {'id': '272766', 'name': 'Spreads'}, {'id': '273760', 'name': 'Dairy free spreads'}, {'id': '387882', 'name': 'Buttery taste & sunflower spreads'}, {'id': '387886', 'name': 'Dairy free spreads'}, {'id': '387935', 'name': 'All dairy free'}, {'id': '387938', 'name': 'Dairy free spreads'}, {'id': '387879', 'name': 'All butter & spreads'}, {'id': '428879', 'name': 'Buttery taste & sunflower spreads'}, {'id': '428884', 'name': 'Dairy free spreads'}, {'id': '428875', 'name': 'All butter & spreads'}, {'id': '428940', 'name': 'All dairy free'}, {'id': '428945', 'name': 'Dairy free spreads'}, {'id': '462852', 'name': 'All vegan'}, {'id': '461862', 'name': 'Food'}, {'id': '461871', 'name': 'Vegetarian & plant based'}, {'id': '534859', 'name': 'Vegan'}, {'id': '534861', 'name': 'Dairy free'}, {'id': '461853', 'name': 'Cheese & dairy alternatives'}, {'id': '461852', 'name': 'All milk free'}, {'id': '453883', 'name': 'Lactose free'}, {'id': '453881', 'name': 'All gluten free'}, {'id': '450494', 'name': 'Cross sell audit - Do not leave on'}]      {'brand': ['Pure']}  [Chilled, Vegan]  /shop/ProductDisplay?storeId=10151&langId=44&productId=112694
....

[60 rows x 33 columns]

Just urls:

print(df['full_url'])
0     https://www.sainsburys.co.uk/shop/gb/groceries...
1     https://www.sainsburys.co.uk/shop/gb/groceries...
2     https://www.sainsburys.co.uk/shop/gb/groceries...
3     https://www.sainsburys.co.uk/shop/gb/groceries...
4     https://www.sainsburys.co.uk/shop/gb/groceries...
5     https://www.sainsburys.co.uk/shop/gb/groceries...
6     https://www.sainsburys.co.uk/shop/gb/groceries...
7     https://www.sainsburys.co.uk/shop/gb/groceries...
8     https://www.sainsburys.co.uk/shop/gb/groceries...
9     https://www.sainsburys.co.uk/shop/gb/groceries...
10    https://www.sainsburys.co.uk/shop/gb/groceries...
11    https://www.sainsburys.co.uk/shop/gb/groceries...
12    https://www.sainsburys.co.uk/shop/gb/groceries...
13    https://www.sainsburys.co.uk/shop/gb/groceries...
14    https://www.sainsburys.co.uk/shop/gb/groceries...
15    https://www.sainsburys.co.uk/shop/gb/groceries...
16    https://www.sainsburys.co.uk/shop/gb/groceries...
17    https://www.sainsburys.co.uk/shop/gb/groceries...
18    https://www.sainsburys.co.uk/shop/gb/groceries...
19    https://www.sainsburys.co.uk/shop/gb/groceries...
20    https://www.sainsburys.co.uk/shop/gb/groceries...
21    https://www.sainsburys.co.uk/shop/gb/groceries...
22    https://www.sainsburys.co.uk/shop/gb/groceries...
23    https://www.sainsburys.co.uk/shop/gb/groceries...
24    https://www.sainsburys.co.uk/shop/gb/groceries...
25    https://www.sainsburys.co.uk/shop/gb/groceries...
26    https://www.sainsburys.co.uk/shop/gb/groceries...
27    https://www.sainsburys.co.uk/shop/gb/groceries...
28    https://www.sainsburys.co.uk/shop/gb/groceries...
29    https://www.sainsburys.co.uk/shop/gb/groceries...
30    https://www.sainsburys.co.uk/shop/gb/groceries...
31    https://www.sainsburys.co.uk/shop/gb/groceries...
32    https://www.sainsburys.co.uk/shop/gb/groceries...
33    https://www.sainsburys.co.uk/shop/gb/groceries...
34    https://www.sainsburys.co.uk/shop/gb/groceries...
35    https://www.sainsburys.co.uk/shop/gb/groceries...
36    https://www.sainsburys.co.uk/shop/gb/groceries...
37    https://www.sainsburys.co.uk/shop/gb/groceries...
38    https://www.sainsburys.co.uk/shop/gb/groceries...
39    https://www.sainsburys.co.uk/shop/gb/groceries...
40    https://www.sainsburys.co.uk/shop/gb/groceries...
41    https://www.sainsburys.co.uk/shop/gb/groceries...
42    https://www.sainsburys.co.uk/shop/gb/groceries...
43    https://www.sainsburys.co.uk/shop/gb/groceries...
44    https://www.sainsburys.co.uk/shop/gb/groceries...
45    https://www.sainsburys.co.uk/shop/gb/groceries...
46    https://www.sainsburys.co.uk/shop/gb/groceries...
47    https://www.sainsburys.co.uk/shop/gb/groceries...
48    https://www.sainsburys.co.uk/shop/gb/groceries...
49    https://www.sainsburys.co.uk/shop/gb/groceries...
50    https://www.sainsburys.co.uk/shop/gb/groceries...
51    https://www.sainsburys.co.uk/shop/gb/groceries...
52    https://www.sainsburys.co.uk/shop/gb/groceries...
53    https://www.sainsburys.co.uk/shop/gb/groceries...
54    https://www.sainsburys.co.uk/shop/gb/groceries...
55    https://www.sainsburys.co.uk/shop/gb/groceries...
56    https://www.sainsburys.co.uk/shop/gb/groceries...
57    https://www.sainsburys.co.uk/shop/gb/groceries...
58    https://www.sainsburys.co.uk/shop/gb/groceries...
59    https://www.sainsburys.co.uk/shop/gb/groceries...
Name: full_url, dtype: object

Use the api. Then iterate through the pages with payload parameter:

import requests
import pandas as pd

url = 'https://www.sainsburys.co.uk/groceries-api/gol-services/product/v1/product'
payload ={
'filter[keyword]': 'vegan',
'include[PRODUCT_AD]': 'citrus',
'page_number': '1',
'page_size': '60',
'sort_order': 'FAVOURITES_FIRST'
 }


jsonData = requests.get(url, params=payload).json()
products = jsonData['products']

df = pd.DataFrame(products)

Output:

print(df.head(5).to_string())
  product_uid favourite_uid                                                          eans product_type                                                                    name                                                                                                                                    image image_zoom                                                                                                                          image_thumbnail                                                                                                                    image_thumbnail_small                                                                                                   full_url                                               unit_price                        retail_price  is_available                                                                                                                                                                                                                                                                                                                                                                                         promotions associations  is_alcoholic  is_spotlight  is_intolerant  is_mhra badges                                                                                                                       labels  zone department                                                                                 reviews breadcrumbs                                                                                                       assets description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 important_information attachments                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              categories               attributes     display_icons                                                  pdp_deep_link
0     2289977          None  [8719200006843, 0000050241569, 8719200175297, 5000118078002]        BASIC                                         Flora Lighter Vegan Spread 500g  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/97/8719200175297/8719200175297_L.jpeg       None  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/97/8719200175297/8719200175297_M.jpeg  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/97/8719200175297/8719200175297_S.jpeg                     https://www.sainsburys.co.uk/shop/gb/groceries/product/details/flora-light-spread-500g     {'price': 4.4, 'measure': 'kg', 'measure_amount': 1}   {'price': 2.2, 'measure': 'unit'}          True                                                                                                                                                                                                                                                                                                                                                                                                 []           []         False         False          False    False     []  [{'label_uid': 'Chilled', 'text': 'Chilled', 'alt_text': 'Chilled', 'color': '#005096', 'link_opens_in_new_window': False}]  None       None  {'is_enabled': True, 'product_uid': '2289977', 'total': 113, 'average_rating': 4.1504}          []  {'plp_image': 'https://assets.sainsburys-groceries.co.uk/gol/2289977/image.jpg', 'images': [], 'video': []}          []  [The above details have been prepared to help you select suitable products. Products and their ingredients are liable to change., <strong>You should always read the label before consuming or using the product and never rely solely on the information presented here., If you require specific advice on any Sainsbury's branded product, please contact our Customer Careline on 0800 636262. For all other products, please contact the manufacturer, whose details will appear on the packaging or label. Sainsbury's is therefore unable to accept liability for any incorrect information., You should also note that the picture / images show only our serving suggestions of how to serve or prepare your food – all accessories and additional items and/or ingredients pictured with the product you are purchasing are not included., This information is supplied for your personal use only. It may not be reproduced in any way without the prior consent of Sainsbury's Supermarkets Ltd and due acknowledgement]          []                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       [{'id': '267410', 'name': 'All butter & spreads'}, {'id': '267413', 'name': 'Buttery taste & sunflower spreads'}, {'id': '387882', 'name': 'Buttery taste & sunflower spreads'}, {'id': '387879', 'name': 'All butter & spreads'}, {'id': '428879', 'name': 'Buttery taste & sunflower spreads'}, {'id': '428884', 'name': 'Dairy free spreads'}, {'id': '428875', 'name': 'All butter & spreads'}, {'id': '428940', 'name': 'All dairy free'}, {'id': '462852', 'name': 'All vegan'}, {'id': '461862', 'name': 'Food'}, {'id': '461871', 'name': 'Vegetarian & plant based'}, {'id': '534859', 'name': 'Vegan'}]     {'brand': ['Flora']}  [Chilled, Vegan]  /shop/ProductDisplay?storeId=10151&langId=44&productId=110206
1     6728602          None                 [8000920580806, 8714100635698, 0000012384433]        BASIC  Magnum Vegan Mini Classic Chocolate & Almond Ice Cream Sticks x6 55 ml  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/98/8714100635698/8714100635698_L.jpeg       None  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/98/8714100635698/8714100635698_M.jpeg  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/98/8714100635698/8714100635698_S.jpeg  https://www.sainsburys.co.uk/shop/gb/groceries/product/details/magnum-mini--almond-white-ice-cream-6x60ml  {'price': 0.91, 'measure': 'ml', 'measure_amount': 100}     {'price': 3, 'measure': 'unit'}          True  [{'promotion_uid': '10513357', 'icon': 'https://www.sainsburys.co.uk/wcsstore/Sainsburys/Promotion assets/Promotion icons/SO_Fixed_Price_S_Icon.gif', 'link': '/shop/PromotionDisplayView?promotionId=10513357&productId=137364&langId=44&storeId=10151', 'strap_line': 'Only £3.00: Save £1.00', 'start_date': '2022-04-12T23:00:00Z', 'end_date': '2022-04-25T23:00:00Z', 'original_price': 4}]           []         False         False          False    False     []     [{'label_uid': 'Frozen', 'text': 'Frozen', 'alt_text': 'Frozen', 'color': '#005096', 'link_opens_in_new_window': False}]  None       None  {'is_enabled': True, 'product_uid': '6728602', 'total': 140, 'average_rating': 4.7214}          []  {'plp_image': 'https://assets.sainsburys-groceries.co.uk/gol/6728602/image.jpg', 'images': [], 'video': []}          []  [The above details have been prepared to help you select suitable products. Products and their ingredients are liable to change., <strong>You should always read the label before consuming or using the product and never rely solely on the information presented here., If you require specific advice on any Sainsbury's branded product, please contact our Customer Careline on 0800 636262. For all other products, please contact the manufacturer, whose details will appear on the packaging or label. Sainsbury's is therefore unable to accept liability for any incorrect information., You should also note that the picture / images show only our serving suggestions of how to serve or prepare your food – all accessories and additional items and/or ingredients pictured with the product you are purchasing are not included., This information is supplied for your personal use only. It may not be reproduced in any way without the prior consent of Sainsbury's Supermarkets Ltd and due acknowledgement]          []                                                                                                                                                                                                                                                                                                                                                     [{'id': '478363', 'name': 'Ben & Jerry's, Carte D'Or, Magnum & Wall's'}, {'id': '474596', 'name': 'Frozen essentials'}, {'id': '248078', 'name': 'Summer'}, {'id': '267389', 'name': 'Ice cream shop'}, {'id': '269815', 'name': 'All ice creams'}, {'id': '269819', 'name': 'All'}, {'id': '269820', 'name': 'Ice cream'}, {'id': '279839', 'name': 'Special offers'}, {'id': '385374', 'name': 'Quick tea time'}, {'id': '410376', 'name': 'Special offers'}, {'id': '410372', 'name': 'Special offers'}, {'id': '419866', 'name': 'Magnum'}, {'id': '419879', 'name': 'Handheld ice cream & lollies'}, {'id': '432368', 'name': 'Special offers'}, {'id': '431864', 'name': 'Summer is coming'}, {'id': '474586', 'name': 'All Summer'}, {'id': '465367', 'name': 'Frozen takeaway'}, {'id': '465352', 'name': 'Nectar prices'}]    {'brand': ['Magnum']}          [Frozen]  /shop/ProductDisplay?storeId=10151&langId=44&productId=137364
2     2001722          None                                [8719200088986, 0000050241293]        BASIC                                           Stork Vegan Baking Block 250g  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/86/8719200088986/8719200088986_L.jpeg       None  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/86/8719200088986/8719200088986_M.jpeg  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/86/8719200088986/8719200088986_S.jpeg               https://www.sainsburys.co.uk/shop/gb/groceries/product/details/stork-vegan-baking-block-250g       {'price': 5, 'measure': 'kg', 'measure_amount': 1}  {'price': 1.25, 'measure': 'unit'}          True                                                                                                                                                                                                                                                                                                                                                                                                 []           []         False         False          False    False     []  [{'label_uid': 'Chilled', 'text': 'Chilled', 'alt_text': 'Chilled', 'color': '#005096', 'link_opens_in_new_window': False}]  None       None       {'is_enabled': True, 'product_uid': '2001722', 'total': 5, 'average_rating': 4.2}          []  {'plp_image': 'https://assets.sainsburys-groceries.co.uk/gol/2001722/image.jpg', 'images': [], 'video': []}          []  [The above details have been prepared to help you select suitable products. Products and their ingredients are liable to change., <strong>You should always read the label before consuming or using the product and never rely solely on the information presented here., If you require specific advice on any Sainsbury's branded product, please contact our Customer Careline on 0800 636262. For all other products, please contact the manufacturer, whose details will appear on the packaging or label. Sainsbury's is therefore unable to accept liability for any incorrect information., You should also note that the picture / images show only our serving suggestions of how to serve or prepare your food – all accessories and additional items and/or ingredients pictured with the product you are purchasing are not included., This information is supplied for your personal use only. It may not be reproduced in any way without the prior consent of Sainsbury's Supermarkets Ltd and due acknowledgement]          []                                                                                                                                                                                                                                                                                                                                                                                                           [{'id': '474592', 'name': 'Dairy & chilled essentials'}, {'id': '267417', 'name': 'Cooking & baking'}, {'id': '267410', 'name': 'All butter & spreads'}, {'id': '267412', 'name': 'Block butter'}, {'id': '267413', 'name': 'Buttery taste & sunflower spreads'}, {'id': '387881', 'name': 'Block butter'}, {'id': '387882', 'name': 'Buttery taste & sunflower spreads'}, {'id': '387883', 'name': 'Cooking & baking'}, {'id': '387879', 'name': 'All butter & spreads'}, {'id': '428879', 'name': 'Buttery taste & sunflower spreads'}, {'id': '428880', 'name': 'Cooking & baking butter & spreads'}, {'id': '428875', 'name': 'All butter & spreads'}, {'id': '428877', 'name': 'Block butter'}, {'id': '462852', 'name': 'All vegan'}, {'id': '461862', 'name': 'Food'}]     {'brand': ['Stork']}         [Chilled]   /shop/ProductDisplay?storeId=10151&langId=44&productId=80536
3     7267574          None                                               [5013683305442]        BASIC                                          Cauldron Vegan Tofu Block 396g  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/42/5013683305442/5013683305442_L.jpeg       None  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/42/5013683305442/5013683305442_M.jpeg  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/42/5013683305442/5013683305442_S.jpeg                 https://www.sainsburys.co.uk/shop/gb/groceries/product/details/cauldron-original-tofu-396g    {'price': 5.05, 'measure': 'kg', 'measure_amount': 1}     {'price': 2, 'measure': 'unit'}          True                                                                                                                                                                                                                                                                                                                                                                                                 []           []         False         False          False    False     []  [{'label_uid': 'Chilled', 'text': 'Chilled', 'alt_text': 'Chilled', 'color': '#005096', 'link_opens_in_new_window': False}]  None       None      {'is_enabled': True, 'product_uid': '7267574', 'total': 36, 'average_rating': 4.5}          []  {'plp_image': 'https://assets.sainsburys-groceries.co.uk/gol/7267574/image.jpg', 'images': [], 'video': []}          []  [The above details have been prepared to help you select suitable products. Products and their ingredients are liable to change., <strong>You should always read the label before consuming or using the product and never rely solely on the information presented here., If you require specific advice on any Sainsbury's branded product, please contact our Customer Careline on 0800 636262. For all other products, please contact the manufacturer, whose details will appear on the packaging or label. Sainsbury's is therefore unable to accept liability for any incorrect information., You should also note that the picture / images show only our serving suggestions of how to serve or prepare your food – all accessories and additional items and/or ingredients pictured with the product you are purchasing are not included., This information is supplied for your personal use only. It may not be reproduced in any way without the prior consent of Sainsbury's Supermarkets Ltd and due acknowledgement]          []                                                                                                                                                                                                                                                                                                                                                                                                            [{'id': '267524', 'name': 'All pies, pasties & quiche'}, {'id': '274776', 'name': 'Special offers'}, {'id': '275319', 'name': 'All vegetarian & meatfree'}, {'id': '283832', 'name': 'Tofu'}, {'id': '371855', 'name': 'Quorn'}, {'id': '400867', 'name': 'Vegetarian'}, {'id': '400868', 'name': 'Vegan'}, {'id': '410376', 'name': 'Special offers'}, {'id': '428954', 'name': 'Pies'}, {'id': '438352', 'name': '5x Nectar points'}, {'id': '428991', 'name': 'Top rated'}, {'id': '448380', 'name': 'Simple meals'}, {'id': '461871', 'name': 'Vegetarian & plant based'}, {'id': '462852', 'name': 'All vegan'}, {'id': '461862', 'name': 'Food'}, {'id': '534857', 'name': 'All meat free'}, {'id': '534859', 'name': 'Vegan'}, {'id': '489876', 'name': 'Meat free'}]  {'brand': ['Cauldron']}         [Chilled]  /shop/ProductDisplay?storeId=10151&langId=44&productId=140978
4     4108351          None                                               [5010219501705]        BASIC                             Pure Dairy Free Vegan Sunflower Spread 500g  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/05/5010219501705/5010219501705_L.jpeg       None  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/05/5010219501705/5010219501705_M.jpeg  https://www.sainsburys.co.uk/wcsstore/ExtendedSitesCatalogAssetStore/images/catalog/productImages/05/5010219501705/5010219501705_S.jpeg       https://www.sainsburys.co.uk/shop/gb/groceries/product/details/pure-dairy-free-sunflower-spread-500g     {'price': 3.6, 'measure': 'kg', 'measure_amount': 1}   {'price': 1.8, 'measure': 'unit'}          True                                                                                                                                                                                                                                                                                                                                                                                                 []           []         False         False          False    False     []  [{'label_uid': 'Chilled', 'text': 'Chilled', 'alt_text': 'Chilled', 'color': '#005096', 'link_opens_in_new_window': False}]  None       None   {'is_enabled': True, 'product_uid': '4108351', 'total': 75, 'average_rating': 3.3733}          []  {'plp_image': 'https://assets.sainsburys-groceries.co.uk/gol/4108351/image.jpg', 'images': [], 'video': []}          []  [The above details have been prepared to help you select suitable products. Products and their ingredients are liable to change., <strong>You should always read the label before consuming or using the product and never rely solely on the information presented here., If you require specific advice on any Sainsbury's branded product, please contact our Customer Careline on 0800 636262. For all other products, please contact the manufacturer, whose details will appear on the packaging or label. Sainsbury's is therefore unable to accept liability for any incorrect information., You should also note that the picture / images show only our serving suggestions of how to serve or prepare your food – all accessories and additional items and/or ingredients pictured with the product you are purchasing are not included., This information is supplied for your personal use only. It may not be reproduced in any way without the prior consent of Sainsbury's Supermarkets Ltd and due acknowledgement]          []  [{'id': '267416', 'name': 'Dairy free spreads'}, {'id': '267410', 'name': 'All butter & spreads'}, {'id': '272766', 'name': 'Spreads'}, {'id': '273760', 'name': 'Dairy free spreads'}, {'id': '387882', 'name': 'Buttery taste & sunflower spreads'}, {'id': '387886', 'name': 'Dairy free spreads'}, {'id': '387935', 'name': 'All dairy free'}, {'id': '387938', 'name': 'Dairy free spreads'}, {'id': '387879', 'name': 'All butter & spreads'}, {'id': '428879', 'name': 'Buttery taste & sunflower spreads'}, {'id': '428884', 'name': 'Dairy free spreads'}, {'id': '428875', 'name': 'All butter & spreads'}, {'id': '428940', 'name': 'All dairy free'}, {'id': '428945', 'name': 'Dairy free spreads'}, {'id': '462852', 'name': 'All vegan'}, {'id': '461862', 'name': 'Food'}, {'id': '461871', 'name': 'Vegetarian & plant based'}, {'id': '534859', 'name': 'Vegan'}, {'id': '534861', 'name': 'Dairy free'}, {'id': '461853', 'name': 'Cheese & dairy alternatives'}, {'id': '461852', 'name': 'All milk free'}, {'id': '453883', 'name': 'Lactose free'}, {'id': '453881', 'name': 'All gluten free'}, {'id': '450494', 'name': 'Cross sell audit - Do not leave on'}]      {'brand': ['Pure']}  [Chilled, Vegan]  /shop/ProductDisplay?storeId=10151&langId=44&productId=112694
....

[60 rows x 33 columns]

Just urls:

print(df['full_url'])
0     https://www.sainsburys.co.uk/shop/gb/groceries...
1     https://www.sainsburys.co.uk/shop/gb/groceries...
2     https://www.sainsburys.co.uk/shop/gb/groceries...
3     https://www.sainsburys.co.uk/shop/gb/groceries...
4     https://www.sainsburys.co.uk/shop/gb/groceries...
5     https://www.sainsburys.co.uk/shop/gb/groceries...
6     https://www.sainsburys.co.uk/shop/gb/groceries...
7     https://www.sainsburys.co.uk/shop/gb/groceries...
8     https://www.sainsburys.co.uk/shop/gb/groceries...
9     https://www.sainsburys.co.uk/shop/gb/groceries...
10    https://www.sainsburys.co.uk/shop/gb/groceries...
11    https://www.sainsburys.co.uk/shop/gb/groceries...
12    https://www.sainsburys.co.uk/shop/gb/groceries...
13    https://www.sainsburys.co.uk/shop/gb/groceries...
14    https://www.sainsburys.co.uk/shop/gb/groceries...
15    https://www.sainsburys.co.uk/shop/gb/groceries...
16    https://www.sainsburys.co.uk/shop/gb/groceries...
17    https://www.sainsburys.co.uk/shop/gb/groceries...
18    https://www.sainsburys.co.uk/shop/gb/groceries...
19    https://www.sainsburys.co.uk/shop/gb/groceries...
20    https://www.sainsburys.co.uk/shop/gb/groceries...
21    https://www.sainsburys.co.uk/shop/gb/groceries...
22    https://www.sainsburys.co.uk/shop/gb/groceries...
23    https://www.sainsburys.co.uk/shop/gb/groceries...
24    https://www.sainsburys.co.uk/shop/gb/groceries...
25    https://www.sainsburys.co.uk/shop/gb/groceries...
26    https://www.sainsburys.co.uk/shop/gb/groceries...
27    https://www.sainsburys.co.uk/shop/gb/groceries...
28    https://www.sainsburys.co.uk/shop/gb/groceries...
29    https://www.sainsburys.co.uk/shop/gb/groceries...
30    https://www.sainsburys.co.uk/shop/gb/groceries...
31    https://www.sainsburys.co.uk/shop/gb/groceries...
32    https://www.sainsburys.co.uk/shop/gb/groceries...
33    https://www.sainsburys.co.uk/shop/gb/groceries...
34    https://www.sainsburys.co.uk/shop/gb/groceries...
35    https://www.sainsburys.co.uk/shop/gb/groceries...
36    https://www.sainsburys.co.uk/shop/gb/groceries...
37    https://www.sainsburys.co.uk/shop/gb/groceries...
38    https://www.sainsburys.co.uk/shop/gb/groceries...
39    https://www.sainsburys.co.uk/shop/gb/groceries...
40    https://www.sainsburys.co.uk/shop/gb/groceries...
41    https://www.sainsburys.co.uk/shop/gb/groceries...
42    https://www.sainsburys.co.uk/shop/gb/groceries...
43    https://www.sainsburys.co.uk/shop/gb/groceries...
44    https://www.sainsburys.co.uk/shop/gb/groceries...
45    https://www.sainsburys.co.uk/shop/gb/groceries...
46    https://www.sainsburys.co.uk/shop/gb/groceries...
47    https://www.sainsburys.co.uk/shop/gb/groceries...
48    https://www.sainsburys.co.uk/shop/gb/groceries...
49    https://www.sainsburys.co.uk/shop/gb/groceries...
50    https://www.sainsburys.co.uk/shop/gb/groceries...
51    https://www.sainsburys.co.uk/shop/gb/groceries...
52    https://www.sainsburys.co.uk/shop/gb/groceries...
53    https://www.sainsburys.co.uk/shop/gb/groceries...
54    https://www.sainsburys.co.uk/shop/gb/groceries...
55    https://www.sainsburys.co.uk/shop/gb/groceries...
56    https://www.sainsburys.co.uk/shop/gb/groceries...
57    https://www.sainsburys.co.uk/shop/gb/groceries...
58    https://www.sainsburys.co.uk/shop/gb/groceries...
59    https://www.sainsburys.co.uk/shop/gb/groceries...
Name: full_url, dtype: object
回复 原文
~没有更多了~

关于作者

意犹

暂无简介

文章
评论
28 人气
发私信

相关话题

更多

热门标签

操作系统 程序设计 IT运维 Linux系统管理 JavaScript 服务器应用 solaris C/C++ PHP Shell BSD Vue.js aix Oracle Python HTML 系统管理 HTML5 CSS 前端
更多

推荐作者

夢野间

文章 0 评论 0

百度③文鱼

文章 0 评论 0

小草泠泠

文章 0 评论 0

zhuwenyan

文章 0 评论 0

weirdo

文章 0 评论 0

坚持沉默

文章 0 评论 0

更多

友情链接

文江博客
    我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站，即表示您同意使用 Cookies 和您的相关数据。
    原文