jsondecodeerror：期望＆＃x27;，＆＃x27;长json字符串中的定界符

Question

我正在尝试解析以下JSON，但我总是面临“ jsondecodeerror：期望”，“定界符”

这是我正在做的代码：

import requests
from bs4 import  BeautifulSoup
import json

page_link="https://www.indeed.com/cmp/Ocean-Beauty-Seafoods/reviews?start=0"
page_response = requests.get(page_link, verify=False)
soup = BeautifulSoup(page_response.content, 'html.parser')
strJson=soup.findAll('script')[16].text.replace("\n    window._initialData=JSON.parse(\'","").replace("');","")
json.loads(strJson)

非常感谢

Answer 1

JSON是无效的。尝试首先使用 ast.code> ast.literal_eval ：

import json
import requests
from ast import literal_eval
from bs4 import BeautifulSoup


page_link = "https://www.indeed.com/cmp/Ocean-Beauty-Seafoods/reviews?start=0"
page_response = requests.get(page_link, verify=False)
soup = BeautifulSoup(page_response.content, "html.parser")

strJson = (
    soup.findAll("script")[16]
    .text.replace("\n    window._initialData=JSON.parse('", "")
    .replace("');", "")
)

s = literal_eval("'''" + strJson + "'''")
data = json.loads(s)

print(json.dumps(data, indent=4))

打印：

{
    "breadcrumbs": {
        "breadcrumbs": [
            {
                "name": "Companies",
                "noFollow": false,
                "url": "https://www.indeed.com/companies"
            },
            {
                "name": "Ocean Beauty Seafoods",
                "noFollow": false,
                "url": "https://www.indeed.com/cmp/Ocean-Beauty-Seafoods"
            },
            {
                "name": "Employee Reviews",
                "noFollow": false
            }
        ]
    },
    "companyPageFooter": {
        "enabledToShowUserFeedbackForm": false,
        "encodedFccId": "a9c95405fb0cdb1c",
        "stickyJobsTabLink": {
            "jobsLink": "/cmp/Ocean-Beauty-Seafoods/jobs"
        }
    },
    "companyPageHeader": {
        "auroraLogoUrl": "https://d2q79iu7y748jz.cloudfront.net/s/_squarelogo/64x64/147cafc3914ffb4693dc99df6ad0b169",
        "auroraLogoUrl2x": "https://d2q79iu7y748jz.cloudfront.net/s/_squarelogo/128x128/147cafc3914ffb4693dc99df6ad0b169",
        "brandColor": "#FFFFFF",
        "companyHeader": {
            "name": "Ocean Beauty Seafoods",
            "rating": 3.7,
            "reviewCount": 114,
            "reviewCountFormatted": "114",
            "reviewsUrl": "/cmp/Ocean-Beauty-Seafoods/reviews"
        },

...and so on.