httpClient请求得到的页面和Chrome监控到的返回页面相比，少了一些内容？

Question

背景

用httpClient爬取页面的时候发现爬到的页面和chrome监控到的少了几行。

Chrome监控到的：

我抓到的：

可以看到下面比上面少了一些内容。

问题

1. 我第一感觉是js生成的，请问chrome监控到的页面是经过js脚本加工之后的吗，不应该请求返回的原始页面吗？
2. 我在source中看了所有的js脚本，都没有查找到相关的添加元素的脚本代码。请问这个问题的可能原因是什么，我这样找对吗？

补充1

以下是我模拟参数的代码

/以下参数除去了抓包时看到的undefined的值
                nvps.add(new BasicNameValuePair("businessType", "PCIS"));
                nvps.add(new BasicNameValuePair("businessNo", ePosPaymentReturnVO.getBusinessNo()));
                nvps.add(new BasicNameValuePair("customerName", ePosPaymentReturnVO.getCustomerName()));
                nvps.add(new BasicNameValuePair("currencyNo", ePosPaymentReturnVO.getCurrencyNo()));
                nvps.add(new BasicNameValuePair("amount", ePosPaymentReturnVO.getAmount()));
                nvps.add(new BasicNameValuePair("regionCode", ePosPaymentReturnVO.getRegionCode()));
                nvps.add(new BasicNameValuePair("insuredName", ePosPaymentReturnVO.getInsuredName()));
                nvps.add(new BasicNameValuePair("tellerNo", ePosPaymentReturnVO.getTellerNo()));
                nvps.add(new BasicNameValuePair("branchNo", ePosPaymentReturnVO.getBranchNo()));
                nvps.add(new BasicNameValuePair("documentNo", ""));
                nvps.add(new BasicNameValuePair("callBackInfo", ""));
                nvps.add(new BasicNameValuePair("certPubKey", ePosPaymentReturnVO.getCertPubKey()));
                nvps.add(new BasicNameValuePair("signData", ePosPaymentReturnVO.getSignData()));
                nvps.add(new BasicNameValuePair("applicantName", ePosPaymentReturnVO.getApplicantName()));
                nvps.add(new BasicNameValuePair("applicantCertificateType", ePosPaymentReturnVO.getApplicantCertificateType()));
                nvps.add(new BasicNameValuePair("applicantCertificateNo", ePosPaymentReturnVO.getApplicantCertificateNo()));
                nvps.add(new BasicNameValuePair("ncpInsuredName", ePosPaymentReturnVO.getNcpInsuredName()));
                nvps.add(new BasicNameValuePair("insuredCertificateType", ePosPaymentReturnVO.getInsuredCertificateType()));
                nvps.add(new BasicNameValuePair("insuredCertificateNo", ePosPaymentReturnVO.getInsuredCertificateNo()));
                nvps.add(new BasicNameValuePair("networkFlag", ePosPaymentReturnVO.getNetworkFlag()));
                nvps.add(new BasicNameValuePair("applicantTelephone", ePosPaymentReturnVO.getApplicantTelephone()));
                nvps.add(new BasicNameValuePair("channelSource", ePosPaymentReturnVO.getChannelSource()));
                nvps.add(new BasicNameValuePair("businessTranChnl", ePosPaymentReturnVO.getBusinessTranChnl()));
                nvps.add(new BasicNameValuePair("businessTranCode", ePosPaymentReturnVO.getBusinessTranCode()));
                nvps.add(new BasicNameValuePair("prePayAmount", ""));
                nvps.add(new BasicNameValuePair("isSocialSecurityPay", ePosPaymentReturnVO.getIsSocialSecurityPay()));
                nvps.add(new BasicNameValuePair("customerCertType", ePosPaymentReturnVO.getCustomerCertType()));
                nvps.add(new BasicNameValuePair("customerCertId", ePosPaymentReturnVO.getCustomerCertId()));
                nvps.add(new BasicNameValuePair("userName", ePosPaymentReturnVO.getUserName()));

以下是在chrome中看到的请求参数

businessType:PCIS
businessNo:12010088000293602413
customerName:(unable to decode value)
currencyNo:RMB
amount:10429
regionCode:000000
insuredName:(unable to decode value)
tellerNo:YWSHP-00001
branchNo:2120971
documentNo:
callBackInfo:
certPubKey:LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVqRENDQTNTZ0F3SUJBZ0lNTjRBVzdPVDM0
dXVkZDFHRk1BMEdDU3FHU0liM0RRRUJCUVVBTUNzeEN6QUoKQmdOVkJBWVRBa05PTVF3d0NnWURW
UVFLREFOVFJFSXhEakFNQmdOVkJBTU1CVk5FUWtOQk1CNFhEVEUwTURVeApNekF5TlRjd09Gb1hE
VEU1TURVeE1qQXlOVGN3T0Zvd2J6RUxNQWtHQTFVRUJoTUNRMDR4Q3pBSkJnTlZCQWdUCkFsTmFN
UTR3REFZRFZRUUhEQVZNVDBOQlRERU5NQXNHQTFVRUNnd0VVRUZKUXpFVE1CRUdBMVVFQ3d3S1JV
VmYKVTBsSFRrbE9SekVmTUIwR0ExVUVBd3dXWlhCamFYTXRZWFYwYnk1d1lXbGpMbU52YlM1amJq
Q0JuekFOQmdrcQpoa2lHOXcwQkFRRUZBQU9CalFBd2dZa0NnWUVBbWN6WmdnOUx4UTZLN1pvWVhS
b2ZXcEYxLzY3MGttekFWSlZQCjF3WXQzVVZVMXNXTDB0RGhZUFdmZHp3ZzNFbzNVL0toTldDNzFS
dm8xSXRsQWlqUWI1cDZ0Rk1vL0xXZmtxOU8KMHpBS00xM2l3MjNlZytkcjI4RHhLKzZIbXJBNkVa
dHkwKzFkVWhzWUZiMXN2S3dDdUFIWjJ5WWYvU2ZrcDBIVwpwSkZoa3lrQ0F3RUFBYU9DQWU0d2dn
SHFNQThHQTFVZEV3RUJBQVFGTUFNQkFRQXdEZ1lEVlIwUEFRRUFCQVFECkFnRENNQlFHQ1dDR1NB
R0crRUlCQVFFQkFBUUVBd0lBNGpBZ0JnTlZIU1VCQVFBRUZqQVVCZ2dyQmdFRkJRY0QKQVFZSUt3
WUJCUVVIQXdJd0lnWURWUjBqQVFFQUJCZ3dGb0FVN0tZVEM2QkIyTU9DZEtpN2RrTW9IenBjRkJj
dwpnWjhHQ0NzR0FRVUZCd0VCQVFFQUJJR1BNSUdNTUlHSkJnZ3JCZ0VGQlFjd0FvWjliR1JoY0Rv
dkx6RXdMakl1Ck1UQXhMakUxT2pRNE9TOURUajFUUkVKRFFTeERUajFUUkVKRFFTeFBWVDFqUVVO
bGNuUnBabWxqWVhSbGN5eHYKUFhOa1ltTmhMR005WTI0L1kwRkRaWEowYVdacFkyRjBaVDlpWVhO
bFAyOWlhbVZqZEVOc1lYTnpQV05sY25ScApabWxqWVhScGIyNUJkWFJvYjNKcGRIa3dnYVlHQTFV
ZEh3RUJBQVNCbXpDQm1EQ0JsYUNCa3FDQmo0YUJqR3hrCllYQTZMeTh4TUM0eUxqRXdNUzR4TlRv
ME9Ea3ZRMDQ5VTBSQ1EwRXNRMDQ5VTBSQ1EwRXNiM1U5UTFKTVJHbHoKZEhKcFluVjBaVkJ2YVc1
MGN5eHZQWE5rWW1OaExHTTlZMjQvWTJWeWRHbG1hV05oZEdWU1pYWnZZMkYwYVc5dQpUR2x6ZEQ5
aVlYTmxQMjlpYW1WamRHTnNZWE56UFdOU1RFUnBjM1J5YVdKMWRHbHZibEJ2YVc1ME1DQUdBMVVk
CkRnRUJBQVFXQkJTODBieXE5ejF0eEx2cWRLYmJldElOWnkxU2hUQU5CZ2txaGtpRzl3MEJBUVVG
QUFPQ0FRRUEKYjRBSVljRnExRXF0OHhJZ1pleDczdGx6aGFmNEgyTXlyS0Z0dG1ubFRFc3RZbkpi
UEdJQ0x6MitNcU1FV0JYcAp4amw3M0JNYUROR0xvMkIremZrVTJxVlk3eEh4b1dNcUR1Zm9EUVRR
Sm5SU2xHRGFKM3QyMDV2dmZ2NWEramFnCk9KeW9kZ0hCUXhrbEJ5alNqSityaFpRdEV1UGNJVlhI
U2t6MVYzK0hFL3piWE9QUVdkUVVUQWZjU25kMlpIOFYKbmxrTHFBTU0rbWI3ZlJTUDJCelFkVmIv
ZFVqSlZpckhuMGN5SzZ3MUwzVTY0dEhXWm5jdE9NKzdWUGx0V09jaQpwcFF2M21mOEwzUmxmRnVC
dXE3aUdBQlBlMFZyT2IwVXdEczFGSGNRSHN1L21kdW5kenU5YURESnc2UHVpQTVuCnlKbWU2R29D
eWQ2a0tTMzFxRFova3c9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0t
signData:J5sgq0DZJj4GK6cVdQe3B0Pt+KuM1mLMUSapFxHq5XrpHbsv1KHWfGikQn9mjez7W9p12cq9QOTS
+5UNkzO/dVoSv92GwN8i115vwIMkpA3Zp5PksOn66obtCOLD7CdZAeB57QtOYvlFqEI4RDZ8oqLk
eYROa0P4f5AKxa1BRu8=
callBackURL:undefined
applicantName:(unable to decode value)
applicantCertificateType:01
applicantCertificateNo:330183199009264314
ncpInsuredName:(unable to decode value)
insuredCertificateType:01
insuredCertificateNo:330183199009264314
prepaidAccountId:undefined
dataSource:33
payType:undefined
prepaidAccountType:undefined
networkFlag:02
applicantTelephone:15057179897
channelSource:G
businessTranChnl:EPO
businessTranCode:000052
prePayAmount:
isContainTax:undefined
vehicleNum:undefined
bargainNum:undefined
departmentName:undefined
operationByName:undefined
inputByName:undefined
undrContractName:undefined
payNo:undefined
payMode:undefined
paBankAccount:undefined
paBankAccountName:undefined
vehicleTaxCountStr:undefined
collectAmountStr:undefined
paymentEndDate:undefined
circPaymentNo:undefined
printFlag:undefined
departmentCode:undefined
isSocialSecurityPay:N
socialSecurityPayInfo:undefined
customerCertType:01
customerCertId:330183199009264314

可以看undefined的参数是没有加的。这样有问题吗？

请各位大佬相助。

先谢过了。

Answer 1

一直忘了，现在自答一下，之所以请求页面不完整。是因为请求参数没有模拟成功，当前post请求的参数，是由上一步的请求得到的数据进行模拟的，其中有一个netFlag的参数，在上一步请求的返回参数中值是"Internet"，但是在实际页面请求时，在页面上被转成了对应的code值"02"，我在模拟这个值得时候直接就是拿了上一步返回参数里的值，也就是"internet"，而实际上应该是要返回"02"。修改了之后就请求成功了。

Answer 2

ctrl+u 看一下他的源代码，就是不包含 js 生成的源代码，

这个就是你抓取所获取的内容，应该是不包含这部分的：

chrome监控到的页面是经过js脚本加工之后的吗?

当然，没有经过 js 修饰过得，多难用；

我在source中看了所有的js脚本，都没有查找到相关的添加元素的脚本代码。请问这个问题的可能原因是什么，我这样找对吗？

如果你前面 ctrl+u 确实证明我说的是对的话，那这段多出来的代码就是 js 生成的，你没有找到很正常呀，那么多代码你一行行看了，都明白什么意思？不太可能吧

Answer 3

对于httpclient而言，是在io层面，数据都只是文件，字符。而浏览器的作用就是解释HTML，css，js代码，他会执行页面嵌入的脚本，渲染页面等等。