撰写DF时，AWS胶工作将无效指针异常投掷
我正在尝试将作业写入读取来自S3的数据并写入bq db （使用连接器），在其他表中运行相同的脚本，并且可以正常工作，但是对于其中一张表来说，不起作用。 它正在进行第一次运行，但是在第一次加载后，增量运行会引发此空指针异常错误。我的书签已启用可以获取在S3中添加的新数据并写入BQ数据库。
我已经在处理新的数据检查，如果有要处理的文件，请继续删除作业。 在作业日志中，df正在打印，计数也在打印，一切似乎都在起作用，但是当它运行写入DF命令时，作业失败了。 我不确定原因是什么。试图通过将源的无效属性设置为True与Target相同，但仍会失败。
无法理解被抛出的空指针异常。
错误：引起的：java.lang.nullpointerexception在com.google.google.cloud.bigquery.connector.common.bigqueryclient.loaddataintotable（bigqueryclient.java：532）at com.cloud.cloud.cloud.cloud.cloud.bigquery.bigquery.bigquery.bigquery.bigquery.bigquerywrlateflof。 （BigQueryWriteHelper.Scala：87）in com.google.cloud.spark.bigquery.bigquerywritehelper.writedataframetobigquery（bigquerywritehelper.scala：66）...
I am trying to write a job to read data from S3 and write to BQ db (using connector), running the same script for other tables and it is working correctly, but for one of the tables the write is not working.
It is working on the first run, but after first load the incremental runs throws this null pointer exception error. I have bookmarks enabled to fetch new data added in S3 and write to BQ database.
I am already handling the new data check, if there are files to process then proceed else abort job.
In the job logs df is printing and count is printing too, everything seems to be working but as it runs the write df command the job fails.
I am not sure what is the cause. Had tried to make the nullability of source and target to be same too, by setting the nullable property of source to True same as target, but it still fails.
Unable to understand the null pointer exception thrown.
Error: Caused by: java.lang.NullPointerException at com.google.cloud.bigquery.connector.common.BigQueryClient.loadDataIntoTable(BigQueryClient.java:532) at com.google.cloud.spark.bigquery.BigQueryWriteHelper.loadDataToBigQuery(BigQueryWriteHelper.scala:87) at com.google.cloud.spark.bigquery.BigQueryWriteHelper.writeDataFrameToBigQuery(BigQueryWriteHelper.scala:66) ... 42 more
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
AWS的BQ连接器有一个错误。当我联系AWS团队时，这是解决的，他们建议使用以前的连接器版本。
因此，使用以前版本的连接器帮助我解决了问题。
The BQ connector by AWS had a bug. This was resolved when I contacted the AWS team and they suggested to use previous version of the connector.
So, using previous version of connector helped me resolve the issue.