如何在AWS胶水工作中覆盖Couchbase查询超时？
我正在尝试使用 Spark Couchbase Connector 。我的查询是一个简单的N1QL查询，以针对现有的couchbase存储桶：
var queryResultRDD: RDD[CouchbaseQueryRow] = spark.sparkContext.couchbaseQuery(N1qlQuery.simple(CbN1qlQuery))
如果我的运行时间很长，我会收到以下错误消息：
Caused by: com.couchbase.client.java.error.QueryExecutionException: Timeout 1m15s exceeded
at com.couchbase.spark.connection.QueryAccessor$$anonfun$compute$1$$anonfun$apply$2$$anonfun$4.apply(QueryAccessor.scala:56)
at com.couchbase.spark.connection.QueryAccessor$$anonfun$compute$1$$anonfun$apply$2$$anonfun$4.apply(QueryAccessor.scala:53)
at rx.lang.scala.Observable$$anon$32.call(Observable.scala:1324)
1M15S超时设置可能来自 couchbase的75s默认查询超时，所以我尝试直接添加查询超时设置默认超时设置：
var queryResultRDD: RDD[CouchbaseQueryRow] = sc.couchbaseQuery(N1qlQuery.simple(CbN1qlQuery), "bucket-name", Some(Duration(130, SECONDS)))
将该持续时间降低到不可能的低点（例如1 ms）导致不同的超时错误。但是，如果我按如上图所示的持续时间更长的时间，我仍然收到了超时仍1m15s的相同QueryExeCutionException。我还尝试在胶水脚本中设置系统属性：
System.setProperty("com.couchbase.env.timeout.queryTimeout", "1ms")
但是我仍然收到相同的1M15S超时错误。我还尝试以相同的方式设置spark.couchbase.timeout.queryTimeout属性，并获得相同的结果。我还尝试在Sparksession Builder中设置查询超时配置而没有任何更改：
val Spark = SparkSession
.builder()
.appName(DefaultName)
.config("spark.couchbase.nodes", CbNodes)
.config(s"spark.couchbase.bucket.$SourceBucketName", SourceBucketPassword)
.config("spark.couchbase.username", SourceBucketUserName)
.config("spark.couchbase.password", SourceBucketPassword)
.config("spark.ssl.enabled", CbSslEnabled)
.config("spark.ssl.keyStore", CbKeyStore)
.config("spark.ssl.keyStorePassword", CbKeyStorePassword)
.config("spark.couchbase.timeout.queryTimeout", "1ms")
.getOrCreate()
如何覆盖此1M15S查询超时设置？
I'm trying to run a Couchbase query in an aws glue job, using the spark couchbase connector. My query is a simple N1Ql query against the existing Couchbase bucket:
var queryResultRDD: RDD[CouchbaseQueryRow] = spark.sparkContext.couchbaseQuery(N1qlQuery.simple(CbN1qlQuery))
If I have a long running query, I receive the following error message:
Caused by: com.couchbase.client.java.error.QueryExecutionException: Timeout 1m15s exceeded
at com.couchbase.spark.connection.QueryAccessor$anonfun$compute$1$anonfun$apply$2$anonfun$4.apply(QueryAccessor.scala:56)
at com.couchbase.spark.connection.QueryAccessor$anonfun$compute$1$anonfun$apply$2$anonfun$4.apply(QueryAccessor.scala:53)
at rx.lang.scala.Observable$anon$32.call(Observable.scala:1324)
The 1m15s timeout setting likely comes from Couchbase's 75s default query timeout, so I tried to add the query timeout setting directly to the query call hoping that it would override the default timeout setting:
var queryResultRDD: RDD[CouchbaseQueryRow] = sc.couchbaseQuery(N1qlQuery.simple(CbN1qlQuery), "bucket-name", Some(Duration(130, SECONDS)))
Dropping that duration down to something impossibly low like 1 ms resulted in a different query timeout error. However, if I made the duration longer as shown above I still received the same QueryExecutionException where the timeout was still 1m15s. I also tried to set the System Property in the Glue job script:
System.setProperty("com.couchbase.env.timeout.queryTimeout", "1ms")
Yet I still received the same 1m15s timeout error. I also tried setting the spark.couchbase.timeout.queryTimeout property in the same way and got the same result. I also tried setting the query timeout configuration in the sparkSession builder with no change:
val Spark = SparkSession
.builder()
.appName(DefaultName)
.config("spark.couchbase.nodes", CbNodes)
.config(s"spark.couchbase.bucket.$SourceBucketName", SourceBucketPassword)
.config("spark.couchbase.username", SourceBucketUserName)
.config("spark.couchbase.password", SourceBucketPassword)
.config("spark.ssl.enabled", CbSslEnabled)
.config("spark.ssl.keyStore", CbKeyStore)
.config("spark.ssl.keyStorePassword", CbKeyStorePassword)
.config("spark.couchbase.timeout.queryTimeout", "1ms")
.getOrCreate()
How do I override this 1m15s query timeout setting?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
我看了看 couchbaseconfig class config应该是什么：
＆nbsp; ＆nbsp;
QueryTimeOut值在MS中的位置。
I figured it out after looking at the CouchbaseConfig class what the config is supposed to be:
where the querytimeout value is in ms.