需要将多个蜂巢分区合并为Spark中的一个分区
我在蜂巢桌上有大约50个分区。我需要将每组分区合并为一个分区。我尝试使用重命名分区命令。但是获取错误消息。
需要帮助将多个蜂巢分区合并为Spark中的一个分区
ALTER TABLE db.table PARTITION (appname='SCORING',indicator='segment_id:1|process_date:20220417') RENAME TO PARTITION (appname='SCORING',indicator='process_date:20220417')
ALTER TABLE db.table PARTITION (appname='SCORING',indicator='segment_id:3|process_date:20220417') RENAME TO PARTITION (appname='SCORING',indicator='process_date:20220417')
ALTER TABLE db.table PARTITION (appname='SCORING',indicator='segment_id:4|process_date:20220417') RENAME TO PARTITION (appname='SCORING',indicator='process_date:20220417')
org.apache.hadoop.hive.ql.metadata.hiveException：无法重命名分区。分区已经存在：db.table
I have around 50 partitions in hive table. I need to merge each set of partitions into one partition. I tried to use rename partition command. But getting error message.
Need help in merging multiple hive partitions into one partition in spark
ALTER TABLE db.table PARTITION (appname='SCORING',indicator='segment_id:1|process_date:20220417') RENAME TO PARTITION (appname='SCORING',indicator='process_date:20220417')
ALTER TABLE db.table PARTITION (appname='SCORING',indicator='segment_id:3|process_date:20220417') RENAME TO PARTITION (appname='SCORING',indicator='process_date:20220417')
ALTER TABLE db.table PARTITION (appname='SCORING',indicator='segment_id:4|process_date:20220417') RENAME TO PARTITION (appname='SCORING',indicator='process_date:20220417')
org.apache.hadoop.hive.ql.metadata.HiveException: Unable to rename partition. Partition already exists:db.table
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
您可以使用 sql语句
通过。在Spark ProgramMign语言中，还有更多更改分区的工具。
您可以使用 partitionby 在Spark中重新分配。
或者，您可以编写选择以获取分区数据。然后，您可以使用 cocee或coalece或repartition 创建1个分区。
You can do this by using a sql statement
distribute by.In spark programmign language there are more tools to change the partitions.
You can use partitionby to repartition in spark.
or you could write a select to grab the partitioned data. Then you can use coalece or repartition to create 1 partition.