如何在AWS Athena分区表2上查询
问题摘要
我还检查了以下链接来修复我的问题,但这对我没有帮助。
有4种类型我的日志表中的分区。
- aws_account_id
string
- log_type
string
- aws_region
string
- date
string
我尝试了一个简单的 SELECT
查询分区表。但收到以下错误消息。
错误消息
GENERIC_INTERNAL_ERROR: No value present
This query ran against the "default" database, unless qualified by the query.
我尝试的 SELECT 查询
SELECT * FROM "logs" LIMIT 10;
并且
SELECT * FROM "default"."logs" LIMIT 10;
由于出现有关No value present
的错误消息,我检查了分区结果。
SHOW PARTITIONS logs;
结果格式
aws_account_id=${aws_account_id}/log_type=${log_type}/aws_region=${aws_region}/date=${date}
我非常感谢您的帮助。
更多信息
的CREATE TABLE
命令
我使用创建表
CREATE EXTERNAL TABLE `logs`(
`account_id` string,
`interface_id` string,
`srcaddr` string,
`dstaddr` string,
`srcport` int,
`dstport` int,
`traffic_protocol` bigint,
`packets` bigint,
`bytes` bigint,
`start_time` bigint,
`end_time` bigint,
`traffic_action` string,
`log_status` string,
`vpc_id` string,
`subnet_id` string,
`instance_id` string,
`tcp_flags` int,
`flow_direction` string)
PARTITIONED BY (
`aws_account_id` string,
`log_type` string,
`aws_region` string,
`date` string)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ' '
WITH SERDEPROPERTIES (
'EXTERNAL'='true',
'skip.header.line.count'='1')
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
's3://default/path/'
TBLPROPERTIES (
'last_modified_by'='hadoop',
'last_modified_time'='1646821386',
'projection.aws_account_id.type'='injected',
'projection.aws_region.type'='injected',
'projection.date.format'='yyyy/MM/dd',
'projection.date.interval'='1',
'projection.date.interval.unit'='DAYS',
'projection.date.range'='2019/11/27, NOW-1DAYS',
'projection.date.type'='date',
'projection.enabled'='true',
'projection.log_type.type'='string',
'projection.log_type.values'='logs',
'storage.location.template'='s3://default/logs/${aws_account_id}/${log_type}/${aws_region}/${date}',
'transient_lastDdlTime'='1646821386')
Summary of the Question
I also checked the below link to repair my issue, but it doesn't helped me.
How to query on AWS Athena partitioned table
There are 4 types of partitions, in my log table.
- aws_account_id
string
- log_type
string
- aws_region
string
- date
string
I tried a simple SELECT
query on partitioned table. But got the below error message.
ERROR Message
GENERIC_INTERNAL_ERROR: No value present
This query ran against the "default" database, unless qualified by the query.
SELECT Query that I Tried
SELECT * FROM "logs" LIMIT 10;
AND
SELECT * FROM "default"."logs" LIMIT 10;
Since error message about No value present
, I checked the partitions results.
SHOW PARTITIONS logs;
Result format
aws_account_id=${aws_account_id}/log_type=${log_type}/aws_region=${aws_region}/date=${date}
I would greatly appreciate your help.
Further Info
CREATE TABLE
command that I used
Create Table
CREATE EXTERNAL TABLE `logs`(
`account_id` string,
`interface_id` string,
`srcaddr` string,
`dstaddr` string,
`srcport` int,
`dstport` int,
`traffic_protocol` bigint,
`packets` bigint,
`bytes` bigint,
`start_time` bigint,
`end_time` bigint,
`traffic_action` string,
`log_status` string,
`vpc_id` string,
`subnet_id` string,
`instance_id` string,
`tcp_flags` int,
`flow_direction` string)
PARTITIONED BY (
`aws_account_id` string,
`log_type` string,
`aws_region` string,
`date` string)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ' '
WITH SERDEPROPERTIES (
'EXTERNAL'='true',
'skip.header.line.count'='1')
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
's3://default/path/'
TBLPROPERTIES (
'last_modified_by'='hadoop',
'last_modified_time'='1646821386',
'projection.aws_account_id.type'='injected',
'projection.aws_region.type'='injected',
'projection.date.format'='yyyy/MM/dd',
'projection.date.interval'='1',
'projection.date.interval.unit'='DAYS',
'projection.date.range'='2019/11/27, NOW-1DAYS',
'projection.date.type'='date',
'projection.enabled'='true',
'projection.log_type.type'='string',
'projection.log_type.values'='logs',
'storage.location.template'='s3://default/logs/${aws_account_id}/${log_type}/${aws_region}/${date}',
'transient_lastDdlTime'='1646821386')
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论