Julia可以连接到S3/Athena并使用SQL吗?

2024-09-27 00:13:33 发布

您现在位置:Python中文网/ 问答频道 /正文

是否可以运行SQL脚本从AWS的S3/Athena中提取Julia中的数据?我的Julia实例位于AWS Sagemaker Jupyter笔记本上

在SageMaker中轻松使用python:

import boto3
region = boto3.Session().region_name

from pyathena import connect
import pandas as pd

conn = connect(s3_staging_dir='s3://sagemaker-examplebucket/',
               region_name = region)

df = pd.read_sql("""SELECT 
 something1,
 something2
FROM "customer_data"."sagemaker_data"
WHERE 
    something = 0;""", conn)

对于使用Julia的SageMaker笔记本实例,我可以连接并放置/获取现有文件,但找不到任何允许新查询的内容,例如Python的pyathena库:

using AWS, AWSS3, Serialization
struct SampleData
  a::Int
  b::String
end

d=SampleData(1,"sss")
aws = global_aws_config(; region="us-west-2")
b = IOBuffer()
serialize(b, d)

s3_put(aws, "sagemaker-examplebucket","myfile.bin", b.data)

ddat = s3_get(aws, "sagemaker-examplebucket","myfile.bin")
d2 = deserialize(IOBuffer(ddat))

@assert d == d2

Tags: 实例nameimportawsdatas3笔记本boto3

热门问题