![]() ![]() Add the s3 whl file paths in the Python library path.awscli-1.18.205-py2.p圓-none-any.whl (as of writing this blog). ![]() boto3-1.16.45-py2.p圓-none-any.whl (as of writing this blog).Download the latest whl files from boto3 files and awscli files.This option is slow as it has to download and install dependencies. Here for my notes I am simply pasting the solution for AWS Glue Python Shell with Internet with minor changes – AWS Glue Python Shell with InternetĪdd awscli and boto3 whl files to Python library path during Glue Job execution. User sarath-mec has provided the solution in the comment section for both AWS Glue Python Shell with and without Internet. I was pretty sure I wasn’t the first person to get into this issue, so I searched it and came across the open issue. The current boto3 version available is 1.16.45 whereas AWS Glue Python shell is importing version 1.9.203. Surprisingly the output is – boto3 version : 1.9.203 Print('boto3 version :', boto3._version_) I was quite surprised with the error and it made me think to check the Boto3 version which AWS Glue Python Shell script is importing. The job failed with UnknownServiceError: Unknown service: 'redshift-data'. To test the redshift-data API I wrote a simple AWS Glue Python Shell job to execute a sql. The Amazon Redshift Data API can be used to run SQL queries on Amazon Redshift tables. When you add a AWS Glue job, you can choose the job to be either Spark or Spark Streaming or Python shell type.įor one of my use-case I wanted to try the new “ redshift-data” api service in AWS Glue Python Shell script. AWS Glue is a fully managed extract, transform, and load (ETL) service to process large amount of datasets from various sources for analytics and data processing.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |