Override S3 endpoint using Boto3 configuration file

OVERVIEW: I'm trying to override certain variables in boto3 using the configuration file (~/aws/confg). In my use case I want to use fakes3 service and send S3 requests to the...

Boto3 Error: botocore.exceptions.NoCredentialsError: Unable to locate credentials

When I simply run the following code, I always gets this error. s3 = boto3.resource('s3') bucket_name = "python-sdk-sample-%s" % uuid.uuid4() print("Creating new bucket with name:",...

Example of update_item in dynamodb boto3

Following the documentation, I'm trying to create an update statement that will update or add if not exists only one attribute in a dynamodb table. I'm trying this response = table.update_item( ...

AWS CLI S3 A client error (403) occurred when calling the HeadObject operation: Forbidden

I'm trying to setup a Amazon Linux AMI(ami-f0091d91) and have a script that runs a copy command to copy from a S3 bucket. aws --debug s3 cp...

Ansible ec2: "boto required for this module"

When I run this simple Ansible playbook: - name: EC2 Test Example hosts: localhost connection: local gather_facts: False tasks: - name: EC2 Instance ec2: # Amazon EC2 key pair...

Is there a way to tell aws-cli to run a command to get a new session token if the current token is expired?

At my work, our AWS authentication is integrated with our corporate sign-sign-on (SSO) system. In other words, when we want to access AWS, we do something that authenticates to our corporate...

Python - InvocationError in tox running py.test and coverage test

My project has the following structure: my_project | setup.py tox.ini src | core_functions.py client.py server.py sql_database.py tests.py The tests.py file contains the...

In macOS Sierra, How Configure AWS CLI to Use Python3.x Instead of the OS Default Python2.7?

AWS CLI Mavens, Via macOS' Terminal, trying to install and configure AWS CLI on macOS Sierra 10.12.6 to use Python 3.6.2 instead of macOS' default, Python 2.7.10. Although I rigorously followed...

Failed building wheel for xmlsec - mac

I'm installing my mac on python virtualenv environment. Whenever I install requirement packages command "pip install -r requirements.txt" i got this messages. pip install -r requirements.txt...

Docker build taking too long when installing grpcio via pip

I have a Dockerfile which installs a few packages via pip. Some of them are requiring grpcio, and it takes a few minutes only to build this part. Does anyone have a tip to speed up this...

from urllib3.util.ssl_ import ( ImportError: cannot import name ssl

My resources: Python 2.7, Ubunutu 18.04, Pycharm, virtual box oracle I have an automation solution built in python. The solution can be run from both cmd or pycharm of course. 2 options to run...

Unable to connect to S3 from Lambda/Python/Boto3 when VPC is enabled

I have a very simple python function in a lambda which runs fine if I leave VPC disabled. import json import boto3 import botocore def lambda_handler(event, context): s3 = boto3.client('s3',...

Unable to upload artifact None referenced by CodeUri parameter of HelloWorldFunction resource

I'm following this tutorial to learn how to use SAM. Here's the code I have: template.yml: AWSTemplateFormatVersion : '2010-09-09' Transform: AWS::Serverless-2016-10-31 Resources: ...

Always getting error: You must be logged in to the server (Unauthorized) EKS

I am currently playing around with AWS EKS But I always get error: You must be logged in to the server (Unauthorized) when trying to run kubectl cluster-info command. I have read a lot of AWS...

Celery consumer does not receive messages from SQS queue on LocalStack

I have a SQS queue on a LocalStack server and I'm trying to consume messages from it with a Celery consumer. It seams that the consumer is properly attached to the queue, for example the queue...

List subnets in an AWS by VPC ID

I need to list the subnets in a specific VPC in AWS. I am getting this error when I do: Traceback (most recent call last): File ".\aws_create_ec2.py", line 253, in <module> main() File...

Serverless wsgi unable to import werkzeug

I'm having issues deploying my serverless application to AWS. In AWS the logs show: Unable to import module 'wsgi_handler': No module named 'werkzeug' I have explicitly specified werkzeug in my...

Connect to S3 compatible storage with boto3

I'm struggling to get boto3 to connect to an S3 compatible resource, I can't seem to find a decent example of this on the boto3 pages so my attempt is below (taken from...

Error with Python3 to connect to snowflake: snowflake.connector.errors.OperationalError::Failed to execute:'SSLSocket' object has no attribute 'conn

It seems there is some issue with latest snowflake connector . I am trying to install it on Ubuntu OS and trying to connect to snowflake but it gives error AttributeError: 'SSLSocket' object has...

AWS Step Functions does not catch error when Lamba function returns an error

I have been struggling with AWS Step Functions for hours now. The use case is quite simple as I want to get gradually familiar with AWS Step Functions. However, I think I do not understand how...

How to install aws-cli on alpine?

I'm installing aws-cli on a docker swarm manager node running alpine (Linux 0317632a4ad9 4.9.59-moby #1 SMP Thu Mar 1 20:54:00 UTC 2018 x86_64 Linux). The aws-cli package for Alpine is currently...

Botocore error: HTTP Client raised an unhandled exception: sys.meta_path must be a list of import hooks

I am running this small snippet to upload a panda dataframe to s3 using parquet. But I get the error: Exception botocore.exceptions.HTTPClientError: HTTPClientError(u'An HTTP Client raised an...

Can't list bucket objects on Scaleway using boto3

I saw a few similar posts, but unfortunately none helped me. I have an s3 bucket (on scaleway), and I'm trying to simply list all objects contained in that bucket, using boto3 s3 client as...

Failed to import the required Python library (botocore or boto3)

I'm running a playbook on localhost, and I've already installed ansible and the required packaged like boto3. The playbook works fine when it is performing tasks on a remote host, but outputs the...

Amazon VPC Endpoint for AWS Appflow

We have a Glue Python Shell Pipeline Job trying to invoke AWS AppFlow through boto3 APIs. The challenge we have is that we need to design our solution without the need for internet access within...

Django-Compressor won't work in offline mode with custom S3 domain

I get the following error whenever a custom domain for the S3 endpoint is used. # WORKS AWS_S3_CUSTOM_DOMAIN = 'example.fra1.digitaloceanspaces.com/{}'.format(AWS_STORAGE_BUCKET_NAME) # DOES...

Using apache-airflow-providers-snowflake on airflow (no module named Snowflake)

I have installed package apache-airflow-providers-snowflake on airflow on docker and i am getting error No module named Snowflake Please refer attachment (check the error mentioned for the...

unable to install awscli using pip on dind 18.09.6-dind due to cython dependency

I saw a strange issue just most of yesterday where while running a simple jenkins build that uses pod template with container docker:18.09.6-dind (alpine linux) the build would fail while trying...

Add packages from requirements.txt to Docker image to minimize cold start time on EC2?

When deploying a machine learning model on EC2 from a Docker image, the cold start time is high because the instance downloads the packages and files from requirements.txt even though the...

Zappa: UpdateFunctionConfiguration operation: The operation cannot be performed at this time. An update is in progress for resource

I am try update a lambda by zappa, I created virtualenv and active virtualenv and install libraries, but in the moment run zappa update enviroment, I have this problem: How can i fix this :( ...