Error registering: NoCredentialProviders: no valid providers in chain ECS agent error

Im trying to use EC2 Container service. Im using terraform for creating it. I have defined a ecs cluster, autoscaling group, launch configuration. All seems to work. Except one thing. The ec2...

Why are my gunicorn Python/Flask workers exiting from signal term?

I have a Python/Flask web application that I am deploying via Gunicorn in a docker image on Amazon ECS. Everything is going fine, and then suddenly, including the last successful request, I see...

Error with Jenkins ECS and EFS

I'm trying to use Jenkins deploy to ECS with EFS. (/var/jenkins_home is mounted) It seems all fine except when I deploy a service I get following error in the log: touch: cannot touch...

AWS ECS tasks keep starting and stopping

I am trying to use ECS for deployment with travis. At one point everything was working but now it stopped. I am following this tutorial https://testdriven.io/part-five-ec2-container-service/ There...

ECS CLI key pair error when calling up

I am trying to use Amazon's ECS cli to create a cluster. I keep getting the error: reason="The key pair 'my-key-pair' does not exist" resourceType="AWS::AutoScaling::LaunchConfiguration" I have...

Airflow 1.9.0 is queuing but not launching tasks

Airflow is randomly not running queued tasks some tasks dont even get queued status. I keep seeing below in the scheduler logs [2018-02-28 02:24:58,780] {jobs.py:1077} INFO - No tasks to consider...

airflow webserver starting - gunicorn workers shutting down

I am running airflow 1.8 on centos7 on docker and my webserver is not getting to the browser. I installed airflow via pip2.7. Flower ui is displaying fine, initdb ran connecting to a postgres...

How to avoid the configuration error while using AWS API Gateway with VPC Link?

I have created the VPC Link using the Network Load Balancer (NLB) as per the AWS documentation and attached the same with the API Gateway resource / method. But it throws "Internal Server Error"...

AWS ECS: Monitoring the status of a service update

I am trying to migrate a set of microservices from Docker Swarm, to AWS ECS using Fargate. I have created an ECS cluster. Moreover, I have initialized repositories using the ECR, each of which...

How to diagnose ECS Fargate task failing to start?

I'm trying to launch/run a Dockerfile on AWS using their ECS service. I can run my docker image locally just fine, but it's failing on the Fargate launch type. I've uploaded my Docker image to...

Container is not able to call S3 in Fargate

I'm not able to synchronize a log-folder to s3 inside a container. I'm trying to get the following setup: Docker Container with installed awscli there are logfiles and other files generated...

AWS::Event::Rule FailedInvocation debug info?

I have a AWS::Event::Rule that routes a S3 put event to a ECS task. I can see the rule is being triggered from the metrics, but also see FailedInvocation on every trigger. I suspect that's a...

Best approach for sending logs from ECS Fargate into Elasticsearch

We have a setup with multiple containers running NodeJS services(node:11-alpine docker image) deployed in AWS ECS Fargate. We already have a running ElasticSearch instance collecting logs from...

AWS ECS service Tasks getting replaced with (reason Request timed out)

We are running ECS as container orchestration layer for more than 2 years. But there is one problem which we are not able to figure out the reason for, In few of our (node.js) services we have...

how to get keycloak to export realm users and then exit

We run Keycloak docker image in AWS ECS and we need a way to export a realm and all users for automation purposes using ansible. We can run the following command with ansible to run the...

How do I troubleshoot "failed to initialize logging driver" (ECS)

I've created 3 tasks on a cluster. 1 is running fine but I'm getting CannotStartContainerError: Error response from daemon: failed to initialize logging driver: failed to create Cloudwatch log...

How to put multiline commands in ECS Task definition

I am trying to run a django application inside of a docker container (ECS - Fargate) However, I am having trouble figuring out how to run multiple commands in the Command section of a task...

WordPress in a container fails to connect to a MySQL container on an ECS Fargate cluster

WordPress is failing to authenticate the bn_wordpress mysql user with the MySQL database when the two containers run on an ECS cluster on Fargate. How do I fix this? Logs The WordPress ECS Task...

Not able to retrieve AWS_CONTAINER_CREDENTIALS_RELATIVE_URI inside the container from fargate task

I'm running a docker container in Fargate ECS Task. And my docker container, I have enabled ssh server, so that I can login to container directly, if I have to debug something. Which is working...

AWS ALB Client TLS Negotiation Error Occurring on the Hour or Half Hour

I have a Scala application managed by ecs that makes a call through an alb to various python applications also managed by ecs. This application sends a significant amount of traffic through the...

ResourceInitializationError: failed to validate logger args: : signal: killed

Suddenly getting the message " ResourceInitializationError: failed to validate logger args: : signal: killed" while starting AWS ECS Fargate Service. Same service was running fine couple of days...

How to configure AWS CDK Account and Region to look up a VPC

I am learning the AWS CDK, and this is a problem I can't seem to figure out. JS/Node are not languages I use often, so if there is some obvious native thing that I am missing, please don't be too...

AWS APP MESH ENVOY sidecar [error] Could not retrieve credentials listing from the instance metadata

I'm using AWS App Mesh with ECS Fargate. Unfortunately in the logs of the envoy sidecar I see errors like: [error][aws] [source/extensions/common/aws/credentials_provider_impl.cc:94] Could not...

CloudWatch event has been triggered but my ECS task didn't run. why?

I have an ECS task that's being invoked everyday by a cron schedule form a CloudWatch event. I noticed that my ECS task hasn't run today (never happened in the past) so I looked at the CloudWatch...

StoppedReason in ECS Fargate is truncated

In ECS Fargate, when a task fails, there is a "Stopped Reason" field which gives some useful logging. However I have noticed that it gets truncated after 255 symbols (screenshot below). I checked...

ECS Task Definition - When overriding ENTRYPOINT, Docker image's CMD is dropped

I have a Docker Image built with the following CMD # Dockerfile ... CMD ["nginx", "-g", "daemon off;"] When my task definition does not include entryPoint or command the task successfully enters...

AWS Fargate ResourceInitializationError: unable to pull secrets or registry auth: pull command failed: : signal: killed

Slightly tearing my hair out with this one... I am trying to run a Docker image on Fargate in a VPC in a Public subnet. When I run this as a Task I get: ResourceInitializationError: unable to pull...

CDK deploy results in "Bucket named 'x' exists, but not in account 067685711111. Wrong account?

I am modifying a CDK pipeline that works. I am adding a lambda, and when this code modification is made, the pipeline fails with an error. import * as cdk from '@aws-cdk/core'; import * as lambda...

Is injecting secrets as environment variables in docker secure? Why does ECS and EKS support it?

I'm having a hard time reconciling some online advice that injecting secrets (usually passwords) as environment variables into docker containers is "not secure" with the native features of AWS ECS...

Filebeat is not sending logs to logstash on kubernetes

I'm trying to send kubernetes' logs with Filebeat and Logstash. I do have some deployment on the same namespace. I tried the suggested configuration for filebeat.yml from elastic in this...