3

Assume I have two clusters devcluster and qacluster on ECS. How to remove an EC2 instance (ecs-optimized amazon-linux) from qacluster and add to devcluster?

sithumc
  • 272
  • 2
  • 11

3 Answers3

6

You need to follow these steps:

To make sure it uses correct cluster, check logs - docker logs ecs-agent. It should have a message like:

[INFO] Registration completed successfully. I am running as 'arn:aws:ecs:eu-west-1:ACCOUNT_ID:container-instance/CLUSTER_ID' in cluster 'devcluster'
Tensibai
  • 11,366
  • 2
  • 35
  • 62
lagivan
  • 176
  • 1
  • 4
  • 1
    You should avoid publishing any ressource arn without masking your account id and the ressource id in my opinion. I've edited and cleared the the revision history. – Tensibai Jan 24 '18 at 15:54
  • @Tensibai you're totally right. But it was actually masked as I have replaced last 6 digits with zeros ;) But this change would not harm. – lagivan Jan 24 '18 at 22:31
  • There's a small chance it match someone else account number, prefer replacing with xxxx than with 0 to avoid bots scanning the internet launching attacks :) – Tensibai Jan 24 '18 at 23:13
2

The way a cluster becomes aware of the EC2 instances associated to it, is a configuration file used by the ECS agent. You can modify this file, located at /etc/ecs/ecs.config, and name a different cluster. Then restart the ECS agent. This will effectively "move" the EC2 instance to the other cluster.

More information about ecs.config in the documentation - http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-agent-config.html

After the modification, it is possible to verify by sending an HTTP request to the ECS Agent metadata service and see the new cluster displayed. http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-agent-update.html

The ECS cluster itself also needs to be notified. Can use the Register/Deregister API calls for this, for example using the AWS CLI:

More housekeeping might be required on the ECS agent itself. Like removing cached checkpoint files, etc. https://github.com/aws/amazon-ecs-agent/issues/139#issuecomment-123061463

Evgeny Zislis
  • 8,963
  • 5
  • 38
  • 72
  • i modified /etc/ecs/ecs.config to ECS_CLUSTER=devcluster and restarted ec2 instance. but the instance still appear on qacluster – sithumc Sep 18 '17 at 05:19
  • @sith I updated the answer with details about registration of instances in the ECS Cluster metadata. Might need to be updated as well, or else the cluster does not know about a new instance or about a removed one. – Evgeny Zislis Sep 18 '17 at 07:06
0

Updated for ECS optimized AMI image as of May 2021. The sudo start command is not present and the checkpoint file /var/lib/ecs/data/ecs_agent_data.json changed to agent.db.

  1. Edit /etc/ecs/ecs.config as previously described (add ECS_CLUSTER=devcluster).
  2. Rename the agent state file with mv /var/lib/ecs/data/agent.db /var/lib/ecs/data/agent.db.old
  3. Restart the ECS agent using sudo systemctl restart ecs

Check the status of the agent with systemctl status ecs, you want it to display Active: active (running).

This was a bit frustrating to figure out because the Amazon online documentation I found still displays the old information. Couldn't have done it without devops!

Mathieu Dugré
  • 1