Resetting SSH key access to your EC2 Instance through Systems Manager Automation

Looking for a simpler way to view your EC2 instance and troubleshoot your lost SSH Key in your AWS account? BlueChipTek Professional Services Engineer, Kyle Newton, will cover how AWS Systems Manager can help this process of resetting an SSH Key.

Although the industry trend is moving our applications to containers and serverless environments, we’re well aware that there’s always going to be a few instances that are still treated as pets, especially legacy applications. With this in mind, Amazon created the EC2 Systems Manager, a suite of features that brings a fair share of Quality of Life tools to manage instances that aren’t up to par with the bleeding edge. In this blog we’ll be addressing Systems Manager’s solution for the typically grueling process of changing a lost SSH Key to restore SSH access for an instance.

Getting Started

IAM User Policy:

To start working with Systems Manager, you will need user permissions to do so. You will either need to be an administrator or will need to have the AmazonSSMFullAccess policy attached to your user or assumed role.

SSM Agent:

If your instance is any of the following, it already comes with SSM agent:

  • Amazon Linux
  • Amazon Linux 2
  • Ubuntu Server 16.04
  • Ubuntu Server 18.04
  • Windows Server 2016 (also 2003-2012 R2 AMIs published Nov 2016 and later)

If you’re not on the list above, and haven’t previously installed an SSM agent onto your instance, unfortunately this blog post cannot help you. Amazon does have a guide here, that walks through the manual recovery steps.

SSM Instance Profile:

In order for Systems Manager to perform actions on your instances, you’ll need to use an IAM instance profile that allows this. You can either create a new role with the AWS-managed policy AmazonEC2RoleForSSM, or can attach this policy to an existing role. We can add this role to other instance profiles easily in the future.

Injecting a new SSH key

Now that we’ve finished the preliminary steps, we can see how easy Systems Manager is to use. Although we can access most Systems Manager features via the EC2 console, we’ll be using the newer Systems Manager AWS Service interface found under Management Tools or under this link.

AWS-systems-manager-console.png#asset:3639


From here, you can go to Managed Instances under the Shared Resources section on the left column. Here you can see all the instances that have both SSM agent, and an instance profile that allows SSM access. If you just replaced or updated the Instance Profile, it may take about 5 minutes to appear here.

AWS-systems-manager-managed-instances.png#asset:3640

Let’s say we’ve lost the SSH Key to get into our Windows SSM Testing instance, and want to inject a new SSH key in. This process typically involves shutting down the instance, un-attaching the volume, attaching it to another instance, editing files, etc. It’s a tedious manual process that is prone to human error. We’re going to use the Automation feature of Systems Manager to handle this instead. The process is only a few clicks. It will run for about 15 minutes, leaving the tedium and human error out of the equation.

In the left column, we will be going to Actions > Automation, and clicking the orange Execute automation button. From here we are looking for AWSSupport-ResetAccess.

Systems-Manager-Execute-Automation.png#asset:3641

Here we have several options, but we can leave all of them default. The only thing we need to fill out is the InstanceId input parameter. Then just click on the Execute automation button. Please realize that this will NOT work on instance store volumes, as it will stop and then start the instance. If you do not have an EIP, your IP address will also change.

Systems-Manager-Input-Parameters.png#asset:3642

Once you execute the command, you can look at the progress and a complete list of all tasks and actions that take place during this process by clicking individual step ids. Here’s the input parameters of the EC2RescueAutomationWithNewVPC step.

Systems-Manager-Parameters-Code.png#asset:3643

Do not be concerned if a few tasks stay as Failed or Pending, as this document is meant for both Windows and Linux instances. If you run it on Windows, the Linux tasks will detect they don’t need to run, and vice versa. You’ll be waiting to see this lovely logo to know it’s all done.

Systems-manager-overall-status.png#asset:3645

When it’s done, we can go back on the left column and under Shared resources go to Parameter Store to find our secure string. It will be listed as /ec2rl/openssh/[InstanceId]/key where we can then view details. You can click “Show” under Value to see the full RSA key. You can now use that key to connect to your instance.

This is a much better solution to the old manual methods, and you can repeat the automation step to do several instances at once with little work involved. However, this is only really useful if you absolutely need private SSH key access, perhaps if you have a script that needs to occasionally SSH in to perform certain tasks.

In our next blog, we’ll tell you why you might just want to close port 22 for good and use another great feature of Systems Manager called Session Manager. We’ll see you then!

If you would like assistance or have questions on setting up AWS Systems Manager, feel free to reach out to our Cloud Services team.