I got locked out of a google compute instance due to ssh packets being dropped

By Iain Cuthbertson

Silly thing to happen really.

Some system on the instance detected too many connections in a short amount of time - likely due to some automated tasks driven by ansible.

That resulted in iptables dropping all connections to port 22 🙁

Thankfully, with google compute, one can access the serial console via the web UI.

Unfortunately, all of my users have no passwords - this is to ensure that ssh logins are via keypairs only.

The work around was eventually found on https://medium.com/google-cloud/resolving-getting-locked-out-of-a-compute-engine-85800251890b

TLDR;

  • Edit the instance
  • Tick “Enable connecting to serial ports”
  • Scroll down to “Custom metadata”
  • Add a new item with the key “startup-script”:
#!/bin/bash
useradd --groups google-sudoers tempuser
echo "tempuser:password" | chpasswd
  • Save the instance and hit “Reset”
  • When the instance is back up, connect to it via the serial console and login as the “tempuser”
  • Fix things!
  • Remove the tempuser
  • Remove the startup-script from custom metadata