System Password Management: JUST DO IT.

One minute to midnight

It’s December 2013. Do you know where your system access credentials are?

The cloud and Internet services have developed quickly. Many providers and services are antiquated in how they handle password repositories – and they’re even further behind in implementing modern Identity and Access Management Services.

Meanwhile, more corporate data and proprietary business processes are operating on the cloud and Internet services.

For most organizations, access to the “crown jewels” has evolved from a few simple tokens that you shared with a small team, to a password quagmire of emails, wiki pages, sticky notes, and text files. Passwords are still the most important protection that you have on your valuable data and systems.  Poor management of these “keys to the kingdom” can cripple small and large organizations alike – and is not something that you want to deal with reactively. It’s time to fix this problem – proactively.

What’s a system password?

A “system password” includes the keys, credentials and other secrets that protect your services, data and infrastructure – anything you have that you wouldn’t share with someone you don’t trust. For example:

  • The public and private keys (and passwords) that you use to log in to servers
  • SSL certificates that protect your web servers
  • Cloud identities that you use to make cloud API calls
  • API keys and tokens that your apps use to call third-party services
  • Encryption keys used for files and storage volumes

What’s wrong with ad hoc management? (As if I really had to explain 😉 )

Of course, mismanaged access keys and credentials can lead to operational failures and data loss. But if I think about all the different ways this could possibly happen, the list would look something like this:

  • You can lose passwords, and lock yourself out of things.
  • Team members can accidentally apply damaging updates to the wrong environments, because of confusion over which credentials to use and excessive access privileges.
  • You can leave your system vulnerable to attacks in which an intruder gains an initially low privilege level, and gradually escalates by finding loose passwords and using them.
  • Your ex-employees still have shared passwords that you gave them – and may be using them
  • You can’t on-board people like temporary contractors easily because you don’t have a way to “sandbox” them appropriately. This costs you time and opportunity.
  • You’re unprepared when a big customer is interested in your technology and asks for a compliance audit. (Wouldn’t it be great to be ready when that day comes?)
  • You can’t easily rotate credentials since you have no way to track down all the places where the old credential is in use.
  • You can expire credentials that you think you aren’t using any more, but find out that you actually are still using them – when your important systems stop working.
  • You can’t decommission a user account, because that user is sharing a password that everyone else still needs to use.

Clearly, trying to protect yourself from each of these different scenarios – traditional ad hoc password management – is impractical.  Where do you start?  Where’s the source? And how do you know?

So what’s the alternative?

Let’s look at this by analogy. Not so long ago, it was common to find small software teams working without source control. They ran into nasty problems like:

  • Out-of-sync code between developers
  • Potentially valuable history being overwritten by new code and lost forever
  • Loss of entire bodies of work due to hard drive crashes, computer damage, theft, etc.
  • Difficulty in bug fixing because the team couldn’t figure out which code the customer was running

Medium and large enterprises learned these lessons the hard way, and invested to solve them with enterprise source control solutions like Subversion, Git, ClearCase, Perforce and PVCS. Specifically, the community learned that, because code is so important, it must be managed with a system that’s:

  • Centralized – and therefore authoritative
  • Durable – so that it’s always available and never lost
  • Multi-user – so that teams can collaborate effectively
  • Versioned – to preserve history
  • Role-based – for separation between code authors (developers) and code consumers (build + test infrastructure)
  • API-accessible – to facilitate automation
  • Searchable – so that things can be found

Today, there’s a universal understanding of the importance of source control in software development. And, thanks in part to today’s excellent, inexpensive source control systems (GitHub), even the smallest projects use source control from the start.

We need the same kind of approach for the management of access and key credentials.

What are good password management practices?

Password management in the software industry is all over the place, with few good strategies and technologies. In most organizations, leaky password management is still the norm. It’s easy to think of good password management as simply locking away a single copy of a password. However, this strategy is laughably insufficient because passwords need to be recoverable and frequently used. What happens if you lose the only copy of a key, or the team member who has all the keys becomes unreliable?

To protect yourself adequately, your password management system should have the following characteristics:

  • Centralized location – so you can protect and secure your password “vault.”
  • Highly available and durable – so they are always there when you need them.
  • Encrypted – to avoid accidental exposure.
  • Group-accessible – passwords should NOT be owned by one single user; they should be owned by a group. You don’t want critical system access to depend on one individual.
  • Versioned – so that critical keys won’t accidentally be over-written.
  • Searchable – to prevent loss, duplication, and general inefficiency.
  • No credential sharing to enable identity tracking, revocation, and rotation.
  • API-accessible – so they can be distributed into the infrastructure via scripts and programs.
  • Separation of duties – so that one group can update the passwords and another group can use them.

What to do about it?

You can build out your own modern password management system. Or you can outsource it to an expert — just like you’ve outsourced other important supporting functions of your business to the cloud or Internet services.

My suggestion: Go to and either sign up for the hosted service (my personal preference) or get ahold of the AMI and run it on your own AWS instance.

There’s NO excuse for not plugging this hole right now.  Just do it.

This entry was posted in Enterprise Software, Information Technology, Start-Ups. Bookmark the permalink.

1 Response to System Password Management: JUST DO IT.

  1. Nick E says:

    Nice post, Andy. Check out Mitro ( from former Googlers.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s