TL;DR: Terraform is cool, but like every tool managing the production infrastructure where your applications run, it's on you to be careful and thoughtful

The beginning

It's very common nowadays to go to conferences and hear people mentioning terraform when talking about cloud infrastructures. As an automation geek, the first thing you do is to download the terraform binary and start reading the online documentation learning all the magics of HCL, the Hashicorp Configuration Language. It takes a while to appreciate its power and expressiveness. But it's exactly at this point that few questions arise:

  • Can I really use this to manage my production infrastructure?
  • How can I use this with my team?
  • Is this secure? Is this safe to use?
  • Is there any unspoken rule I should know before start using it?

The idea

Terraform was created for:

  • Managing your infrastructure as code
  • One syntax to rule them all (the "clouds")
  • To give idempotence to unidempotent APIs (try repeating this 3 times faster)

Flow

Terraform is just a command-line tool and a pretty easy one to use: there is no global configuration and only files in the working directory are kept into consideration by the program.
Most of the work is done with only three sub-commands:

  • terraform init checks-out or initializes the remote state, installs the required providers and modules
  • terraform plan compare the actual state with the desired state and computes the difference
  • terraform apply same as plan but actually performs the changes needed to converge to current state of the infrastructure to the desired one.

All of these commands are safe to run multiple times with no risk of causing damage. The output of every command is clear and self explicable.

API mindset

When first approaching a cloud provider, the easiest thing to do is to start using the web interface: it's clean and simple and gives instant feedback on the actions taken. But after a while using it with real-world projects, it gets frustrating having to point-and-click multiple times and not been able to automate with the GUI: this is where terrafrom shines.
But (yes, there's is a but), the GUI usually abstracts the complexity of the cloud provider: it wraps together many API objects.
With Terraform, using the restAPI that your cloud provider offers you, it forces you to understand which objects play a role in the action you are performing and which not, those that combine naturally and those that don't.
There are no more buttons that create networks, internet gateways, security groups, routes, floating IPs, and finally the simple VM that you wanted. Now you'll have to create all this resources by yourself and associate them to each others.

Mind your state

Terraform has a concept called state: which is basically a file where it keeps track of all the resources it manages for you. The state file contains all input data used to create the infrastructure, including passwords or tokens (in clear text). Moreover, terraform only manages the resources present in its state file: if you accidentally delete the file, you will not be able to manage anymore these resources via terraform. This is why it's critical to keep the state in a secure place, locked and encrypted, accessible only by who/what needs to run terraform. For this purpose having a state persistent on the local disk makes sense only in the initial exploration steps of terraform. When things start to get serious, you should really start using remote state. There are multiple backend available available to store safely your infrastructure state, with few things to keep in mind when choosing yours as it should support:

  • versioning, so that you can rollback to any given moment in time
  • locking, it's easy to see how things can go wrong when multiple people try to terraform apply at the same time. This is why only the first person in a team running the command, can acquire the lock on the remote state. This is a core feature that will probably save you a headache.
  • encryption, inside the state file you have passwords and tokens stored in clear text. It's therefore mandatory to have encrypted and secured remote state.

It might sound really scary and complex but actually using remote state is nothing more than something like this:

terraform {
  backend "gcs" {
    bucket  = "example-foobar"
    prefix  = "prod"
  }
  required_version = "> 0.11.0"
}

Keep your friends close and your credentials closer

To be able to manage your resources, terraform must have access to your authentication credentials, in a form of username/password, API tokens, service account file.
While this is possible in principle, it's highly recommended to not keep secrets in your .tf files at all. Most of the providers (which are the entities you're authenticating against; e.g. aws, gcp,azure) have the ability to retrieve credentials from environment variables, like AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY for aws.
In case the used provider does not directly implement secrets discovery from the environment variables, you can easily avoid committing your clear-text credentials using the embedded capability of terraform to read generic variables from the environment. This feature is really helpful when setting up databases or encrypted object storage.

Know thy dependencies

Terraform has awareness of infrastructural dependencies: for example, when defining a VM attached to a specific network and declaring the network, including its properties, terraform knows that it first needs to create the network and then the VM, same with disk and IPs. Despite all the magic built in, however, terraform sometimes might still fail: for these special situations the depends_on keyword can be used for explicit dependency declaration.
More on this topic can be found here.

Documentation

Everything you need is neatly written here
At the beginning it may seem overwhelming but it is complete: a rare feature for an open-source project.
The list of available providers can be found here instead.

If you like all the features but don't want to manage complexity there is, as always, a managed solution by Hashicorp with their Enterprise offering providing SCM integration, policy and user management.