continuous delivery implementation : getting started with aws
This article is part of the Continuous Delivery Blueprints series. It discusses how to go from no cloud infrastructure and no continuous integration set up to having a functioning Continuous Delivery Pipeline in Amazon Web Services. It discusses high level topics while also providing a reference implementation so it’s easy to follow along with.
Everyone is talking about migrating to the cloud these days, and getting started with Amazon Web Services is super simple to do. However, most people just rush in, creating headaches for themselves down the road. There are some best practices you should take at the beginning of your cloud migration that will make things easier, more secure, and allow you to scale up and out better.
What we’re going to do today:
- • Create an AWS Account
- • Turn On AWS CloudTrail
- • Turn On Programmatic Billing
- • Create IAM Users and Groups
- • Add MFA for New Users
Create your AWS Account
It all starts here: aws.amazon.com. Find the big sign up button and just follow the prompts. A couple of things to note before getting started:
- It’ll prompt you for your information (name, email, address, etc) and credit card info, so you should get that figured out first.
- You’ll need to verify your account via a phone call, so have your phone handy.
- You don’t need to sign up for support just yet.
Once you’re signed up, just login into the AWS console. The console allows you to interact with most AWS services. Most people will start building their servers in the sky right away, but there’s a bit of information you should probably know up front, and some account set up we recommend before getting started. Let’s go over that first.
What You Need To Know About AWS Before Setting Stuff Up
Amazon Web Services offers a lot of different services, from virtual computing instances and storage to transcoding and streaming. Going over each service would take a whole series of blog posts, but an understanding of how AWS is laid out will be helpful when getting started.
AWS has data centers all over the world, and has two ways of grouping them. At global scale there are regions, representing parts of or entire continents. Inside each region are availability zones. Regions are completely distinct entities, and you can only work in one at a time. Availability zones are designed to talk to each other, and AWS will automatically spread your resources across availability zones. Availability zones, however, can only speak to other zones within the same region.
Choosing a region is important, though these directions are the same more-or-less in every region. However, be aware that not all services are available in all regions, and pricing does vary by region. In addition to that, US-East-1 is the “default” zone when you start with AWS, and has been around the longest. For that reason, it’s also the most popular, and sometimes you won’t be able to allocate resources in certain Availability Zones in the US-East-1 region due to those zones being at capacity.
AWS provides lots of documentation on how to choose a region, so definitely look through that to decide the best place to host your infrastructure. If you’re just doing initial investigation into AWS and aren’t sure what region to use, just pick one close to you.
Making a Name For Yourself
We’ll be talking about several AWS services in this section, and many of them make use of AWS Simple Storage Service, or S3. S3 allows you to store objects in the cloud with a high degree of durability. Where S3 objects are stored are called “buckets”. S3 bucket names have to be unique, not just across you account, but across the entire world. A bucket name is globally unique. By the time we’re done, we’ll have created a couple buckets, as well globally unique login URL. For that reason, you should come up with a unique identifier now. For example, when we tested this documentation, we used the identifier “stelligent-cdblueprints.” Just note it down now and we’ll refer to it as we go on.
Turn on CloudTrail
First thing is to turn on CloudTrail. CloudTrail is basically logging for your AWS account. It will generate JSON files and store them in an S3 bucket (Amazon’s cloud storage solution) every time an action is performed on the account. While we won’t be doing a lot with CloudTrail right away, we’re turning it on now because it’s not retroactive — you can only see logs after you’ve turned it on. So let’s turn it on first.
(Quick note: CloudTrail is a relatively new service, and at the time of this writing is only available in two regions: US-East-1 and US-West-2. If you’re using a different region, you might not be able to turn CloudTrail on. If that’s the case, just skip on to the next step.)
- Find CloudTrail panel from the main AWS Console,
- Click Get Started and just punch in an S3 Bucket name. (As was mentioned above, the S3 bucket name has to be globally unique. One approach is to take the unique identifier you came up with before, and just append -cloudtrail to it. We’ve named our bucket “stelligent-cdblueprints-cloudtrail”.)
- Click OK and you’re done. That was easy.
Turn on Programmatic Billing
Next, we’ll want to turn on Programmatic Billing. This will store your AWS billing in JSON files in another S3 bucket, so that other services can analyze your spending and plot trends over time. We’ll be visiting those kind of tools later on, but we want to enable programmatic billing now because (just like CloudTrail) it only generates data from the present — there’s no way to go back and generate historical data. By turning it on now, when we do start parsing that data for trends, you’ll have a good amount of data to go back through.
Unlike CloudTrail, you’ll need to create and permission the bucket for this yourself.
- Go to the S3 console so we can create a new bucket. (Taking your previous unique identifier and just appending -billing to it isn’t a bad idea. We’ve named ours “stelligent-cdblueprints-billing” to keep with the theme.)
- Click Create Bucket and punch that name in.
- We’ll need to get a bucket permissions policy. Luckily, AWS will generate that for us at this page (we’ll need to flip back to the S3 page in a second, so open this in a new tab): https://portal.aws.amazon.com/gp/aws/developer/account?ie=UTF8&action=billing-preferences
- Go down the list and turn everything on one and a time.
- When you get to to Programmatic billing, punch in the name of your bucket, and click “sample policy.” Just copy that policy, then flip back to your S3 bucket.
- Click on the bucket, then properties, then Permissions, and you’ll see an option for setting an access policy.
- Click into that, paste the policy you just copied, and save.
- Now, flip back to the Billing Preferences page, click save there
- Continue to enable everything else on this page.
If CloudTrail and Programmatic Billing are so important, why aren’t they turned on by default?
One thing to be aware of with these two services is that they will put data into your S3 buckets. S3 storage is very cheap, and while it is pretty close, it is not free. You’ll be paying between nine and fifteen cents a gig for storage, depending on region. For more details, check out the S3 pricing page. The services themselves don’t cost anything, though; you only pay for storing the data they generate.
Create IAM Users
Now that the bookkeeping is taken care of, let’s set up some users. A lot of new AWS users will start doing everything as the root account, which besides being a bit of a security risk, also poses some issues when you try to have multiple developers building solutions in your cloud. That’s why we strongly recommend setting up IAM users and roles from the beginning.
We’re going to use the AWS Identity and Access Management (IAM) console. IAM allows you to create users, groups, and roles so that you can manage users and access to your AWS account. For the first section, we’ll only be creating one user (for you) and one group (admins) but as your usage of the cloud increases and you need to add more users, you’ll be able to control that from here.
To create a new admins group, head to the IAM console
- Click Create Group, and follow the prompts.
- We’ll name the group “admins” and give it Administrator access.
Now that we have an admins group, go to the Users panel and create a new user for yourself to log in as. It’s pretty straightforward, and if you hit any bumps in the road, AWS has some pretty good documentation about it.
After you create the user, add it to the admins group. Then, for each user we want to set up two types of authentication. The first is a simple password. Under each users’ Security Credentials tab, click the “Manage Passwords” button and you’ll be able to assign a password.
After each user logs in, you’ll want to require them to add a multi-factor authentication (MFA) device to their account. To add an MFA device
- the user will need to login and go to the IAM console
- find their username
- under the security credentials tab, select “Manage MFA device.”
- Then follow the steps to add your virtual MFA device to the account.
Having MFAs set up for all accounts helps ensure that AWS accounts won’t be compromised, keeping your data safe. Also, it helps ensure that your account won’t be used for malicious purposes (DDOS attacks, spam emails, etc) which would at best would increase your AWS bill and worst case have your entire account disabled. We strongly recommend enabling MFAs for all user accounts.
Now that users are able to log in, we’ll need to give them a URL to do so. If you go to the main IAM console, you’ll find a IAM User Sign-In URL section. Remember the unique identifier you came up with your CloudTrail and Programatic Billing buckets? That’s probably a good option for your sign in URL. Changing it is optional, though highly recommended.
Wrapping Up
Using AWS is easy; using it well takes some thought. By setting up logging of your usage and billing information, you’ll be able to identify trends as time goes on. By setting up groups and users, your account is prepared to scale as you bring on more developers. And by giving those users multi-factor authentication, you’ve helped ensure the security of the account. You’re in a great place to start using the cloud. In our next post, we’ll lay the foundations for building a continuous delivery pipeline.