Nuts and Bolts

One of the biggest game-changers to spot instances are spot fleets. I mentioned how you could save 50%-90% when you take advantage of spot instance pricing in: How Does Spot Pricing Work? Spot fleets is where things get even more interesting. With a normal spot instance request, you place a bid for a specific instance type in one specific AZ and hope you get it. With spot fleets, you can request a variety of different instance types that meet your requirements. Additionally, you can spread your spot fleet bet across multiple AZs to increase the likelihood of getting your instance fulfilled. It’s like playing rollette and being able to bet a single chip on multiple numbers. Insanely cool! The method dramatically increases the chances of you getting instances available at your spot bid price. Like I said, game-changer.

Spot instance pricing is a fascinating EC2 pricing model that many people have not heard of yet. It is not only interesting from the 50%-90% savings perspective, which is already huge. It is also interesting because it encourages you to think about building your system with the best practice of high availability in mind.

Just like renting or leasing a house, when you pay for servers from AWS, there are many, many different options. The plethora of options is so vast that it can be overwhelming staring at them. I’m hoping to cover the pricing options in a useful way. With the options, you get exactly the same server, but you pay a different price because of the different commitment levels from either you or from AWS.

Have you ever been asked to deploy a branch of code to the staging or uat environment but cannot because the environment is currently in used by someone else or another feature? Usually, you end up having to wait until the environment free. Ultimately, after this happens often enough a common request is to build additional environments. This can take some time though, so you still have to wait.

This is an introductory guide to ufo, an ECS deployment tool, with AWS Fargate. Ufo helps you deploy Docker images to AWS ECS. With Fargate, you can run docker containers without having to manage servers, for an interesting “serverless” option.

This is an introductory guide to ufo, an ECS deployment tool. Ufo helps you deploy Docker images to AWS ECS quickly. One pretty neat thing about ufo is that it provides direct access and control to the ECS Task Definition. So you can customize your ECS container options to your heart’s content. We’ll also talk about some of the resources that ufo creates.

There are some pretty big changes for ufo version 4. Here’s what’s new:

• Fuller CLI Toolkit Commands
• Load Balancer Support
• Updated Tutorial Guide
• Security Groups
• Improved Fargate Support
• Extra Env Support
• CloudFormation Implementation
• Upgrade Guide

Recently upgraded ufo to add support for ECS Fargate. As part of this, I had a chance to look at the pricing for Fargate. Found out that ECS Fargate’s pricing is competitive to Heroku’s offering. The pricing makes a lot of sense because they offer a similar value proposition. We do not have to manage the servers.

Docker is kind of like git in a sense that there are only a few commands to learn to for it be useful. This tutorial focuses on the docker commands that I find myself most commonly use. This article assumes that you have a general idea of what docker is and have it installed and set up already. Hopefully, it serves as handy reference or cheatsheet for useful common docker commands.

Looking into AWS Step Functions. Here are some initial thoughts.

AWS Step Functions involves translating your logic into a declarative form. In this sense, it’s like CloudFormation. For some, the change in logical constructs can be a bit daunting at first. Here are some benefits and disadvantages.

If you are working with AWS heavily, you should look into a powerful tool called CloudFormation. Taking the time to learn CloudFormation is an investment that is easily returned to you in the form of powerful automation. In this guide, we’ll provide a gentle introduction to CloudFormation, and by the end, you’ll have the skills to start using it in your own AWS workflow.

The common advice to speeding up your Docker development workflow focuses on reducing your Docker image size. At the surface, the logic makes a lot of sense. By making your Docker images smaller, you win. With smaller image sizes, there are fewer bytes to pull down and push up to the registry, thus saving you a boatload of time. Smaller images = solution. What the advice does not take into account is a precious resource: developer’s time.

One way to think about Docker image layers is to think of them as git commits. While the two are technically different, this article uses this analogy to point out an interesting commonality between both of them.

There are a ton of ssh tips out there, and I thought that I surface 3 of them. Here are 3 ssh tips that I’ve learned over the years. Hope you find them useful 🎉

It can be difficult to understand the CloudFormation resources and their dependencies when working with a complicated CloudFormation template. We’ll run through some commands to try to help understand their dependencies.

In the previous posts, we went over some jq tips with CloudFormation. We used jq to quickly summarize CloudFormation template parameters and resources. lono summary automates that process and makes the command short and sweet.

Shortly after the last video CloudFormation Templates and jq Tips, I wanted to know what the required parameters in a CloudFormation template were. This is also an easy task with jq. I’ll show you how to use jq to quickly summarize the required and optional parameters in a CloudFormation template in this post. It’s a one-liner 😁

I don’t remember exactly where I got these one-liners from anymore. It’s been in my expander for a while. Here are useful oneliners to convert JSON to YAML and vice versa.

I’ll go over a useful way to summarize CloudFormation templates resources with jq. This is useful when you are looking at a CloudFormation template and trying to understand it. We’ll download an example CloudFormation template and use run through some useful jq commands summarize the resources defined in a CloudFormation template. Note, I’ll only show the output that is useful for understanding.

Embarrassingly, I’ve been converting raw CloudFormation templates to lono CloudFormation templates manually. Probably one of the reasons why it’s been neglected is because this process is so simple, but it is ripe for automation. This should have been done this a long ago.

Update 9/5/2018: The video is old and demos the older version of the tool called thor_template. The new tool is called cli-template. I’ve updated the blog post but not the video.

These two tools seem pretty cool too:

• http://piotrmurach.github.io/tty/: Thor based also.
• davetron5000.github.io/methadone/: Has it’s own CLI parser.

Thanks @eashman for showing me the tools.

In this post, we’ll build a CLI project that is based on Thor in under a second.

The cli-template tool builds a fully functional CLI command based on Thor. The commands immediately work and there are even specs.

Learn about the shortcuts menu tray at the top of the AWS Console. Jam in as many shortcuts as you can 🎉

Once a developer came up to me and told me that he was ready to deploy, but the deploy required migrations be run. He asked me an innocent question, how should he run migrations. Embarrassingly, because we were still evolving our tooling for the new infrastructure, we did not have a process for this simple task. We came up with an ad-hoc process that was honestly pretty terrible. It went like this:

I’ll provide a basic introduction to the mysterious VPC world. I’ll explain terms using various diagrams. We’ll also build a simple VPC network out manually to help understand VPCs.

ECS is the AWS Docker container service that handles the orchestration and provisioning of Docker containers. This is a beginner level introduction to AWS ECS. I’ve seen some nightmare posts and some glowing reviews about the ECS service, so I knew it was going to interesting to get my hands dirty and see what ECS was all about.