Since I seem to be on a kick posting AWS articles, here’s one more!
Rather than using shell shenanigans (`grep`, `awk`, `sed`, and friends) to filter AWS CLI output from `text` output type, the integrated JMESPath query command makes it easy to filter the output before `aws` command even spits it out. However, a JMESPath query can become pretty complex when using it to dig some information deep in the output structure. Of course, while a query can be complex, digging the same information out with a string of piped `greps`, `seds`, and `awks` is even more messy, not to mention fragile.
A very handy trick exists that makes building JMESPath queries a lot easier! This technique can be used for ad hoc queries when you simply need to find some information from a large output, but it can just as well be used as a testing ground for a query to be embedded in a script. Here’s how it works (assuming you’re on some kind of *nix command line… macOS, Ubuntu, something; though it may even work in Windows command prompt since it’s Python based). First, install `jmespath-terminal`:
sudo pip install jmespath-terminal
Now (with `aws` CLI obviously configured), retrieve the unfiltered output from which you wish to dig the important nuggets of information. Here I’m listing all the security groups. Note that the output format needs to be set to `json` (if you have configured output format to something else (i.e. `table` or `text`) in `~/.aws/config`, here `–output json` will override it (on the other hand, `json` is the default output format if you haven’t set it otherwise). Save the output into a file. Then launch `jpterm` with the file name as the sole argument.
aws ec2 describe-security-groups --output json > ~/securitygroups.json jpterm ~/securitygroups.json
And…
Now you’ll have an interactive text-based console, where you can experiment with JMESPath queries! In the left pane is your original data, while on the right you’ll see the result of the JMESpath query immediately as you type!
Check out the official JMESPath Specification for everything you can do with the embedded JMESPath queries. And while you’re reading, also check out James Lawson’s 2015 article on JMESPath queries in the AWS CLI for more insights.
One more thing. I mentioned above that you must use `–output json` to get the output in a format that `jsonpath-terminal` can understand. That is accurate, but if you’re building the JMESPath query for a script, once you’re satisfied with the query, you can switch the output format to `–output text`, and AWS will use the query just fine, while the final output will be in `text` format (i.e. no quotes around a string, no JSON artifacts). This can be useful especially if the final output result set is very small, such as a single value, or a simple list of IDs, as such output may then be easier to process further with your favorite shell commands.