• My Links at Diigo
  • Ergonomics Resource

RANDOM.AC/CESS

Musings of a Web Technologist

Home Archives for CLI

Interactive AWS CLI Query Filtering with JSONPath

Posted on 31 October 2017 by Ville 1 Comment

Since I seem to be on a kick posting AWS articles, here’s one more!

Rather than using shell shenanigans (grep, awk, sed, and friends) to filter AWS CLI output from text output type, the integrated JMESPath query command makes it easy to filter the output before aws command even spits it out. However, a JMESPath query can become pretty complex when using it to dig some information deep in the output structure. Of course, while a query can be complex, digging the same information out with a string of piped greps, seds, and awks is even more messy, not to mention fragile.

A very handy trick exists that makes building JMESPath queries a lot easier! This technique can be used for ad hoc queries when you simply need to find some information from a large output, but it can just as well be used as a testing ground for a query to be embedded in a script. Here’s how it works (assuming you’re on some kind of *nix command line… macOS, Ubuntu, something; though it may even work in Windows command prompt since it’s Python based). First, install jmespath-terminal:

Shell
1
sudo pip install jmespath-terminal

Now (with aws CLI obviously configured), retrieve the unfiltered output from which you wish to dig the important nuggets of information. Here I’m listing all the security groups. Note that the output format needs to be set to json (if you have configured output format to something else (i.e. table or text) in ~/.aws/config, here --output json will override it (on the other hand, json is the default output format if you haven’t set it otherwise). Save the output into a file. Then launch jpterm with the file name as the sole argument.

Shell
1
2
aws ec2 describe-security-groups --output json > ~/securitygroups.json
jpterm ~/securitygroups.json

And…

jmespath

Now you’ll have an interactive text-based console, where you can experiment with JMESPath queries! In the left pane is your original data, while on the right you’ll see the result of the JMESpath query immediately as you type!

Check out the official JMESPath Specification for everything you can do with the embedded JMESPath queries. And while you’re reading, also check out James Lawson’s 2015 article on JMESPath queries in the AWS CLI for more insights.

One more thing. I mentioned above that you must use --output json to get the output in a format that jsonpath-terminal can understand. That is accurate, but if you’re building the JMESPath query for a script, once you’re satisfied with the query, you can switch the output format to --output text, and AWS will use the query just fine, while the final output will be in text format (i.e. no quotes around a string, no JSON artifacts). This can be useful especially if the final output result set is very small, such as a single value, or a simple list of IDs, as such output may then be easier to process further with your favorite shell commands.

Filed Under: AWS, DevOps, Technical Tagged With: aws, CLI, JMESpath, query

Blog Author, Ville Walveranta

Information Architect, Application Developer, Web Technologist

Social

Follow me on:

StackExchange

profile for Ville on Stack Exchange, a network of free, community-driven Q&A sites

Recent Posts

  • macOS: ‘dig’ vs. ‘dscacheutil’ while using split DNS with Viscosity VPN client
  • Remove DRM Easily (?) from Your Audible Purchases
  • Exploring GitHub Flavored Markdown local preview
  • Interactive AWS CLI Query Filtering with JSONPath
  • Easy MFA and Profile Switching in AWS CLI

Tags

2fa automatic automount aws bash beyond compare boot centos co-lo co-location comparison diff DNS enforcer esxi freebsd ftp fusemail Hardware iam install key Linux mailtrust microsoft monitoring multi-factor nfs RELEASE-7.0 rotation script security self-healing shell software sublime tbe trackball ubuntu unix vista vmware Windows windows 7 workflow

Blog archive

February 2019
M T W T F S S
« Apr    
 123
45678910
11121314151617
18192021222324
25262728  

PGP/GPG

Keybase: vwal
(PGP Key ID 2E99 86D7 7ED9 9C13)

Copyright © Ville Walveranta 2019 - All Rights Reserved · Powered by Wordpress and Genesis Framework