macOS: ‘dig’ vs. ‘dscacheutil’ while using split DNS with Viscosity VPN client

If you’re using Viscosity VPN client on Mac, and have enabled the split DNS for the VPN domains, `dig` doesn’t work quite as someone familiar with Linux/*NIX would expect. This is because Apple has replaced the traditional resolver on macOS with something “more advanced”. Granted, it handles the split DNS gracefully, but as a result, using `dig` for the VPN domains without defining a DNS server for a query doesn’t produce any result even though resolution works otherwise in macOS.

This works:

~ dig +short random.ac a
52.4.179.30

But this does not:

~ dig +short internal.ville.sh a

However, this does:

~ dscacheutil -q host -a name internal.ville.sh
name: internal.ville.sh
ip_address: 10.50.50.10

To make things easier, I have created the following `zsh` alias:

alias dnsquery='dscacheutil -q host -a name'

However, I’ve forgotten this a few times when some time has passed since the last encounter with dig on internal domains. And then time goes down the drain trying to figure out if something is wrong with the DNS.. but it’s working all along! So I added a reminder for myself in form of another `zsh` alias:

dig() { echo && echo -e '\033[0;97m\033[41m Remember, this is a Mac! Use "dnsquery" instead! \033[0m' && /usr/bin/dig $@ } #macOS

And so:

~ dig +short internal.ville.sh a

 Remember, this is a Mac! Use "dnsquery" instead!

#oh right! :-)

~ dnsquery internal.ville.sh
name: internal.ville.sh
ip_address: 10.50.50.10

Exploring GitHub Flavored Markdown local preview

I’ve been writing various lengthy documents for GitHub repositories recently, and as I push them to GitHub, and then view them in the browser, there’s almost always something wrong with the layout resulting in additional commits or amends/forced updates which I would rather avoid. I was initially using MacDown to edit/preview Markdown, but its rendering of GitHub Flavored Markdown (GFM) was way off. There had a to be a better way, and there is! I found two solutions that work reasonably well: the first one is Node-based vmd for general use (it watches the given file, and thus works with any editor), and the second is a set of plugins for Sublime Text 3. The former is now my choice for offline GFM previewing (such as on a plane), while the latter is my preferred choice because 1) Sublime Text 3 is already my go-to editor, 2) the Sublime Text plugins provide incredibly handy macros for, and context-highlighting of the Markdown being edited and, most importantly 3) the GFM is rendered through the GitHub API, ensuring that the rendered output matches the way GitHub renders it 100%. Both `vmd` and the Sublime Text solution work on macOS, Windows, and Linux Desktop.

Some other options that I looked at included Grip (a cool idea, but it kept crashing on me), or Atom‘s GFM preview (which, even with the right extensions installed, wasn’t accurately GFM, which is strange considering that Atom is made by GitHub; if you know how to make Atom render GFM through the GitHub API, please comment this article).

First, `vmd`. It’s easy to set up, simply first install Node.js on your platform, then add `vmd` with `npm` as a global:

npm install -g vmd

Now simply point `vmd` to the file you’re editing, and it gets re-rendered every time the file is saved as long as `vmd` is running.

vmd SomeFileName

That’s all. You can use any editor to edit the file.

However, if you’re using Sublime Text 3, give the following a try; it makes Markdown editing outright fun. The following instructions are for ST3.

  1. Install Sublime package manager if it’s not yet installed. Then restart Sublime.
  2. Install the following packages with the Package Manager:
    `Ctrl-Shift-p [Win] / Cmd-Shift-p [macOS] > Package Control: Install Package`

    Restart Sublime.

  3. Install monokaiC theme (read the installation instructions on its GitHub page).The target location depends on the platform, see this StackOverflow answer for details, but use the User subdir under the directory listed for your platform.
  4. SETTINGS

    1. In Markdown Preview User Settings:
      `Preferences > Package Settings > Markdown Preview > Settings — User`

      Add:

      {
         "git_oauth_token": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
         "github_mode": "gfm",
         "enable_autoreload": true
      }

      ** Get the `git_oauth_token` from GitHub. Log in to your GitHub account, then:
      `Settings > Developer Settings > Personal Access Tokens`, or use generatetoken.sh script. No scope permissions are required for a token for it to be valid for Markdown rendering. Note that without a token set you’re limited to about 60 renders per hour, while with the token set you can make 5000 requests per hour (which you’ll never hit, since the re-render only occurs on file save).

      ** Make sure you have a comma at the end of each line, except the last parameter line in the settings

    2. In Markdown Editing User Settings for GFM:
      `Preferences > Package Settings > Markdown Editing > Markdown GFM Settings — User`

      Add:

      {
        "color_scheme": "Packages/User/ME-MonokaiC.tmTheme",
        "extensions":
        [
          "md"
        ]
      }
    3. In Key bindings
      `Preferences > Key Bindings — User`

      Add:

      { "keys": ["alt+l"], "command": "markdown_preview", "args": {"target": "browser", "parser":"markdown"} },
      { "keys": ["alt+m"], "command": "markdown_preview", "args": {"target": "browser", "parser":"github"} }
      

    Restart Sublime.

  5. Enable Simple Reload:
    `Ctrl-Shift-p [Win] / Cmd-Shift-p [macOS] > Live Reload: Enable/Disable Plugins > Enable Simple Reload` You need to reactivate Simple Reload like this whenever you restart Sublime, and want to use the auto-reload.

All set! Open an existing markdown (.md) file, or start a new file and save it with `.md` extension. Then hit `alt-m` for GitHub-rendered preview, or `alt-l` for locally rendered (but less accurate for GFM) Markdown preview. When you make changes and save the file, the preview gets automatically refreshed in your browser using the originally selected method (i.e. GitHub API or local).

Be sure to read the MarkdownEditing documentation for awesomely useful keyboard macros and other support for Markdown! This will also give you syntax highlighting in fenced code blocks when you postfix the first triple-backtick with supported language identifier, like so:

```sql

Happy GitHub Markdown editing! Octocat

Update 2018-03-02: The `git_oauth_token` must be set in Markdown Editing package settings, not in global Sublime settings. This correction has been reflected in the above instructions.

Easy MFA and Profile Switching in AWS CLI

UPDATE 2019-01-15: I have released a much-improved v2.0 of the awscli-mfa.sh script. Its location has also changed; it can now be found at https://github.com/vwal/awscli-mfa

An updated blog entry is forthcoming; the article below discusses version 1.0 which was completely rewritten in this new iteration.

As I alluded in the previous post, I have released another AWS script that works in conjunction with the AWS CLI Key Rotation script. This one is for easily activating an MFA session while on the command line. It also makes it easy to switch between the different configured AWS profiles.

While MFA is perhaps more frequently used with the web console sessions, in some ways the API keys are even more of a security risk. Whereas people usually memorize the console password, or keep it in a password manager, the API keys sit in plaintext in `~/.aws/credentials` file. All it takes is a malware that dumps the content of that known file location to a hacker, or perhaps a stolen laptop, or a stolen unencrypted flash drive (that you used to transfer the API keys to your new laptop, say), and a third party will have access to as much of your AWS assets as the IAM policy associated with the key allows. Admin keys? Full access! For this reason, I advocate the use of MFA especially for the keys with wide access, and that are intended for interactive use. Of course roles should be used as much as possible, and as much as possible should be automated, but still, in most environments people use the API keys for various command line tasks and to run utility scripts from the admins’ laptops.

Enter awscli-mfa.sh! This script makes it easy to select a profile that has MFA configured, to initialize an MFA session, and to make the MFA session default for the current console session (via an environment variable). The script is obviously interactive since it’ll prompt for an MFA one-time passcode. When you run it, it’ll query the AWS profiles configured in `~/.aws/credentials`, and list them out, along with the actual IAM username associated with each profile (since the profile names are arbitrary, and as such can be set to anything). The script will also indicate whether a virtual MFA has been configured for the profile. You can select to activate a profile that doesn’t have MFA configured, but obviously then an MFA code is not prompted for, and MFA is not used. When the chosen profile does have MFA configured, the script will prompt for the one-time passcode (e.g. from Authy or Google Authenticator), initialize the session, and output the environment export variable for easy cut-and-paste to the console (when ran on a Mac, or in a Linux GUI that has `xclip` installed, the export string is automatically copied on the clipboard). Paste on the command line, hit Enter, and the MFA profile is active!

Once an MFA session has been activated and you run the script again, you’ll see the newly initialized MFA profile under its base profile. You can switch away from it to another profile, and then back again. The MFA profile is indicated either in “OK” or “LIMITED” status. The latter may suggest that the IAM policy session duration has been set to a shorter period than what was configured in this script (you can adjust the token lifetime duration in the script on line 16. It should be set to match the session duration set in the MFA enforcement policy (if the policy duration times out first, the key appears still valid, but it’s no longer authorized for any transaction on the AWS). Note that if the IAM policy for the profile is limited so that it isn’t allowed to query `aws iam get-user` for itself, you’ll see the “LIMITED” status even when the MFA session is valid. Once the session token expires, the MFA profile disappears from the session list (even though you can see it still if you `cat ~/.aws/credentials` file.

A good write-up and an example MFA enforcement policy can be found here. I used their policy with a slight modification: I set the maximum MFA session duration to six hours by altering the Condition for the last two Sids in the policy, like so:

"Condition": {
  "NumericGreaterThanIfExists": {
    "aws:MultiFactorAuthAge": "21600"
  }
}

As I mentioned in the previous post (“AWS CLI Key Rotation Script for IAM Users revisited“), you can use an MFA session initialized with this script to rotate the keys of the MFA session’s base profile that doesn’t have permissions for anything without an active associated MFA session. As long as you have an MFA session initialized for the profile whose keys you want to rotate, the key rotation script will detect the presence of the MFA session, and ask if you would like to use it to execute the key rotation for the base profile. The MFA sessions are not visible in the key rotation session list because their keys are transient, only valid for the length of an MFA session.

The script was written for macOS, but portability for Ubuntu (and so, probably most/all Linux distros) was added.

The awscli-mfa.sh script is available from its GitHub repository.

I recommend Authy for the virtual MFA (it’s available both for iOS and Android).

As always, comments or improvement suggestions are welcome (either as comments to this post, or as PRs to the repository)!

UPDATE 2018-02-19: I’ve updated the script, fixed some minor bugs for some edge cases, cleaned up its formatting, clarified the output, and added a companion script (source-to-clear-AWS-envvars.sh) that can be used to clear any AWS secrets/variables that may be set in the environment through the use of the awscli-mfa.sh script.

AWS CLI Key Rotation Script for IAM Users revisited

In April of this year I published a Bash script for rotating the default AWS API keys configured in `~/.aws/credentials` file. Now I have improved on the original script, adding the following functionality:

  • The script is now fully interactive, and supports multiple profiles. It was an interactive script before, potentially requiring user input during its run, and as such was intended for manual execution. Now the script presents a list of configured profiles from which you can choose a profile whose keys you want to rotate.
  • The script works with Multi-Factor Authentication. At work we have been moving increasingly to MFA/2FA, and have also started enforcing MFA use from the users (this policy works great for that purpose). Since MFA enforcement for AWS console and for the CLI API cannot be separated for a given IAM user for the most part (since the console is just a GUI for the API), there had to be a solution for relatively convenient use of MFA on the command line. My second script (more detail in the next post) offers that capability, and now the key rotation script is aware of it. If you’re rotating the keys for an IAM account whose MFA use is enforced, the script now detects an existing MFA profile (created by my CLI MFA script), and can use it to authorize key rotation for the base profile, which might otherwise not be authorized to execute the key rotation operation.
  • The listing of the configured profiles includes the current keys (two concurrent keys is the recommended maximum by policy), the ages of the keys, the actual IAM username (since the profile name is arbitrary, and as such can be set to anything), and the access status of the profile (‘OK’, or ‘LIMITED’; the latter is displayed when the profile doesn’t appear to have normal access during the query process — for example, it may result from MFA enforcement)
  • Many AWS errors are also masked and translated to more user-friendly outputs. If a profile doesn’t have a valid key, the script handles it gracefully and displays: “CHECK CREDENTIALS!” next to that profile (and obviously it cannot give more detail about such profile, or offer the option to rotate its keys).
  • The script was originally written for macOS, but it has now been tested on Ubuntu, and portability has been added (hence it likely works on other Linux distros as well).

The latest version of the aws-iam-rotate-keys.sh script is available from the same repository as before.