Improving My Git Workflow

Through various projects I’ve been working on in the last couple of years I’ve been digging deeper into Git. While many of Git’s basic operations are pretty straightforward, for someone getting used to Git it soon becomes apparent that some operations aren’t as simple as one might first expect, especially when using Git on the command line rather than with a GUI client.

I’ll expand on the topic later, but I wanted to create this stub article pointing to the GitHub repo I published a few days ago. It contains my annotated Git aliases (many of which I use as a quick reference as I haven’t yet committed all of them to memory), and a collection of helper scripts, many of which I have written myself, while few others, like git-wtf, I have not, but have included them for convenience. Also check out the README in the repo for alias dependencies and for other favorite Git extensions I frequently use.

Besides the enhanced Git command line, gitk with its visual blame tool (accessible from gitk, and also directly from the command line with gui blame when gitk is installed), Atlassian SourceTree, Scooter Software’s BeyondCompare (for diffs and merges), and tig for repository viewing on the command line are now solidly part of my everyday Git toolkit.

Update 2016-12-26: If you like to use a graphical Git client, be sure to check out the awesome GitKraken (Windows/macOS/Linux, free + pro subscription). Tower is now also available for Windows, but with GitKraken having established itself, it’s now too little too late, especially for the Windows users who previously didn’t have a really good GUI Git client (Atlassian SourceTree is ok, but not really stellar).

Old-school link lists, the purpose of the web, and Google’s quest against “Unnatural links” (R.I.P. LinkResource.com)

For many years in the 2000’s I maintained an online link list, LinkResource.com, that I originally started to have an easy access to many of the resouces from any web-connected location. Over the years the significance of such resource waned, but I still kept it online since it received a decent number of hits (last update to the list was posted in 2009). Within the last year I’ve received number of emails from people whose site I had featured on the list, begging me to remove the link as Google penalizes their site for “Unnatural links”. Granted, LinkResource was an outright list of links, but wasn’t that a significant part of what the web was built for: to be easily able to access cross-referenced resources. Wikipedia articles, for instance, are rife with links to other Wikipedia articles and external resources.

Of course, this action by Google was brought on by abuse of cross-linking sites in the form of link-exchange networks. But in the end the innocuous links that site owners often have no control over (such as were the links on my now-discontinued LinkResource.com) can create a SEO disaster lowering the search-index score and hence reducing or even destroying business. Fortunately, Google announced the Disawow Tool for webmasters in the fall of 2012 to disconnect links that Google deems harmful from the site indexing score.

LinkResource.com 2000-2013
LinkResource.com 2000-2013

Encrypted Vault in Ubuntu for Your Valuable Data

Recently I set up Bitnami Cloud Tools for AWS to facilitate AWS configuration and use from the command line. After creating an administrative IAM (as not to use the main AWS login), and created and uploaded/associated the necessary X.509 credentials for that IAM login, I realized that anyone who would gain access to the local dev server would also gain full access to several AWS Virtual Private Cloud configurations. Not a terribly likely occurrence, but would I like to risk it? Say, when I have the cloud tools configured on Ubuntu on my laptop, someone could conceivably steal the laptop, and with a little technical expertise, gain access to the Ubuntu instance (running in a VM), and hence to the AWS VPCs.

At least in this case having the IAM credentials and the X.509 keys on a USB drive would be impractical (and would probably increase the likelihood that the keys would get misplaced and end up in the wrong hands). On Windows it’s a simple task to set up an encrypted vault using one of many available utilities to achieve such. But how to do that on Linux? After some digging I came across a Wiki entry Ubuntu: Make a secure vault. It worked fine, but via cut-and-paste that appeared rather cumbersome for daily operations. So I set out to write couple of scripts to make things easier.

First, you need to have cryptsetup package installed. Then you can make use of the setup-crypt script below. These scripts are quick utility scripts that don’t have a separate configuration file; you may want to edit some of the variables on top of the script, namely “CRYPT_HOME” (depending on where you want to place your encrypted vault file), “CRYPT_MOUNTPOINT” (depending on where you want to mount it), and “CRYPT_DISK_SIZE” (the capacity of the encrypted vault in megabytes).

#!/bin/bash

CRYPT_HOME=/root/crypto
CRYPT_DISK=cryptdisk
CRYPT_DISK_FQFN=${CRYPT_HOME}/${CRYPT_DISK}
CRYPT_DISK_SIZE=64	# size in megabytes
CRYPT_LABEL=crypt-disk
CRYPT_MOUNTPOINT=/mnt/crypto
LOOPBACK_DEVICE=`losetup -f`

CRYPTSETUP=`which cryptsetup`
if [ $? -ne 0 ] ; then
  echo "ERROR - cryptsetup not found! Install it first with 'apt-get install cryptsetup'."
  exit 1
fi

IAM=`whoami`
if [ ! "${IAM}" = "root" ]; then
  echo "ERROR - Must be root to continue."
  exit 1
fi 

SETUP_INCOMPLETE=true

function cleanup {
  if [ ! "$1" = "called" ] && [ ! "$1" = "nodelete" ]; then
    echo
    echo
    echo "Crypto-disk setup interrupted. Cleaning up."
  fi
  if [ -b /dev/mapper/${CRYPT_LABEL} ]; then
    cryptsetup luksClose /dev/mapper/${CRYPT_LABEL}
  fi
  
  losetup -d ${LOOPBACK_DEVICE} > /dev/null 2>&1
  if [ "$1" = "nodelete" ]; then
    exit 0
  else
    rm -rf ${CRYPT_HOME}
    exit 1
  fi 
}

mkdir ${CRYPT_HOME} > /dev/null 2>&1

# Capture errors
if [ $? -ne 0 ]; then 
  if [ -d ${CRYPT_HOME} ]; then
    REASON="Directory already exists."
  else
    REASON=""
  fi
  echo "ERROR - Could not create directory '${CRYPT_HOME}'. ${REASON}"
  echo "Continuing..."
else
  echo
  echo "OK - '${CRYPT_HOME}' directory created."
fi

cd /root/crypto

if [ -f $CRYPT_DISK_FQFN ]; then
  echo "ERROR - Crypt disk already exists. Cannot continue."
  exit 1
fi

trap cleanup INT

dd if=/dev/zero of=cryptdisk bs=1M count=${CRYPT_DISK_SIZE}

# Capture errors
if [ $? -ne 0 ]; then 
  echo "ERROR - Could not create raw container. Cannot continue."
  cleanup called
  exit 1
else
  echo
  echo "OK - ${CRYPT_DISK_SIZE}MB raw device created."
fi

losetup ${LOOPBACK_DEVICE} ${CRYPT_DISK_FQFN}

# Capture errors
if [ $? -ne 0 ]
then
  echo "ERROR - Loopback device in use. Cannot continue."
  cleanup called
  exit 1
fi

cryptsetup luksFormat ${LOOPBACK_DEVICE}

# Capture errors
if [ $? -ne 0 ]
then
  echo "ERROR - Could not format the raw container. Cannot continue."
  cleanup called
  exit 1
fi

echo
echo "NOTE: Use the same password you set above!"
cryptsetup luksOpen ${LOOPBACK_DEVICE} ${CRYPT_LABEL}

# Capture errors
if [ $? -ne 0 ]; then
  echo "ERROR - Could not open LUKS CryptoFS. Cannot continue."
  cleanup called
  exit 1
else
  echo "OK - LUKS CryptoFS Opened."
fi

mkfs.ext4 /dev/mapper/${CRYPT_LABEL}

# Capture errors
if [ $? -ne 0 ]
then
  echo "ERROR - File system creation failed. Cannot continue."
  cleanup called
else
  echo "OK - Encrypted file system created."
  echo "Closing handles."
  cleanup nodelete
  exit 0
fi

After you save the above script to a file, and make the file executable (chmod 500 filename), you’re good to go. If you don’t want the encrypted vault file located at /root/crypto/, or want a vault of a different size than the rather small default of 64MB (I’m just saving a handful of AWS keys, so I didn’t need a larger vault file), edit the variables on top of the script before running it. Once started, follow the prompts and the encrypted vault file is created for you. If an error occurs during the vault creation process, if the vault file already exists, or if you cancel the script, any changes made up to that point are rolled back.

To mount and access the vault, save the following two scripts for mounting and unmounting the vault respectively:

#!/bin/bash

CRYPT_MOUNTPOINT=/mnt/crypto
CRYPT_DISK_FQFN=/root/crypto/cryptdisk
CRYPT_LABEL=crypt-disk
LOOPBACK_DEVICE=`losetup -f`

if [ ! -f ${CRYPT_DISK_FQFN} ]; then
  echo "Crypt disk '${CRYPT_DISK_FQFN}' missing. Cannot continue."
  exit 1
fi

if [ ! -d ${CRYPT_MOUNTPOINT} ]; then
  echo "Mountpoint '${CRYPT_MOUNTPOINT}' missing. Cannot continue."
  exit 1
fi

function check_mounted {
  if grep -qsE "^[^ ]+ $1" /proc/mounts; then
    _RET=true
  else
    _RET=false
  fi
}

check_mounted $CRYPT_MOUNTPOINT
if ${_RET} ; then
  echo "Mountpoint '${CRYPT_MOUNTPOINT}' already mounted. Cannot continue."
  exit 1
fi

losetup ${LOOPBACK_DEVICE} ${CRYPT_DISK_FQFN} > /dev/null 2>&1

# Capture errors
if [ $? -ne 0 ]; then
  echo "ERROR - Loopback device in use."
  exit 1
else
  echo "OK - Loopback device mapped."
fi

cryptsetup luksOpen ${LOOPBACK_DEVICE} ${CRYPT_LABEL} > /dev/null 2>&1

# Capture errors
if [ $? -ne 0 ]; then
  echo "ERROR Opening LUKS CryptoFS. Removing the loopback device."
  losetup -d ${LOOPBACK_DEVICE}
  exit 1
else
  echo "OK - LUKS CryptoFS Opened."
fi

mount /dev/mapper/${CRYPT_LABEL} ${CRYPT_MOUNTPOINT} > /dev/null 2>&1

# Capture errors
if [ $? -ne 0 ]; then
  echo "ERROR mounting CryptoFS."
  cryptsetup luksClose /dev/mapper/${CRYPT_LABEL}
  losetup -d ${LOOPBACK_DEVICE}
  exit 1
else
  echo "OK - Mounted CryptoFS."
  exit 0
fi
#!/bin/bash

CRYPT_MOUNTPOINT=/mnt/crypto
CRYPT_DISK=/root/crypto/cryptdisk
CRYPT_LABEL=crypt-disk

LOOPBACK_DEVICE=`losetup -j ${CRYPT_DISK} | awk '{print $1}' | sed '$s/.$//'`

CAN_RELEASE=true
if grep -qsE "^[^ ]+ ${CRYPT_MOUNTPOINT}" /proc/mounts; then
  umount ${CRYPT_MOUNTPOINT} > /dev/null 2>&1
  
  if [ $? -ne 0 ]; then
    echo "WARNING - Could not unmount ${CRYPT_MOUNTPOINT}! Device busy."
    CAN_RELEASE=false
  else
    echo "Crypto-disk was unmounted."
  fi  
else 
  echo "Crypto-disk was not mounted."
fi

if $CAN_RELEASE; then
  if [ -b /dev/mapper/${CRYPT_LABEL} ]; then
    cryptsetup luksClose /dev/mapper/${CRYPT_LABEL} > /dev/null 2>&1
  fi

  losetup -d ${LOOPBACK_DEVICE} > /dev/null 2>&1
fi

Similarly make these scripts executable before running them. If you modified the encrypted vault location/name, or the mount point location during the creation process, you’ll want to make corresponding changes the the variable atop these scripts.

You can place these utility scripts in /usr/local/bin or other location on your path (or symlink from a location on your path) to avoid having to type the full path every time.

With the encrypted vault created using setup-crypt, you can then mount the vault using mount-crypt and access the contents of the vault at /mnt/crypto, and finally unmount the vault with umount-crypt. Since the vault is protected by a single passoword, be sure to set an appropriately safe password to match the required security level.

To further improve the security, you probably want to unmount the vault whenever you’re not logged in. Most likely contents of a vault such as this are intended for interactive use. You can always unmount and hence “lock” the vault with umount-crypt command, but it is a good idea to run umount-crypt automatically at logout. Depending on your shell you can crete/edit .zslogout (zsh), .bash_logout (bash), or .logout (tcsh/csh) at the user home directory (likely in “/root” since opening/closing loopback handles can only be done by the root), and place the following code in it:

#!/bin/zsh
# NOTE: You need to adjust the path to the login shell above

/opt/crypto/umount-crypt

I also close the vault at system shutdown/reboot, by symlinking the following from /etc/rc6.d/S40umount-crypto:

#!/bin/bash
#
# umount-crypto - Unmounts a crypto-drive if mounted
# -> convenience script to be called in the shutdown/reboot sequence of Ubuntu
#    from /etc/rc6.d, e.g. as "/etc/rc6.d/S40umount-crypto"

start() {
	echo "umount-crypto: nothing to do!"
}

stop() {
	echo "Unmounting LUKS CryptoFS filesystem..."
	umount /mnt/crypto> /dev/null 2>&1 
        cryptsetup luksClose /dev/mapper/crypt-disk > /dev/null 2>&1
        losetup -d /dev/loop0 > /dev/null 2>&1
}

status() {
	echo "No status available."
}

restart() {
	echo "restart ..."
	start
}

reload() {
	echo "start ..."
	start
}

force_reload() {
	echo "force-reload ..."
	start
}

case $1 in
	start)
	start
	;;

	stop)
	stop
	;;

	status)
	status
	;;

	restart)
	restart
	;;

	reload)
	reload
	;;

	force-reload)
	force_reload
	;;

	*)
	echo "This is a non-interactive crypto-disk unmount script."
	;;

esac

exit 0

And that’s all there is to it! With your files safely inside a locked, encrypted vault, only you and the NSA have access to them! 😉

P.S.
To utilize the vault with Bitnami Cloud Tools, I have created folders for each AWS account I want to access under /mnt/crypto/, e.g. /mnt/crypto/aws_account_a, /mnt/crypto/aws_account_b, etc. Each folder contains similarly named files (as found in bitnami-awstools-x.x-x/config folder), like so:

aws-config.txt
aws-credentials.txt
ec2.crt
ec2.key

To switch from account to another I (re-)symlink the contents of the desired account from bitnami-awstools-x.x-x/config/, for example:

ln -sf /mnt/crypto/aws_account_b/* /opt/bitnami-awstools-x.x-x/config/

This way, once the vault is locked, the access to any and all of the AWS accounts via cloud tools goes away. Switching between the accounts could, of course, be scripted easily as well.

OpenVPN with FreeRADIUS: How To Use the CN from the User Cert as the Login Name (i.e. the reverse of “username-as-common-name”)

I recently set up handful of OpenVPN servers to provide access to various LAN and AWS VPC resources. Initially I had just the certificate validation configured, but I felt slightly uneasy about not having a password. Especially in the environments where multiple people need access to a resource, in the event one of them no longer should have access (such as when leaving an organization) the only way to block such user would be to add their cert into the CRL. While that should be done anyway when a user’s privilege needs to be revoked, a password would provide a more immediate and easy way to make such changes.

The next step was to install FreeRADIUS which proved to be a very easy task. I’m initially running it with just text-based back-end and will later add MySQL, perhaps with daloRADIUS GUI to make user administration even easier. On Ubuntu/Debian there is a package “openvpn-auth-radius”, which makes it possible to add FreeRADIUS authentication to OpenVPN server with one simple line:

plugin /usr/lib/openvpn/radiusplugin.so /etc/openvpn/endpoint_server_radiusplugin.conf

Of course, the client side also needs the auth-user-pass statement in their OpenVPN client configuration.

But there is a problem: The user cert can be that of Bob while the login username/password is that of Alice, and the login would still be valid. Apparently I’m not the only one who has thought about this. While I didn’t want to hack the pam auth plugin, the post had enough clues to help creating a simple bash script that sets the username based on the common-name from the validated user’s certificate:

#!/bin/bash

# $1 provides the temp file name provided by OpenVPN
# file has two lines: username and password, as entered by the user.
# We get the username from the user cert's CN (available via an envvar).

export PATH="/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin"

if [ ! -z ${common_name} ]; then
  username=`echo ${common_name}`
else
  username="-"
fi

if [ ! -z $1 ] && [ -f $1 ]; then
  password=`tail -1 $1`
else
  password="-"
fi

radius_server=localhost
# shared secret for localhost (or your RADIUS server) from /etc/freeradius/
shared_secret="XXXXXXXX"

AUTHCHECK=`cat << EOF | /usr/bin/radclient -s ${radius_server} auth ${shared_secret} | grep approved | tr -d 'n' | tail -c 1
User-Name=${username}
User-Password=${password}
EOF`

if [[ $AUTHCHECK = 1 ]]; then
  exit 0
else
  exit 1
fi

To use this script, simply save it to /etc/openvpn/endpoint_server_radius_auth.sh, make it executable, and edit the file to add the shared secret for the RADIUS server from /etc/freeradius/clients. Finally, add the following lines in your OpenVPN server configuration that already authenticates the users by their certificates:

tmp-dir /dev/shm
auth-user-pass-verify /etc/openvpn/endpoint_server_radius_auth.sh via-file

Now the login name for RADIUS authentication is taken from the CommonName (CN) of the user's certificate and, in fact, the username that the user enters when prompted for auth-user-pass username/password is ignored, only the password is significant.

The bottom line of this script: It utilizes RADIUS to provide a server-side password validation for the certificate's CN. A user can always remove the password protection from their private key, so this approach functions as an extra layer of security while making it easier to quickly revoke user's access to a resource.

Note: For this to work, the CommonName set in user certificates obviously must be a valid RADIUS login name. A user can't modify the CN in their certificate (unless they're NSA since they apparently have access to RSA-keys, too 🙁 ), so they're locked to use that specific username.

Also note that I wrote this script on Ubuntu, and did not necessarily observe portability, so you may need to modify the script some for other platforms. It is primarily intended as an example (although it does work), as finding something like this would have saved me a few hours of work.