Hardware Frustrations, part II

How did the issues I wrote in the previous post turn out? Well, Dataprobe shipped the iBootBar back promptly (two day turn-around at their end which I suppose is reasonable) with the settings reset, and with the new firmware that now has a hardware-based password recovery mechanism. The project was delayed by a few days, but such happens. I’m ok with that and will buy and recommend Dataprobe products in the future.

Meanwhile Adaptec could not (or would not) find the drive activity LED cable. I also found that the “kit” for AAR-2410SA is simply not just the card with more software and cables—it’s the controller with a drive enclosure kit at more than three times cost of the controller by itself! So if you install 2410SA to a chassis without Adaptec’s enclosure kit, Adaptec has decided you don’t need to have a visual cue as to whether the array is active. Adaptec clearly suffers from the »large business syndrome» where customers aren’t always accommodated, even if they’re willing to pay. Individual end-users are not significant to the bottom line. 🙁

I’ve also been trying to source the connector, but so far the attempts have been fruitless. A picture of the cable shipped with the kit can be found on this page. Below is a picture of the [male] connector on the controller itself. If anyone happens to know where to find a cable with such connector (Adaptec says it’s “not proprietary”.. but clearly it is not very commonly available), I’d like to know. It doesn’t matter much what’s in the other end of the cable as the right kind of connector can be easily attached with some solder.

Since the controller supports four drives and there’re four pins on the controller, I assume each pin, representative of a drive, provides +5v to a corresponding LED with ground being supplied from the PSU.

mini-connector.jpg
The problematic drive activity LED connector on Adaptec AAR-2410SA. The female connector that would match the male connector on the card would be about 5.6mm wide.

Hardware Frustrations

Instead of the usual software development work I have spent last few weeks doing some hard-core hardware stuff: integrating a new LAN server, and finalizing my employer’s data center hardware setup. In doing so, I have encountered two items that should not have been as frustrating as what they turned out to be. In both cases, I think the equipment manufacturer could have averted the resulting problem with more careful design to begin with. Or, at least, the problematic hardware designs could have been ameliorated with better documentation and/or product packaging.

Item 1: Dataprobe iBootBar
This is basically a rack-mountable iBoot (remote reboot unit) with eight outlets. The product shipped couple of months behind the schedule, which is probably why the firmware shipped with the first batch didn’t include a feature to allow hardware-reset of the device. The manual, of course, omitted this fact, and also failed to mention that serial port connection to the device (which was in the initial firmware release the only means of resetting the device to the factory defaults) would also require the admin password. You might guess what happened. I changed the admin password, logged out, logged back in, and then decided to rename the “admin” user to something less obvious. The user rename process is all cleartext, so I saw what the new user name was when I typed it. I much doubt I would’ve mistyped it, but I suppose it is possible. Whatever happened, once I renamed the admin user, locked out I was. No problem, I thought, I just connect to the serial port and reset the device. Once I figured I needed a standard serial cable (like for a modem) and not a null-modem cable (this was my bad; I’d forgotten that “DCE” as mentioned in the manual, is not the same as “DTE” 😉 ), I got to the prompt at the serial port. It said “username:”. And then it wanted a password! I was just as locked out as I had been before. Dataprobe’s tech support was glad to help — only I had to FedEx the device to them in New Jersey for resetting (plus they’ll upload the more recent firmware which will allow firmware reset). Just time and FedEx shipping fees lost.

If the manual had included a line: “Be careful not to lose the admin user name and/or password! If you lose them, the only way to regain access to the device is to send it back to us.”, I would first have created the new admin user and made sure I can log in with it before proceeding to rename or delete the original one.

Item 2: Adaptec AAR-2410SA SATA-RAID
Over the years I’ve learned not to use no-name or not-well-known name components for system integration. Just as the motherboard needs to have ASUS, Tyan, or Supermicro label on it, the hard drive controller must be AMCC/3Ware or Adaptec. This time I opted for Adaptec’s four-disk PCI-X SATA-RAID controller, 2410SA. I don’t normally order “kits” because they usually contain junk I don’t need (SATA cables which I already have, miscellaneous unnecessary software, etc.). This time, however, in order to connect the drive activity LED to the system front panel, a proprietary cable was needed… and of course it only ships in the kit! So I called Adaptec to find out that the cable is not for sale/available separately (tech support people are currently looking to see if they could locate one for me from some scrap), and the only suggestion the customer service rep had was to return the bare card and acquire the kit instead. Oh-uh! Why to make the LED connector proprietary in the first place? I’m sure there would’ve been space at the edge of the card for some PCB jumper pins. Or, if a proprietary connector was necessary to use for some design reason, why not bundle a cord with the bare card—it already comes bundled with the low profile PCI bracket (rarely needed), a useless installation manual, and a driver CD which is not necessary as the drivers are available from the website. And if it’s totally not possible to bundle the cord for some unknown reason, why not at least make it purchaseable? I would’ve paid $15.00 for it for “not having realized” to buy the kit.

One last word.. Both products are good products. Their overall design is solid and they work well. Just if the companies paid some attention to detail, or at least patched the deficiencies in the documentation, or in customer service (there shouldn’t be a spare part that is “not available” for a current product), there would not be a reason to complain as I’m sure it’s very difficult to design and manufacture a completely flawless product.

Scripting tools for Windows

Many years ago, before the time of Windows, I had a large set of DOS utilities to accomplish various file management tasks. After Windows 95 showed up, use of many of those tools gradually faded as they were one by one replaced various GUI tools. In recent times, however, I have again accumulated a set of DOS-based tools, this time mostly used for scripting. Below are some of my favorite current-day Windows command line tools:

  • JPSoft 4NT – a complete replacement for Windows standard command interpreter, cmd.exe. 4NT introduces countless additional commands and features that make scripting much more pleasant. 4NT is to cmd.com sort of like bash is to sh. JP Software’s other product, more “modern” graphical shell TakeCommand isn’t as attractive because it doesn’t run native DOS applications but opens a separate DOS box for them instead—exactly what one wants to avoid when opening a DOS box for command entry.
  • XXCOPY – a massively enhanced copy command. I use it to copy directory structures (massive directory structures copy much more reliably using xxcopy than DOS copy), and for some more specialized tasks which would be otherwise hard to accomplish, such as creating intersection between the contents of two directories.
  • Tar for Windows and Gzip for Windows – UNIX-like tar and gzip for DOS. Great for creating archives on Windows system, to be exported to a UNIX system for easy processing. GnuWin32-project has also several other UNIX-commands ported for Windows.
  • Renamer – allows mass-rename of files matching a regex pattern, to a new regex pattern
  • For scriptable communications from Windows command line (besides the built in telnet and ftp), PuTTY provides a command line SCP client which you can find from their download page under the name “PSCP”. Also WinSCP can be used from the command line. VanDyke Software’s high quality SecureFX (SFTP) utility also offers a command line application that can be scripted.
  • I should also mention Cygwin which is “a Linux-like environment for Windows”. Personally, I have steered away from it as while it implements many common UNIX/Linux commands on Windows, it makes Windows scripting more difficult — if you dislike Windows so much that you try to make it function like Linux or UNIX, then use Linux or FreeBSD instead :).

Microsoft has also realized that scripting for automation is increasingly important. Microsoft’s extensible PowerShell is available for Windows XP, Windows Server 2003, and Windows Vista, and will be available for the upcoming Windows Server 2008 (Longhorn). For more info on PowerShell see Windows PowerShell @ Wikipedia, Scripting with Windows PowerShell, and Windows PowerShell Blog.  Do note that PowerShell integrates itself rather tightly to the system and this has repercussions: At the time of this blog entry there is no PowerShell version for Windows 2003 SP2, and if you install PowerShell to Win 2003 SP1 and then upgrade the system to SP2, you can’t uninstall PowerShell!  I’m not sure if I want to base my system scripts on technology that is not very transportable and that has dependencies that aren’t kept up to date with the operating system and the service packs. There’s more discussion about this issue on the PowerShell blog.

System Actions with Web Triggers (UNIX)

Recently when the data center of a server I administer needed to change the IP of the box I would’ve needed to make the corresponding DNS change on our DNS server. I knew wasn’t going to be available at the time when the routing change was to occur. Since I didn’t want there to be a service outage I set up a system event that could be triggered from a password-protected web page (on the same server). Here’s a quick outline of the procedure:

  1. I created a shell script file /usr/local/sbin/webaction (this was created for a FreeBSD machine, though with minor adaptations it should work on any UNIX/Linux system), made sure it’s owned by “root:wheel” and set the permissions to 700 (user read/write/execute; could be also 500 for read/execute only, but that makes little difference as root will be able to write either way).  In the case of this script, it copies the new DNS zone files in place of the old ones and restarts the DNS server. If executed multiple times, it won’t do any damage or mess things up (this is important since it’s triggered by an external user who doesn’t know what the internal workings of the procedure are).
  2. Using “visudo” command I added sudo execute-without-password privileges to that file for the user that Apache httpd runs under:

    Cmnd_Alias WEBCMNDS = /usr/local/sbin/webaction
    httpduser ALL= NOPASSWD: WEBCMNDS

  3. I created a PHP command file in a web-accessible directory protected with .htpasswd (I added the password directly into Apache configuration). The PHP file simply executes the command with sudo as it is allowed to do, and displays a message to the user:

    <html>
    <body>
    <?php
    $command = “/usr/local/bin/sudo /usr/local/sbin/webaction”;
    exec($command);
    ?>
    DNS update complete
    </body>
    </html>

Done! Now when the ISP is ready to make the routing change, they simply point their browser to the URL I provided to them, enter the username/password, and the PHP file executes the shell script through sudo. Since the shell script is created so that it won’t make a mess of things if executed multiple times, and since no parameters can be passed from the web-interface to the shell-script, things are fairly secure. Obviously this kind of setup is not appropriate in every setting, but for systems with limited CLI user base it is a handy trick.