Sunday, August 29, 2010

YUI "website top nav" Menu from JavaScript only

I recently had the opportunity to observe someone running into some difficulties trying to implement a YUI 2 Menu. The menu items were to be retrieved from a database, and needed to be dynamically updatable through an AJAX call - so using JSON rather than HTML to build the menu made sense in this case. According to YUI's menu page, "Menus can be created from simple, semantic markup on the page or purely through JavaScript".

In this case, getting the menu built "purely through JavaScript" was looking to be a little bit of a challenge. The "Website Top Nav With Submenus" was the type of menu desired. YUI's available menu examples included this type of menu built both "From Markup" and "From JavaScript". However, the "From JavaScript" version still built the top-level menu with HTML rather than JavaScript. While certainly possible, dynamically generating both HTML and JavaScript from the server-side certainly didn't seem ideal.

Recognizing that YUI is a very robust JavaScript library, I knew that there had to be a way to build the menu with only JavaScript. After starting with a new test script, I quickly found the 2 main parts to the solution:

  1. The MenuBar constructor accepts an "itemData" property as part of its 2nd "config" argument. The menu structure can be configured here for the top level, in a fashion identical to the "aSubmenuData" array of objects visible in the original YUI example.
  2. All menu items - including the top-level menus, submenus, and the menu bar itself - must have a unique ID for the HTML DOM, otherwise the item will not activate. For example, if a top-level menu is missing an ID, it will display, but not activate to display the submenus.

    Ideally, it'd be nice to have the "id" property optional. However, the "YAHOO.util.Dom.generateId()" function works well for this - and at least for the purposes of this example, I aliased it as simply "id".

The complete updated example is shown below. For lack of better data, the same example data from the original YUI example is used:

JavaScript source code:

  YAHOO.util.Event.onDOMReady(function(){
    
    var id = YAHOO.util.Dom.generateId;
    
    var menuData = [
      {
        text: "Communication",
        submenu: {id: id(), itemdata: [ 
          {text: "360", url: "http://360.yahoo.com"},
          {text: "Alerts", url: "http://alerts.yahoo.com"},
          {text: "Avatars", url: "http://avatars.yahoo.com"},
          {text: "Groups", url: "http://groups.yahoo.com"},
          {text: "Internet Access", url: "http://promo.yahoo.com/broadband"},
          {
            text: "PIM", 
            submenu: { 
              id: id(), 
              itemdata: [
                {text: "Yahoo! Mail", url: "http://mail.yahoo.com"},
                {text: "Yahoo! Address Book", url: "http://addressbook.yahoo.com"},
                {text: "Yahoo! Calendar",  url: "http://calendar.yahoo.com"},
                {text: "Yahoo! Notepad", url: "http://notepad.yahoo.com"}
              ] 
            }
          }, 
          {text: "Member Directory", url: "http://members.yahoo.com"},
          {text: "Messenger", url: "http://messenger.yahoo.com"},
          {text: "Mobile", url: "http://mobile.yahoo.com"},
          {text: "Flickr Photo Sharing", url: "http://www.flickr.com"},
        ]}
      },
      {
        text: "Shopping",
        submenu: {id: id(), itemdata: [
          {text: "Auctions", url: "http://auctions.shopping.yahoo.com"},
          {text: "Autos", url: "http://autos.yahoo.com"},
          {text: "Classifieds", url: "http://classifieds.yahoo.com"},
          {text: "Flowers & Gifts", url: "http://shopping.yahoo.com/b:Flowers%20%26%20Gifts:20146735"},
          {text: "Real Estate", url: "http://realestate.yahoo.com"},
          {text: "Travel", url: "http://travel.yahoo.com"},
          {text: "Wallet", url: "http://wallet.yahoo.com"},
          {text: "Yellow Pages", url: "http://yp.yahoo.com"}
        ]}
      },
      {
        text: "Entertainment",
        submenu: {id: id(), itemdata: [
          {text: "Fantasy Sports", url: "http://fantasysports.yahoo.com"},
          {text: "Games", url: "http://games.yahoo.com"},
          {text: "Kids", url: "http://www.yahooligans.com"},
          {text: "Music", url: "http://music.yahoo.com"},
          {text: "Movies", url: "http://movies.yahoo.com"},
          {text: "Radio", url: "http://music.yahoo.com/launchcast"},
          {text: "Travel", url: "http://travel.yahoo.com"},
          {text: "TV", url: "http://tv.yahoo.com"}
        ]}
      },
      {
        text: "Information",
        submenu: {id: id(), itemdata: [
          {text: "Downloads", url: "http://downloads.yahoo.com"},
          {text: "Finance", url: "http://finance.yahoo.com"},
          {text: "Health", url: "http://health.yahoo.com"},
          {text: "Local", url: "http://local.yahoo.com"},
          {text: "Maps & Directions", url: "http://maps.yahoo.com"},
          {text: "My Yahoo!", url: "http://my.yahoo.com"},
          {text: "News", url: "http://news.yahoo.com"},
          {text: "Search", url: "http://search.yahoo.com"},
          {text: "Small Business", url: "http://smallbusiness.yahoo.com"},
          {text: "Weather", url: "http://weather.yahoo.com"}
        ]}
      }
    ];
    
    new YAHOO.util.YUILoader({require: ["menu"], onSuccess: function(){
      var oMenuBar = new YAHOO.widget.MenuBar(id(), {
        autosubmenudisplay: true, 
        hidedelay: 750, 
        itemdata: menuData});
      
      oMenuBar.render("com.ziesemer.demos.yuiMenuJsOnly.global_menu_parent");
    }}).insert();
  });

HTML source code:

  <div class="yui-skin-sam">
    <div id="com.ziesemer.demos.yuiMenuJsOnly.global_menu_parent" class="yuimenubarnav"></div>
  </div>

Output:

Sunday, July 11, 2010

Updated Blogger Tools

I've updated the "Blog Archive" and "Labels" gadgets on this site - as visible in the right-hand margin of this page. Please leave a comment with any issues or suggestions.

Over a year ago, I had already replaced Blogger's default Labels gadget with the Yahoo! UI Library (YUI)'s TreeView component, which provided:

  • A view that is collapsed by default, saving screen space for other features unless the Labels are clicked and expanded for use.
  • A complete and compact list of tagged posts under each label, without having to request a new label search page - while still providing these links.

This tree is populated with an AJAX request to Google's GData Blogger API, using the JSON Alt Type and YUI's Get Utility. Since the returned data is a list of posts, a limited amount of JavaScript is needed to group and sort the posts into the labels. As part of this last update, a label will now again automatically expand to show the related posts if the current page being viewed is a label search page. (This is similar to the default functionality, but is determined by the current URL.)

Also now just completed, I've also replaced Blogger's default Blog Archive gadget with another YUI TreeView, providing:

  • A consistent look-and-feel between the Blog Archive and Labels features.
  • A more compact display, while maintaining readability.
  • Post titles were previously truncated after 50 characters. Complete post titles are now displayed, along with the specific date posted.
  • As the same data retrieved to populate the Labels is re-used, the page loading performance should actually be improved as the HTML is now dynamically generated client-side - instead of downloading what was essentially duplicate information as additional HTML from the server.

As with the Labels, the archive will also automatically expand to show the related posts for the time period being viewed - similar to the default functionality, but again determined by the current URL.

Also just added are "loading bar" graphics for the short period of time between the initial page load and the population of the trees.

A few other quick things to note:

  • The source JavaScript code used for these features is available for reference in this page's HTML source (among the rest of Blogger's default and somewhat cluttered code), using your browser's View/Source feature. Even though it is only ~250 lines or ~7 KB of code, this is not the most efficient as this code is reloaded with every page load. I'd like to be able to properly move this code into a separate and cacheable *.js file - but can't currently find a usable solution that matches the reliability and pricing (free) of the rest of Blogger. I attempted using versions stored on Google Code and Google Sites, but each had various issues - including being able to properly make updates, as well as having download landing pages interfere - partially complicated by not being hosted on the blogspot.com domain. Amazon's S3 storage service looks like a promising option that I may still consider. However, while it would likely only cost pennies / month to use for this, it's another bill to take care of.

  • The JSON data feed received is a bit large - currently ~200 KB - but unlike the JavaScript code, this data is returned with proper HTTP headers by Google, and should be cacheable by the browser. Much of this data is information that is not necessary or used, such as post summaries and comments. Hopefully, these extra fields will soon be selectively disabled - as soon as the Blogger GData API supports partial responses.

  • As visible in the source code, I also needed to implement a custom "clickEvent" handler for YUI's TreeView - otherwise the HTML links contained within the tree nodes were not clickable, as any clicks would instead expand/collapse the node instead of activating the link.

A few ideas for future enhancements:

  • Options to alternatively sort the labels, including by post count.
  • Having the displayed labels under each post interact with the Labels tree.

Monday, July 5, 2010

JavaScript conditional oversights and solutions

While this is nothing new or particularly noteworthy, I felt this post would be good to share after fixing a number of conditionals in various JavaScript code - as well as having received some interesting and surprising feedback after suggesting the same fixes to others.

The code in question looked something like this:

var x;
// …
if(x == ""){
  // …
}

While the exact validity of this code depends on the exact requirements being implemented, there are a few potential issues in what is otherwise a very simple block of code:

  • null is not checked for, which is often desirable as - while accepted as a best programming practice or not - null and an empty string ("") are usually treated (or expected to be treated) the same.
  • Looking from the other side of things, the if block will also be entered if x is any of the following (not guaranteed to be a complete list): 0 (a number), "0" (a string representation of the number), or false (a boolean).

Stated another way, when using the standard equals operator (==) in JavaScript, all of ["", 0, "0", false] are considered equal. The same applies with a separate set of [null, (undefined)], though no elements between sets are considered equal.

Verify the results for yourself, by running this code in your browser:

log(false == false);
log("" == false);
log(0 == false);
log("0" == false);

var x = {};
// x.y is undefined:
log(x.y == null);
// Another way is to obtain an undefined result is to use the void operator:
log(void(0) == null);

// Show that items between these 2 groups are not considered equal:
log(null != false);

My recommendation to all JavaScript developers is to instead use one of the following approaches:

  • To really test against an empty string (""), and only an empty string, use the strict equals operator (===). For example:
    var x;
    // …
    if(x === ""){
      // …
    }
    The same also applies for testing against 0, "0", or false.
  • From the other side of things, don't use an equals operator at all. Just use the variable in a conditional without an explicit operator or comparison, which is equivalent to wrapping with the Boolean(…) function. To test for a defined, non-null, and non-"false" value, the following 4 statements are all functionally identical, with the first and highlighted version being the preferred version:
    var x;
    // …
    if(x){
      // …
    }
    if(Boolean(x)){ /*…*/ }
    if(x != null && x == true){ /*…*/ }
    if(x !== null && typeof(x) != "undefined" && x !== "" && x !== 0 && x !== "0" && x !== false){ /*…*/ }
    Or for the reverse, by using the negated form:
    if(!x){ /*…*/ }
    if(!Boolean(x)){ /*…*/ }
    if(x == null || x == false || isNaN(x)){ /*…*/ }
    if(x === null || typeof(x) == "undefined" || x === "" || x === 0 || x === "0" || x === false || isNaN(x)){ /*…*/ }
    Assuming one of these code blocks meet the functional requirement, the first line forms are the preferred usage.

Monday, June 7, 2010

Eclipse Web Tools Spell Check Broken

Just a quick warning to anyone looking to upgrade to the upcoming release of Eclipse 3.6 / "Helios" and uses the HTML Editor or Web Page Editor. As of all the release candidates, including RC4 of Classic with the WTP editors installed and RC3 of the JEE / "Eclipse IDE for Java EE Developers" package, spell check is broken. Quite simply, HTML content is no longer being checked for misspellings. The spell checking is working as expected in 3.5.2.

I reported this in bug 316025. I'm not sure if it can or will be fixed before the official release, but it is a fairly significant issue in my opinion. If this issue will affect you, please sign-in to Eclipse's Bugzilla and vote, or otherwise contribute.

Sunday, June 6, 2010

OpenLDAP Password Permissions Configuration Example

Following my post on OpenLDAP under Ubuntu Linux Lucid Lynx 10.04, I'd like to address a question submitted as a comment submitted by "raerek":

I plan to set up things like this:
-Group A: members can change the password of anyone, except the password of other Group A members.
-Group B: members can change their own passwords, and the passwords of Group C members.
-Group C: members can only change their own password.

Can you outline how to write the "olcAccess: to" lines - If I do not have to, I would not like to set up three OU-s - I'd rather have everyone in the ou=people.

While I cannot guarantee a response to every question, this one caught my interest - especially due to some of the well-advised reasons I'm guessing are behind the stated requirements.

The "Group A" requirement in particular looked rather complicated when I first looked at this, as the "... anyone, except ..." would normally be addressed with compound conditionals - and OpenLDAP doesn't provide for this - at least not as many would expect. As with many technical issues, the best approach here seems to be in simplifying the functional requirements:

  1. Anyone can change their own password.
    • Is specified by both the original "Group A" and "Group B" requirements, and completely satisfies the original "Group C" requirement.
    • "Anyone" should not necessarily apply to "service" or "machine" accounts, unless such an account is specifically known or required to change its own password.
  2. "Group A" members cannot change the passwords of other "Group A" members.
  3. "Group A" members can change the passwords of everyone else (besides other "Group A" members, as listed above in #2).
  4. "Group B" members can change the passwords of "Group C" members.
  5. Unless otherwise specified above, deny any other access to passwords, except for accounts to read their own password (probably not always even necessary) and for anonymous (an account before authenticated) to authenticate against.

Here is my entire proposed solution, with notes to follow. First, an LDIF export of a setup to demonstrate the example:

dn: dc=example,dc=com
dc: Example
description: LDAP Example 
o: Example Organization
objectclass: top
objectclass: dcObject
objectclass: organization

dn: ou=groups,dc=example,dc=com
objectclass: organizationalUnit
ou: groups

dn: cn=groupA,ou=groups,dc=example,dc=com
cn: groupA
description: members can change the password of anyone, except the password 
 of other Group A members.
member: cn=groupOfNamesPlaceHolder,ou=groups,dc=example,dc=com
member: cn=groupA_user1,ou=people,dc=example,dc=com
member: cn=groupA_user2,ou=people,dc=example,dc=com
objectclass: groupOfNames

dn: cn=groupB,ou=groups,dc=example,dc=com
cn: groupB
description: members can change their own passwords, and the passwords of Gr
 oup C members.
member: cn=groupOfNamesPlaceHolder,ou=groups,dc=example,dc=com
member: cn=groupB_user1,ou=people,dc=example,dc=com
member: cn=groupB_user2,ou=people,dc=example,dc=com
objectclass: groupOfNames

dn: cn=groupC,ou=groups,dc=example,dc=com
cn: groupC
description: members can only change their own password.
member: cn=groupOfNamesPlaceHolder,ou=groups,dc=example,dc=com
member: cn=groupC_user1,ou=people,dc=example,dc=com
member: cn=groupC_user2,ou=people,dc=example,dc=com
objectclass: groupOfNames

dn: cn=groupOfNamesPlaceHolder,ou=groups,dc=example,dc=com
cn: groupOfNamesPlaceHolder
objectclass: applicationProcess

dn: ou=people,dc=example,dc=com
objectclass: organizationalUnit
ou: people

dn: cn=groupA_user1,ou=people,dc=example,dc=com
cn: groupA_user1
objectclass: inetOrgPerson
objectclass: simpleSecurityObject
sn: Test
userpassword: secret

dn: cn=groupA_user2,ou=people,dc=example,dc=com
cn: groupA_user2
objectclass: inetOrgPerson
objectclass: simpleSecurityObject
sn: Test
userpassword: secret

dn: cn=groupB_user1,ou=people,dc=example,dc=com
cn: groupB_user1
objectclass: inetOrgPerson
objectclass: simpleSecurityObject
sn: Test
userpassword: secret

dn: cn=groupB_user2,ou=people,dc=example,dc=com
cn: groupB_user2
objectclass: inetOrgPerson
objectclass: simpleSecurityObject
sn: Test
userpassword: secret

dn: cn=groupC_user1,ou=people,dc=example,dc=com
cn: groupC_user1
objectclass: inetOrgPerson
objectclass: simpleSecurityObject
sn: Test
userpassword: secret

dn: cn=groupC_user2,ou=people,dc=example,dc=com
cn: groupC_user2
objectclass: inetOrgPerson
objectclass: simpleSecurityObject
sn: Test
userpassword: secret

Note that the userpassword's are in clear text for the purposes of this example. They should normally be stored using a cryptographic hash such as salted SHA (SSHA) for proper security.

Next are the olcAccess rules:

# Req. #1 - Any user in the people OU can change their own password.
olcaccess: to dn.subtree="ou=people,dc=example,dc=com"
  attrs=userPassword
  by self write
  by * break
# Req. #2 - "Group A" members cannot change the passwords of other "Group A" members.
olcaccess: to filter=(memberof=cn=groupA,ou=groups,dc=example,dc=com)
  attrs=userPassword
  by group.exact="cn=groupA,ou=groups,dc=example,dc=com" none
  by * break
# Req. #3 - "Group A" members can change the passwords of everyone else.
olcaccess: to attrs=userPassword
  by group.exact="cn=groupA,ou=groups,dc=example,dc=com" write
  by * break
# Req. #4 - "Group B" members can change the passwords of "Group C" members.
olcaccess: to filter=(memberof=cn=groupC,ou=groups,dc=example,dc=com)
  attrs=userPassword
  by group.exact="cn=groupB,ou=groups,dc=example,dc=com" write
  by * break
# Req. #5 - Unless otherwise specified above, deny any other access to passwords,
#   except for accounts to read their own password (probably not always even necessary)
#   and for anonymous (an account before authenticated) to authenticate against.
olcaccess: to attrs=userPassword
  by self read
  by anonymous auth
  by * none

First, note that this requires use of the "memberof" overlay, as detailed in section "12.8. Reverse Group Membership Maintenance" in the OpenLDAP Administrator's Guide. Without it, the "memberof" attributes will not exist, and the required filters will not work. Also note that as provided by the current version of the overlay, the "memberof" attributes are stored rather than simply calculated when requested (probably for performance reasons), and are only updated when the entry is involved in a modification of group membership. As such, if the overlay is added to a database with existing populated groups, the groups will likely have to be re-populated for the "memberof" attribute to reflect the current and expected results. In testing, any update to the "member" attribute on a group - even only the addition or removal of one member - will cause the overlay to update the "memberof" attribute on all other members in the group. Also related to performance, make sure that "memberof" is indexed after enabling the overlay.

Next, note that most rules are terminated by a "by * break" clause. This allows following rules to be considered, without first terminating with the "by * none" clause that is implicitly added at the end of every rule, as detailed in section "8.2.4. Access Control Evaluation" in the OpenLDAP Administrator's Guide.

No work is complete without a good test script. This is also a good reference for how to change account passwords using ldapmodify. Here is the test source as a bash shell script:

#!/bin/bash

# Mark A. Ziesemer, http://www.ziesemer.com
# 2010-06-06

_cpw(){
  # "Change PassWord"
  local bindUser=$1
  local bindPW=$2
  local changeUser=$3
  local newPW=$4
  declare -i expected=$5 # optional, defaults to 0

  declare -i result=0
  echo "% _cpw $*"
  echo "
    dn: $changeUser
    changetype: modify
    replace: userpassword
    userpassword: $newPW" | ldapmodify -D "$bindUser" -xw "$bindPW" || result=$?
  if [ $result -ne $expected ]; then
    echo -e "\t* Test FAILED, exit code: $result"
    if [ $result -gt 0 ]; then return $result; fi
    return 1
  fi
  echo -e "\t* Test passed, exit code: $result"
  return 0
}

_testTitle(){
  echo -e "\n*** $1\n"
}

_defSuffix="dc=example,dc=com"
_peopleOU="ou=people,${_defSuffix}"
_groupA_user1="cn=groupA_user1,${_peopleOU}"
_groupA_user2="cn=groupA_user2,${_peopleOU}"
_groupB_user1="cn=groupB_user1,${_peopleOU}"
_groupB_user2="cn=groupB_user2,${_peopleOU}"
_groupC_user1="cn=groupC_user1,${_peopleOU}"
_groupC_user2="cn=groupC_user2,${_peopleOU}"

_testTitle "Testing requirement #1: Anyone can change their own password."
_cpw "${_groupA_user1}" "secret" "${_groupA_user1}" "secret1"
_cpw "${_groupA_user1}" "secret1" "${_groupA_user1}" "secret"
_cpw "${_groupB_user1}" "secret" "${_groupB_user1}" "secret1"
_cpw "${_groupB_user1}" "secret1" "${_groupB_user1}" "secret"
_cpw "${_groupC_user1}" "secret" "${_groupC_user1}" "secret1"
_cpw "${_groupC_user1}" "secret1" "${_groupC_user1}" "secret"

_testTitle "Testing requirement #2: \"Group A\" members cannot change the passwords of other \"Group A\" members."
_cpw "${_groupA_user1}" "secret" "${_groupA_user2}" "secret1" 50

_testTitle "Testing requirement #3: \"Group A\" members can change the passwords of everyone else."
_cpw "${_groupA_user1}" "secret" "${_groupB_user1}" "secret1"
_cpw "${_groupA_user1}" "secret" "${_groupB_user1}" "secret"
_cpw "${_groupA_user1}" "secret" "${_groupC_user1}" "secret1"
_cpw "${_groupA_user1}" "secret" "${_groupC_user1}" "secret"

_testTitle "Testing requirement #4: \"Group B\" members can change the passwords of \"Group C\" members."
_cpw "${_groupB_user1}" "secret" "${_groupC_user1}" "secret1"
_cpw "${_groupB_user1}" "secret" "${_groupC_user1}" "secret"
_cpw "${_groupB_user1}" "secret" "${_groupB_user2}" "secret1" 50

_testTitle "Testing requirement #5: Deny other access."
_cpw "${_groupC_user1}" "secret" "${_groupA_user1}" "secret1" 50
_cpw "${_groupC_user1}" "secret" "${_groupB_user1}" "secret1" 50
_cpw "${_groupC_user1}" "secret" "${_groupC_user2}" "secret1" 50

With the associated output from a successful run:

*** Testing requirement #1: Anyone can change their own password.

% _cpw cn=groupA_user1,ou=people,dc=example,dc=com secret cn=groupA_user1,ou=people,dc=example,dc=com secret1
modifying entry "cn=groupA_user1,ou=people,dc=example,dc=com"

        * Test passed, exit code: 0
% _cpw cn=groupA_user1,ou=people,dc=example,dc=com secret1 cn=groupA_user1,ou=people,dc=example,dc=com secret
modifying entry "cn=groupA_user1,ou=people,dc=example,dc=com"

        * Test passed, exit code: 0
% _cpw cn=groupB_user1,ou=people,dc=example,dc=com secret cn=groupB_user1,ou=people,dc=example,dc=com secret1
modifying entry "cn=groupB_user1,ou=people,dc=example,dc=com"

        * Test passed, exit code: 0
% _cpw cn=groupB_user1,ou=people,dc=example,dc=com secret1 cn=groupB_user1,ou=people,dc=example,dc=com secret
modifying entry "cn=groupB_user1,ou=people,dc=example,dc=com"

        * Test passed, exit code: 0
% _cpw cn=groupC_user1,ou=people,dc=example,dc=com secret cn=groupC_user1,ou=people,dc=example,dc=com secret1
modifying entry "cn=groupC_user1,ou=people,dc=example,dc=com"

        * Test passed, exit code: 0
% _cpw cn=groupC_user1,ou=people,dc=example,dc=com secret1 cn=groupC_user1,ou=people,dc=example,dc=com secret
modifying entry "cn=groupC_user1,ou=people,dc=example,dc=com"

        * Test passed, exit code: 0

*** Testing requirement #2: "Group A" members cannot change the passwords of other "Group A" members.

% _cpw cn=groupA_user1,ou=people,dc=example,dc=com secret cn=groupA_user2,ou=people,dc=example,dc=com secret1 50
modifying entry "cn=groupA_user2,ou=people,dc=example,dc=com"
ldap_modify: Insufficient access (50)

        * Test passed, exit code: 50

*** Testing requirement #3: "Group A" members can change the passwords of everyone else.

% _cpw cn=groupA_user1,ou=people,dc=example,dc=com secret cn=groupB_user1,ou=people,dc=example,dc=com secret1
modifying entry "cn=groupB_user1,ou=people,dc=example,dc=com"

        * Test passed, exit code: 0
% _cpw cn=groupA_user1,ou=people,dc=example,dc=com secret cn=groupB_user1,ou=people,dc=example,dc=com secret
modifying entry "cn=groupB_user1,ou=people,dc=example,dc=com"

        * Test passed, exit code: 0
% _cpw cn=groupA_user1,ou=people,dc=example,dc=com secret cn=groupC_user1,ou=people,dc=example,dc=com secret1
modifying entry "cn=groupC_user1,ou=people,dc=example,dc=com"

        * Test passed, exit code: 0
% _cpw cn=groupA_user1,ou=people,dc=example,dc=com secret cn=groupC_user1,ou=people,dc=example,dc=com secret
modifying entry "cn=groupC_user1,ou=people,dc=example,dc=com"

        * Test passed, exit code: 0

*** Testing requirement #4: "Group B" members can change the passwords of "Group C" members.

% _cpw cn=groupB_user1,ou=people,dc=example,dc=com secret cn=groupC_user1,ou=people,dc=example,dc=com secret1
modifying entry "cn=groupC_user1,ou=people,dc=example,dc=com"

        * Test passed, exit code: 0
% _cpw cn=groupB_user1,ou=people,dc=example,dc=com secret cn=groupC_user1,ou=people,dc=example,dc=com secret
modifying entry "cn=groupC_user1,ou=people,dc=example,dc=com"

        * Test passed, exit code: 0
% _cpw cn=groupB_user1,ou=people,dc=example,dc=com secret cn=groupB_user2,ou=people,dc=example,dc=com secret1 50
modifying entry "cn=groupB_user2,ou=people,dc=example,dc=com"
ldap_modify: Insufficient access (50)

        * Test passed, exit code: 50

*** Testing requirement #5: Deny other access.

% _cpw cn=groupC_user1,ou=people,dc=example,dc=com secret cn=groupA_user1,ou=people,dc=example,dc=com secret1 50
modifying entry "cn=groupA_user1,ou=people,dc=example,dc=com"
ldap_modify: Insufficient access (50)

        * Test passed, exit code: 50
% _cpw cn=groupC_user1,ou=people,dc=example,dc=com secret cn=groupB_user1,ou=people,dc=example,dc=com secret1 50
modifying entry "cn=groupB_user1,ou=people,dc=example,dc=com"
ldap_modify: Insufficient access (50)

        * Test passed, exit code: 50
% _cpw cn=groupC_user1,ou=people,dc=example,dc=com secret cn=groupC_user2,ou=people,dc=example,dc=com secret1 50
modifying entry "cn=groupC_user2,ou=people,dc=example,dc=com"
ldap_modify: Insufficient access (50)

        * Test passed, exit code: 50

Additional references:

Sunday, May 9, 2010

OpenLDAP under Ubuntu Linux Lucid Lynx 10.04

Introduction

Somewhat as a follow-up to my Ubuntu Linux router upgrade project over a year and a half ago, I've worked on extending my home setup to include a complete LDAP solution.

My router upgrade project was completed under the latest version of Ubuntu Linux at the time, 8.04 ("Hardy Heron"), which also happened to be a Long Term Support (LTS) release. That guide has held up very well over the last 2 years and the following 4 Ubuntu releases, and is still my working reference as I choose to rebuild with each new release every 6 months. This includes the latest release, Ubuntu Linux 10.04, "Lucid Lynx". Lucid also happens to be the next LTS release, and is the version I am basing this guide on.

This guide will include setup of an LDAP server, as well as setting up client authentication and administration tools.

LDAP stands for "Lighweight Directory Access Protocol". In the simplest setup, it is a database of usernames, passwords, and other information, and is commonly used for shared authentication and authorization. It can be extended with any set of additional fields, allowing for extra uses.

For my purposes within a home network, LDAP is probably a bit of an overkill. However, with an increasing number of computers and virtual machines, it will make user maintenance easier once complete. I even plan on using it with RADIUS and WPA 2 Enterprise for keeping better control around wireless access. Additionally, this will be a good learning experience, as well as hopefully a good working reference for others needing a similar setup.

OpenLDAP

OpenLDAP (Wikipedia) is a free and open source LDAP server. It is highly configurable and very performant. As detailed at http://www.openldap.org/pub/hyc/LDAPcon2007.pdf:

OpenLDAP is the premier implementation of LDAP client and server software, providing full support of LDAPv3 and most popular standard and draft (work in progress) LDAP extensions. It has evolved over the years from its origins in the University of Michigan's reference implementation of LDAPv2 as a vehicle for experimentation into a mature, commercial grade package capable of supporting the most demanding environments. The current release has been proven to scale to hundreds of millions of objects in data volumes in excess of a terabyte, with performance in excess of 22,000 queries per second at sub-millisecond latencies. Reliability in production deployments has been flawless, with hardware failure being the principal cause of unscheduled downtime.

Also refer to "An OpenLDAP Update" (Marty Heyman, 2007-09-13, onlamp.com) for additional details. This was as of 2007, and these results should easily be multiplied through replicated nodes. (See "What is a directory service?" and "Replication" in the Administrator's Guide, and the "Directories vs. Relational Database Management Systems" FAQ entry.) OpenLDAP is also the only LDAP server provided in Ubuntu that is also maintained / supported by the Ubuntu developers.

The only alternatives I even considered were:

  • The 389 Directory Server (Wikipedia). Part of Red Hat's Fedora project, and identical to the Red Hat Directory Server. Like OpenLDAP, the 389 Directory Server is also based off of the University of Michigan project, and later, Netscape Directory Server. Unfortunately, the 389 Directory Server looks rather exclusive to Fedora and Red Hat Enterprise Linux (RHEL), with no binary packages readily available for other systems. There is some outdated documentation for Ubuntu, that only applied to older versions of both the directory server (1.1 vs. 1.2) and Ubuntu (9.10 vs. 10.04). Even following the notes, installation under a 9.10 VM no longer works as documented. Compiling from source would probably be the most successful attempt for current versions, but just the number of dependencies involved doesn't make this a very appealing option.
  • Apache Directory Server (Wikipedia). I had no specific issues against, but also no current compelling reasons to consider Apache Directory Server over OpenLDAP.

OpenLDAP installation under Ubuntu

Unfortunately, as of Ubuntu Linux 9.10 (Karmic Koala), there is no longer any automated, prompted configuration when installing the Ubuntu-packaged OpenLDAP server, slapd. The issues are visible in Ubuntu Bug # 463684, https://help.ubuntu.com/community/OpenLDAPServer, http://ubuntuforums.org/showthread.php?t=1313472, and http://ubuntuforums.org/showthread.php?t=1295934. Fortunately, most of these issues appear to have been corrected in the latest server guide for 10.04 (Lucid Lynx) at http://doc.ubuntu.com/ubuntu/serverguide/C/openldap-server.html.

My installation is based off of the latest server guide (the last link above), with a few noted changes (highlighted) for the backend*.ldif file:

# Load dynamic backend modules
dn: cn=module,cn=config
objectClass: olcModuleList
cn: module
olcModulepath: /usr/lib/ldap
olcModuleLoad: back_hdb
olcModuleLoad: back_monitor

# Database settings
dn: olcDatabase=hdb,cn=config
objectClass: olcDatabaseConfig
objectClass: olcHdbConfig
olcDatabase: {1}hdb
olcSuffix: dc=example,dc=com
olcDbDirectory: /var/lib/ldap
olcRootDN: cn=admin,dc=example,dc=com
olcRootPW: secret
olcDbConfig: set_cachesize 0 2097152 0
olcDbConfig: set_lk_max_objects 1500
olcDbConfig: set_lk_max_locks 1500
olcDbConfig: set_lk_max_lockers 1500
olcDbIndex: objectClass eq
olcDbIndex: uid eq
olcDbIndex: cn eq
olcDbIndex: uidNumber eq
olcDbIndex: gidNumber eq
olcDbIndex: memberUid eq
olcDbIndex: uniqueMember eq
olcLastMod: TRUE
olcMonitoring: TRUE
olcDbCheckpoint: 512 30
olcAccess: to dn.subtree="ou=people,dc=example,dc=com" attrs=userPassword,shadowLastChange by dn="cn=admin,dc=example,dc=com" write by anonymous auth by self write by * none
olcAccess: to attrs=userPassword,shadowLastChange by dn="cn=admin,dc=example,dc=com" write by anonymous auth by * none
# Below line should already exist by default in frontend.
#olcAccess: to dn.base="" by * read
# Below line modified from "*" to "users" to prevent anonymous access.
olcAccess: to * by dn="cn=admin,dc=example,dc=com" write by users read

dn: olcDatabase={0}config,cn=config
changetype: modify
add: olcRootDN
olcRootDN: cn=admin,dc=example,dc=com
-

dn: olcDatabase={-1}frontend,cn=config
changetype: modify
add: olcAccess
olcAccess: to dn.subtree="cn=Monitor" by dn="cn=admin,dc=example,dc=com" read
-

# http://www.openldap.org/doc/admin24/monitoringslapd.html
dn: olcDatabase=monitor,cn=config
objectClass: olcDatabaseConfig
olcDatabase: {2}monitor

My updates include enabling monitoring, addition of commonly used indicies, and some security changes. The security changes are somewhat commented above, and include disabling anonyous access by default, removing a redudant entry that already exists in the default-created /etc/ldap/slapd.d/cn=config/olcDatabase={-1}frontend.ldif file, and only allowing "people" (vs. system accounts, etc.) to change their own passwords. Though fixed later in the server guide under "Setting up ACL", the initial permissions on the userPassword and shadowLastChange attributes are treated differently in the server guide. The above gives them the same permissions, and matching those under "Setting up ACL" later in the server guide - from the start.

Starting with OpenLDAP 2.3, the LDAP configuration, by default, including the schema, is stored in LDAP itself. This allows for dynamic configuration at runtime, and typically without restarting the service. At least in Ubuntu's default configuration, access to this configuation, stored in cn=config, is only available to the root user on the operating system. My updated configuration (above) also allows access to the same configured root user (olcRootDN) as specified in the backend database. Since no seperate olcRootPW is provided, the same password from the backend database is also used. Interestingly, I found that if only a olcRootPW is provided for the configuration database without a olcRootDN, a currently-undocumented default olcRootDN of "cn=admin" is used. I filed an issue report addressing this oversight in the documentation at http://www.openldap.org/its/index.cgi?findid=6546.

These settings and changes may not and probably will not be perfect for everyone, but at least this should point out some things to be aware of. Please modify as appropriate to fit your needs.

Security

Be sure to configure TLS (transport layer security). This appears to be covered quite well in the Ubuntu server guide, and I don't plan to provide any additional details around TLS for LDAP here.

Backup Plan

A backup plan is not currently covered in the Ubuntu server guide, so be sure to at least review the Maintenance section of the OpenLDAP Administrator's Guide. However, even the OpenLDAP guide doesn't even mention the possibility of hot database backups, but does conclude the backup section with "MORE on actual Berkeley DB backups later covering db_recover etc.".

A hot database backup doesn't require any downtime, while offering good protection with minimal space overhead. Unlike the other documented approach using slapcat, a hot database backup also guarantees consistency by capturing an exact point in time, regardless of how long the backup takes.

Here I am assuming that you are using a Berkeley Database for the backend, either bdb or hdb. Details of these backends are available under "Berkeley DB Backends" in the OpenLDAP Administrator's Guide. Since most of the database is stored within this database, we can defer to the database's backup documentation. For the Berkeley Database, this is documented quite well at http://www.oracle.com/technology/documentation/berkeley-db/db/gsg_txn/C/backuprestore.html.

The best available option for backing up, at least for a hot backup, seems to be the db_hotbackup command line utility referenced in the Berkeley DB documentation. At least in Ubuntu, this script is not installed by default, but is available in the db4.x-util package. Unfortunately, the OpenLDAP implementation provided by the Ubuntu package is still linked against version 4.7 of the Berkeley DB instead of the version 4.8 that was standardized on by almost every other package depending upon Berkeley DB in the Lucid release, so the db4.7-util package is probably the one you need to install. I filed a bug report addressing this version issue in Ubuntu Bug # 572489.

Once the utility package is installed, a hot backup can simply be executed as:

sudo db4.7_hotbackup -c -h /var/lib/ldap/ -b <backupDir>/db

Also backup the /etc/ldap/slapd.d/ directory, which contains the server configuration, including the LDAP schema and configuration of the backend database: (On my systems, I already backup all of /etc/.)

sudo cp -R /etc/ldap/slapd.d <backupDir>/slapd.d

Schedule this to happen regularly and automatically through cron or another job scheduler, and ensure a copy is written to tape or another equivilent media and kept off-site, following typical best practices.

To be continued...

I'll be following up with a number of posts. To be included:

Wednesday, April 7, 2010

MarkUtils-JMX

Following my recent post on JMX Secure Connections / Avoiding Java System Properties, I am making another addition to MarkUtils: MarkUtils-JMX.

JMX Management Bean Metadata

My primary inspiration for this library was that JMX provides a generous amount of metadata along with each management bean, attribute, operation, and parameter - including names, descriptions, and impacts (INFO, ACTION_INFO, ACTION, or UNKNOWN) for operations. Parameter names must be provided through metadata, otherwise only the generated defaults of p0, p1, etc. are displayed. Unfortunately, associating this data with a management bean is currently a pain, and the only support available for this through the JDK itself is by subclassing StandardMBean, or completely instantiating a MBeanInfo yourself, and returning from a subclass of StandardMBean or from a custom DynamicMBean implementation.

MarkUtils-JMX provides a MBeanInfoBuilder class that serves as an alternative to the non-public classes used to create MBeanInfo's within the JDK. The most significant feature of MBeanInfoBuilder is support for including names, descriptions, and impacts that are read from Java annotations. The supported annotations are provided as part of this library in the com.ziesemer.utils.jmx.beanInfo package. Similar functionality is planned for release with Java 7 as part of JMX 2.0 / JSR 255, as detailed by Eamonn McManus in Playing with the JMX 2.0 API (2008-08-06, weblogs.java.net). Unfortunately, while I'm sure the annotations will not be compatible with my own (even just due to mine being in a com.ziesemer package), mine are available immediately, and support both Java 1.5 / 5.0 and Java 1.6 / 6.

MBeanInfoBuilder is not intended to be a complete replacement, as it currently doesn't support constructors, notifications, and descriptors, mostly as I currently have no use for them. However, these limitations are almost completely mitigated by MBeanInfoCombiner, which allows the metadata from a custom MBeanInfo built by MBeanInfoBuilder to supplement the default MBeanInfo built by the JDK's StandardMBean - effectively adding support for the custom annotations.

SimpleMBean, also included, is a alternative to the JDK's StandardMBean (as with MBeanInfoBuilder, and doesn't depend upon non-public classes. Along with utilizing MBeanInfoBuilder by default, this allows for easy extension / customization of the implementation. Additionally, both MBeanInfoBuilder and SimpleMBean are designed and written with performance as a primary focus. Performance is more of a concern for the actual MBean implementation than the building of the MBeanInfo, as the information should only have to be built once, where as the MBean implementation will likely be called repeatedly and often throughout the lifetime of the hosting application.

Authentication and Authorization

Also included in this library are classes to handle most of the work around securing JMX access by providing authentication and authorization.

Authentication is the simpler half, and is provided by com.ziesemer.utils.jmx.server.authentication.BaseJmxAuthenticator. This abstract class handles all the validation and breaking-down of the Object input parameter. An implementing class must only implement Subject authenticate(String username, String password). The authentication implementation is then registered as a value to the JMXConnectorServer.AUTHENTICATOR property, and passed-in as part of the environment map to JMXConnectorServerFactory.newJMXConnectorServer.

Authorization is a little more complex. Limited support is built-in to the JDK through Java's security policy and MBeanPermission. Unfortunately, the JDK approach seems overly-complex and did not meet several of my requirements. This library provides a high-performance and flexible alternative, BaseJmxAuthorizer.

BaseJmxAuthorizer and the other classes in com.ziesemer.utils.jmx.server.authorization are based on "Use Case 2" in Luis-Miguel Alventosa's blog post, Authentication and Authorization in JMX RMI connectors (2006-09-25, blogs.sun.com), where an InvocationHandler is used to selectively proxy requests to the management bean. Again, BaseJmxAuthorizer was designed and written with performance as a primary focus, along with flexibility. It is based on a Map (implemented with a HashMap), associating incoming method names (e.g. getAttribute or invoke) with an associated handler. This Map is publicly exposed, allowing for easy customization through modifications made to the map. Several default handlers, including the simple and unconditional AllowHandler and DenyHandler, are included in the package.

A ReadOnlyInvokeHandler is also provided, and allows all calls to operations / methods that are considered to be "read-only". The request impact (MBeanOperationInfo.getImpact()) is first consulted, and the request is allowed if INFO is returned. The request is otherwise denied, unless the impact returned is UNKNOWN, in which case an attempt to determine if the method has a "read-only" name is made, which checks for if the method name starts with "get", "is", "list", "query", or is equal to "hashCode". By default, it also allows for calls to ThreadMXBean.dumpAllThreads. These allowances allow for full and easy use of all "read-only" functionality provided by JConsole by default, without compromising on security.

Unfortunately, under Java 1.5 / 5.0, this currently denies access to get* methods on ThreadMXBean and LoggingMXBean, as these operations are marked as ACTION_INFO rather than UNKNOWN (or more preferably / correctly, INFO). This appears to have been somewhat fixed in Java 1.6 / 6, as per Sun Bug 6320104, it seems that UNKNOWN was simply not supported by the infrastructure in 1.5 / 5.0 at the time. A ReadOnlyJavaFixInvokeHandler is provided in the package as a work-around for use under Java 1.5 / 5.0. I reported Sun Bug 6933325 to address that these operations should now return INFO instead of ACTION_INFO, now that it is possible. Please view the Javadocs and/or source for additional details.

BaseJmxAuthorizer handles basic logging of method invocations through SLF4J by default.

This library's BaseJmxAuthorizer also simplifies authentication and improves performance by associating permissions with a subclass of JMXPrincipal, JmxPermissionPrincipal. This allows permissions to be obtained for a user once during authorization, without having to re-check permissions on each JMX call. Just be sure that this caching is accounted for, such that a user doesn't maintain unattended access if / once a permission is removed by the underlying security directory. A few options could be implementing a session timeout, or adding a call to a customized implementation to also remove the permissions from the runtime if / once removed from the user.

JmxPermissionPrincipal maintains an instance of a subclass of PermissionCollection, JmxPermissions, which contains one or more JmxPermission (subclass of Permission instances. Default provided JmxPermission's are CONNECT, READ_EX, INVOKE, and ADMIN. Other subclasses may be created and used, but should be kept as singletons. Please view the Javadocs and/or source for additional details.

Packaging with Apache Maven, Java versions

As with all my other MarkUtils libraries, MarkUtils-JMX is configured, compiled, tested, and packaged using Apache Maven. This library posed a little bit of a challenge, however, as I wanted to provide 2 versions, one built for Java 1.5 / 5.0, and one for Java 1.6 / 6. This is one of the few times I wish that Java had standard support for conditional compilation directives, such as those supported by the C preprocessor.

The primary reason for requiring dual versions is for Java 6's support of MXBean's, which were not supported in Java 5. This includes use of the additional 2-arg constructor added to StandardMBean, which as far as I can see, is impossible to implement without being available at compile-time - even if considering reflection tricks.

The solution I decided on for now is to offer dual packages / builds: com.ziesemer.utils.jmx.java5 and com.ziesemer.utils.jmx.java6. The Java 6 version includes the source paths from the Java 5 version. To clarify, only the Java 5 or the Java 6 version is required, as the Java 6 version is a superset of the Java 5 version. The Java 5 version includes a StandardMBeanInfoCombiner5 class that is a subclass of StandardMBean and utilizes MBeanInfoBuilder and MBeanInfoCombiner. The Java 6 version includes a StandardMBeanInfoCombiner6, which is the same as the Java 5 version, but exposes the added constructor and allows use of MXBeans. This setup actually worked with Maven fairly well, including the ability to automatically run Maven builds against both versions in one operation, and including packaging the build outputs together into the final distribution.

The Java 6 version also includes a DefaultRegistration class, which registers a few additional custom MXBeans that I found helpful and that provide functionality not currently offered by the MXBeans provided by the JDK (ManagementFactory). These include my custom CharsetMXBean, SecurityMXBean, and SystemMXBean. (As these custom beans themselves don't require any Java 6-specific features or calls at compile time, they are actually included in the Java 5 package. They just require Java 6 to be registered as MXBeans rather than regular MBeans.)

Download

com.ziesemer.utils.jmx is available on ziesemer.dev.java.net under the GPL license, complete with source code, a compiled .jar, generated JavaDocs, and a suite of 40+ JUnit tests. Download the com.ziesemer.utils.jmx-*.zip distribution from here. Please report any bugs or feature requests on the java.net Issue Tracker.

Monday, March 8, 2010

Thoughts on Google Fiber for Appleton / ISPs

Tonight I attended a public hearing at Appleton City Hall (PDF) regarding the city's consideration to submit a response to Google's request for information on the Google Fiber for Communities experiment. (Don't miss Google's project overview and other linked pages.) Also, please join the Google Fiber for Appleton Facebook group.

I was pleasantly surprised to see this public hearing bring the attention of Green Bay TV stations WFRV / CBS channel 5 and WGBA / NBC channel 26. The public hearing started late due to another meeting in the same room. There were probably a few more than 20 people in attendance, and the response was overwhelmingly positive. This echoed the current status of the City of Appleton's Survey, which was mentioned to also be overwhelmingly in favor of submitting a proposal. In general, the responses given at the hearing were mostly focused on the points that Appleton should proceed with submitting a favorable proposal in order to remain competitive as a community, to bring additional competition and choices for Internet service, and many other convincing reasons.

There were only 3 responses against: 2 from AT&T representatives in attendance for somewhat obvious reasons in concern for their business, and 1 gentleman concerned with the physical cost necessary to connect to connect gigabit networking to his Apple computer. (He apparently assumed that the fiber would need to be connected directly to his computer. Most computers sold in the past few years already have gigabit Ethernet cards, or they are readily available for less than $50. Additionally, there are many potential uses beyond a computer, such as video and other multimedia.) As also mentioned by another resident at the meeting, even being able to connect at only 100 Mbps (vs. the 1 Gbps / 1,000 Mbps being advertised) would still be about 10x faster than most consumer broadband connections available today.

My Own Thoughts

In general, additional competition for residential Internet service can only be a good thing - whether that competition is from Google or another provider. At my current residence in an apartment just outside the official city limits (in Grand Chute, near the Fox River Mall), my choices for broadband Internet are currently limited to Time Warner Cable and DSL. Wireless / "3G" was tried, and is not current viable for primary / serious use as previously detailed. Interestingly, the reps from AT&T at the hearing used their wireless network as one of their primary arguments against submitting a proposal to Google. After looking into Time Warner Cable, they presented themselves as one of the shadiest operations in town, at least based upon my experience at their local office. I'm currently using AT&T's DSL. While I would like to sign-up for AT&T U-Verse, I'm told that it is not available to my particular building, despite several of my neighbors in very close proximity having the service.

Unlike many of the comments and hype, I would hope that Google's offering is not all about the speed - even though, unfortunately, this is all many residential consumers are aware of or take into consideration. A few questions to consider of any ISP:

  • In addition to the speed, what is the latency (or lag)?
  • What is the support for IPv6?
  • Are industry standards properly followed, such as RFC 2308 - Negative Caching of DNS Queries? Or is the ISP involved in DNS hijacking?
  • Is service advertised as "unlimited" truly unlimited, or are there limits involved that are only shown in fine print, if at all?
  • What is the ISP's stance on network neutrality?
  • What support / allowance is there for operating as a server - either a web server, or something more "residential" such as allowing remote desktop connections, allowing for peer-to-peer file transfers, or playing games that require being able to accept incoming network connections?
  • What are the results from the ICSI Netalyzer hosted by UC Berkeley, which tests for most of the above as well other issues?

Google, in particular, has a positive record for properly supporting the above requirements and avoiding the listed issues:

I can't imagine that Google wouldn't uphold the same principals in providing their own Internet service.

(On a humorous note, I can't help but recall Google's previous ISP / fiber offering: Google TiSP.)

Even if Appleton submits a proposal for and is accepted as a location for Google Fiber, there is no guarantee that I would be in the service area - especially being in a neighboring town. However, I have no doubt that I would still benefit from the increased competition. Additionally, once we're back in the market to buy a a house in the area, the availability of Google Fiber would be a serious consideration. (Someone please buy our house for sale in Wausau! - which happens to be wired with Cat-6 for Gigabit networking.)

The Need for IPv6

Almost a year ago, I brought IPv6 connectivity to my home network / LAN.

Background

Similar to the past Y2K issue, the Internet is facing a similar issue that just hasn't been publicized too much in the mainstream media yet: Exhaustion of the IPv4 addresses currently being used. I found a very interesting and detailed IPv4 Address Report by Geoff Huston that is auto-generated daily. There are various estimates as to the numbers and dates, but all the predictions are currently falling in the range of years 2011-2012. This shortage of IPv4 addresses will certainly be a much larger issue than other predictions and myths for the year 2012.

The only real solution to the IPv4 address shortage is upgrading to IPv6. IPv4 allowed for 232, or 4,294,967,296 addresses. With most computers, servers, and even cell phones each being assigned a unique address, the shortage should not be surprising. However, back when the IPv4 specification was published back in 1981 (RFC 791), I'm sure 4+ billion addresses was considered more than sufficient. IPv6 solves this shortage by increasing the number of possible addresses to 2128, or 3.4×1038. Written out, this is 340,282,366,920,938,463,463,374,607,431,768,211,456 addresses. Beyond the increased address space, IPv6 also brings a number of other features, including mandatory support for advanced security, simplified processing, and support for network mobility.

IPv6 became active for production use on the Internet in June 2006. Unfortunately, it seems that many organizations and much of the Internet has not yet committed to converting, and the shortage will have to be dealt with. I see this causing more problems for regular at-home users than anyone else. Most individuals are not aware of the issue, and have little choice than to accept however their ISP handles the issue - short of possibly switching providers. Already, most consumers are only leased one IP address per Internet account, which usually must be shared between several computers and other Internet-connected devices. This is almost always accomplished through network address translation (NAT). This already causes complications and issues with file transfers, remote assistance applications, VPN software, online gaming, and many other typical Internet uses. As the shortage becomes more significant, expect for an increasing number of ISPs to no longer lease a public IP address at all, but instead only lease a private IP, where multiple private IPs share one public IP - essentially nesting one NAT network within another, and will only further complicate matters. The same "public IP per Internet account" that we are accustomed to today may still be available - but only for an added fee.

Use of NAT and private IPs are also in conflict with the fundamental design of the Internet and prevent end-to-end connectivity. Overall, it increasingly seems that Internet providers are only guaranteeing limited "web access" vs. fuller "Internet access". I.E., if it doesn't run in a web browser, it is probably not supported. I already experienced this first-hand in my previous dealings with Alltel / Verizon in regards to my wireless Internet issues. Consumers need to start demanding more from their ISPs, and need to know and have a limited understanding of the facts to do so. One tool that can help with this is the ICSI Netalyzer hosted by UC Berkeley. Guarantee of a public IP - either IPv4 or IPv6 - is also something that should be investigated and demanded.

IPv6 Choices

The best and easiest way to utilize IPv6 is by connecting to an ISP that provides IPv6 support. Unfortunately, finding such an ISP is still a nearly impossible task - especially when limited to those that provide local access. There are a number of transition mechanisms that should be able to provide IPv6 even without ISP support, but all have their own issues. For example, Microsoft Windows Vista, Windows 7, and most other modern operating systems support 6to4, Teredo, and ISATAP as tunneling mechanisms. However, I have not had any real success with any of these - at least not under Windows and while behind a NAT.

6to4 actually seems like an ideal solution to provide IPv6 access to a LAN, as long as there is a capable device to serve as a router that also has access to a public IPv4 address. Unfortunately, the address of the IPv6 subnet is based on the IPv4 address. While this may be a feasible solution for those with static IPv4 addresses (rare, more expensive, and only becoming worse), use on a dynamic IPv4 address requires an insanely short lifetime on IPv6 addresses, and requires the entire LAN to be re-addressed whenever the hosting IPv4 address is updated.

This pretty much leaves me with tunneled IPv6 access through a tunnel broker, using either configured 6in4 or AYIYA protocols. The best I have found - at least for free - are Hurricane Electric's Tunnel Broker, SixXS, and gogoNET (previously go6.net).

SixXS has the largest list of available "Points of Presence" - 35 over 18 countries. However, access pretty much requires the AICCU client, which is becoming a bit outdated and has a number of issues under Windows. (As of this writing, the last update for Windows was 2008-05-25.) Additionally, while free, SixXS has had much difficulty maintaining uptimes - particularly the one in Chicago as well as other POPs in the US.

gogoNET currently has much better support for Windows (using their gogoClient - with versions for most *nix versions as well), but the available tunnels are limited to 3, and with nothing local to the US: Montreal, Amsterdam, and Sydney.

Overall, I've had the most success with Hurricane Electric. HE provides 24 tunnels across 10 countries, including 12 within the US. However, unlike SixXS and gogoNET, HE provides no visible support for use behind a non-owned firewall, such as for mobile use on other public networks.

Saturday, February 20, 2010

JMX Secure Connections / Avoiding Java System Properties, RMI

I spent much of my weekend working on adding support for Java Management Extensions (JMX) into a large enterprise application. Security was appropriately a primary concern, and I needed to ensure that all connections were properly encrypted. The most significant observation I've made during this work is that Java system properties are often overly depended upon / misused. Dependencies within RMI are a prime example of where the use of system properties cause some severe limitations, and are an area that probably could certainly use some improvement.

System properties are global to a JVM. Especially in a large application, conflicts can quickly arise if alternate configuration methods aren't available. For example, a necessary configuration may require one section of code or a referenced library to have a given system property set to one value, while another section or library requires the same system property set to another value. This is possible if they are split into separate programs, running on separate JVMs, but not within the same JVM. System properties can be set and changed at runtime by calling System.setProperty(…), but this should not be taken lightly and should usually be avoided. When they do need to be set outside of the java command-line, system properties should only be set within an applications "main" method, or other top-level code. I previously had to fix an issue where a JSP was switching the "javax.xml.transform.TransformerFactory" value between the interpretive and compiled (XSLTC) versions, which caused interesting issues (a.k.a. failures) elsewhere throughout the application, as the switch was causing different processors to be used for various functions, depending upon the timing between calls to the JSP (review: concurrency, thread safety). The same primary issue is shared with environment variables, as they are also globally shared by the JVM (or any process), However, unlike Java's system properties, the environment variables of the current runtime are not modifiable by Java.

Specific to my work were the Java Secure Socket Extension (JSSE) customizations, particularly the "javax.net.ssl.keyStore*" and "javax.net.ssl.trustStore*" system properties. Note that these are referred to as "defaults", with some default values provided for the default parameters, meaning that there should be a way to use a non-default value when needed. Another limitation of these specific customizations is that they are only single-valued, with no way to provide support for multiple key stores, etc., short of providing an overridden implementation class, which is full of issues in itself. Especially in a large enterprise application, calls need to be made to different services that require different certificates, and especially with limitations around automatic certificate selection, there needs to be a way to hook into this through flexible code when required.

HttpsURLConnection is an excellent example of a class that provides exactly this. In addition to the static (JVM-global) setDefaultSSLSocketFactory(…) method, it also provides a setSSLSocketFactory(…) method that can be used to provide customized SSL socket factories on a per-connection basis.

Unfortunately, JMX and RMI currently provide no such hooks, relying exclusively on system properties or the default socket factory. In the case of RMI, things only get more interesting and complicated. Everything exposed through the RMI public API is protocol-generic, with no concept of TCP, IP addresses, or port numbers. These are only handled by internal UnicastRef, LiveRef, and eventually TCPEndpoint classes. Only a "stub" is communicated to the client, through a registry or other means, and this stub contains an optional, serilaized RMICClientSocketFactory for creating a connection from the client to the host. That's right - the server controls the socket factory that will be used by the client to connect back to the server. I haven't found any clean way to override this behavior at the client. For JMX over RMI, both the server and client factories are set through an environment map with property keys defined on RMIConnectorServer. (RMI is the only JMX connector included with Java 5 and 6.)

For the application I was working on, changing the system properties to control the certificates used for secure communications is simply not feasible, if even an option at all. So I created an alternate SslRMIClientSocketFactory implementation, as even mentioned in the JDK's source code:

// If someone needs to have different SslRMIClientSocketFactory factories
// with different underlying SSLSocketFactory objects using different key
// and trust stores, he can always do so by subclassing this class and
// overriding createSocket(String host, int port).

This alternate implementation returned sockets from a socket factory on a custom-initialized SSLContext. This worked great when connecting between different instances of the same application (different JVMs on the same node, as well as different nodes). However, this requires the alternate class (and any and all other references classes) to be available on the classpath of any client making the connection - making things difficult for connections from other standard clients such as JConsole or VisualVM. It is possible, however, by setting the classpath with the "-J-Djava.class.path=…" arguments, which work the same for both JConsole and VisualVM. Both these programs utilize native launchers, so the "-J" prefix means to pass the trailing argument to the JVM and not the native launcher itself. When doing this, "<java.home>/lib/jconsole.jar" must be included as well for JConsole, or JConsole won't even start. tools.jar is also necessary for connecting to local processes and possibly other features, but apparently isn't required for remote connections like those being attempted here.

The solution I'm proceeding with is two-fold. First, I'm registering two JMXConnectorServers per JVM - one that uses the standard SslRMIClientSocketFactory, and one that uses a customized factory class. The first can be used by any client, assuming that the certificates are valid in the default trust store, or that the proper references using system properties can be made. The second can be used by any client that has the customized class available on the classpath, including other instances of the application itself. Fortunately, this does not require any additional network ports to be kept open. Each instance shares the same port, with a non-visible (at least not publicly or easily) ObjID being used to distinguish between them - which is also included in the serialized connection stub used by the client.

For my customized SslRMIClientSocketFactory class, simplest is best. By using only classes native to the JDK, only the one class is necessary to be available on the client's classpath so that it can be deserialized from the RMI stub. To avoid the issues with global system properties as described above, it also needed to be customizable, ideally being able to provide alternate socket factory implementations from within the client. Unfortunately, even if this class had a setSocketFactory method available, the rest of the RMI and JMX API doesn't provide for any apparent opportunity to call such a method. While a bit of a hack, my solution was to use a ThreadLocal. Here is my entire class:

import java.io.IOException;
import java.io.Serializable;
import java.net.Socket;
import java.rmi.server.RMIClientSocketFactory;

import javax.rmi.ssl.SslRMIClientSocketFactory;

public class ThreadLocalSslRmiClientSocketFactory
    implements RMIClientSocketFactory, Serializable{
  
  private static final long serialVersionUID = 1L;
  
  public static final ThreadLocal<RMIClientSocketFactory> SOCKET_FACTORY
      = new InheritableThreadLocal<RMIClientSocketFactory>(){
    @Override
    protected RMIClientSocketFactory initialValue(){
      return new SslRMIClientSocketFactory();
    }
  };
  
  public Socket createSocket(String host, int port) throws IOException{
    return SOCKET_FACTORY.get().createSocket(host, port);
  }
  
}

This allows for the socket factory to be configured by calling ThreadLocalSslRmiClientSocketFactory.SOCKET_FACTORY.set(…) somewhere within the current thread prior to making the connection, and without having requirements on or otherwise impacting the rest of the application. If this customization is not called, then the default SslRMIClientSocketFactory is used, and the global system properties should be referenced to determine the trust store, etc. Starting the servers then looks like this:

import java.lang.management.ManagementFactory;
import java.util.HashMap;
import java.util.Map;

import javax.management.MBeanServer;
import javax.management.remote.JMXConnectorServer;
import javax.management.remote.JMXConnectorServerFactory;
import javax.management.remote.JMXServiceURL;
import javax.management.remote.rmi.RMIConnectorServer;
import javax.rmi.ssl.SslRMIClientSocketFactory;

import org.slf4j.Logger;

// …

MBeanServer mbs = ManagementFactory.getPlatformMBeanServer();
JMXServiceURL url = new JMXServiceURL("rmi", null, 0);

Map<String, ? super Object> serverEnv = new HashMap<String, Object>();
serverEnv.put(
  RMIConnectorServer.RMI_SERVER_SOCKET_FACTORY_ATTRIBUTE,
  new JmxSslRmiServerSocketFactory());
serverEnv.put(
  RMIConnectorServer.RMI_CLIENT_SOCKET_FACTORY_ATTRIBUTE,
  new SslRMIClientSocketFactory());

JMXConnectorServer sslConnector = JMXConnectorServerFactory.newJMXConnectorServer(url, serverEnv, mbs);
sslConnector.start();
LOGGER.info("JMX SSL server started: {}", sslConnector.getAddress());

serverEnv.put(
  RMIConnectorServer.RMI_CLIENT_SOCKET_FACTORY_ATTRIBUTE,
  new ThreadLocalSslRmiClientSocketFactory());

JMXConnectorServer threadLocalSslConnector = JMXConnectorServerFactory.newJMXConnectorServer(url, serverEnv, mbs);
threadLocalSslConnector.start();
LOGGER.info("JMX thread-local SSL server started: {}", threadLocalSslConnector.getAddress())

What about other connectors? As listed in the connector chapter of the JMX overview documentation:

An optional part of the JMX Remote API, which is not included in the Java SE platform, is a generic connector. This connector can be configured by adding pluggable modules to define the following:

  • The transport protocol used to send requests from the client to the server, and to send responses and notifications from the server to the clients
  • The object wrapping for objects that are sent from the client to the server and whose class loader can depend on the target MBean

The JMX Messaging Protocol (JMXMP) connector is a configuration of the generic connector where the transport protocol is based on TCP and the object wrapping is native Java serialization. Security is more advanced than for the RMI connector. Security is based on the Java Secure Socket Extension (JSSE), the Java Authentication and Authorization Service (JAAS), and the Simple Authentication and Security Layer (SASL).

The generic connector and its JMXMP configuration are optional, which means that they are not always included in an implementation of the JMX Remote API. The Java SE platform does not include the optional generic connector.

There is also a Web Services (WS) connector being worked on, as described in JSR 262: Web Services Connector for Java Management Extensions (JMX) Agents and the reference implementation project at https://ws-jmx-connector.dev.java.net/. I've read that the WS connector is planned to be included with Java 7. Fortunately, it appears that the WS connector supports a custom SSLContext, configurable on the client, as shown in Securing JMX Web Services Connector (Jean-Francois Denise, 2007-08-16, blogs.sun.com).

Other related JMX references that have already proved useful:

Wednesday, February 17, 2010

MarkUtils-IO: Performant Java Streams, Readers, and Writers

MarkUtils-IO is another high-performance addition to MarkUtils, and is a collection of utility classes that I've found myself frequently reusing over the past number of years.

Work on this library was also the driving factor for another recent post concerning some of Java's built-in code, Redundant argument validation code in Java IO classes.

com.ziesemer.utils.io is available on ziesemer.dev.java.net under the GPL license, complete with source code, a compiled .jar, generated JavaDocs, and a suite of 40+ JUnit tests. Download the com.ziesemer.utils.io-*.zip distribution from here. Please report any bugs or feature requests on the java.net Issue Tracker.

Saturday, January 30, 2010

DynDNS Update Client Shell Script

I'm sharing the shell script I wrote and have been using for the past number of months to update my Dynamic DNS account on DynDNS.com with my dynamic IP address on my DSL connection, and previously with Alltel / Verizon Wireless.

There are many other update clients available. However, I had specific issues with every other one I tried, and this one meets my needs perfectly. Additionally, I wrote this script with a few particular design goals, as commented in the code below.

This is a Unix/Linux shell script, and is not designed to work with other environments such as Windows. It may work under Cygwin or another such environment as long as all the dependencies are available and any necessary changes are made. These dependencies include: wget, date, logger, sed, at, and parseable output from ifconfig - all available by default on most Linux installations.

#!/bin/sh

# Mark A. Ziesemer, http://www.ziesemer.com
# 2009-08-30, 2009-10-26
# http://blogger.ziesemer.com/2010/01/dyndns-update-client-shell-script.html
# With thanks to "ferret" and "pgas" on #bash (IRC) for some general bash-related questions.

# Design goals, in order of priority / importance:
#    1) 100% compliance with DynDNS Update API (http://www.dyndns.com/developers/specs/),
#      including all policies and required timings.
#    2) Maintain and update IP as fast as allowed by the specification, minimizing "downtime".
#     Properly update in cases currently missed by other update clients, e.g.
#        when multiple updates are requested within an otherwise arbitrarily-defined time limit.
#    3) Run efficiently, respecting CPU, RAM, and disk requirements.
#      Use external scheduling (atd) and hooks to lessen required in-memory process time as much as possible.
#      Should be re-usable on embedded systems, e.g. OpenWrt, with only minor modifications necessary.
#    4) No dependencies on other "large" runtime libraries, e.g. Perl or Python.

# Exit status codes:
#   0   Update completed successfully.
#   11  Update completed successfully, but unnecessarily. (NOCHG)
#   21  IP has not changed.
#   31  Recognized temporary failure.
#   41  Assumed (unrecognized) temporary failure.
#   51  Temporary failure due to $lastUpdateResult="TEMP_FAIL".
#   101  Permanent failure.
#   111  Permanent failure due to $lastUpdateResult="FATAL".
#   112  Permanent failure due to unrecognized $lastUpdateResult.
#   201  Couldn't kill existing script.
#   255  Other unexpected failure.

### variables accepted as command line arguments
configFile="/etc/ddIpUpdate/ddIpUpdate.config"
forceUpdate=
forceRetry=

### variables persisted in $configFile
dynIF=

username=
password=
hostname=

varDir="/var/lib/ddIpUpdate"
stateFile="${varDir}/ddIpUpdate.state"
pidFile="/var/run/ddIpUpdate.pid"

### variables persisted in $stateFile

lastIP=
lastUpdateTime=
# GOOD, TEMP_FAIL, FATAL
lastUpdateResult=
futureJobNum=

### other internal variables
userAgent="com.ziesemer.ddIpUpdate 2009.10.26"
callback=$0

### Helper functions.

_log(){
  printf "$(date --rfc-3339=seconds) $*\n" >&2
  logger $0 "$*"
}

_setArgs(){
  while [ "$1" != "" ]; do
    case $1 in
      "-c" | "--configFile")
        shift
        configFile=$1
        ;;
      "-f" | "--forceUpdate")
        forceUpdate=true
        ;;
      "-r" | "--forceRetry")
        forceRetry=true
        ;;
    esac
    shift
  done
}

_exitErr(){
  local exitStatus=$?
  _log "Error! $exitStatus"
  _exitNormal
  return $exitStatus
}

_exitNormal(){
  trap - EXIT
  _writeConfig
  rm $pidFile
}

_checkRun(){ # cmd
  local cmd status out
  cmd=$*
  out=$(eval $cmd)
  status=$?
  if [ $status -ne 0 ]; then
    _log \
      "\nUnexpected return status: $status, exiting." \
      "\nCommand: $cmd" \
      "\nOutput: $out"
    return $status
  else
    echo "$out"
  fi
}

_schedule(){ # cmd, time
  if [ -n "$futureJobNum" ]; then
    _log "Removing existing scheduled job: $futureJobNum"
    atrm $futureJobNum
  fi

  local at_out
  at_out=$(_checkRun "echo $1 | at $2 2>&1")
  _log "Scheduled command: \"$1\", $(echo "$at_out" | tail -n 1)"
  local at_id=$(echo "$at_out" | sed -n "s/^job \([0-9]*\) at .*$/\1/p")
  echo $at_id
}

### Core functions.

_findIP(){
  local ip=$(ifconfig $dynIF | sed -n 's/ *inet addr:\([0-9.]*\).*/\1/p')
  _log "Detected IP $ip on interface $dynIF."
  echo $ip
}

_checkInstances(){
  if [ -e $pidFile ]; then
    local pid=$(cat $pidFile)
    command kill -TERM $pid
    # 'wait' only works for child processes
    sleep 1
    if [ kill -0 "$pid" ]; then
      _log "Existing script with PID $pid didn't stop; exiting..."
      exit 201
    fi
  fi

  echo $$ > $pidFile
}

_checkLastStatus(){
  case "$lastUpdateResult" in
    "FATAL")
      _log "Last update resulted in a fatal condition; user intervention required."
      exit 111
      ;;
    "TEMP_FAIL")
      if [ -z "$forceRetry" ] ; then
        if [ $(( $(date +%s) < ($lastUpdateTime + 1800) )) -ne 0 ] ; then
          _log "Temporary timeout not yet expired, or user intervention required."
          _rescheduleTempFail
          exit 51
        fi
      fi
      ;;
    "GOOD" | "")
      # Continue
      ;;
    *)
      _log "Unrecognized lastUpdateResult: $lastUpdateResult; exiting..."
      exit 112
      ;;
  esac
}

_readConfig(){
  case "$configFile" in
    *"/"*) . $configFile ;;
    *) . ./$configFile ;;
  esac
  if [ -r $stateFile ]; then
    case "$stateFile" in
      *"/"*) . $stateFile ;;
      *) . ./$stateFile ;;
    esac
  fi
}

_writeConfig(){
  echo "#This file is automatically re-written!" >$stateFile
  for name in "lastIP" "lastUpdateTime" "lastUpdateResult" "futureJobNum"; do
    echo $name=\"$(eval "echo \$$name")\" >> $stateFile
  done
  echo >> $stateFile
}

_checkIPChanged(){ # ip
  if [ "$lastIP" = "$1" ]; then
    _log "IP has not changed from $lastIP; exiting..."
    exit 21
  fi
}

_rescheduleTempFail(){
  futureJobNum=$(_schedule "$0 -c $configFile" "now + 30 minutes")
}

_updateIP(){ # ip
  local returnStatus=255
  lastIP=

  # Could do without writing the temporary files, but good to save anyway for debugging / troubleshooting.
  local updateResult=0
  echo "https://$username:$password@members.dyndns.org/nic/update?hostname=$hostname&myip=$1" \
    | wget -i - -O - -U "$userAgent" --save-headers \
      2>${varDir}/response.err >${varDir}/response.out || updateResult=$?
  
  if [ $updateResult -eq 0 ]; then
    # DynDNS requires action based on return codes only, not HTTP status:
    #    http://www.dyndns.com/developers/specs/guidelines.html
    :
  else
    _log "Error result from web service: $updateResult"
  fi

  # Previous bashism (bash array):
  # declare -a updateResponse=($(tail -n 1 ${varDir}/response.out))
  # ${updateResponse[0]}

  local updateResponse="$(tail -n 1 ${varDir}/response.out)"
  local updateResponse1="$(echo $updateResponse | awk '{print $1}')"
  # 2nd token is the returned IP, which really doesn't offer anything.
  # local updateResponse2="$(echo $updateResponse | awk '{print $2}')"

  _log "Received response: $updateResponse"

  case "$updateResponse1" in
    "good")
      lastUpdateResult="GOOD"
      returnStatus=0
      ;;
    "nochg")
      lastUpdateResult="GOOD"
      returnStatus=11
      ;;
    "dnserr" | "911")
       # Temporary issue, prevent any further requests for at least 30 minutes or until user manually clears error.
      lastUpdateResult="TEMP_FAIL"
      returnStatus=31
      ;;
    "badauth" | "!donator" | "notfqdn" | "nohost" | "numhost" | "abuse" | "badagent" | *)
      if ( [ $updateResult -ne 0 ] && [ -z "$updateResponse1" ] ); then
        # Temporary network failure or other assumed-temporary issue.
        lastUpdateResult="TEMP_FAIL"
        returnStatus=41
      else
        # Known permanent failure, or other completely unexpected result.
        # Prevent any further requests until user manually clears error.
        lastUpdateResult="FATAL"
        returnStatus=101
      fi
      ;;
  esac

  lastUpdateTime=$(date +%s)
  case "$lastUpdateResult" in
    "GOOD")
      lastIP=$1
      futureJobNum=$(_schedule "$0 -f -c $configFile" "now + 28 days")
      ;;
    "TEMP_FAIL")
      _rescheduleTempFail
      ;;
  esac

  return $returnStatus
}

_runUpdate(){
  _checkLastStatus
  local ip=$(_findIP)
  if [ -z "$forceUpdate" ]; then
    _checkIPChanged $ip
  fi
  _updateIP $ip
  return $?
}

### "Main"

set -e
trap _waitAbort TERM
trap _exitErr EXIT

_setArgs $*
_readConfig
_checkInstances

mkdir -p $varDir

result=0
_runUpdate || result=$?

_exitNormal
_log "Exiting with status: $result"
exit $result

To use, just save as an executable file somewhere at the location of your choice. Create a configuration file containing the 4 required parameters: dynIF, username, password, and hostname, e.g.:

dynIF="ppp0"
username="someUsername"
password="somePassword"
hostname="someDomainName.dyndns.org"

View the source above for the other optional parameters accepted, as well as their default values. The permissions for this file should be set so that it is only readable by the user account that will execute the script, in order to protect the password. This file will be looked for at "/etc/ddIpUpdate/ddIpUpdate.config" by default, but can be overridden with the "--configFile" or "-c" command line arguments.

When run, this script maintains state in a file, the location of which defaults to "/var/lib/ddIpUpdate/ddIpUpdate.state", but can be overriden in the config file. This file is created automatically on first run, and looks like this:

#This file is automatically re-written!
lastIP="1.2.3.4"
lastUpdateTime="1264001234"
lastUpdateResult="GOOD"
futureJobNum="1"

There are many options available for having this script executed. On my Ubuntu Karmic (9.10) system, I created a link to this script in the "/etc/ppp/ip-up.d" directory so that it is executed every time after my PPP connection starts, or is restarted.

I don't write shell scripts for a living, so while I have been using this myself for several months now without an issue, it is certainly possible that there may be a bug or other room for improvement. Please leave a comment here if you have a correction or a suggestion, but please "cite your source" to a supporting reference related to the issue, particularly for any shell script semantics. Please also remember to follow typical best practices for bug reporting.