Sunday, October 30, 2011

Ubuntu Linux 11.10 Install Notes

This past week, I finally got around to installing the latest version of Ubuntu Linux on my home server - 11.10, "Oneiric Ocelot" (released 2011-10-13). There shouldn't be anything too significant here - these are mainly notes for myself, but posted here in case something is useful to others. I always do a full, clean re-installation - so several of the notes listed here won't be of concern for an in-place upgrade.

For some additional reference, this is the same server that I have setup as part of my Ubuntu Linux Router project, is setup with LDAP, and is the subject of several other Linux-related posts.

As I am using and now dependent on VLANs on this server for my network setup, I need to have a few necessary related packages readily available for the upgraded OS before reinstalling. These are not currently included on either the CD or DVD installation images, and I need them to be able to regain Internet connectivity - so simply using "apt-get" to re-install them from the Internet is clearly not an option. For the 11.10 upgrade, I downloaded the following amd64 *.deb packages:

After the fresh installation, the server fails to boot. This isn't anything new to to 11.10, but has been something to remember since I converted to a GUID Partition Table (GPT) from the legacy MBR partitioning scheme. I already have the special BIOS boot partition, as detailed under "GPT" at http://www.gnu.org/software/grub/manual/grub.html#BIOS-installation, which doesn't need to be re-configured with an OS upgrade. However, after the Ubuntu installation finishes, I do need to reset this partition as "active" in the legacy MBR partition table using fdisk - as I apparently have one of the Intel motherboards with a "buggy BIOS" that otherwise shouldn't require this. Additional details are at http://ubuntuforums.org/showpost.php?p=9787109&postcount=22 and http://www.rodsbooks.com/gdisk/bios.html.

All of my posts for the Ubuntu Linux Router project, while written for 8.04, are still almost completely valid and applicable for all releases since - including 11.10. One edit for Configuring Persistent PPP is that instead of attempting to disable NetworkManager, it is easier to just remove it: "apt-get remove network-manager". Install the networking *.deb packages downloaded above, then configure the DSL. Uncomment the "usepeerdns" line from the /etc/ppp/peers/dsl-provider restored from backup until BIND can be setup.

Once online, download and install some additional packages:

apt-get install vim build-essential ssh gnome-panel nautilus-open-terminal dkms linux-server
apt-get upgrade

After getting LDAP re-configured, 11.10 has some problems with graphical / xorg logins, as detailed at https://bugs.launchpad.net/ubuntu/+source/at-spi2-core/+bug/870874, and somewhat, "Problems with LightDm on Ubuntu 11.10 Oneiric? Here's how to solve" (2011-10-16, linux-software-news-tutorials.blogspot.com). Unfortunately, in this case, both LightDm and gdm exhibit the same issue - so my current work-around is to disable the problem script, /etc/X11/Xsession.d/90qt-a11y, by prefixing its name with a non-numeric character.

Finally, I don't need the extra help that Ubuntu provides at every login, so I disable it:

$ sudo chmod -x /etc/update-motd.d/10-help-text

Saturday, October 1, 2011

Fixing UPnP/DLNA sharing on 2nd drive under Windows 7

Background

Though I may be a few years late to the party, I finally have a capable Home Theater PC (HTPC) and HDTV setup in my living room. For simplicity for the entire family, it's running Windows 7's Media Center.

My favorite feature of a HTPC-solution is the digital video recorder (DVR) functionality - including the options to pause and replay live TV. My family previously had a hard-drive enabled DVD player / DVR with this functionality built-in - but without an ATSC digital tuner, its use is now limited to analog cable. I since had a newer Philips DVD player / DVR with similar features - including an ATSC digital tuner - but it was only standard-definition (SD), not high-definition (HD), and recorded content had issues with volume levels as well as not supporting closed captioning and other broadcast features.

Disappointingly, these types of devices are appearing to be pushed out of the market in favor of versions that require monthly fees. Other than a new subscription to Netflix and our DVDs, most of what we watch is free broadcast television - without paying monthly fees for cable or satellite television service. I'm excited to have a HTPC-based DVR that is also fee-free. Much of the broadcast television we watch is also in HD, for free - something most paid-for TV services are still behind in for not offering without yet additional fees!

Besides having a direct HDMI connection between the PC and the TV, both devices support UPnP/DLNA. I'll prefer the HDMI connection as the primary link between the HTPC and its connected TV - mostly as it doesn't require to have an extra encode/decode step to stream the video over Ethernet. The DLNA method is also missing a few features that are natively available from the HTPC - including some simpler features such as closed captioning. However, the DLNA feature is certainly nice to have as a convenience for quickly streaming videos, pictures, or music from any other computer connected to the network - without needing to first add a wired connection, or otherwise transfer the media to a USB flash drive or another connected device.

UPnP/DLNA issue

I ran into some difficulties, however, when trying to stream content from my primary laptop - also running Windows 7 x64, Professional. My laptop simply would not show itself as a media server device. It wasn't a problem with the TV, as other clients on the network could also see all the other servers - just not my laptop. I eventually found that I could create new user accounts in Windows, and then my laptop became visible as a server - but only for the new user accounts, and not my primary user account. Unfortunately, this was something a bit more complicated than adjusting the "Media streaming options" in the Control Panel for my user account.

UPnP uses Simple Service Discovery Protocol (SSDP) for discovery. Not finding anything significant within the Windows Event Log, and not finding any other tools within Windows that could help troubleshoot the issue, I used Wireshark to try to determine what was causing the issue. I found several SSDP HTTP NOTIFY packets, with a URL of something like http://[fe80::5ddf:5cb8:a7e1:bbcb]:2869/upnphost/udhisapi.dll?content=uuid:eb1c9c01-e5c3-4eea-bea2-d9f0a6fb2bcf. Following the URL, XML was returned that described the media servers for all other media servers and users on the network - including from my own laptop, but not for my primary user account. This confirmed that the issue was with my server, and not the network or clients.

I eventually ran across several posts of other users having very similar symptoms, including:

Unfortunately, they were all unanswered (both replied to - especially with many non-solutions to the first). However, they all had something in common with my laptop setup: The user profile directory was moved off of the system drive. This should be a common setup - using disk partitions to separate the user data from the program data. Another common scenario for this today is to have the OS on a high-performance SSD, and a larger HDD for the user data. However, as Windows doesn't easily provide for configuring this setup, it isn't surprising that this isn't a more visible issue.

My Solution

In both threads, the users seemed to infer that the problem must have been in the file copy / move of the user profile directory between drives. Fortunately, at least in my case, this had nothing to do with the content of the files - but with the security permissions / Access Control Lists (ACLs).

Even after moving my user profile to a new drive / partition, folders such as "My Videos" have an added "Read" permission granted to "WMPNetworkSvc" (which is obviously the account for the "Windows Media Player network service"). This permission does not exist on (nor does it need to exist on) parent folders. However, there is a quirk in Windows where at least limited access needs to be granted to the root of the hosting drive, otherwise accounts can't access any data within the drive - regardless of what child ACLs may be present. This was not an issue on the C:\ drive, as by default, the "Read" permission is granted to "Users", and additionally has other special permissions granted to "Authenticated Users" - both which should apply to the "WMPNetworkSvc" account. Such permissions, however, did not exist on my data drive - which only had permissions defined for my personal account, the "Administrators" group, and the "SYSTEM" group.

The fix is rather simple - give the "WMPNetworkSvc" limited access to the root of the alternate drive hosting the user profile. By limited, I'm not kidding - and keeping access to the absolute minimum only follows best security practices. (You'll need to use the "Advanced" button to open the "Advanced Security Settings" dialog for this):

  1. From the drive properties, "Security" tab, click "Advanced".
  2. From the "Advanced Security Settings" dialog, on the bottom of the "Permissions" tab, click "Change Permissions...".
  3. On the 2nd dialog that pops-up, click "Add...".
  4. Type in "NT Service\WMPNetworkSvc" (without the quotes), click "Check Names" to make sure it is recognized, then click "OK".
    • The "NT Service" prefix is necessary to resolve the Windows service account. There is no other known way to be able to select it through the UI options.
  5. In the "Permission Entry" dialog that pops-up:
    1. Under "Apply to", select "This folder only".
    2. Under "Permissions:", check only the "Allow" checkbox for "Traverse folder / execute file".
    3. Click "OK".
  6. Click "OK" to close all 3 remaining dialogs.
  7. Restart the "Windows Media Player Network Sharing Service" (WMPNetworkSvc) Windows service.

All of the above UI steps can also be easily replaced with the following commands. Simply replace "D:\" with the drive letter that the Windows Media Player network service needs access to:

icacls D:\ /grant "NT Service\WMPNetworkSvc":(X)
net stop WMPNetworkSvc
net start WMPNetworkSvc

This is certainly not meant to nor will this help fix the many different types of issues that exist with network media sharing. However, hopefully this will benefit those who have organized their systems on to separate drives or partitions, and are experiencing the same issue.

HP LaserJet PCL Errors and Driver Availability

My primary printer for the past 10 years has been a trusted Hewlett-Packard (HP) LaserJet 2200D, with an added network adapter making it the equivalent of a 2200DN. 12,600 pages later, it is still working like new (though recognizing in a shared office environment, this same print volume would probably be used within a fraction of a year.)

After recently completing a Windows 7 x64 reinstall, I installed the latest driver linked to from the product support page, which is the Universal Print Driver (UPD), version 5.3.1. Interestingly, the "Previous version" column on the download page is empty - and I was unable to find any archive of driver downloads on HP's site.

This wouldn't have been a problem - except for having the following printed in place of the expected output whenever I attempted to print:

PCL XL error

        Subsystem:  KERNEL

        Error:      UnsupportedProtocol

        Operator:   0x0

        Position:   0

Fortunately, I still had the last known-working version of the driver saved, 5.0.3.37 from December 2009. This worked, but now I was curious as to if this was an issue new to the 5.3.1 drivers and if there were still other updates I could be benefiting from.

Kudos to HP for still releasing drivers for a 10-year-old product, and for the concept of a universal driver. However, these versions shouldn't replace existing ones on the product support page - especially if there isn't at least minimal testing with the related product to ensure that it still works.

From HP's Readme for the HP Universal Print Driver v5.3.1.10527:

Each release is a full release of the product for all printer description languages for both 32bit and 64bit platform. Version history of UPD builds released on CD-ROM or posted to www.hp.com/go/upd.

Unfortunately, there is nothing visible at www.hp.com/go/upd for previous versions of the drivers. Even after searching HP's FTP site, I couldn't find any "official" archive of the UPD driver for versions prior to 5.3.1. However, I did find copies elsewhere on the Internet, and here are some details of each that should be useful to anyone else trying to find the same versions. Note that these are for Windows x64, PCL 6 only:

VersionFile NameSize (bytes)SHA-1 Checksum
5.0.3.37 / December 2009 upd-503-pcl664.exe 16,289,336 733cd725b962b99a25b9b7c94cdd5aa3a06a7116
5.1.1.8232 / August 2010 upd-5_1_1_8232-pcl6_winxp-vista-x64.exe 16,329,016 2ead57774607bec316e55ad707b841fe8dfe6cfa
5.2.6.9321 / February 2011 upd-PCL6-X64-5_2_6_9321.exe 16,633,400 bcb70eb684d1f5b684bcc2873a30df72293416b0
5.3.1.10527 / July 2011 upd-pcl6-x64-5.3.1.10527.exe 17,794,616 a789f2bab5bf9d48897c8dd99dbae9a46bd7031f

None of the drivers newer than 5.0.3.37 printed anything other than the "UnsupportedProtocol" error shown above on my 2200. This seems to correlate to HP's version history, which shows that Unidrv*.dll - which was at version 6.0.6001.22127 from 4.7.0.30 in November 2008 through 5.0.3.37, was updated to 6.1.7600.16385. It seems probable that the upgrade of Unidrv*.dll caused an incompatibility with the printer.

I've emailed this to HP support (incident # 14612070), and beyond receiving the automated "One of our Technical Support Specialists will be responding to your inquiry" message, I'm skeptical that I'll receive a response, or that this issue will be addressed in future versions of the UPD driver.

I did receive a few responses:

  • According to your product serial number, the warranty on your HP LaserJet 2300d printer has expired.
  • Thank you for contacting Hewlett-Packard's Commercial Solutions Center.

    Mark, unfortunately, Hewlett-Packard does not offer support via e-mail for your product.

    We are sorry for the inconvenience caused to you in this regard.

    The issue seems to be with the printer driver you are using. Hence, to resolve the issue, please install the UPD driver which mmight resolve the issue.

    http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?lang=en&cc=us&prodTypeId=18972&prodSeriesId=28861&prodNameId=28862&swEnvOID=4063&swLang=8&mode=2&taskId=135&swItem=lj-95991-2

    We hope that the above information provides a quick solution to your inquiry.

    Thank you again for contacting HP e-Solutions.

    • (Apparently, HP didn't comprehend the part about the problem being with latest version of the UPD driver - and that no previous versions were available for download from HP.)
    • My response:

      I understand that my previous email was not even read, beyond the fact that yes, I'm using an out-of-warranty LaserJet 2200D. However, I'm not looking for support - I just want you to update your web site to reflect correct information.

      The problem that I'm attempting to bring to your attention is that I'm not the only one having this issue, and that the product page for this product that HP is still hosting should either be updated to reflect this, or remove the page entirely. I already found the solution - using the 5.0 version of the UPD instead of using the 5.1.3 one. Simply, please update the page at http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareIndex.jsp?lang=en&cc=us&prodNameId=28862&prodTypeId=18972&prodSeriesId=28861&swLang=8&taskId=135&swEnvOID=4063 to reflect this, and to assist other users who may be having the same problem - or remove the page as to not promote dis-information.

  • Thank you for contacting HP eSolutions.

    We really apologize for the inconveniance caused to you in regards to this issue.

    Thank you for updating on this. We will escalate this to the concerned team.

    If you have any other queries, please feel free to get back to us and we would be glad to be of assistance.

    Once again, thank you for contacting HP eSolutions.

    • (Received 2011-09-19. No communications since.)

Saturday, April 16, 2011

Improving code and quality with Checkstyle

I've always been picky about the quality my code and the code that I work with. For good reason. It makes the code easier to read and work with. Consistency makes the program flow easier to understand, bugs and other potential issues easier to spot, and difference comparisons between files and versions more effective. Sun Microsystems (original designer of the Java platform, now part of Oracle) seems to agree. From their "Code Conventions for the Java Programming Language":

This Code Conventions for the Java Programming Language document contains the standard conventions that we at Sun follow and recommend that others follow. It covers filenames, file organization, indentation, comments, declarations, statements, white space, naming conventions, programming practices and includes a code example.

  • 80% of the lifetime cost of a piece of software goes to maintenance.
  • Hardly any software is maintained for its whole life by the original author.
  • Code conventions improve the readability of the software, allowing engineers to understand new code more quickly and thoroughly.

For years, I haven't really used any special tools to assist with code formatting and standards, other than the Eclipse Java Formatter for automatically and completely reformatting code as needed, and Eclipse's Compiler Errors/Warnings for pointing out potential issues. One limitation with the compiler errors/warnings in Eclipse is that they really don't include any formatting-specific checks.

I've had experience with Checkstyle in the past (official site | Wikipedia). However, I'm not sure how much thought was put into the configuration of the standards that were in place, and combined with a high level of false positives, most of the developers found ourselves working against the tool rather than working with it. Standards are great, but they need to be thought through, and have buy-in from the project team.

Over the past few months, I rediscovered a need for Checkstyle for the group of developers I was working with. We were doing an satisfactory job addressing the compiler errors and warnings, but the code itself was a sore sight to look at. The code formatting was addressed during code reviews, but without having the formatting issues also appear as warnings in the Eclipse "Problems" view as it was being developed, the formatting issues were quickly "out of sight, out of mind", and quickly forgotten by much of the team. (I'm not sure how the group would have managed with a language like Python, where proper indentation of the code is actually part of the language syntax.)

Fortunately, since I last worked with Checkstyle, the tools have only improved. eclipse-cs now integrates seamlessly with both Eclipse and the Checkstyle XML configurations, and is fully configurable to projects on both local and global levels - using either internal, external, remote, or project-relative configurations.

For my own reference, as well as others to benefit from, I'm sharing the Checkstyle configuration I've worked with over the past few months. The surprising thing is how many inconsistencies were reported and that I was able to fix even in my own code after running the Checkstyle validations.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE module PUBLIC "-//Puppy Crawl//DTD Check Configuration 1.3//EN" "http://www.puppycrawl.com/dtds/configuration_1_3.dtd">

<!--
    This configuration file was written by the eclipse-cs plugin configuration editor
-->
<!--
    Checkstyle-Configuration: MAZ
    Description: none
-->
<module name="Checker">
  <property name="severity" value="warning"/>
  <module name="TreeWalker">
    <property name="tabWidth" value="2"/>
    <module name="JavadocMethod">
      <property name="severity" value="ignore"/>
      <property name="suppressLoadErrors" value="true"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="JavadocType">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="JavadocVariable">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="JavadocStyle">
      <property name="checkFirstSentence" value="false"/>
    </module>
    <module name="ConstantName"/>
    <module name="LocalFinalVariableName"/>
    <module name="LocalVariableName"/>
    <module name="MemberName"/>
    <module name="MethodName"/>
    <module name="PackageName"/>
    <module name="ParameterName"/>
    <module name="StaticVariableName">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="TypeName"/>
    <module name="AvoidStarImport"/>
    <module name="IllegalImport"/>
    <module name="RedundantImport"/>
    <module name="UnusedImports"/>
    <module name="LineLength">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="MethodLength">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="ParameterNumber">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="EmptyForIteratorPad"/>
    <module name="MethodParamPad"/>
    <module name="NoWhitespaceAfter">
      <property name="tokens" value="BNOT,DEC,DOT,INC,LNOT,UNARY_MINUS,UNARY_PLUS"/>
    </module>
    <module name="NoWhitespaceBefore"/>
    <module name="OperatorWrap"/>
    <module name="ParenPad"/>
    <module name="TypecastParenPad"/>
    <module name="WhitespaceAfter">
      <property name="tokens" value="COMMA,SEMI"/>
    </module>
    <module name="WhitespaceAround">
      <property name="tokens" value="BAND,BAND_ASSIGN,BOR,BOR_ASSIGN,BSR,BSR_ASSIGN,BXOR,BXOR_ASSIGN,COLON,DIV,DIV_ASSIGN,EQUAL,GE,GT,LAND,LE,LOR,LT,MINUS,MINUS_ASSIGN,MOD,MOD_ASSIGN,NOT_EQUAL,PLUS,PLUS_ASSIGN,QUESTION,SL,SL_ASSIGN,SR,SR_ASSIGN,STAR,STAR_ASSIGN,TYPE_EXTENSION_AND"/>
    </module>
    <module name="ModifierOrder"/>
    <module name="RedundantModifier"/>
    <module name="AvoidNestedBlocks">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="EmptyBlock">
      <property name="option" value="text"/>
      <property name="tokens" value="LITERAL_DO,LITERAL_ELSE,LITERAL_FINALLY,LITERAL_IF,LITERAL_FOR,LITERAL_TRY,LITERAL_WHILE,STATIC_INIT"/>
    </module>
    <module name="LeftCurly"/>
    <module name="NeedBraces"/>
    <module name="RightCurly"/>
    <module name="AvoidInlineConditionals">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="DoubleCheckedLocking"/>
    <module name="EmptyStatement"/>
    <module name="EqualsHashCode"/>
    <module name="HiddenField">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="IllegalInstantiation"/>
    <module name="InnerAssignment">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="MagicNumber">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="MissingSwitchDefault">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="RedundantThrows">
      <property name="suppressLoadErrors" value="true"/>
    </module>
    <module name="SimplifyBooleanExpression"/>
    <module name="SimplifyBooleanReturn"/>
    <module name="DesignForExtension">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="FinalClass"/>
    <module name="HideUtilityClassConstructor">
      <metadata name="net.sf.eclipsecs.core.comment" value="MAZ: Disabled, until this module provides an option to ignore abstract classes."/>
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="InterfaceIsType"/>
    <module name="VisibilityModifier">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="ArrayTypeStyle"/>
    <module name="FinalParameters">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="TodoComment">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="UpperEll"/>
    <module name="RegexpSinglelineJava">
      <metadata name="net.sf.eclipsecs.core.comment" value="MAZ"/>
      <property name="format" value="^\t* +\t*\S"/>
      <property name="message" value="Line has leading space characters; indentation should be performed with tabs only."/>
      <property name="ignoreComments" value="true"/>
    </module>
    <module name="RegexpSinglelineJava">
      <metadata name="net.sf.eclipsecs.core.comment" value="MAZ"/>
      <property name="format" value="^[^&quot;]*(catch|for|if|switch|synchronized|while)\s+\("/>
      <property name="message" value="No whitespace after block keyword."/>
      <property name="ignoreComments" value="true"/>
    </module>
    <module name="RegexpSinglelineJava">
      <metadata name="net.sf.eclipsecs.core.comment" value="MAZ"/>
      <property name="format" value="^\s*(&quot;([^&quot;])*(?&lt;!\\)&quot;|[^&quot;\t])+\)\s+\{"/>
      <property name="message" value="No whitespace before '{'."/>
      <property name="ignoreComments" value="true"/>
    </module>
    <module name="RegexpSinglelineJava">
      <metadata name="net.sf.eclipsecs.core.comment" value="MAZ"/>
      <property name="format" value="^\s*(&quot;([^&quot;])*(?&lt;!\\)&quot;|[^&quot;\t])+(\t|  )"/>
      <property name="message" value="No tabs or multiple whitespace beyond indentation."/>
      <property name="ignoreComments" value="true"/>
    </module>
    <module name="AnnotationUseStyle">
      <metadata name="net.sf.eclipsecs.core.comment" value="javac currently does not accept a trailing comma (http://bugs.sun.com/view_bug.do?bug_id=6337964)."/>
      <property name="severity" value="error"/>
      <property name="elementStyle" value="ignore"/>
      <property name="closingParens" value="ignore"/>
    </module>
    <module name="MissingDeprecated"/>
    <module name="MissingOverride"/>
  </module>
  <module name="JavadocPackage">
    <property name="severity" value="ignore"/>
    <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
  </module>
  <module name="NewlineAtEndOfFile">
    <property name="lineSeparator" value="lf"/>
  </module>
  <module name="Translation"/>
  <module name="FileLength">
    <property name="severity" value="ignore"/>
    <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
  </module>
  <module name="FileTabCharacter">
    <property name="severity" value="ignore"/>
    <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
  </module>
  <module name="RegexpSingleline">
    <metadata name="net.sf.eclipsecs.core.comment" value="MAZ"/>
    <property name="format" value="[^\s]+\s+$"/>
    <property name="message" value="Line has trailing whitespace."/>
  </module>
</module>

This configuration is based off of the "Sun Checks" configuration, with a few enhancements, and tweaks for personal preference.

One of the noteworthy settings is "trailingArrayComma" setting under Annotations/Annotation Use Style. Per the Java Language Specification (JLS), a trailing comma in an annotation array should be allowed / ignored. The Eclipse compiler respects this, but attempting to use javac (e.g. as part of a Maven build) with a trailing comma results in a "illegal start of expression" failure, due to an acknowledged bug with javac. Using Checkstyle here helps to ensure consistent, stable builds across compilers.

Additionally, I extended the configuration to perform limited checks on non-Java files as well. First, there is a "exclude from checking" configuration section that includes a "all files types except:" rule that is enabled by default, and set to "java, properties". This filter needs to be expanded to include any file types that should even be considered for processing by Checkstyle. For basic web development, I added "htm, html, js, css, jsp". Next, disable the "Use simple configuration" option, which enables "Advanced - configure file sets to control which files get checked by which configuration". I then configured 3 file sets:

  • Java - Uses the configuration shown above. Includes regular expression matches for "\.java$" and "\.properties$".
  • Web - Provides basic whitespace rules for non-Java files. Includes regular expression matches for each of the "web development" types mentioned above, or "\.(html?|js|jsp|css)$" as a version to accomplish it with one expression.
  • JSP - Provides additional checks specific to the scriptlet tags used in JSP files. Includes a regular expression match for "\.jsp$". Note that the above Web file set is configured to also apply to JSP files.

In each of these configurations are a number of additional regular expression checks, that primarily focus on whitespace rules that aren't currently checked for or configurable by the other Checkstyle rules:

File Set Module Pattern Message
Java RegexpSingleLineJava ^\t* +\t*\S Line has leading space characters; indentation should be performed with tabs only.
Java RegexpSingleLineJava ^\s*("([^"])*(?<!\\)"|[^"\t])+(\t|  ) No tabs or multiple whitespace beyond indentation.
Java RegexpSingleLine [^\s]+\s+$ Line has trailing whitespace.
Java RegexpSingleLineJava ^\s*("([^"])*(?<!\\)"|[^"\t])+\)\s+\{ No whitespace before '{'.
Java RegexpSingleLineJava ^\s*}\s+ No whitespace after '}'.
Java RegexpSingleLineJava ^[^"]*(catch|for|if|switch|synchronized|while)\s+\( No whitespace after block keyword.
Web RegexpSingleLine ^\t* +(?!\*)+\t*\S Line has leading space characters; indentation should be performed with tabs only.
Web RegexpSingleLine ^\s*(?!\*)([^\s"]|"[^"]*(?<!\\)")+(\t|  ) No tabs or multiple whitespace beyond indentation.
Web RegexpSingleLine [^\s]+\s+$ Line has trailing whitespace.
JSP RegexpSingleLine (?<=.+)(?<!%>)<%(?!=|.+;%>) Scriptlet begin tag must be at beginning of line.
JSP RegexpSingleLine <%[^@=](?!;%>)+$ Content after scriptlet begin tag must be on new line.

These rules are not all infallible. They are designed to catch the majority of the targeted issues, but without causing false positives in the process. (For example, ignoring spacing within multi-line comments in JSP files - as Checkstyle's Java interpretation doesn't work in JSP files, and forbidden sequences that are allowed within String constants are a bit tricky to work with.) In order to prevent regressions as the rules are expanded upon, all of these regular expressions are included in a local suite of JUnit tests that I plan to make available at a future date. There are currently 22 tests in 3 test classes (one per file set), executing 33 base comparisons that result in a total of 993 assertions. Recently, some issues with the JSP rules were observed, causing Checkstyle to fail on "minified" JavaScript files that consisted of single lines of 10,000+ characters. Recursion within Java's regular expression support resulted in stack overflow exceptions, until I tweaked the JSP expressions to make use of different zero-width lookahead and lookbehind constructs. A test case on 10,000+ characters in a single line still takes several seconds to execute, but at least it consistently completes without exception now.

After getting the source code and formatting in order (and to not drown developers in too many warnings at once), additional tools can be considered for use, e.g. FindBugs (official site | Wikipedia). While I have nothing against FindBugs, I just haven't seen as urgent of a need for it. Most of the bugs I've seen are either due to things that are already covered by properly-configured errors and warnings in the Eclipse compiler, or are functional type issues (not properly following business requirements) or other higher-level issues that not even FindBugs can do a satisfactory job with detecting - but are things that are better caught during a proper code review process and comprehensive unit testing. Additionally, FindBugs only searches against the compiled Java bytecode, so it can't help report the source code level issues, like Checkstyle.

Sunday, January 2, 2011

LDAP authentication for Samba

As part of my OpenLDAP under Ubuntu Linux project, this post documents configuring Samba to use LDAP - as a storage back-end, as well as for authentication and authorization. (Samba is a free software re-implemenation of the SMB networking protocol, and is useful for providing network file shares that are recognized by Microsoft Windows.) As with my previous posts, this post was written against Ubuntu Linux's latest release, 10.10 ("Maverick Meerkat").

This post is focused on integrating SMBD (Samba's SMB/CIFS server) with LDAP, not with Samba or SMB itself. A few resources for Samba include:

Additionally, there are already a few references specific to combining Samba and LDAP, which proved useful when I first started working on this:

  • "Samba & LDAP" (wiki.samba.org)
  • The Linux Samba-OpenLDAP Howto (Jérôme Tournier & Olivier Lemaire, 2007-07-12, gna.org)
    • This is titled as revision 1.10. I also found references to a revision 1.21, but it actually appears older. There is no "print date", but I've only found it as a PDF on mirrors with a created metadata date of 2006-06-26.

As documented by Samba, Samba supports a few different account / password backends - including Ubuntu's default tdbsam (a "trivial database"), and for the purposes of this post, ldapsam.

First, we need to extend the LDAP schema to include Samba's object classes and their dependencies. Following my notes in OpenLDAP under Ubuntu Linux, the LDAP configuration, by default, including the schema, is stored in LDAP itself (a.k.a. "cn=config"). Unfortunately, documentation and examples using cn=config are still quite rare. This is complicated by the fact that schema additions or changes, just as with data, are now handled through LDIF files. Previously, schema files were available in ".schema" files, which used a custom format. When using cn=config, LDIF files need to be used. At least in Ubuntu, OpenLDAP comes with both versions (.schema and .ldif) for many of the "core" schemas, including "core", "cosine", "inetorgperson", "misc", "nis", and "openldap". However, this list doesn't include the "samba" schema, or other schemas that may be necessary in the future - so they need to be converted.

As discussed in Debian bug # 190162, finding even the samba.schema source file is a bit difficult. The easiest (but somewhat inefficient) way is to install the samba-doc package, then uncompress "/usr/share/doc/samba-doc/examples/LDAP/samba.schema.gz". (Copy the file to a workspace directory, then run "gunzip samba.schema.gz".)

The best process I found for converting individual schema files to LDIF format is as follows (similar to the steps documented in OpenLDAP Server in the Ubuntu Server Guide):

  1. In a new "workspace" directory, create a "schemaConvert.conf" file. Add "include" lines for each schema to be converted, including any dependencies:
    include /etc/ldap/schema/core.schema
    include /etc/ldap/schema/cosine.schema
    include /etc/ldap/schema/nis.schema
    include /etc/ldap/schema/inetorgperson.schema
    
    include samba.schema
    
  2. Use slaptest (part of OpenLDAP), which includes functionality to convert to "config directory" (cn=config) format:
    $ mkdir config && slaptest -f schemaConvert.conf -F config
    configuration file testing succeeded
    
  3. Find the generated .ldif files in 'config/cn=config/cn=schema'.
  4. Copy the necessary file(s) to the root of the workspace directory for convenience and reference:
    cp diff 'config/cn=config/cn=schema/cn={4}samba.ldif' samba.ldif samba.ldif
    
  5. For each needed file, some edits are necessary:
    1. On the 1st line (beginning with "dn:"), remove the index surrounded by curly braces, and append ",cn=schema,cn=config" to the end of the line.
    2. On the 3rd line (beginning with "cn: "), remove the index surrounded by curly braces.
    3. At the end of the file, remove the line beginning with "structrualObjectClass", and any following lines - there should be 7 total.
    Here is a diff showing the necessary edits:
    $ diff 'config/cn=config/cn=schema/cn={4}samba.ldif' samba.ldif
    1c1
    < dn: cn={4}samba
    ---
    > dn: cn=samba,cn=schema,cn=config
    3c3
    < cn: {4}samba
    ---
    > cn: samba
    186,192d185
    < structuralObjectClass: olcSchemaConfig
    < entryUUID: a0d0349c-a737-102f-97c0-cf017cbccf86
    < creatorsName: cn=config
    < createTimestamp: 20101229013514Z
    < entryCSN: 20101229013514.341084Z#000000#000#000000
    < modifiersName: cn=config
    < modifyTimestamp: 20101229013514Z
    

Add the converted and modified samba.ldif using ldapadd, or your favorite LDAP editor e.g. phpLDAPadmin. (This assumes that the "core", "cosine", "inetorgperson", and "nis" schemas were already included in the default OpenLDAP configuration.)

sudo ldapadd -Y EXTERNAL -H ldapi:/// -f samba.ldif

Add some appropriate security controls. This is what my "olcAccess" looked like on my "olcDatabase={1}hdb,cn=config" after the initial setup of OpenLDAP:

{0}to dn.subtree="ou=people,dc=example,dc=com" attrs=userPassword,shadowLastChange by group.exact="cn=ldap.admins,ou=groups,dc=example,dc=com" write by anonymous auth by self write by * none
{1}to attrs=userPassword,shadowLastChange by group.exact="cn=ldap.admins,ou=groups,dc=example,dc=com" write by anonymous auth by * none
{2}to * by group.exact="cn=ldap.admins,ou=groups,dc=example,dc=com" write by users read

I recommend making the following additions and updates, partially inspired by my previous password permissions configuration example:

{0}to * by group.exact="cn=ldap.admins,ou=groups,dc=example,dc=com" write by * break
{1}to dn.one="dc=example,dc=com" filter=(objectClass=sambaDomain) by group.exact="cn=samba.admins,ou=groups,dc=example,dc=com" write by * break
{2}to attrs=@sambaSamAccount,userPassword by group.exact="cn=samba.admins,ou=groups,dc=example,dc=com" write by * break
{3}to dn.subtree="ou=people,dc=example,dc=com" attrs=userPassword by self write by * break
{4}to attrs=userPassword,shadowLastChange,sambaNTPassword,sambaLMPassword,sambaPwdLastSet,sambaPwdMustChange by self read by anonymous auth by * none
{5}to * by users read

The "ldap.admins" and "samba.admins" groups need to be created using the "groupOfNames" objectClass. Add a "samba.proxy" user (use the "applicationProcess" and "simpleSecurityObject" objectClasses), and make it a member of the "samba.admins" group, as well as "ldap.admins" temporarily. (Unfortunately, the above {1} access rule only allows for modification of the sambaDomain, not its creation. For only a one-time use, I haven't yet found a proper access rule that allows for its restricted creation.)

Modify "/etc/samba/smb.conf" to use the "ldapsam" backend. Find the existing "passdb backend = tdbsam", comment it out, then add the following lines:

passdb backend = ldapsam:ldap:///
ldap ssl = Off
ldap suffix = dc=example,dc=com
ldap admin dn = cn=samba.proxy,ou=serviceAccounts,dc=example,dc=com
ldap passwd sync = only
#ldap debug level = 1

For any serious usage, or any usage with real user accounts, "ldap ssl" needs to be properly configured to protect user credentials between the SMBD and LDAP servers.

The above "ldap passwd sync = only", per the ldapsam documentation, means:

Only update the LDAP password and let the LDAP server worry about the other fields. This option is only available on some LDAP servers and only when the LDAP server supports LDAP_EXOP_X_MODIFY_PASSWD.

OpenLDAP supports this, but the SambaNTPasword and SambaLMPassword fields will only be updated with an OpenLDAP overlay such as smbk5pwd. Unfortunately, this overlay is not currently installed with Ubuntu's distribution - see Ubuntu bug # 82853 for details. A few additional notes if building this overlay from source yourself:

  1. By default, this overlay is configured to support both Kerberos and Samba, both at compile-time and run-time. The dependencies for Kerberos are complex and numerous, at least under Ubuntu. If you don't have a need for Kerberos, I'd recommend compiling without Kerberos support. This can be done by running make with a DEFS argument of only "-DDO_SAMBA" from the smbk5pwd source directory. Unfortunately, this currently does not exclude the Kerberos includes or libraries dependencies from the make file. To remedy this, I also set "HEIMDAL_INC" and "HEIMDAL_LIB" to empty strings.
    • If dependencies are met at compile-time, and not at run-time - even if not enabled, slapd will fail to load with "lt_dlopenext failed: (smbk5pwd) file not found" visible in "/var/log/syslog". This is a bit misleading as it isn't "/usr/lib/ldap/smbk5pwd.so" that is missing, but one of its dynamically linked libraries, as viewable with ldd.
  2. While unrelated to Samba, part of the shadowAccount" objectClass is a "shadowLastChange" attribute. While it is apparently seldom or no longer used, if it exists, and if smbk5pwd needs to be compiled anyway, I thought it would be beneficial to have smbk5pwd also reset this attribute along with any password changes. I submitted this request to OpenLDAP, along with a completed patch, a ITS 6550. While this seemed to receive some support from Michael Michael Ströder (one of the developers), it was abruptly closed by Howard Chu. In case anyone else is looking for a similar solution, and in case the patch becomes unavailable from the OpenLDAP tracker for some reason, it is included below:
    Index: contrib/slapd-modules/smbk5pwd/Makefile
    ===================================================================
    RCS file: /repo/OpenLDAP/pkg/ldap/contrib/slapd-modules/smbk5pwd/Makefile,v
    retrieving revision 1.7
    diff -u -r1.7 Makefile
    --- contrib/slapd-modules/smbk5pwd/Makefile 13 Apr 2010 20:17:37 -0000 1.7
    +++ contrib/slapd-modules/smbk5pwd/Makefile 14 May 2010 00:50:46 -0000
    @@ -16,8 +16,8 @@
     OPT=-g -O2
     CC=gcc
     
    -# Omit DO_KRB5 or DO_SAMBA if you don't want to support it.
    -DEFS=-DDO_KRB5 -DDO_SAMBA
    +# Omit DO_KRB5, DO_SAMBA, or DO_SHADOW if you don't want to support it.
    +DEFS=-DDO_KRB5 -DDO_SAMBA -DDO_SHADOW
     
     HEIMDAL_INC=-I/usr/heimdal/include
     SSL_INC=
    Index: contrib/slapd-modules/smbk5pwd/README
    ===================================================================
    RCS file: /repo/OpenLDAP/pkg/ldap/contrib/slapd-modules/smbk5pwd/README,v
    retrieving revision 1.6
    diff -u -r1.6 README
    --- contrib/slapd-modules/smbk5pwd/README 13 Apr 2010 20:17:37 -0000 1.6
    +++ contrib/slapd-modules/smbk5pwd/README 14 May 2010 00:50:46 -0000
    @@ -1,6 +1,6 @@
     This directory contains a slapd overlay, smbk5pwd, that extends the
    -PasswordModify Extended Operation to update Kerberos keys and Samba
    -password hashes for an LDAP user.
    +PasswordModify Extended Operation to update Kerberos keys, Samba
    +password hashes, and the shadowLastChange attribute for an LDAP user.
     
     The Kerberos support is written for Heimdal using its hdb-ldap backend.
     If a PasswordModify is performed on an entry that has the krb5KDCEntry
    @@ -17,6 +17,10 @@
     objectclass, then the sambaLMPassword, sambaNTPassword, and sambaPwdLastSet
     attributes will be updated accordingly.
     
    +The Shadow support updates the shadowLastChange attribute to the current
    +date if a PasswordModify is performed on an entry that has the
    +shadowAccount objectclass.
    +
     To use the overlay, add:
     
      include <path to>/krb5-kdc.schema
    @@ -40,8 +44,8 @@
      smbk5pwd-enable  <module>
     
     can be used to enable only the desired one(s); legal values for <module>
    -are "krb5" and "samba", if they are respectively enabled by defining
    -DO_KRB5 and DO_SAMBA.
    +are "krb5", "samba", and "shadow", if they are respectively enabled by defining
    +DO_KRB5, DO_SAMBA, and DO_SHADOW.
     
     The samba module also supports the
     
    @@ -60,15 +64,16 @@
      olcOverlay: {0}smbk5pwd
      olcSmbK5PwdEnable: krb5
      olcSmbK5PwdEnable: samba
    + olcSmbK5PwdEnable: shadow
      olcSmbK5PwdMustChange: 2592000
     
    -which enables both krb5 and samba modules with a password expiry time
    -of 30 days.
    +which enables all the krb5, samba, and shadow modules with a password
    +expiry time of 30 days.
     
    -The provided Makefile builds both Kerberos and Samba support by default.
    -You must edit the Makefile to insure that the correct include and library
    -paths are used. You can change the DEFS macro if you only want one or the
    -other of Kerberos or Samba support.
    +The provided Makefile builds all of Kerberos, Samba, and Shadow support by
    +default. You must edit the Makefile to insure that the correct include and
    +library paths are used. You can change the DEFS macro if you only want partial
    +support.
     
     This overlay is only set up to be built as a dynamically loaded module.
     On most platforms, in order for the module to be usable, all of the 
    Index: contrib/slapd-modules/smbk5pwd/smbk5pwd.c
    ===================================================================
    RCS file: /repo/OpenLDAP/pkg/ldap/contrib/slapd-modules/smbk5pwd/smbk5pwd.c,v
    retrieving revision 1.34
    diff -u -r1.34 smbk5pwd.c
    --- contrib/slapd-modules/smbk5pwd/smbk5pwd.c 13 Apr 2010 20:17:37 -0000 1.34
    +++ contrib/slapd-modules/smbk5pwd/smbk5pwd.c 14 May 2010 00:50:47 -0000
    @@ -17,6 +17,7 @@
     /* ACKNOWLEDGEMENTS:
      * Support for table-driven configuration added by Pierangelo Masarati.
      * Support for sambaPwdMustChange and sambaPwdCanChange added by Marco D'Ettorre.
    + * Support for shadowLastChange added by Mark A. Ziesemer <www.ziesemer.com>.
      */
     
     #include <portable.h>
    @@ -81,14 +82,21 @@
     static ObjectClass *oc_sambaSamAccount;
     #endif
     
    +#ifdef DO_SAMBA
    +static AttributeDescription *ad_shadowLastChange;
    +static ObjectClass *oc_shadowAccount;
    +#endif
    +
     /* Per-instance configuration information */
     typedef struct smbk5pwd_t {
      unsigned mode;
     #define SMBK5PWD_F_KRB5  (0x1U)
     #define SMBK5PWD_F_SAMBA (0x2U)
    +#define SMBK5PWD_F_SHADOW (0x4U)
     
     #define SMBK5PWD_DO_KRB5(pi) ((pi)->mode & SMBK5PWD_F_KRB5)
     #define SMBK5PWD_DO_SAMBA(pi) ((pi)->mode & SMBK5PWD_F_SAMBA)
    +#define SMBK5PWD_DO_SHADOW(pi) ((pi)->mode & SMBK5PWD_F_SHADOW)
     
     #ifdef DO_KRB5
      /* nothing yet */
    @@ -100,6 +108,10 @@
      /* How many seconds after allowing a password change? */
      time_t  smb_can_change;
     #endif
    +
    +#ifdef DO_SHADOW
    + /* nothing yet */
    +#endif
     } smbk5pwd_t;
     
     static const unsigned SMBK5PWD_F_ALL =
    @@ -110,6 +122,9 @@
     #ifdef DO_SAMBA
      | SMBK5PWD_F_SAMBA
     #endif
    +#ifdef DO_SHADOW
    + | SMBK5PWD_F_SHADOW
    +#endif
     ;
     
     static int smbk5pwd_modules_init( smbk5pwd_t *pi );
    @@ -653,6 +668,34 @@
       }
      }
     #endif /* DO_SAMBA */
    +
    +#ifdef DO_SHADOW
    + /* Shadow stuff */
    + if ( SMBK5PWD_DO_SHADOW( pi ) && is_entry_objectclass(e, oc_shadowAccount, 0 ) ) {
    +  struct berval *keys;
    +  
    +  ml = ch_malloc(sizeof(Modifications));
    +  ml->sml_next = qpw->rs_mods;
    +  qpw->rs_mods = ml;
    +
    +  keys = ch_malloc( 2 * sizeof(struct berval) );
    +  keys[0].bv_val = ch_malloc( LDAP_PVT_INTTYPE_CHARS(long) );
    +  keys[0].bv_len = snprintf(keys[0].bv_val,
    +   LDAP_PVT_INTTYPE_CHARS(long),
    +   "%ld", slap_get_time() / 60 / 60 / 24 );
    +  BER_BVZERO( &keys[1] );
    +  
    +  ml->sml_desc = ad_shadowLastChange;
    +  ml->sml_op = LDAP_MOD_REPLACE;
    +#ifdef SLAP_MOD_INTERNAL
    +  ml->sml_flags = SLAP_MOD_INTERNAL;
    +#endif
    +  ml->sml_numvals = 1;
    +  ml->sml_values = keys;
    +  ml->sml_nvalues = NULL;
    + }
    +#endif /* DO_SHADOW */
    +
      be_entry_release_r( op, e );
      qpw->rs_new.bv_val[qpw->rs_new.bv_len] = term;
     
    @@ -715,6 +758,7 @@
     static slap_verbmasks smbk5pwd_modules[] = {
      { BER_BVC( "krb5" ),  SMBK5PWD_F_KRB5 },
      { BER_BVC( "samba" ),  SMBK5PWD_F_SAMBA },
    + { BER_BVC( "shadow" ),  SMBK5PWD_F_SHADOW },
      { BER_BVNULL,   -1 }
     };
     
    @@ -860,6 +904,16 @@
       }
     #endif /* ! DO_SAMBA */
     
    +#ifndef DO_SHADOW
    +  if ( SMBK5PWD_DO_SHADOW( pi ) ) {
    +   Debug( LDAP_DEBUG_ANY, "%s: smbk5pwd: "
    +    "<%s> module \"%s\" only allowed when compiled with -DDO_SHADOW.\n",
    +    c->log, c->argv[ 0 ], c->argv[ rc ] );
    +   pi->mode = mode;
    +   return 1;
    +  }
    +#endif /* ! DO_SHADOW */
    +
       {
        BackendDB db = *c->be;
     
    @@ -882,66 +936,78 @@
      return rc;
     }
     
    +typedef struct smbk5pwd_verify_schema_t {
    + const char  *name;
    + AttributeDescription **adp;
    +} smbk5pwd_verify_schema_t;
    +
     static int
    -smbk5pwd_modules_init( smbk5pwd_t *pi )
    +smbk5pwd_modules_verify_schema(const char *ocName, ObjectClass **oc, smbk5pwd_verify_schema_t *ad)
     {
    - static struct {
    -  const char  *name;
    -  AttributeDescription **adp;
    + int i, rc;
    + 
    + *oc = oc_find( ocName );
    + if ( !*oc ) {
    +  Debug( LDAP_DEBUG_ANY, "smbk5pwd: "
    +   "unable to find \"%s\" objectClass.\n",
    +   ocName, 0, 0 );
    +  return -1;
      }
    + 
    + for ( i = 0; ad[ i ].name != NULL; i++ ) {
    +  const char *text;
    +
    +  *(ad[ i ].adp) = NULL;
    +
    +  rc = slap_str2ad( ad[ i ].name, ad[ i ].adp, &text );
    +  if ( rc != LDAP_SUCCESS ) {
    +   Debug( LDAP_DEBUG_ANY, "smbk5pwd: "
    +    "unable to find \"%s\" attributeType: %s (%d).\n",
    +    ad[ i ].name, text, rc );
    +   *oc = NULL;
    +   return rc;
    +  }
    + }
    +}
    +
    +static int
    +smbk5pwd_modules_init( smbk5pwd_t *pi )
    +{
    +
     #ifdef DO_KRB5
    - krb5_ad[] = {
    + smbk5pwd_verify_schema_t krb5_ad[] = {
       { "krb5Key",   &ad_krb5Key },
       { "krb5KeyVersionNumber", &ad_krb5KeyVersionNumber },
       { "krb5PrincipalName",  &ad_krb5PrincipalName },
       { "krb5ValidEnd",  &ad_krb5ValidEnd },
       { NULL }
    - },
    + };
     #endif /* DO_KRB5 */
     #ifdef DO_SAMBA
    - samba_ad[] = {
    + smbk5pwd_verify_schema_t samba_ad[] = {
       { "sambaLMPassword",  &ad_sambaLMPassword },
       { "sambaNTPassword",  &ad_sambaNTPassword },
       { "sambaPwdLastSet",  &ad_sambaPwdLastSet },
       { "sambaPwdMustChange",  &ad_sambaPwdMustChange },
       { "sambaPwdCanChange",  &ad_sambaPwdCanChange },
       { NULL }
    - },
    + };
     #endif /* DO_SAMBA */
    - dummy_ad;
    -
    - /* this is to silence the unused var warning */
    - dummy_ad.name = NULL;
    +#ifdef DO_SHADOW
    + smbk5pwd_verify_schema_t shadow_ad[] = {
    +  { "shadowLastChange",  &ad_shadowLastChange },
    +  { NULL }
    + };
    +#endif /* DO_SHADOW */
     
     #ifdef DO_KRB5
      if ( SMBK5PWD_DO_KRB5( pi ) && oc_krb5KDCEntry == NULL ) {
       krb5_error_code ret;
       extern HDB  *_kadm5_s_get_db(void *);
     
    -  int  i, rc;
    -
    -  /* Make sure all of our necessary schema items are loaded */
    -  oc_krb5KDCEntry = oc_find( "krb5KDCEntry" );
    -  if ( !oc_krb5KDCEntry ) {
    -   Debug( LDAP_DEBUG_ANY, "smbk5pwd: "
    -    "unable to find \"krb5KDCEntry\" objectClass.\n",
    -    0, 0, 0 );
    -   return -1;
    -  }
    -
    -  for ( i = 0; krb5_ad[ i ].name != NULL; i++ ) {
    -   const char *text;
    -
    -   *(krb5_ad[ i ].adp) = NULL;
    -
    -   rc = slap_str2ad( krb5_ad[ i ].name, krb5_ad[ i ].adp, &text );
    -   if ( rc != LDAP_SUCCESS ) {
    -    Debug( LDAP_DEBUG_ANY, "smbk5pwd: "
    -     "unable to find \"%s\" attributeType: %s (%d).\n",
    -     krb5_ad[ i ].name, text, rc );
    -    oc_krb5KDCEntry = NULL;
    -    return rc;
    -   }
    +  int rc = smbk5pwd_modules_verify_schema("krb5KDCEntry", &oc_krb5KDCEntry, krb5_ad);
    +  if ( rc != LDAP_SUCCES ) {
    +   return rc;
       }
     
       /* Initialize Kerberos context */
    @@ -980,32 +1046,21 @@
     
     #ifdef DO_SAMBA
      if ( SMBK5PWD_DO_SAMBA( pi ) && oc_sambaSamAccount == NULL ) {
    -  int  i, rc;
    -
    -  oc_sambaSamAccount = oc_find( "sambaSamAccount" );
    -  if ( !oc_sambaSamAccount ) {
    -   Debug( LDAP_DEBUG_ANY, "smbk5pwd: "
    -    "unable to find \"sambaSamAccount\" objectClass.\n",
    -    0, 0, 0 );
    -   return -1;
    +  int rc = smbk5pwd_modules_verify_schema("sambaSamAccount", &oc_sambaSamAccount, samba_ad);
    +  if ( rc != LDAP_SUCCESS ) {
    +   return rc;
       }
    + }
    +#endif /* DO_SAMBA */
     
    -  for ( i = 0; samba_ad[ i ].name != NULL; i++ ) {
    -   const char *text;
    -
    -   *(samba_ad[ i ].adp) = NULL;
    -
    -   rc = slap_str2ad( samba_ad[ i ].name, samba_ad[ i ].adp, &text );
    -   if ( rc != LDAP_SUCCESS ) {
    -    Debug( LDAP_DEBUG_ANY, "smbk5pwd: "
    -     "unable to find \"%s\" attributeType: %s (%d).\n",
    -     samba_ad[ i ].name, text, rc );
    -    oc_sambaSamAccount = NULL;
    -    return rc;
    -   }
    +#ifdef DO_SHADOW
    + if ( SMBK5PWD_DO_SHADOW( pi ) && oc_shadowAccount == NULL ) {
    +  int rc = smbk5pwd_modules_verify_schema("shadowAccount", &oc_shadowAccount, shadow_ad);
    +  if ( rc != LDAP_SUCCESS ) {
    +   return rc;
       }
      }
    -#endif /* DO_SAMBA */
    +#endif /* DO_SHADOW */
     
      return 0;
     }
    Index: contrib/slapd-modules/README
    ===================================================================
    RCS file: /repo/OpenLDAP/pkg/ldap/contrib/slapd-modules/README,v
    retrieving revision 1.6
    diff -u -r1.6 README
    --- contrib/slapd-modules/README 13 Apr 2010 20:17:33 -0000 1.6
    +++ contrib/slapd-modules/README 14 May 2010 00:50:46 -0000
    @@ -49,8 +49,8 @@
      Proxy Authorization compatibility with obsolete internet-draft.
     
     smbk5pwd (overlay)
    - Make the PasswordModify Extended Operation update Kerberos
    - keys and Samba password hashes as well as userPassword.
    + Make the PasswordModify Extended Operation update Kerberos keys,
    + Samba password hashes, and shadowLastChange, as well as userPassword.
     
     trace (overlay)
      Trace overlay invocation.
    
    To compile with the "shadow" patch, and without Kerberos, I used the following command:
    make DEFS="-DDO_SAMBA -DDO_SHADOW" HEIMDAL_INC="" HEIMDAL_LIB=""
    

To provide Samba with the password for the admin DN, use the smbpasswd tool:

$ sudo smbpasswd -W
Setting stored password for "cn=samba.proxy,ou=serviceAccounts,dc=example,dc=com" in secrets.tdb
New SMB password:
Retype new SMB password:

Now, enable SMB for the first user. This will also create a new "sambaDomain" child under the configured suffix:

$ sudo smbpasswd -a mark
New SMB password:
Retype new SMB password:
Added user mark.

Once complete, remove the "samba.proxy" user from the "ldap.admins" group, as the elevated permissions are no longer necessary. Users and the Samba domain can continue to be modified with the remaining permissions.

Finally, for some additional and advised security, add something along the following to "/etc/samba/samba.conf":

security = user
valid users = @samba.users

Per the Samba documentation:

"security = user" is always a good idea. This will require a Unix account in this server for every user accessing the server.

Finally, "valid users" points to a valid Unix group name. This can be anything that is resolved locally, and visible through "getent groups". This group may be declared locally to the same server running Samba, or a posixGroup declared in LDAP, as covered previously in Linux client authentication with LDAP, PAM, and NSS.

Wednesday, December 29, 2010

Linux client authentication with LDAP, PAM, and NSS

As part of my OpenLDAP under Ubuntu Linux project, this post documents configuring Linux client authentication and authorization through LDAP, using Pluggable Authentication Modules (PAM) and Name Service Switch (NSS). As with my previous posts, this post was written against Ubuntu Linux's latest release, 10.10 ("Maverick Meerkat").

As Linux uses numeric IDs for users and groups (in separate domains), I highly recommend creating a numbering strategy to keep things organized. Per existing Debian policy:

  • 9 - 99: Reserved for local system / package accounts, and should be consistent across systems. (For example, 0 is "root".)
  • 100 - 999: Typically for dynamically allocated system accounts.
  • 1,000 - 59,999: Dynamically allocated user accounts.
    • (For the first user created on an Ubuntu system, the user ID and primary group ID are probably both 1,000.)
  • 65,000 - 65,533: Reserved for global allocation by the Debian project.
  • 65,534: The "nobody" user and "nogroup" group.

Before any attempts are considered to organize the available user IDs into blocks, according to the above list, this would leave only 59,000 valid IDs for actual users. This shouldn't be a problem for many organizations, but what if a larger organization needed all of their employees on a large network in the same domain? (There are quite a few companies with over 250,000 employees.) Apparently, the 16-bit limit is only a recommendation, with a current maximum limit of 32 bits. However, 32-bit systems may treat this as signed, allowing for a safe total of 31 bits, or just over 2 billion accounts. So I'm not going to worry about this any further for now, knowing that IDs 65,536 - 2,147,483,648 should be available for future use if necessary.

For now, I'm breaking down the above 1,000+ range as follows:

  • 1,000 - 1,999: Local accounts, not in LDAP.
  • 2,000 - 2,999: Administrative groups, in LDAP.
  • 3,000 - 3,999: Service accounts (users and groups), in LDAP.
  • 4,000 - 4,999: Reserved for future use.
  • 5,000 - 59,999: Actual users.

At a minimum, this offers one significant benefit. As LDAP is a network service, the unfortunate reality is that it will go down on occasion. If all the accounts were in LDAP, there would be no way to access the systems while LDAP is unavailable. Linux can be configured to look for accounts both locally, as well as in LDAP, which will be detailed below. There should be at least one account created locally in the 1,000 - 1,999 range for emergency use independent of LDAP.

Before making any configuration changes, review what is returned by the "getent passwd" and "getent group" commands. Nothing should be visible from LDAP yet, only locally-defined users and groups.

For simplicity, I'd like to install only the NSS support (provided by libnss-ldap) to start with, then review the outputs of getent before continuing with PAM. However, libnss-ldap includes libpam-ldap (for PAM) in its dependency hierarchy, so, I guess we'll just take everything together in one batch:

sudo apt-get install libpam-ldap libnss-ldap 

During configuration of ldap-auth-config, answer the following prompts:

  • LDAP server Uniform Resource Identifier: ldapi:/// (Default)
    • This assumes that the LDAP server is running on the same machine. Any valid LDAP URL can be used here.
    • For any serious usage, or any usage with real user accounts hosted on a seperate LDAP server, this needs to be properly configured to use ldaps:// (SSL/TLS) to protect user credentials between the client and the LDAP server.
    • See below for a current issue with using ldapi:///.
  • Distinguished name of the search base: dc=example,dc=com (Default uses "dc=net")
  • LDAP version to use: 3 (Default)
  • Make local root Database admin: No (Default is Yes)
  • Does the LDAP database require login? Yes (Default is No)
  • Unprivileged database user: cn=ldap.proxy,ou=serviceAccounts,dc=example,dc=com (Default is "cn=proxyuser,dc=example,dc=net")
  • Password for database login account:
  • Local crypt to use when changing passwords: exop (Default is md5)

This added /etc/ldap.conf, which did not exist prior to this configuration. (The closest base reference I can find is /usr/share/doc/libnss-ldap/examples/ldap.conf.gz). To re-run the prompted configuration (if necessary), run "dpkg-reconfigure ldap-auth-config".

Now NSS needs to be configured. Edit /etc/nsswitch.conf to use "files ldap" for the "passwd", "group", and "shadow" databases instead of the existing "compat." This can be done automatically by running "sudo auth-client-config -t nss -p lac_ldap". Re-run the getent commands, and combined results from both the local accounts as well as LDAP should now be visible.

It appears that simply by installing one of the above packages (probably either libpam-ldap or ldap-auth-config), PAM was also already configured with default LDAP extensions. Logins can be attempted by using "su <username>". Attempting to login as a user that does not exist either locally or in LDAP should return a "Unknown id: " error message. Otherwise, a "Password: " prompt should be displayed, and using a password that matches an account configured in LDAP should successfully change to that user. (Alternatively, use "sudo login" for a more comprehensive login and test.)

To see what was configured in PAM for LDAP, here is a diff comparing "/etc/pam.d" before and after this configuration, basically showing the additions using "pam_ldap.so":

diff -ur before/common-account after/common-account
--- before/common-account       2010-12-29 17:14:02.264282406 -0600
+++ after/common-account        2010-12-29 17:14:36.554657351 -0600
@@ -14,7 +14,8 @@
 #

 # here are the per-package modules (the "Primary" block)
-account        [success=1 new_authtok_reqd=done default=ignore]        pam_unix.so
+account        [success=2 new_authtok_reqd=done default=ignore]        pam_unix.so
+account        [success=1 default=ignore]      pam_ldap.so
 # here's the fallback if no module succeeds
 account        requisite                       pam_deny.so
 # prime the stack with a positive return value if there isn't one already;
diff -ur before/common-auth after/common-auth
--- before/common-auth  2010-12-29 17:14:02.264282406 -0600
+++ after/common-auth   2010-12-29 17:14:36.554657351 -0600
@@ -14,7 +14,8 @@
 # pam-auth-update(8) for details.

 # here are the per-package modules (the "Primary" block)
-auth   [success=1 default=ignore]      pam_unix.so nullok_secure
+auth   [success=2 default=ignore]      pam_unix.so nullok_secure
+auth   [success=1 default=ignore]      pam_ldap.so use_first_pass
 # here's the fallback if no module succeeds
 auth   requisite                       pam_deny.so
 # prime the stack with a positive return value if there isn't one already;
diff -ur before/common-password after/common-password
--- before/common-password      2010-12-29 17:14:02.264282406 -0600
+++ after/common-password       2010-12-29 17:14:36.554657351 -0600
@@ -22,7 +22,8 @@
 # pam-auth-update(8) for details.

 # here are the per-package modules (the "Primary" block)
-password       [success=1 default=ignore]      pam_unix.so obscure sha512
+password       [success=2 default=ignore]      pam_unix.so obscure sha512
+password       [success=1 user_unknown=ignore default=die]     pam_ldap.so use_authtok try_first_pass
 # here's the fallback if no module succeeds
 password       requisite                       pam_deny.so
 # prime the stack with a positive return value if there isn't one already;
diff -ur before/common-session after/common-session
--- before/common-session       2010-12-29 17:14:02.264282406 -0600
+++ after/common-session        2010-12-29 17:14:36.554657351 -0600
@@ -22,5 +22,6 @@
 session        required                        pam_permit.so
 # and here are more per-package modules (the "Additional" block)
 session        required        pam_unix.so
+session        optional                        pam_ldap.so
 session        optional                        pam_ck_connector.so nox11
 # end of pam-auth-update config
diff -ur before/common-session-noninteractive after/common-session-noninteractive
--- before/common-session-noninteractive        2010-12-29 17:14:02.264282406 -0600
+++ after/common-session-noninteractive 2010-12-29 17:14:36.554657351 -0600
@@ -22,4 +22,5 @@
 session        required                        pam_permit.so
 # and here are more per-package modules (the "Additional" block)
 session        required        pam_unix.so
+session        optional                        pam_ldap.so
 # end of pam-auth-update config

Here are some additional (manual) modifications, in addition to the above, and described below:

diff -ur after//common-account custom//common-account
--- after//common-account       2010-12-29 17:14:36.554657351 -0600
+++ custom//common-account      2010-12-29 17:17:22.626472292 -0600
@@ -14,8 +14,13 @@
 #

 # here are the per-package modules (the "Primary" block)
-account        [success=2 new_authtok_reqd=done default=ignore]        pam_unix.so
-account        [success=1 default=ignore]      pam_ldap.so
+#account       [success=2 new_authtok_reqd=done default=ignore]        pam_unix.so
+#account       [success=1 default=ignore]      pam_ldap.so
+
+account [success=1 default=ignore] pam_succeed_if.so uid < 2000 quiet
+account [success=2 user_unknown=ignore default=bad]     pam_ldap.so
+account [success=1 new_authtok_reqd=done default=ignore]  pam_unix.so
+
 # here's the fallback if no module succeeds
 account        requisite                       pam_deny.so
 # prime the stack with a positive return value if there isn't one already;
diff -ur after//common-password custom//common-password
--- after//common-password      2010-12-29 17:14:36.554657351 -0600
+++ custom//common-password     2010-12-29 17:17:22.626472292 -0600
@@ -22,8 +22,13 @@
 # pam-auth-update(8) for details.

 # here are the per-package modules (the "Primary" block)
-password       [success=2 default=ignore]      pam_unix.so obscure sha512
-password       [success=1 user_unknown=ignore default=die]     pam_ldap.so use_authtok try_first_pass
+#password      [success=2 default=ignore]      pam_unix.so obscure sha512
+#password      [success=1 user_unknown=ignore default=die]     pam_ldap.so use_authtok try_first_pass
+
+password [success=1 default=ignore] pam_succeed_if.so uid < 2000 quiet
+password [success=2 user_unknown=ignore default=die] pam_ldap.so try_first_pass
+password [success=1 default=ignore] pam_unix.so obscure sha512
+
 # here's the fallback if no module succeeds
 password       requisite                       pam_deny.so
 # prime the stack with a positive return value if there isn't one already;
diff -ur after//common-session custom//common-session
--- after//common-session       2010-12-29 17:14:36.554657351 -0600
+++ custom//common-session      2010-12-29 17:17:22.626472292 -0600
@@ -22,6 +22,7 @@
 session        required                        pam_permit.so
 # and here are more per-package modules (the "Additional" block)
 session        required        pam_unix.so
+session required       pam_mkhomedir.so
 session        optional                        pam_ldap.so
 session        optional                        pam_ck_connector.so nox11
 # end of pam-auth-update config

As mentioned above, and partially inspired by http://wiki.debian.org/LDAP/PAM, the edits in common-account and common-password configure LDAP to only be used for accounts with a UID below 2,000. However, the above approach seems a little cleaner, as pam_succeed_if is used first as the conditional, and then channels the request to either pam_ldap or pam_unix as appropriate, without having to ignore "user_known" return statuses from LDAP for local users. I've tested this and can confirms that it also successfully allows local logins if the LDAP server is unavailable. (I haven't tested the other configuration, but it looks like if LDAP can't return "user_unknown", it will default to a "bad" result, causing local logins to also fail.) The final edit in common-sesson simply adds pam_mkhomedir.so to create home directories at login as necessary.

Two other references that I found helpful were "LDAPClientAuthentication" in the Ubuntu Community Documentation, and Brian White's "LDAP Authentication" LinuxWiki.

Now that everything should be mostly working, there are a few further enhancements and fixes that can be made:

First, it may not be desirable for everyone in LDAP to have access to every LDAP-enabled client machine. /etc/ldap.conf provides several options for limiting which accounts can be used from the current machine. Options include "pam_filter", "pam_check_host_attr", "pam_groupdn", "pam_min_uid", and "pam_max_uid". I find "pam_groupdn" to be the most effective and useful, especially after setting "pam_member_attribute" to "memberUid" to allow reusing posixGroup's.

Next (as mentioned above), is an issue with using ldapi:///. This is really only applicable (possible) when running the LDAP server on the same machine as a client. Basically, "connection_read(): no connection!" warnings are logged when using ldapi:///, which are resolved simply by switching over to ldap:///. This actually appears to be an issue with the libldap client library provided by OpenLDAP itself (2.4.21), and not the slapd daemon. This was reported in full detail as OpenLDAP ITS # 6548. Even after being confirmed on the same ticket with a follow-up by Gerald Turner, the ticket was promptly closed without proper justification, as far as I can understand. This is now open as Ubuntu bug # 594840, where it is marked as confirmed and affecting at least 7 people as of this writing.

Finally, Ubuntu bug # 297408 may prove beneficial for some setups. From Edward Murrell's original bug description:

Binary package hint: libpam-modules

The pam_group module allows assigning groups based on group membership. This is handy, if for example you want to automagically enable various rights on a specific server dependent on a users membership in a given netgroup. The following line in /etc/security/group.conf makes a user part of the admin group on the machine if they are part of the net group developers.

*;*;@developers;Al0000-2400;admin

Unfortunately, this only works for NIS net groups. This isn't much help if your network groups are held in LDAP, which are recognized as normal NSS groups.

Patch is attached which fixes this. NSS groups are recognized by '%'. Ie;

*;*;%someldapgroup;Al0000-2400;admin

Reportedly, this functionality has been included in PAM 1.1.2. Thanks to Edward for his involvement in providing a solution for this! (As of this writing, Ubuntu Maverick is still using version 1.1.1.)

Tuesday, December 28, 2010

LDAP authentication for Apache HTTP Server

As part of my OpenLDAP under Ubuntu Linux project, this post documents configuring the Apache HTTPD Server to use LDAP for authentication and authorization. The Apache HTTPD Server will simply be referred to as "Apache" for the remainder of this post. As with my previous post on phpLDAPadmin, this post was written against Apache 2.2.16, and Ubuntu Linux's latest release, 10.10 ("Maverick Meerkat").

Before configuring Apache to accept credentials, ensure that sensitive pages and the prompts for credentials will only be accepted on SSL/TLS-protected sites, otherwise users' credentials will be passed over the network in clear-text where they may easily be compromised. (It doesn't make much sense to use SSL/TLS encryption between Apache and the LDAP server, if the communication between the web browser and the web server isn't protected.) Documentation for using SSL/TLS encryption for the Apache web server is readily available elsewhere (e.g. the "HTTPS Configuration" section under "HTTPD" in the Ubuntu Server Guide), and is beyond the scope of this post.

At least under Ubuntu Maverick, Apache is installed with the mod_authnz_ldap module, but it is not enabled by default. To enable, use "sudo a2enmod authnz_ldap", or manually create the symbolic links in "/etc/apache2/mods-enabled". (a2enmod is a Debian-specific script shared with Ubuntu.)

Next, add a <Location> directive into the Apache configuration that contains configuration directives as documented in mod_authnz_ldap. Assuming that only one LDAP directory will be used for the Apache instance, I just place most of the configuration under a root Location, like this:

<Location "/">
    AuthBasicProvider ldap
    AuthType Basic
    AuthName "example.com"
    AuthzLDAPAuthoritative off
    AuthLDAPURL "ldaps://ldap.example.com/dc=example,dc=com?uid?sub"
    AuthLDAPBindDN "cn=ldap.proxy,ou=serviceAccounts,dc=example,dc=com"
    AuthLDAPBindPassword "secret123"
</Location>

I need the bind distinguished name (DN) and password, as I have OpenLDAP configured to not allow anonymous binds, which is what Apache 2.2 defaults to if these are not specified. I don't particularly like the need for the proxy account, however, for security and maintenance reasons. I would think that ideally, Apache could just use the username and password of the authenticating user to perform the bind. If the user's bind fails, then access should be denied anyway. However, this would require Apache to know how to convert a username (e.g. "mark") into a DN (e.g. "cn=mark,ou=people,dc=example,dc=com") that is acceptable for the bind. As configured above, Apache currently determines this with a query performed using the specified bind credentials, searching for an LDAP entry where the uid (user identifier) attribute matches the user's entered username. Interestingly, it appears that "binding as the user" will be available in Apache 2.3 (currently in alpha), as seen in the 2.3 documents for AuthLDAPInitialBindAsUser and AuthLDAPInitialBindPattern. However, while sometimes beneficial, this method of translating the username is limited to string matching and substitution only (using regular expressions) as LDAP access is not yet available, so continuing to use a configured bind DN and password will still be more flexible for several scenarios.

Securing the LDAP queries with SSL/TLS is necessary to protect the credentials being passed over the network, as accomplished with the "ldaps://" AuthLDAPURL above. For testing or troubleshooting during initial setup, it can be temporarily replaced with simply "ldap://" to use unencrypted connections. Alternatively, if the LDAP server is operating on the same host as Apache, "ldapi:///" can be used (with the hostname ommitted) to use *NIX sockets, bypassing the IP layer entirely. Interestingly, while I can't find any documentation to support this (only Apache Bug # 44302 which reports that it wasn't working at least as of Apache 2.2.8 on 2008-01-27), "ldapi:///" is working for me as of this writing.

For SSL communications to be successful, the LDAP server's SSL certificate must be trusted by the calling server. This is currently accomplished by providing one or more trusted cerver certificates, and setting LDAPTrustedGlobalCert in the Apache configuration. If multiple certificates must be supported, they can simply be concatenated into a single file when using the Base64 / PEM (CA_BASE64) certificate format. Currently, this must be configured globally per Apache instance, not per directory or location. Alternatively, this can be set globally per server through the TLS_CACERT option in /etc/ldap/ldap.conf. Setting "TLS_REQCERT never" should also work, but this is not recommended for security reasons, nor does it appear to work currently.

Now, just add require directives as appropriate. Available options for LDAP include ldap-user, ldap-group, ldap-dn, ldap-attribute, and ldap-filter. ldap-user should be avoided for reasons of maintainability and security best practices. One of these directives may be placed at the same root <Location> block as above, or in child locations. (Per the Apache Configuration Sections documentation, <Directory> directives should be used over <Location> directives for access control of filesystem-based resources, as improper configurations may otherwise easily result in circumventions.) Either Location or Directory directives will inherit the above-configured Auth* directives from a parent.

For example, to protect phpLDAPadmin as previously configured:

  <Location "/phpldapadmin">
    Options +ExecCGI
    AddHandler fcgid-script .php
    FCGIWrapper /usr/lib/cgi-bin/php5 .php

    require ldap-group cn=ldap-admins,ou=groups,dc=example,dc=com
  </Location>

Finally, reload the Apache configuration ("sudo /etc/init.d/apache2 reload"), and test away!

Monday, December 27, 2010

LDAP web administration with phpLDAPadmin

As part of my OpenLDAP under Ubuntu Linux project, I wanted to find a good web administration tool for the directory. The best option I found was phpLDAPadmin (Wikipedia), a.k.a. PLA.

I meant to complete several other LDAP-related posts since I started this project in April, but other priorities took precedence. However, a side benefit is that my previous configurations have now been successfully repeated and tested under Ubuntu Linux's latest release, 10.10 ("Maverick Meerkat").

Apache Configuration

As phpLDAPadmin is a web application, it depends on a web server that can serve PHPs. My choice is the Apache HTTPD Server (Wikipedia) version 2.2, which will simply be referred to as "Apache" for the remainder of this post.

phpLDAPadmin is available in the Ubuntu package repositories. However, the default configuration (as packaged for Ubuntu) could use some improvement, so I'd recommend against installing it from the repository. I already have an Apache instance running for some other uses, which uses Apache's default worker MPM. However, Ubuntu's packaged version depends on libapache2-mod-php5, which depends on apache2-mpm-prefork, which reconfigures Apache to use the prefork MPM. The differences between these Multi-Processing Modules is documented by Apache in Apache Performance Tuning, and a few other references that I stumbled upon:

Now granted, my uses of Apache and/or LDAP aren't currently anywhere near demanding enough for a significant difference in performance between the MPMs. However, this is a good exercise, and I can be confident that this setup will be able to scale with fewer issues in the future.

My existing Apache installation consisted of the "apache2" Ubuntu (meta)package. To support PLA, the "php5-cgi" and "php5-ldap" packages also need to be installed. (Do not install the base "php5" package, as this also has the same dependencies listed above, and will reconfigure Apache to use the prefork MPM.)

The primary reason for libapache2-mod-php5 depending on the prefork MPM is due to thread-safety issues. As such, a suitable way is necessary to run PHP under Apache in a thread-safe fashion. Standard CGI would be one option, but is terribly inefficient. FastCGI is an option that provides for isolation between PHP (and/or other modules and languages) and Apache, while avoiding the inefficiencies of CGI. Support for FastCGI in Apache was originally available through mod_fastcgi, but is discouraged due to various stability and license compatibility issues. mod_fcgid is the preferred alternative, provides binary compatibility, is reportedly faster, and is available as the libapache2-mod-fcgid package in Ubuntu.

To summarize the above paragraph, also install libapache2-mod-fcgid. The module should be automatically enabled. If not, use "sudo a2enmod fcgid", or manually create the symbolic links in "/etc/apache2/mods-enabled". (a2enmod is a Debian-specific script shared with Ubuntu.)

Download the latest "phpldapadmin-php5" package from http://sourceforge.net/projects/phpldapadmin/files/phpldapadmin-php5/. As of this post, I am using version 1.2.0.5. I extracted the download into "/var/www/phpldapadmin" (the default DocumentRoot). Then add a <Location> directive into the Apache configuration, utilizing fcgid:

  <Location "/phpldapadmin">
    Options +ExecCGI
    AddHandler fcgid-script .php
    FCGIWrapper /usr/lib/cgi-bin/php5 .php
  </Location>

As phpLDAPadmin deals with credentials (including passwords for both administrative login, as well as user accounts), proper measures should be taken to secure this site. It should only be accessible over SSL/TLS (HTTPS). I've also configured Apache to require authorization to access PLA at all - including its login page (detailed later).

phpLDAPadmin Configuration

phpLDAPadmin provides an example configuration in config/config.php.example. Copy it to config/config.php, then customize it from there. If you are running the LDAP server (e.g. OpenLDAP) on the same host, no changes should be required, but setting the server "friendly" name is probably desired at a minimum. Details are available at Installation, Config, Config.php, and LDAP server definitions in the PLA Wiki.

At this point, phpLDAPadmin should be mostly usable. However, as of version 1.2.0.5 of PLA, there are a few quirks that I found and addressed:

  1. One unloadable DN causes others to not expand in the tree, due to a buggy early return from the draw function in lib/HTMLTree.php that skips over draw_javascript() from being called, and ajax_tree.js and layersmenu-browser_detection.js not being loaded into the browser. I reported this in bug # 2997938, and also provided a patch (attached to the bug report). As of 2010-12-27, there has been no acnowledgement or other activity taken by the PLA developers on this bug, so anyone also affected will need to manually apply the patch.

  2. When using phpLDAPadmin over HTTPS (as recommended above), all page resources are properly loaded over HTTPS except the SourceForge logo in the page footer. This causes security warnings in the browser due to "partial encryption". I reported this in bug # 2997703, along with a work-around of using the "remoteurls" option that is mentioned but otherwise undocumented at http://phpldapadmin.sourceforge.net/wiki/index.php/Config.php, as well as a code fix to properly use proper relative URLs. As of 2010-11-06, a fix has been committed for the next (but not yet available) PLA release.
  3. With every attempt to edit an entry, there is a nuisance "Select a template to edit the entry" page, that prompts for a template to use (either "Default", "Generic: Address Book Entry", or "Generic: Posix Group"). This prompt is repeated for repeat edits, with no visible option to save a default template. After reading this forum post on gentoo.org, and extending upon some of the suggestions offered for other issues in the PLA FAQ, I renamed each of the 2 .xml files under "templates/modification" to add a non-.xml suffix, essentially disabling the modification templates. (Simply renaming or removing the directory seemed like it should be a better option, but causes other errors.)
  4. Similar to the above, and more closely covered by the FAQ, even after removing the "modification" templates, there are still "Automatically removed objectClass from template" warnings displayed when first editing an entry in a PLA session. Renaming "mozillaOrgPerson.xml", "courierMailAccount.xml", and "courierMailAlias.xml" under the "templates/creation" folder seems to have removed all of these warnings, at least for my configured LDAP schema. (I'm not sure why the "creation" templates are even being referenced during modification.)

Alternatives

I had looked through a few alternatives before choosing phpLDAPadmin. At least while limiting the search to free and open-source solutions, I really didn't find anything that I think can compare to PLA. However, one cool, but non-web-based alternative I did find is shelldap, which allows for browsing and manipulating an LDAP directory using familiar shell commands like cat, cd, cp, ls, touch, mkdir, move, and vi.