A faster, "complete" Java HttpServletRequest.getRequestUrl() replacement

As another performance-focused method for reuse: It should be easier to obtain the "complete" URL from a HttpServletRequest in Java. This is even a popular question on Stack Overflow: HttpServletRequest to complete URL - though I have various issues with each of the current answers.

Following is a self-written version that I've been using for a number of years now. I previously had posted it as a Gist, and since incorporated it into my MarkUtils-Web library in WebUtils - Checkstyle-approved and JUnit-tested:

MarkUtils-CLI: Annotations (and more) for Apache Commons CLI

So much of Java development in the enterprise often seems to be focused around web applications and other aspects of JEE. Sometimes it is almost comical to watch another developer who has typically been focused on this type of work try to develop a stand-alone Java program. One of the challenges faced here is typically proper acceptance, handling, and validation of command-line arguments.

Fortunately, Apache Commmons CLI exists to help with this effort. The summary from their home page:

The Apache Commons CLI library provides an API for parsing command line options passed to programs. It's also able to print help messages detailing the options available for a command line tool.

Commons CLI supports different types of options:

• POSIX like options (ie. tar -zxvf foo.tar.gz)
• GNU like long options (ie. du --human-readable --max-depth=1)
• Java like properties (ie. java -Djava.awt.headless=true -Djava.net.useSystemProxies=true Foo)
• Short options with value attached (ie. gcc -O2 foo.c)
• long options with single hyphen (ie. ant -projecthelp)

If anything, I feel that the Apache Commons CLI project is selling themselves short. I've found it to be a very comprehensive, well-designed library for effectively parsing the command-line. The only shortcoming I've observed is that the project was developed before Java 5 - and annotations - were available. As such, the library doesn't offer support for any features that annotations have to offer.

Introducing the latest addition to MarkUtils: MarkUtils-CLI is a library that provides an effective bridge between Apache Commons CLI and Java annotations - without replacing the mature Commons CLI library. Originally developed in 2013 with Commons CLI 1.2, the stability of and between both libraries across multiple releases has been a proven success. Like all of the MarkUtils libraries, there are minimal project dependencies: just commons-cli and slf4j-api.

Of the three stages to command line processing, MarkUtils-CLI should most commonly be used to:

• Replace the "Definition Stage" - wrapping it with functionality driven by annotations.
• Leave the "Parsing Stage" intact - while also offering a convenient wrapper to help "glue" everything together into a seamless process.
• Wrap the "Interrogation Stage" - automatically setting the annotated fields or calling the annotated methods as configured with the values received from the command line.

classParser

As with most languages implementing a "main" method, all command-line arguments coming into a Java program are received simply as Strings. Apache Commons CLI doesn't provide any assistance with parsing or converting these String values to the other data types that may be expected or required by the program being developed. MarkUtils-CLI contains a child "classParser" library to effectively bridging this gap - automatically converting one or more String values to the value type being expected, including:

• All primitive types: boolean, char, byte, short, int, long, float, and double.
• Number Object types: Byte, Short, Integer, Long, Float, Double, BigInteger, BigDecimal.
• Direct assignment to the abstract Number type - using BigDecimal unless a number with a radix != 10 is specified.
• Support for alternate number bases for all integral types listed above:
  • Hexadecimal: Prefixes 0x, 0X, 0h, or 0H.
  • Octal: Prefix 0o.
• Boolean
• Character
• String and CharSequence
• java.io.File
• java.net.URI
• java.net.URL
• java.util.UUID
• Identifiers to java.nio.charset.Charset
• Identifiers to java.util.TimeZone
• Enum types, using the enum type's named identifiers.
• Class (using forName)
• Object (using default instantiation of the passed-in class name, similar to Class above - 0-parameter or default constructor required.)
• java.net.InetAddress and java.net.InetSocketAddress, and their respective subtypes - such as java.net.Inet4Address and java.net.Inet6Address.
  • InetSocketAddresses are expected in the form of <InetSocketAddress>:<int port>. For example: "127.0.0.1:80" or "::1:80".

Boolean (either to the primitive or Object forms) conversion builds upon the default Boolean.valueOf logic, which evaluates to true if and only if the value is case-insensitive equals to "true". The conversion accepts the following values:

• true: Y, YES, T, TRUE, 1, -1, X
• false: N, NO, F, FALSE, 0
• Any other values will result in a detailed ClassParseException.

The parsers are split into a few different groups, with each class implementing IClassParser. ClassParserChain simply chains together these groups, and also implements IClassParser itself. Each parser may either return a definitive result (including null), throw an Exception which will bubble-up to the caller, or return IClassParser.NO_RESULT to defer processing to a later parser in the chain. UnhandledParser is a parser that always throws a detailed "Unhandled type" ClassParseException. ClassParserChain defines a default chain that can be obtained by calling getDefault(), and contains all available parsers within the library (those listed above) - ending with UnhandledParser.

If something in the default parse chain doesn't work for you (or if you simply don't like it) - simply create and use your own chain. For simplicity, the default chain could be obtained and prepended to in order to add a custom parser that takes precedence over all the following (default) parsers - or appended in order to handle any conversions that aren't already handled by the former (default) parsers.

Nothing in the "classParser" package is tied to any of the CLI logic - either that of Commons CLI, or the CLI-specific logic within MarkUtils-CLI. As such, this code may be appropriately reused for other purposes beyond parsing of command lines. (If appropriate, this code may be factored into a separate project outside of MarkUtils-CLI, which would then become a dependency of MarkUtils-CLI.)

cli

Now that we have a comprehensive set of code to handle the String-to-type conversions, it's time to put it to work.

The Parameter annotation can be assigned to any field or method - and provides attributes that map to most of the available options in the Commons CLI Option class. If a parameter name is not specifically defined, the base field/method name is used by default - following standard JavaBean conventions. Additional ParameterGroup and ParameterGroups annotations are used to work with the Commons CLI OptionGroup functionality. To eliminate needing to deal with security manager concerns, all fields or methods must be declared public to be accessed by this library. Proper design of the application code - including using separate "plain-old Java objects" ("POJOs") for containing the mapped command-line arguments - should eliminate most concerns with this.

Mappings support 1 or more argument values, acceptable by a suitable Array type as either a field, or an equivalent set method - including allowing for varargs. If the annotated class wishes to add any special processing - such as additional validations, or storing multiple values in a Collection type instead of an Array - simply annotate an appropriate set method instead of a field, and implement accordingly.

Most of the actual "magic" happens within ClassOptions. To allow for future extensibility and customization, all methods here are non-static, so an instance of this class must be created before use (using the default constructor). General usage is as follows:

1. ClassOptionsData get(Class<?>)
   • Pass-in the type of the "config" class, containing the @Parameter annotations. The returned ClassOptionsData instance contains mappings from each Commons CLI Option to the reflection Member for efficiency - especially if multiple executions are to be evaluated from within the same program (sure, this may be rare - but it is accounted for). Use of this composite class also allows for future, internal extensions - hopefully without breaking the public API.
   • This essentially wraps the Commons CLI "Definition Stage".
2. Options getOptions(ClassOptionsData cod)
   • Using the ClassOptionsData obtained above, returns a fresh Commons CLI Options instance - for use by the Commons CLI CommandLineParser - completing the Commons CLI "Parsing Stage".
3. void autoMap(Object target, ClassOptionsData, CommandLine, IClassParser)
   • Given an instance of the actual "config" class, along with the ClassOptionsData, a Commons CLI CommandLine, and an IClassParser - will read the values from the parsed command-line and automatically map the values to the annotated fields / methods in the "config" target.
   • This completed the Commons CLI "Interrogation Stage" - but besides simply providing the necessary details from the command-line, this should hopefully eliminate most of the "boilerplate" coding that would have to be done otherwise.

CliRunner

As part of the cli package, the CliRunner class aims to further simplify the above. While the separate calls to the various ClassOptions methods provide for almost unbounded flexibility, and may still be required in some cases, CliRunner should help further eliminate any remaining "boilerplate" code.

Click here to view the CliRunner Javadoc. All that is required is instantiation using one of the constructors, and calling its run method. As such, the complete code necessary to handle command-line processing - using both Apache Commons CLI and MarkUtils-CLI - can be as simple as:

package com.ziesemer.utils.cli;

import java.util.concurrent.Callable;

public class CliRunnerTestConfigTarget implements Callable<Integer>{
  
  @Parameter
  public int testReturn;

  public Integer call() throws Exception{
    return testReturn;
  }
  
}

package com.ziesemer.utils.cli.examples;

import org.apache.commons.cli.PosixParser;

import com.ziesemer.utils.cli.CliRunner;
import com.ziesemer.utils.cli.CliRunnerTestConfigTarget;

public class CliRunnerExample{

  public static void main(String[] args){
    CliRunner<CliRunnerTestConfigTarget> runner = new CliRunner<CliRunnerTestConfigTarget>(
      new PosixParser(), CliRunnerTestConfigTarget.class);
    CliRunnerTestConfigTarget configTarget = new CliRunnerTestConfigTarget();
    runner.run(configTarget, configTarget, args);
  }

}

Note that here, "configTarget" is being used both as the "configuration" object - as well as the target to be executed. I.E., CliRunnerTestConfigTarget has @Parameter annotations, and implements Callable<Integer>. The Integer result returned by the callable is used as the return code / exit status for the program (assuming everything else executes "normally" - further details below).

There are a significant number of getters and setters provided that can be used to customize the operation of CliRunner. Any unit of significant processing within CliRunner is factored into its own method - allowing for sub-classing and overriding as desired / required. Many of these methods are specifically documented as extension points. Please view the Javadocs (and possibly the source code) for additional details.

As noted above, an additional advantage of using CliRunner is that it can handle the return code / exit status of the program without any additional coding. As noted in the Javadocs, the following codes are used by default (all of which can be overridden):

• If the command line is successfully parsed, the result returned by the target. Otherwise:
  • 1 if the default help was requested.
  • 254 if a ParseException is generated.
  • 255 for any other exception (considered the "worst").

Note that to properly handle the return code / exit status for most cases, System.exit() will be called (by default).

Download

com.ziesemer.utils.cli is available on java.ziesemer.com under the GPL license, complete with source code, a compiled .jar, generated Javadocs, and a suite of 95+ JUnit tests. Please report any bugs or feature requests on the GitHub Issue Tracker.

MarkUtils Refresh, New Hosting

It's been a few years, but I just completed a much-needed refresh to my open source Java libraries ("MarkUtils").

The code is vastly unchanged, and has held up incredibly well over the Java 1.6, 1.7, and 1.8 upgrades (and soon, 1.9) - probably mostly due to 90%+ unit test coverage on everything. The main driving factor here was to get things re-hosted following the shutdown of java.net. In the process, I rebased against against all the latest dependencies and Maven plugins, and issued a coordinated release. Everything is built on the latest version of Java (and only publicly available secure version), 1.8.0_131 as of this writing - but should continue to run fine on older runtimes as documented and as needed for other purposes. Change logs are included in the Maven site reports of each project. Also included are some new projects, as well as some past efforts not previously hosted - neither of which have been blogged about in further detail here yet.

Downloads and additional details here: https://java.ziesemer.com. HTTPS/TLS/SSL, DNSSEC, CDN, and IPv6 support all now included.

As time permits, I'll work through my collection of old blog posts here to update any broken links to java.net - as well as to the legacy downloads across java.net and Google Code. Future efforts may include making the source code available directly on GitHub, as well as an accessible online Maven repository - either as a hosted directory at the above site, or possibly through Bintray (any input is welcome).

How to (not) Validate an Email Address

A common requirement to a software developer often follows the lines of "accept a valid email address". Somewhat appropriately, use of a regular expression is typically considered to fulfill this requirement. Unfortunately, many times this is made more complicated than necessary - and only causes additional issues due to becoming too restrictive.

This scenario is best explained by "I Knew How To Validate An Email Address Until I Read The RFC" (Phil Haack, 2007-08-21, haacked.com). I highly encourage you to read through the entire article, but here are a few significant quotes:

What I found out was surprising. Nearly 100% of regular expressions on the web purporting to validate an email address are too strict.

These are all valid email addresses!

"Abc\@def"@example.com
"Fred Bloggs"@example.com
"Joe\\Blow"@example.com
"Abc@def"@example.com
customer/department=shipping@example.com
$A12345@example.com
!def!xyz%abc@example.com
_somename@example.com

The article goes on to suggest the following regular expression (complete with unit tests!):

^(?!\.)("([^"\r\\]|\\["\r\\])*"|([-a-z0-9!#$%&'*+/=?^_`{|}~] |(?@[a-z0-9][\w\.-]*[a-z0-9]\.[a-z][a-z\.]*[a-z]$

However - this still misrepresents the real issue. There is only one way to validate an email address: Send something containing a cryptographically secure, one-time token to the email address in question, and prompt the user to provide it (either by input back to the web page, and/or by clicking on a link to invoke the validation). If and only if the user is able to complete this step, it can be reasonably assured that the user owns or at least maintains control of the provided email address.

As such, any related requirement should be relaxed to only ensure that whatever is being entered at least "looks" like an email address. This isn't so much for the purpose of ensuring that the address is valid - but that the proper data is being entered into the correct fields, such that an email address is being entered when prompted, instead of a name or a phone number, for example. To accomplish this, the following regular expression is sufficient:

.+@.+\..+

This basically indicates that in order for a given text to match against this pattern, it must contain one or more characters, followed by the '@' sign, followed by one or more characters, followed by a dot ('.'), followed by one or more characters. (Any of the "one or more characters" will also allow for one or more additional '@' signs or dots ('.').)

So for example, the below is an example validation implemented in JavaScript that "makes sense":

var emailValidation = /.+@.+\..+/;
console.log(emailValidation.exec("user@example.com") != null); // Result: "true"
console.log(emailValidation.exec("www.example.com") != null); // Result: "false"

The only exception to this should probably be for a system that needs to validate email addresses for registration into an email system itself. I.E., for registration of an address into a company's webmail system, and it is well-known that this particular system doesn't accept special characters in the email address, for example. Otherwise, why should a given site be coded to reject what otherwise would be a perfectly valid and legitimate email address, just because the local system's requirements don't "like" certain characters or formats? (Hint: If this is the case, the local system's requirements are probably due for review.)

Real-world issues

One of my reminders of this issue, and part of this inspiration of this post, comes from Liferay Portal. In older versions of Liferay, this was the regular expression (implemented within Java code) for validating an email address:

"([\\w-]+\\.)*[\\w-]+@([\\w-]+\\.)+[A-Za-z]+

This incorrectly caused email addresses containing a '+' sign to be rejected - a practice somewhat common with Gmail, for example. As an enterprise Liferay customer, I requested a patch, which replaced the above with the following regular expression to provide support for the '+' sign (and some other things):

([\\w!#%&-/=_`~\\Q.$*+?^{|}\\E]+)*@([\\w-]+\\.)+[A-Za-z]+

Unfortunately, this patch only led to worse issues. At least with the newer version of the pattern, it is a non-optimized regular expression and is subject to catastrophic backtracking. (Please refer to http://www.regular-expressions.info/catastrophic.html and/or http://www.codinghorror.com/blog/2006/01/regex-performance.html for additional details regarding catastrophic backtracking.) In particular, in these situations, each additional character that needs to be backtracked over will cause an exponential growth in required CPU iterations. This expression seems to "fall apart" starting at about 25 characters. This caused us to run into periodic instances (both production and non-production) where our Liferay JVM would completely hang, causing 100% CPU utilization. I captured a stack trace that a provided to them - but the JVM was basically getting stuck on the following call: com.liferay.portal.kernel.util.Validator.isEmailAddress(String).

I proceeded to escalate this as a critical security issue, due to the potential of an easy-to-execute denial-of-service (DOS) attack for anyone running versions of Liferay using this patch or newer versions containing this code.

A minimal example that can be used to reproduce the issue outside of Liferay - but using the same later regular expression shown above:

 Pattern emailAddressPattern = Pattern.compile("([\\w!#%&-/=_`~\\Q.$*+?^{|}\\E]+)*@([\\w-]+\\.)+[A-Za-z]+");
 // "Accidentally" type an email address in with the wrong punctuation.
 Matcher m = emailAddressPattern.matcher("mark.ziesemer.myexamplecompany@com");
 System.out.println(m.matches());

I'm not even exactly sure what this new regular expression was aiming to accomplish, but quite honestly, I believe Liferay went the wrong direction / took the wrong approach with this modification. I also provided them with what I've provided above, including the minimal regular expression. My advice was not immediately accepted. The latest patch uses the following pattern:

[\w!#$%&'*+/=?^_`{|}~-]+(?:\.[\w!#$%&'*+/=?^_`{|}~-]+)*@(?:[\w](?:[\w-]*[\w])?\.)+[\w](?:[\w-]*[\w])?

... which is almost as bad as Phil Haack's example (above) - actually, probably worse, since I doubt it was tested to the same standards as his (if Liferay even has unit tests around this at all...) To their credit, they did file a feature request around this (LPS-30849) - though it doesn't include any of the rationale for its existence, nor has any effort been demonstrated on the ticket yet.

parent-updates-maven-plugin

I've had the privilege of working with Apache Maven (official site) in most of my projects almost since its official release in 2005. Maven is so much more than a build automation tool - providing dependency resolution, project configurations, and automatically-generated project reports / web sites, just to name a few features. It isn't often that I find something that I can't accomplish with Maven - but hopefully this is my chance to make a slight contribution back to the project.

One of the many plugins I've been using for a while now is the versions-maven-plugin - specifically to find and report upon any available dependencies (artifacts used by a project), as well as plugins (artifacts used by the Maven build for the project). There are goals to display the results of these update checks during the build - see "Checking for new dependency updates" and "Checking for new plugin updates". The plugin also provides even further detailed report pages. For examples of these, see one of the plugin's own reports - which shows available dependency updates for the plugin itself when it was last built: "Dependency Updates Report".

However, one notable omission seems to be a goal to display any available updates to the parent POM (parent project) - which isn't reported as part of either the available dependencies or plugins. I had reported this as a feature request back in March as MVERSIONS-181: Versions Plugin should have a "display-parent-updates" goal:

Similar to how we have "display-dependency-updates", "display-plugin-updates", and "display-property-updates" goals, there should also be a "display-parent-updates" goal.

For example, we have a corporate root POM. This POM already runs "display-dependency-updates" and "display-plugin-updates" as part of its and all child builds. However, if the root POM is updated, any child builds aren't reporting that an updated parent POM is available. I believe that a "display-parent-updates" goal would help with this.

There is already the "update-parent" goal - but there isn't any equivalent "report/display only" functionality.

Unfortunately, since then and as of this writing, the request hasn't seen any activity. In the spirit of open source, I decided to see what it would take to implement this myself. The result is my own Maven plugin that adds "display-parent-updates" and "parent-updates-report" goals.

Shown below are the results of these efforts, as demonstrated against a small set of mock test projects. Each test project was chosen to go against a different parent. Choosing 2 versions of the Apache Maven project itself as parent examples seemed appropriate - and produced good results for demonstration purposes. I also included my own parent project - which had no parent updates available at the time, and a very simple project that doesn't even use a parent project. Click through the provided links, and observe the console output, as well as the generated report for each example. Note that the report template changes between projects, as the theme can be controlled by the parent project:

maven-3.0 | maven-RELEASE | MarkUtils-RELEASE | no-parent

[INFO] --- parent-updates-maven-plugin:2012.08.08:display-parent-updates (default) @ maven-3.0 ---
[INFO] 
[WARNING] The parent project has a newer version:
[WARNING]   org.apache.maven:maven .................................. 3.0 -> 3.0.4
[WARNING] The parent project has a newer version:
[WARNING]   org.apache.maven:maven-parent ............................... 15 -> 21
[WARNING] The parent project has a newer version:
[WARNING]   org.apache:apache ............................................ 6 -> 11

[INFO] --- parent-updates-maven-plugin:2012.08.08:display-parent-updates (default) @ maven-RELEASE ---
[INFO] 
[INFO] The parent project is the newest version:
[INFO]   org.apache.maven:maven ......................................... 3.0.4
[INFO] The parent project is the newest version:
[INFO]   org.apache.maven:maven-parent ..................................... 21
[WARNING] The parent project has a newer version:
[WARNING]   org.apache:apache ........................................... 10 -> 11

[INFO] --- parent-updates-maven-plugin:2012.08.08:display-parent-updates (default) @ MarkUtils-RELEASE ---
[INFO] 
[INFO] artifact com.ziesemer.utils:com.ziesemer.utils.pom: checking for updates from central
[INFO] The parent project is the newest version:
[INFO]   com.ziesemer.utils:com.ziesemer.utils.pom ................. 2011.08.05

[INFO] --- parent-updates-maven-plugin:2012.08.08:display-parent-updates (default) @ no-parent ---
[INFO] Project does not have a parent

The sources for these mock test projects are checked-in and distributed with the plugin sources, as the best alternative to proper automated testing that I was able to put together for this project. I had hoped to include a proper suite of JUnit tests - but only found disappointment in the current state of available testing frameworks for Maven plugins - some details of which are covered at http://docs.codehaus.org/display/MAVENUSER/Review+of+Plugin+Testing+Strategies.

Implementation notes: This plugin actually extends the org.codehaus.mojo.versions.DisplayDependencyUpdatesMojo, DependencyUpdatesReport, and AbstractVersionsReportRenderer classes from versions-maven-plugin. For the display and report goals, I would have liked to have just extended their parent classes - AbstractVersionsUpdaterMojo and AbstractVersionsReport, respectively. However, due to the issues described at MPLUGIN-164, the Javadoc annotations are not visible across projects, causing fields on the parent classes to not be set - resulting in NullPointerExceptions, etc. The work-around I'm using is to use the maven-inherit-plugin. Unfortunately, it only support extending the classes actually bound to goals - and not their parent classes. Fortunately, the goals I'm extending were written such that I could access and override the methods needed to achieve the desired functionality. The only real side-effect of this is that there are some extra configuration options that are exposed from the goals I'm extending, that don't serve any purpose or have any impact on the goals I implemented. As the base versions-maven-plugin maintains compatibility with Java 1.4, I ensured that my parent-updates-maven-plugin does so as well. However, hopefully once everyone can agree to drop support for Java 1.4 and have a minimum Java version requirement of 1.5, proper Java 5 (non-Javadoc) annotations can be used to eliminate these issues, as detailed at MPLUGIN-189.

My intent for this release is to provide a temporary work-around for this missing functionality. I hope that my efforts may be included in a future version of the proper versions-maven-plugin - which would also eliminate some of the issues I noted above.

parent-updates-maven-plugin is available on ziesemer.java.net. It is released with a dual license - GNU GPL v3, as well as The Apache Software License, Version 2.0 - to match the base versions-maven-plugin that this project extends. Download the parent-updates-maven-plugin-*-dist.zip distribution from here. Please report any bugs or feature requests on the java.net Issue Tracker.

Improving code and quality with Checkstyle

I've always been picky about the quality my code and the code that I work with. For good reason. It makes the code easier to read and work with. Consistency makes the program flow easier to understand, bugs and other potential issues easier to spot, and difference comparisons between files and versions more effective. Sun Microsystems (original designer of the Java platform, now part of Oracle) seems to agree. From their "Code Conventions for the Java Programming Language":

This Code Conventions for the Java Programming Language document contains the standard conventions that we at Sun follow and recommend that others follow. It covers filenames, file organization, indentation, comments, declarations, statements, white space, naming conventions, programming practices and includes a code example.

• 80% of the lifetime cost of a piece of software goes to maintenance.
• Hardly any software is maintained for its whole life by the original author.
• Code conventions improve the readability of the software, allowing engineers to understand new code more quickly and thoroughly.

For years, I haven't really used any special tools to assist with code formatting and standards, other than the Eclipse Java Formatter for automatically and completely reformatting code as needed, and Eclipse's Compiler Errors/Warnings for pointing out potential issues. One limitation with the compiler errors/warnings in Eclipse is that they really don't include any formatting-specific checks.

I've had experience with Checkstyle in the past (official site | Wikipedia). However, I'm not sure how much thought was put into the configuration of the standards that were in place, and combined with a high level of false positives, most of the developers found ourselves working against the tool rather than working with it. Standards are great, but they need to be thought through, and have buy-in from the project team.

Over the past few months, I rediscovered a need for Checkstyle for the group of developers I was working with. We were doing an satisfactory job addressing the compiler errors and warnings, but the code itself was a sore sight to look at. The code formatting was addressed during code reviews, but without having the formatting issues also appear as warnings in the Eclipse "Problems" view as it was being developed, the formatting issues were quickly "out of sight, out of mind", and quickly forgotten by much of the team. (I'm not sure how the group would have managed with a language like Python, where proper indentation of the code is actually part of the language syntax.)

Fortunately, since I last worked with Checkstyle, the tools have only improved. eclipse-cs now integrates seamlessly with both Eclipse and the Checkstyle XML configurations, and is fully configurable to projects on both local and global levels - using either internal, external, remote, or project-relative configurations.

For my own reference, as well as others to benefit from, I'm sharing the Checkstyle configuration I've worked with over the past few months. The surprising thing is how many inconsistencies were reported and that I was able to fix even in my own code after running the Checkstyle validations.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE module PUBLIC "-//Puppy Crawl//DTD Check Configuration 1.3//EN" "http://www.puppycrawl.com/dtds/configuration_1_3.dtd">

<!--
    This configuration file was written by the eclipse-cs plugin configuration editor
-->
<!--
    Checkstyle-Configuration: MAZ
    Description: none
-->
<module name="Checker">
  <property name="severity" value="warning"/>
  <module name="TreeWalker">
    <property name="tabWidth" value="2"/>
    <module name="JavadocMethod">
      <property name="severity" value="ignore"/>
      <property name="suppressLoadErrors" value="true"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="JavadocType">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="JavadocVariable">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="JavadocStyle">
      <property name="checkFirstSentence" value="false"/>
    </module>
    <module name="ConstantName"/>
    <module name="LocalFinalVariableName"/>
    <module name="LocalVariableName"/>
    <module name="MemberName"/>
    <module name="MethodName"/>
    <module name="PackageName"/>
    <module name="ParameterName"/>
    <module name="StaticVariableName">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="TypeName"/>
    <module name="AvoidStarImport"/>
    <module name="IllegalImport"/>
    <module name="RedundantImport"/>
    <module name="UnusedImports"/>
    <module name="LineLength">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="MethodLength">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="ParameterNumber">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="EmptyForIteratorPad"/>
    <module name="MethodParamPad"/>
    <module name="NoWhitespaceAfter">
      <property name="tokens" value="BNOT,DEC,DOT,INC,LNOT,UNARY_MINUS,UNARY_PLUS"/>
    </module>
    <module name="NoWhitespaceBefore"/>
    <module name="OperatorWrap"/>
    <module name="ParenPad"/>
    <module name="TypecastParenPad"/>
    <module name="WhitespaceAfter">
      <property name="tokens" value="COMMA,SEMI"/>
    </module>
    <module name="WhitespaceAround">
      <property name="tokens" value="BAND,BAND_ASSIGN,BOR,BOR_ASSIGN,BSR,BSR_ASSIGN,BXOR,BXOR_ASSIGN,COLON,DIV,DIV_ASSIGN,EQUAL,GE,GT,LAND,LE,LOR,LT,MINUS,MINUS_ASSIGN,MOD,MOD_ASSIGN,NOT_EQUAL,PLUS,PLUS_ASSIGN,QUESTION,SL,SL_ASSIGN,SR,SR_ASSIGN,STAR,STAR_ASSIGN,TYPE_EXTENSION_AND"/>
    </module>
    <module name="ModifierOrder"/>
    <module name="RedundantModifier"/>
    <module name="AvoidNestedBlocks">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="EmptyBlock">
      <property name="option" value="text"/>
      <property name="tokens" value="LITERAL_DO,LITERAL_ELSE,LITERAL_FINALLY,LITERAL_IF,LITERAL_FOR,LITERAL_TRY,LITERAL_WHILE,STATIC_INIT"/>
    </module>
    <module name="LeftCurly"/>
    <module name="NeedBraces"/>
    <module name="RightCurly"/>
    <module name="AvoidInlineConditionals">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="DoubleCheckedLocking"/>
    <module name="EmptyStatement"/>
    <module name="EqualsHashCode"/>
    <module name="HiddenField">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="IllegalInstantiation"/>
    <module name="InnerAssignment">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="MagicNumber">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="MissingSwitchDefault">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="RedundantThrows">
      <property name="suppressLoadErrors" value="true"/>
    </module>
    <module name="SimplifyBooleanExpression"/>
    <module name="SimplifyBooleanReturn"/>
    <module name="DesignForExtension">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="FinalClass"/>
    <module name="HideUtilityClassConstructor">
      <metadata name="net.sf.eclipsecs.core.comment" value="MAZ: Disabled, until this module provides an option to ignore abstract classes."/>
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="InterfaceIsType"/>
    <module name="VisibilityModifier">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="ArrayTypeStyle"/>
    <module name="FinalParameters">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="TodoComment">
      <property name="severity" value="ignore"/>
      <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
    </module>
    <module name="UpperEll"/>
    <module name="RegexpSinglelineJava">
      <metadata name="net.sf.eclipsecs.core.comment" value="MAZ"/>
      <property name="format" value="^\t* +\t*\S"/>
      <property name="message" value="Line has leading space characters; indentation should be performed with tabs only."/>
      <property name="ignoreComments" value="true"/>
    </module>
    <module name="RegexpSinglelineJava">
      <metadata name="net.sf.eclipsecs.core.comment" value="MAZ"/>
      <property name="format" value="^[^&quot;]*(catch|for|if|switch|synchronized|while)\s+\("/>
      <property name="message" value="No whitespace after block keyword."/>
      <property name="ignoreComments" value="true"/>
    </module>
    <module name="RegexpSinglelineJava">
      <metadata name="net.sf.eclipsecs.core.comment" value="MAZ"/>
      <property name="format" value="^\s*(&quot;([^&quot;])*(?&lt;!\\)&quot;|[^&quot;\t])+\)\s+\{"/>
      <property name="message" value="No whitespace before '{'."/>
      <property name="ignoreComments" value="true"/>
    </module>
    <module name="RegexpSinglelineJava">
      <metadata name="net.sf.eclipsecs.core.comment" value="MAZ"/>
      <property name="format" value="^\s*(&quot;([^&quot;])*(?&lt;!\\)&quot;|[^&quot;\t])+(\t|  )"/>
      <property name="message" value="No tabs or multiple whitespace beyond indentation."/>
      <property name="ignoreComments" value="true"/>
    </module>
    <module name="AnnotationUseStyle">
      <metadata name="net.sf.eclipsecs.core.comment" value="javac currently does not accept a trailing comma (http://bugs.sun.com/view_bug.do?bug_id=6337964)."/>
      <property name="severity" value="error"/>
      <property name="elementStyle" value="ignore"/>
      <property name="closingParens" value="ignore"/>
    </module>
    <module name="MissingDeprecated"/>
    <module name="MissingOverride"/>
  </module>
  <module name="JavadocPackage">
    <property name="severity" value="ignore"/>
    <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
  </module>
  <module name="NewlineAtEndOfFile">
    <property name="lineSeparator" value="lf"/>
  </module>
  <module name="Translation"/>
  <module name="FileLength">
    <property name="severity" value="ignore"/>
    <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
  </module>
  <module name="FileTabCharacter">
    <property name="severity" value="ignore"/>
    <metadata name="net.sf.eclipsecs.core.lastEnabledSeverity" value="inherit"/>
  </module>
  <module name="RegexpSingleline">
    <metadata name="net.sf.eclipsecs.core.comment" value="MAZ"/>
    <property name="format" value="[^\s]+\s+$"/>
    <property name="message" value="Line has trailing whitespace."/>
  </module>
</module>

This configuration is based off of the "Sun Checks" configuration, with a few enhancements, and tweaks for personal preference.

One of the noteworthy settings is "trailingArrayComma" setting under Annotations/Annotation Use Style. Per the Java Language Specification (JLS), a trailing comma in an annotation array should be allowed / ignored. The Eclipse compiler respects this, but attempting to use javac (e.g. as part of a Maven build) with a trailing comma results in a "illegal start of expression" failure, due to an acknowledged bug with javac. Using Checkstyle here helps to ensure consistent, stable builds across compilers.

Additionally, I extended the configuration to perform limited checks on non-Java files as well. First, there is a "exclude from checking" configuration section that includes a "all files types except:" rule that is enabled by default, and set to "java, properties". This filter needs to be expanded to include any file types that should even be considered for processing by Checkstyle. For basic web development, I added "htm, html, js, css, jsp". Next, disable the "Use simple configuration" option, which enables "Advanced - configure file sets to control which files get checked by which configuration". I then configured 3 file sets:

• Java - Uses the configuration shown above. Includes regular expression matches for "\.java$" and "\.properties$".
• Web - Provides basic whitespace rules for non-Java files. Includes regular expression matches for each of the "web development" types mentioned above, or "\.(html?|js|jsp|css)$" as a version to accomplish it with one expression.
• JSP - Provides additional checks specific to the scriptlet tags used in JSP files. Includes a regular expression match for "\.jsp$". Note that the above Web file set is configured to also apply to JSP files.

In each of these configurations are a number of additional regular expression checks, that primarily focus on whitespace rules that aren't currently checked for or configurable by the other Checkstyle rules:

File Set	Module	Pattern	Message
Java	RegexpSingleLineJava	^\t* +\t*\S	Line has leading space characters; indentation should be performed with tabs only.
Java	RegexpSingleLineJava	^\s*("([^"])*(?<!\\)"|[^"\t])+(\t|  )	No tabs or multiple whitespace beyond indentation.
Java	RegexpSingleLine	[^\s]+\s+$	Line has trailing whitespace.
Java	RegexpSingleLineJava	^\s*("([^"])*(?<!\\)"|[^"\t])+\)\s+\{	No whitespace before '{'.
Java	RegexpSingleLineJava	^\s*}\s+	No whitespace after '}'.
Java	RegexpSingleLineJava	^[^"]*(catch|for|if|switch|synchronized|while)\s+\(	No whitespace after block keyword.
Web	RegexpSingleLine	^\t* +(?!\*)+\t*\S	Line has leading space characters; indentation should be performed with tabs only.
Web	RegexpSingleLine	^\s*(?!\*)([^\s"]|"[^"]*(?<!\\)")+(\t|  )	No tabs or multiple whitespace beyond indentation.
Web	RegexpSingleLine	[^\s]+\s+$	Line has trailing whitespace.
JSP	RegexpSingleLine	(?<=.+)(?<!%>)<%(?!=|.+;%>)	Scriptlet begin tag must be at beginning of line.
JSP	RegexpSingleLine	<%[^@=](?!;%>)+$	Content after scriptlet begin tag must be on new line.

These rules are not all infallible. They are designed to catch the majority of the targeted issues, but without causing false positives in the process. (For example, ignoring spacing within multi-line comments in JSP files - as Checkstyle's Java interpretation doesn't work in JSP files, and forbidden sequences that are allowed within String constants are a bit tricky to work with.) In order to prevent regressions as the rules are expanded upon, all of these regular expressions are included in a local suite of JUnit tests that I plan to make available at a future date. There are currently 22 tests in 3 test classes (one per file set), executing 33 base comparisons that result in a total of 993 assertions. Recently, some issues with the JSP rules were observed, causing Checkstyle to fail on "minified" JavaScript files that consisted of single lines of 10,000+ characters. Recursion within Java's regular expression support resulted in stack overflow exceptions, until I tweaked the JSP expressions to make use of different zero-width lookahead and lookbehind constructs. A test case on 10,000+ characters in a single line still takes several seconds to execute, but at least it consistently completes without exception now.

After getting the source code and formatting in order (and to not drown developers in too many warnings at once), additional tools can be considered for use, e.g. FindBugs (official site | Wikipedia). While I have nothing against FindBugs, I just haven't seen as urgent of a need for it. Most of the bugs I've seen are either due to things that are already covered by properly-configured errors and warnings in the Eclipse compiler, or are functional type issues (not properly following business requirements) or other higher-level issues that not even FindBugs can do a satisfactory job with detecting - but are things that are better caught during a proper code review process and comprehensive unit testing. Additionally, FindBugs only searches against the compiled Java bytecode, so it can't help report the source code level issues, like Checkstyle.

MarkUtils-JMX

Following my recent post on JMX Secure Connections / Avoiding Java System Properties, I am making another addition to MarkUtils: MarkUtils-JMX.

JMX Management Bean Metadata

My primary inspiration for this library was that JMX provides a generous amount of metadata along with each management bean, attribute, operation, and parameter - including names, descriptions, and impacts (INFO, ACTION_INFO, ACTION, or UNKNOWN) for operations. Parameter names must be provided through metadata, otherwise only the generated defaults of p0, p1, etc. are displayed. Unfortunately, associating this data with a management bean is currently a pain, and the only support available for this through the JDK itself is by subclassing StandardMBean, or completely instantiating a MBeanInfo yourself, and returning from a subclass of StandardMBean or from a custom DynamicMBean implementation.

MarkUtils-JMX provides a MBeanInfoBuilder class that serves as an alternative to the non-public classes used to create MBeanInfo's within the JDK. The most significant feature of MBeanInfoBuilder is support for including names, descriptions, and impacts that are read from Java annotations. The supported annotations are provided as part of this library in the com.ziesemer.utils.jmx.beanInfo package. Similar functionality is planned for release with Java 7 as part of JMX 2.0 / JSR 255, as detailed by Eamonn McManus in Playing with the JMX 2.0 API (2008-08-06, weblogs.java.net). Unfortunately, while I'm sure the annotations will not be compatible with my own (even just due to mine being in a com.ziesemer package), mine are available immediately, and support both Java 1.5 / 5.0 and Java 1.6 / 6.

MBeanInfoBuilder is not intended to be a complete replacement, as it currently doesn't support constructors, notifications, and descriptors, mostly as I currently have no use for them. However, these limitations are almost completely mitigated by MBeanInfoCombiner, which allows the metadata from a custom MBeanInfo built by MBeanInfoBuilder to supplement the default MBeanInfo built by the JDK's StandardMBean - effectively adding support for the custom annotations.

SimpleMBean, also included, is a alternative to the JDK's StandardMBean (as with MBeanInfoBuilder, and doesn't depend upon non-public classes. Along with utilizing MBeanInfoBuilder by default, this allows for easy extension / customization of the implementation. Additionally, both MBeanInfoBuilder and SimpleMBean are designed and written with performance as a primary focus. Performance is more of a concern for the actual MBean implementation than the building of the MBeanInfo, as the information should only have to be built once, where as the MBean implementation will likely be called repeatedly and often throughout the lifetime of the hosting application.

Authentication and Authorization

Also included in this library are classes to handle most of the work around securing JMX access by providing authentication and authorization.

Authentication is the simpler half, and is provided by com.ziesemer.utils.jmx.server.authentication.BaseJmxAuthenticator. This abstract class handles all the validation and breaking-down of the Object input parameter. An implementing class must only implement Subject authenticate(String username, String password). The authentication implementation is then registered as a value to the JMXConnectorServer.AUTHENTICATOR property, and passed-in as part of the environment map to JMXConnectorServerFactory.newJMXConnectorServer.

Authorization is a little more complex. Limited support is built-in to the JDK through Java's security policy and MBeanPermission. Unfortunately, the JDK approach seems overly-complex and did not meet several of my requirements. This library provides a high-performance and flexible alternative, BaseJmxAuthorizer.

BaseJmxAuthorizer and the other classes in com.ziesemer.utils.jmx.server.authorization are based on "Use Case 2" in Luis-Miguel Alventosa's blog post, Authentication and Authorization in JMX RMI connectors (2006-09-25, blogs.sun.com), where an InvocationHandler is used to selectively proxy requests to the management bean. Again, BaseJmxAuthorizer was designed and written with performance as a primary focus, along with flexibility. It is based on a Map (implemented with a HashMap), associating incoming method names (e.g. getAttribute or invoke) with an associated handler. This Map is publicly exposed, allowing for easy customization through modifications made to the map. Several default handlers, including the simple and unconditional AllowHandler and DenyHandler, are included in the package.

A ReadOnlyInvokeHandler is also provided, and allows all calls to operations / methods that are considered to be "read-only". The request impact (MBeanOperationInfo.getImpact()) is first consulted, and the request is allowed if INFO is returned. The request is otherwise denied, unless the impact returned is UNKNOWN, in which case an attempt to determine if the method has a "read-only" name is made, which checks for if the method name starts with "get", "is", "list", "query", or is equal to "hashCode". By default, it also allows for calls to ThreadMXBean.dumpAllThreads. These allowances allow for full and easy use of all "read-only" functionality provided by JConsole by default, without compromising on security.

Unfortunately, under Java 1.5 / 5.0, this currently denies access to get* methods on ThreadMXBean and LoggingMXBean, as these operations are marked as ACTION_INFO rather than UNKNOWN (or more preferably / correctly, INFO). This appears to have been somewhat fixed in Java 1.6 / 6, as per Sun Bug 6320104, it seems that UNKNOWN was simply not supported by the infrastructure in 1.5 / 5.0 at the time. A ReadOnlyJavaFixInvokeHandler is provided in the package as a work-around for use under Java 1.5 / 5.0. I reported Sun Bug 6933325 to address that these operations should now return INFO instead of ACTION_INFO, now that it is possible. Please view the Javadocs and/or source for additional details.

BaseJmxAuthorizer handles basic logging of method invocations through SLF4J by default.

This library's BaseJmxAuthorizer also simplifies authentication and improves performance by associating permissions with a subclass of JMXPrincipal, JmxPermissionPrincipal. This allows permissions to be obtained for a user once during authorization, without having to re-check permissions on each JMX call. Just be sure that this caching is accounted for, such that a user doesn't maintain unattended access if / once a permission is removed by the underlying security directory. A few options could be implementing a session timeout, or adding a call to a customized implementation to also remove the permissions from the runtime if / once removed from the user.

JmxPermissionPrincipal maintains an instance of a subclass of PermissionCollection, JmxPermissions, which contains one or more JmxPermission (subclass of Permission instances. Default provided JmxPermission's are CONNECT, READ_EX, INVOKE, and ADMIN. Other subclasses may be created and used, but should be kept as singletons. Please view the Javadocs and/or source for additional details.

Packaging with Apache Maven, Java versions

As with all my other MarkUtils libraries, MarkUtils-JMX is configured, compiled, tested, and packaged using Apache Maven. This library posed a little bit of a challenge, however, as I wanted to provide 2 versions, one built for Java 1.5 / 5.0, and one for Java 1.6 / 6. This is one of the few times I wish that Java had standard support for conditional compilation directives, such as those supported by the C preprocessor.

The primary reason for requiring dual versions is for Java 6's support of MXBean's, which were not supported in Java 5. This includes use of the additional 2-arg constructor added to StandardMBean, which as far as I can see, is impossible to implement without being available at compile-time - even if considering reflection tricks.

The solution I decided on for now is to offer dual packages / builds: com.ziesemer.utils.jmx.java5 and com.ziesemer.utils.jmx.java6. The Java 6 version includes the source paths from the Java 5 version. To clarify, only the Java 5 or the Java 6 version is required, as the Java 6 version is a superset of the Java 5 version. The Java 5 version includes a StandardMBeanInfoCombiner5 class that is a subclass of StandardMBean and utilizes MBeanInfoBuilder and MBeanInfoCombiner. The Java 6 version includes a StandardMBeanInfoCombiner6, which is the same as the Java 5 version, but exposes the added constructor and allows use of MXBeans. This setup actually worked with Maven fairly well, including the ability to automatically run Maven builds against both versions in one operation, and including packaging the build outputs together into the final distribution.

The Java 6 version also includes a DefaultRegistration class, which registers a few additional custom MXBeans that I found helpful and that provide functionality not currently offered by the MXBeans provided by the JDK (ManagementFactory). These include my custom CharsetMXBean, SecurityMXBean, and SystemMXBean. (As these custom beans themselves don't require any Java 6-specific features or calls at compile time, they are actually included in the Java 5 package. They just require Java 6 to be registered as MXBeans rather than regular MBeans.)

Download

com.ziesemer.utils.jmx is available on ziesemer.dev.java.net under the GPL license, complete with source code, a compiled .jar, generated JavaDocs, and a suite of 40+ JUnit tests. Download the com.ziesemer.utils.jmx-*.zip distribution from here. Please report any bugs or feature requests on the java.net Issue Tracker.

JMX Secure Connections / Avoiding Java System Properties, RMI

I spent much of my weekend working on adding support for Java Management Extensions (JMX) into a large enterprise application. Security was appropriately a primary concern, and I needed to ensure that all connections were properly encrypted. The most significant observation I've made during this work is that Java system properties are often overly depended upon / misused. Dependencies within RMI are a prime example of where the use of system properties cause some severe limitations, and are an area that probably could certainly use some improvement.

System properties are global to a JVM. Especially in a large application, conflicts can quickly arise if alternate configuration methods aren't available. For example, a necessary configuration may require one section of code or a referenced library to have a given system property set to one value, while another section or library requires the same system property set to another value. This is possible if they are split into separate programs, running on separate JVMs, but not within the same JVM. System properties can be set and changed at runtime by calling System.setProperty(…), but this should not be taken lightly and should usually be avoided. When they do need to be set outside of the java command-line, system properties should only be set within an applications "main" method, or other top-level code. I previously had to fix an issue where a JSP was switching the "javax.xml.transform.TransformerFactory" value between the interpretive and compiled (XSLTC) versions, which caused interesting issues (a.k.a. failures) elsewhere throughout the application, as the switch was causing different processors to be used for various functions, depending upon the timing between calls to the JSP (review: concurrency, thread safety). The same primary issue is shared with environment variables, as they are also globally shared by the JVM (or any process), However, unlike Java's system properties, the environment variables of the current runtime are not modifiable by Java.

Specific to my work were the Java Secure Socket Extension (JSSE) customizations, particularly the "javax.net.ssl.keyStore*" and "javax.net.ssl.trustStore*" system properties. Note that these are referred to as "defaults", with some default values provided for the default parameters, meaning that there should be a way to use a non-default value when needed. Another limitation of these specific customizations is that they are only single-valued, with no way to provide support for multiple key stores, etc., short of providing an overridden implementation class, which is full of issues in itself. Especially in a large enterprise application, calls need to be made to different services that require different certificates, and especially with limitations around automatic certificate selection, there needs to be a way to hook into this through flexible code when required.

HttpsURLConnection is an excellent example of a class that provides exactly this. In addition to the static (JVM-global) setDefaultSSLSocketFactory(…) method, it also provides a setSSLSocketFactory(…) method that can be used to provide customized SSL socket factories on a per-connection basis.

Unfortunately, JMX and RMI currently provide no such hooks, relying exclusively on system properties or the default socket factory. In the case of RMI, things only get more interesting and complicated. Everything exposed through the RMI public API is protocol-generic, with no concept of TCP, IP addresses, or port numbers. These are only handled by internal UnicastRef, LiveRef, and eventually TCPEndpoint classes. Only a "stub" is communicated to the client, through a registry or other means, and this stub contains an optional, serilaized RMICClientSocketFactory for creating a connection from the client to the host. That's right - the server controls the socket factory that will be used by the client to connect back to the server. I haven't found any clean way to override this behavior at the client. For JMX over RMI, both the server and client factories are set through an environment map with property keys defined on RMIConnectorServer. (RMI is the only JMX connector included with Java 5 and 6.)

For the application I was working on, changing the system properties to control the certificates used for secure communications is simply not feasible, if even an option at all. So I created an alternate SslRMIClientSocketFactory implementation, as even mentioned in the JDK's source code:

// If someone needs to have different SslRMIClientSocketFactory factories
// with different underlying SSLSocketFactory objects using different key
// and trust stores, he can always do so by subclassing this class and
// overriding createSocket(String host, int port).

This alternate implementation returned sockets from a socket factory on a custom-initialized SSLContext. This worked great when connecting between different instances of the same application (different JVMs on the same node, as well as different nodes). However, this requires the alternate class (and any and all other references classes) to be available on the classpath of any client making the connection - making things difficult for connections from other standard clients such as JConsole or VisualVM. It is possible, however, by setting the classpath with the "-J-Djava.class.path=…" arguments, which work the same for both JConsole and VisualVM. Both these programs utilize native launchers, so the "-J" prefix means to pass the trailing argument to the JVM and not the native launcher itself. When doing this, "<java.home>/lib/jconsole.jar" must be included as well for JConsole, or JConsole won't even start. tools.jar is also necessary for connecting to local processes and possibly other features, but apparently isn't required for remote connections like those being attempted here.

The solution I'm proceeding with is two-fold. First, I'm registering two JMXConnectorServers per JVM - one that uses the standard SslRMIClientSocketFactory, and one that uses a customized factory class. The first can be used by any client, assuming that the certificates are valid in the default trust store, or that the proper references using system properties can be made. The second can be used by any client that has the customized class available on the classpath, including other instances of the application itself. Fortunately, this does not require any additional network ports to be kept open. Each instance shares the same port, with a non-visible (at least not publicly or easily) ObjID being used to distinguish between them - which is also included in the serialized connection stub used by the client.

For my customized SslRMIClientSocketFactory class, simplest is best. By using only classes native to the JDK, only the one class is necessary to be available on the client's classpath so that it can be deserialized from the RMI stub. To avoid the issues with global system properties as described above, it also needed to be customizable, ideally being able to provide alternate socket factory implementations from within the client. Unfortunately, even if this class had a setSocketFactory method available, the rest of the RMI and JMX API doesn't provide for any apparent opportunity to call such a method. While a bit of a hack, my solution was to use a ThreadLocal. Here is my entire class:

import java.io.IOException;
import java.io.Serializable;
import java.net.Socket;
import java.rmi.server.RMIClientSocketFactory;

import javax.rmi.ssl.SslRMIClientSocketFactory;

public class ThreadLocalSslRmiClientSocketFactory
    implements RMIClientSocketFactory, Serializable{
  
  private static final long serialVersionUID = 1L;
  
  public static final ThreadLocal<RMIClientSocketFactory> SOCKET_FACTORY
      = new InheritableThreadLocal<RMIClientSocketFactory>(){
    @Override
    protected RMIClientSocketFactory initialValue(){
      return new SslRMIClientSocketFactory();
    }
  };
  
  public Socket createSocket(String host, int port) throws IOException{
    return SOCKET_FACTORY.get().createSocket(host, port);
  }
  
}

This allows for the socket factory to be configured by calling ThreadLocalSslRmiClientSocketFactory.SOCKET_FACTORY.set(…) somewhere within the current thread prior to making the connection, and without having requirements on or otherwise impacting the rest of the application. If this customization is not called, then the default SslRMIClientSocketFactory is used, and the global system properties should be referenced to determine the trust store, etc. Starting the servers then looks like this:

import java.lang.management.ManagementFactory;
import java.util.HashMap;
import java.util.Map;

import javax.management.MBeanServer;
import javax.management.remote.JMXConnectorServer;
import javax.management.remote.JMXConnectorServerFactory;
import javax.management.remote.JMXServiceURL;
import javax.management.remote.rmi.RMIConnectorServer;
import javax.rmi.ssl.SslRMIClientSocketFactory;

import org.slf4j.Logger;

// …

MBeanServer mbs = ManagementFactory.getPlatformMBeanServer();
JMXServiceURL url = new JMXServiceURL("rmi", null, 0);

Map<String, ? super Object> serverEnv = new HashMap<String, Object>();
serverEnv.put(
  RMIConnectorServer.RMI_SERVER_SOCKET_FACTORY_ATTRIBUTE,
  new JmxSslRmiServerSocketFactory());
serverEnv.put(
  RMIConnectorServer.RMI_CLIENT_SOCKET_FACTORY_ATTRIBUTE,
  new SslRMIClientSocketFactory());

JMXConnectorServer sslConnector = JMXConnectorServerFactory.newJMXConnectorServer(url, serverEnv, mbs);
sslConnector.start();
LOGGER.info("JMX SSL server started: {}", sslConnector.getAddress());

serverEnv.put(
  RMIConnectorServer.RMI_CLIENT_SOCKET_FACTORY_ATTRIBUTE,
  new ThreadLocalSslRmiClientSocketFactory());

JMXConnectorServer threadLocalSslConnector = JMXConnectorServerFactory.newJMXConnectorServer(url, serverEnv, mbs);
threadLocalSslConnector.start();
LOGGER.info("JMX thread-local SSL server started: {}", threadLocalSslConnector.getAddress())

What about other connectors? As listed in the connector chapter of the JMX overview documentation:

An optional part of the JMX Remote API, which is not included in the Java SE platform, is a generic connector. This connector can be configured by adding pluggable modules to define the following:

• The transport protocol used to send requests from the client to the server, and to send responses and notifications from the server to the clients
• The object wrapping for objects that are sent from the client to the server and whose class loader can depend on the target MBean

The JMX Messaging Protocol (JMXMP) connector is a configuration of the generic connector where the transport protocol is based on TCP and the object wrapping is native Java serialization. Security is more advanced than for the RMI connector. Security is based on the Java Secure Socket Extension (JSSE), the Java Authentication and Authorization Service (JAAS), and the Simple Authentication and Security Layer (SASL).

The generic connector and its JMXMP configuration are optional, which means that they are not always included in an implementation of the JMX Remote API. The Java SE platform does not include the optional generic connector.

There is also a Web Services (WS) connector being worked on, as described in JSR 262: Web Services Connector for Java Management Extensions (JMX) Agents and the reference implementation project at https://ws-jmx-connector.dev.java.net/. I've read that the WS connector is planned to be included with Java 7. Fortunately, it appears that the WS connector supports a custom SSLContext, configurable on the client, as shown in Securing JMX Web Services Connector (Jean-Francois Denise, 2007-08-16, blogs.sun.com).

Other related JMX references that have already proved useful:

• Sun Developer Network's Java Management Extensions Tutorial
• Luis-Miguel Alventosa's Blog on blogs.sun.com:
  • Mimicking the out-of-the-box JMX management agent in J2SE 5.0 (2006-06-02)
  • Authentication and Authorization in JMX RMI connectors (2006-09-25)
• Chapter 3. JSR 160 (JMX Remoting) Explained in the MX4J documentation.
• I avoided using an RMI registry completely here, but if I had, Eamonn McManus's Blog on java.net:
  • Securing the RMI registry (2006-12-29)
  • Reimplementing the RMI protocol (2007-01-11)

MarkUtils-IO: Performant Java Streams, Readers, and Writers

MarkUtils-IO is another high-performance addition to MarkUtils, and is a collection of utility classes that I've found myself frequently reusing over the past number of years.

• AppendableWriter: A faster alternative to StringWriter. Uses a non-synchronized StringBuilder by default instead of the synchronized StringBuffer, but also accepts any Appendable as the underlying destination.
• CharSequenceReader: An alternative to StringReader, but supports reading from any CharSequence in addition to String. Optimizations included for String, StringBuilder, StringBuffer, and CharBuffer.
• NoFlushBufferedOutputStream, NoFlushBufferedWriter: Fixes / work-arounds for https://issues.apache.org/jira/browse/XALANJ-2500. Overrides flush() to do nothing, unless called as flush(boolean) with true. (Previously discussed: Xalan-J Serialization Performance hindered by Flushing.)
• OutputStreamSplitter, WriterSplitter: Writes to zero or more OuputStreams or Writers as one.
• RandomAccessFileInputStream, RandomAccessFileOutputStream: Provides an InputStream or an OutputStream from a RandomAccessFile.

Work on this library was also the driving factor for another recent post concerning some of Java's built-in code, Redundant argument validation code in Java IO classes.

com.ziesemer.utils.io is available on ziesemer.java.net under the GPL license, complete with source code, a compiled .jar, generated JavaDocs, and a suite of 40+ JUnit tests. Download the com.ziesemer.utils.io-*.zip distribution from here. Please report any bugs or feature requests on the java.net Issue Tracker.