Java and Closures

A lot of people in the Java development community were hoping that closures would be coming to the Java langauge in the next major release, but with each passing day it's looking less likely that Java 7 will have closures included alongside it's many other improvements and bugfixes. And, based on some of the rumblings in the community, it seems more than a few developers aren't very happy about it. Some are even starting to whisper that dreaded "Java's Dead" meme, believing that Java development cannot last without closures as a part of the language.

Of course, they forget about the fact that Java's been doing just fine without this feature for over a decade....but anyway....

Before developers start reaching for their shovels and heading out to bury Java once and for all, I think it's a good idea to really sit back and consider for a moment what's likely holding back the inclusion of closures in the Java language. There is no doubt that getting closures into the Java language has been slower than a lot of people would have liked, but overall this is probably a good thing. It gives everyone involved a chance to really think through the options and ensure that closures are done right.

What The $%&! is a Closure?!?!?
If your programming experience hasn't ventured beyond Java, you might not have a clear understanding of what all of the fuss is about.

When programmers talk about closures, they're usually talking about two features of a programming language together.

First is the ability to create anonymous functions in-line. Think Java's anonymous inner classes only more flexible, and without that annoying need to define a class to contain the function you're creating. In fact, the proliferation of lots of annoying little anonymous inner classes in Java code (Swing, anyone?) is often used as an argument in favor of having closures in the langauge. Being able to declare a small, in-line function would streamline a lot of Java code and make a whole lot of developers' lives happier (and we all like happy developers, don't we?).

Second is the actual "closure" involved with closures: sometimes called variable capture. This is a little more complicated concept to wrap your head around, but it comes down to being able to retain references to variables that are accessed (but not declared) within a closure function, but which have actually gone out of scope.

For example, in Javascript (or ECMAScript) you can do something like this:

function foo() {
  var x = 2;
 return function() { return x*x; };
}

var f = foo();
print(f()); // "4"

In the above code, x has already gone out of scope by the time f (which is a reference to the function returned by foo) is called, yet the value of x is still retained.

Now you're probably asking: "What the hell would I need this for?"

Well, a lot of stuff. I won't go into iterating all the ways that closures can be used, because that would take all day and my poor fingers get enough of a workout from banging out code all day. Let's just say that closures provide ways to create vastly simpler code over what the Java language currently offers for similar functionality. And the fact is Java currently can't even do all of what a proper closure implementation can in languages such as...say...Ruby, for example.

The bottom line is that closures are an immensely useful feature to have in any programming language.

Sticking Point
The problem with the closure implementations that have been proposed for Java so far is that they don't (or can't) address the key feature that you typcially find in languages that support closures: Functions as first-class entities of the langauge.

When I say "first-class entity", I'm using the SICP definition¹ of what constitutes some element of a language as having first-class status:

• They may be named by variables.
• They may be passed as arguments to procedures.
• They may be returned as the results of procedures.
• They may be included in data structures.
  

Basically, if it can be assigned to a variable, passed as a function parameter, returned from a function or be part of some data structure it's a first-class entity in the language.

So, having functions as first-class entities in the languages means that you can declare them on their own and assign them to variables. This is something that you cannot do in Java.

Java is strictly an object (or, if you prefer, class) based language: When writing a program, nothing can exist outside the context of an class definition. As far as the language is concerned, everything must be an object.

So, you can't do something like this:

public String someFunction(int x, int y) {
 //...do something...
}

Instead, you have to do this:

public class SomeClass {

  public String someFunction(int x, int y) {
     //...do something...
  }

}

And you can't assign a function (or method) to it's own variable and call it, like you would with something like JavaScript:

var something = function(x) { return x*x };

var result = something(10);

Now, technically you can do something similar if you use reflection in Java. You can call getClass() to get the Class of the object in question, then get a reference to the Method object for whatever method of the class we want and then assign that to a variable, thereby allowing us to work directly with a method. But unless the method is static, you can't actually invoke the method without a reference to an actual instance of the class it's defined in, so you're right back where you started. Plus, you're not actually working directly with the method itself, but a representation of that method. You can't directly invoke the Method instance like you can in the above example -- you have to explicitly call invoke() and pass in an instance of the class, along with whatever parameters the method is supposed to accept. This is considerably clumsier than just being able to define a function, assign it to a variable, and then calling it as though it's just any other function.

The Workaround
This lack of functions as first-class entities in the language necessarily limits the implementation options for Java, because being able to easily define functions, assign them to variables, pass them around and otherwise treat them like any other data type is at the core of working with closures.

So, if we can't do any of this with functions in Java, how can we have closures?

The simple answer would be to change the language and compiler (and probably even the JVM) to support the ability to have stand-alone functions...which would basically break all existing code. While Python may have gotten away with breaking backward compatibility in 3.0, I think at this point it's highly unlikely we'd see something similar happen in Java.

This leaves us with the only option available: Provide some syntax elements in the language itself that make defining functions simple and give the appearance within the code that you are only defining in-line functions, but have the compiler do some translation and actually convert the syntax to the equivalent Java object notation. Essentially we have to have some syntactic sugar that the programmer can use, while behind the scenes the Java compiler is creating code that's identical to what you would get normally if you tried to do something similar without the special syntax.

Now, if you've been working with Java for a long time this probably sounds strangely familiar to you. It should: It's a similar approach to what was done to add Generics to the language. In order to ensure backward compatibility, Generic notation in Java is (for all intents and purposes) syntactic sugar added to the language to hide all the ugly casting that developers always had to deal with before². The bytecode you get when you compile Generic code is virtually identical to what you would get with non-Generic code that used the old casting-all-over-the-damn-place coding style.

The choice to take the "syntactic sugar" approach to Generics means that the power of Java's Generics is somewhat limited compared to what is possible in other languages. Mind you they don't completely suck, but there are just enough irritations with using Generics that a lot of Java developers -- even the very ones that were pushing heavily for the feature -- have not been particularly happy with the implementation.

And I think this is a the heart of why we haven't seen a big push to get closures into Java. A lot of people working on this feature know how much grief has been caused by the push for Generics and the nagging issues it caused. And I'd find it hard to believe that they aren't keeping those issues in the back of their mind as they work hard to bring closures to Java. It's no easy task trying to retrofit something like this into a language after the fact, and taking the time to get it right (or at least more acceptable than Generics are) is preferable to rushing headlong into a hacked implementation.

I'll admit that I'm not particularly enthused with any of the implementations that have been proposed to date -- none of them adequately address the functions as first-class entities problem I mentioned, so any implementation of closures we get for Java is going to be substandard at best. Nevertheless, I am looking forward to having closures in the language.

And if I have to wait a bit longer to allow the guys working on it plenty of time to get it right then so be it. The alternative will be a lackluster implementation that leaves everyone unhappy.

That's what we got with Generics, and we don't need that for closures, too.

---

1 - Found on page 76, or on the SICP website here.

2-Yes, this is an oversimplification. There were some changes made at the bytecode level (from what I remember), but for the most part that's all Generics are currently.