How can you extend Java to introduce passing by re

Question

Java is pass-by-value. How could you modify the language to introduce passing by reference (or some equivalent behavior)?

Take for example something like

public static void main(String[] args) {
    String variable = "'previous String reference'";
    passByReference(ref variable);
    System.out.println(variable); // I want this to print 'new String reference'
}

public static void passByReference(ref String someString) {
    someString = "'new String reference'";
}

which (without the ref) compiles to the following bytecode

  public static void main(java.lang.String[]);
    Code:
       0: ldc           #2                  // String 'previous String reference'
       2: astore_1
       3: aload_1
       4: invokestatic  #3                  // Method passByReference:(Ljava/lang/String;)V
       7: return

  public static void passByReference(java.lang.String);
    Code:
       0: ldc           #4                  // String 'new String reference'
       2: astore_0
       3: return

The code at 3: loads the reference onto the stack from the variable variable.

One possibility I'm considering is to have the compiler determine a method is pass by reference, possibly with ref, and change the method to accept a Holder object which stores the same reference as our variable. When the method completes, and possibly changes that reference in the holder, the variable on the caller side's value is replaced with the holder reference's value.

It should compile to an equivalent of this

public static void main(String[] args) {
    String variable = "'previous String reference'";
    Holder holder = Holder.referenceOf(variable);
    passByReference2(holder);
    variable = (String) holder.getReference(); // I don't think this cast is necessary in bytecode
    System.out.println(variable);
}

public static void passByReference(Holder someString) {
    someString.setReference("'new String reference'");
}

where Holder might be something like

public class Holder {
    Object reference;
    private Holder (Object reference) {
        this.reference = reference;
    }
    public Object getReference() {
        return this.reference;
    }
    public void setReference(Object reference) {
        this.reference = reference;
    }
    public static Holder referenceOf(Object reference) {
        return new Holder(reference);
    }
}

Where can this fail or how could you improve it?

Answer 1

To answer your question:

Where can this fail?

1. Final variables and enum constants
2. 'Special' references such as this
3. References that are returned from method calls, or constructed inline using new
4. Literals (Strings, integers, etc.)

...and possibly others. Basically, your ref keyword must only be usable if the parameter source is a non-final field or local variable. Any other source should generate a compilation error when used with ref.

An example of (1):

final String s = "final";
passByReference(ref s);  // Should not be possible

An example of (2):

passByReference(ref this);  // Definitely impossible

An example of (3):

passByReference(ref toString());  // Definitely impossible
passByReference(ref new String("foo"));  // Definitely impossible

An example of (4):

passByReference(ref "literal");  // Definitely impossible

And then there are assignment expressions, which seem to me like something of a judgement call:

String s;
passByReference(ref (s="initial"));  // Possible, but does it make sense?

It's also a little strange that your syntax requires the ref keyword for both the method definition and the method invocation. I think the method definition would be sufficient.

Answer 2

Using AtomicReference class as holder object.

public static void main(String[] args) {
    String variable="old";
    AtomicReference<String> at=new AtomicReference<String>(variable);
    passByReference(at);
    variable=at.get();
    System.out.println(variable);
}

public static void passByReference(AtomicReference<String> at) {
  at.set("new");
}

Answer 3

Think about how it might be implemented with a primitive type, say int. Java - the JVM, not just the language - does not have any "pointer" type to a local variable, on the frame (method stack) or the operand stack. Without that, it is not possible to truly pass by reference.

Other languages that support pass-by-reference use pointers (I believe, though I don't see any other possibility). C++ references (like int&) are pointers in disguise.

I've thought of creating a new set of classes that extend Number, containing int, long, etc. but not immutable. This could give some of the effect of passing primitives by reference - but they won't be auto-boxed, and some other features might not work.

Without support in the JVM, you can't have real pass-by-reference. Sorry, but that's my understanding.

BTW, there are already several Reference-type classes (like you'd like for Holder). ThreadLocal<> (which has get() and set()), or the Reference extenders, like WeakReference (which I think only have get()).

Edit: After reading some other answers, I'd suggest that ref be a form of auto-boxing. Thus:

class ReferenceHolder<T> {
    T referrent;
    static <T> ReferenceHolder<T> valueOf(T object) {
        return new ReferenceHolder<T>(object);
    }
    ReferenceHolder(T object) { referrent = object; }
    T get()            { return referrent; }
    void set(T value)  { referrent = value; }
}

class RefTest {
    static void main() {
        String s = "Hello";
        // This is how it is written...
        change(s);
        // but the compiler converts it to...
        ReferenceHolder<String> $tmp = ReferenceHolder.valueOf(s);
        change($tmp);
        s = $tmp.get();
    }
    // This is how it is written...
    static void change(ref Object s) {
        s = "Goodbye";              // won't work
        s = 17;             // *Potential ClassCastException, but not here*
    }
    // but the compiler converts it tothe compiler treats it as:
    static <T> void change(ReferenceHolder<T> obj) {
        obj.set((T) "Goodbye");     // this works
        obj.set((T) 17);    // *Compiler can't really catch this*
    }
}

But see where there is potential for putting the wrong kind of type in the ReferenceHolder? If genericized properly, the compiler may be able to warn sometimes, but as you likely want the new code to resemble normal code as much as possible, there is the possibility of a CCEx with each auto-ref call.

Answer 4

Oddly enough, I've been thinking about this problem myself recently. I was considering whether it might be fun to create a dialect of VB that ran on the JVM - I decided it wouldn't be.

Anyway, there are two main cases where this is likely to be useful and well defined:

• local variables
• object attributes

I'm assuming that you're writing a new compiler (or adapting an existing one) for your new dialect of Java.

Local variables are typically handled by code similar to what you're proposing. I'm most familiar with Scala, which doesn't support pass-by-reference, but does support closures, which have the same issues. In Scala, there's a class scala.runtime.ObjectRef, which resembles your Holder class. There are also similar {...}Ref classes for primitives, volatile variables, and similar.

If the compiler needs to create a closure that updates a local variable, it "upgrades" the variable to a final ObjectRef (which can be passed to the closure in its constructor), and replaces uses of that variable by gets and updates by sets, on the ObjectRef. In your compiler, you could upgrade local variables whenever they're passed by reference.

You could use a similar trick with object attributes. Suppose that Holder implements an interface ByRef. When your compiler sees an object attribute being passed by reference, it could create an anonymous subclass of ByRef that reads and updates the object attribute in its get and set methods. Again, Scala does something similar to this for lazily evaluated parameters (like references, but read-only).

For extra brownie points, you could extend the techique to JavaBean properties and even Map, List and Array elements.

One side effect of this is that at the JVM level, your methods have unexpected signatures. If you compile a method with signature void doIt(ref String), at the bytecode level, you'll end up with the signature void doIt(ByRef) (you might expect this to be something like void doIt(ByRef<String>), but of course generics use type erasure). This can cause problems with method overloading, as all by-ref parameters compile to the same signature.

It may be possible to do this with bytecode manipulation, but there are pitfalls, like the fact that the JVM permits applications to re-use local variables - so at the bytecode level, it may not be clear whether a parameter is being re-assigned, or its slot re-used, if the application was compiled without debugging symbols. Also, the compiler may elide aload instructions if there's no possibility of a value having changed within the outer method - if you don't take steps to avoid this, changes to your reference variable may not be reflected in the outer method.

Answer 5

I think you can accomplish most of what you want by building an agent and using cglib.

Many of the examples given here can work. I'd recommend using the template you proposed because it will compile with the normal compiler.

public void doSomething(@Ref String var)

Then behind the scenes you use cglib to rewrite the annotated methods, which is easy. You'll also have to rewrite the caller, which i think will be much more complicated in cglib. javassist uses more of a "source code" oriented approach, and might be better suited for rewriting the callers.

Answer 6

Answering you question about how to extend the language my pick would be: - Using various holders technics as several other answers describe - Use annotations to attach metadata regarding which arguments should be passed by reference and then start juggling with a byte code manipulation library, like cglib in order to fulfil your ideas in byte code itself.

Though this whole idea seems strange.