JBuilder JOT concepts

General description

What is JOT? JOT stands for the Java Object Toolkit and is a set of interfaces and classes used to parse and generate Java code. It's main use is the parsing of source or class files to extract information about their contents. For example, the Deployment wizard uses JOT to traverse files and determine what object types are actually used by a set of classes or source. Additionally, JOT can be used to generate code. Examples of this include the various Wizards in the Object Gallery (File|New) or the UI Designer.

Detailed description of feature/subsystem

The heart of JOT is a series of interfaces that describe the Java language in detail.

Accessing JOT

You are either working with a file that exists already or you are creating a new file when you are working with JOT.  If the file exists, it is usually part of your project (say, a selected node).  If you are creating a new file, it is usually done in the context of the current project.  To access files with JOT you need to get either a JotSourceFile or a JotFile object from the package manager.  The package manager is represented by the JotPackages interface and you can get an instance of one from JBProject.  The package manager will know about the files and packages that are available to the project and you use the package manager to access source files or class files.

JBProject project; // this is a valid, non-null instance of JBProject
String filename = "c:/dev/test/Unit.java";
JotPackages pkg = project.getJotPackages();
JotFile jfile = pkg.getFile(new Url(new File(filename)));

The above code would return a JotFile object for the file c:/dev/test/Unit.java if it exists and can be found in the classpath available to the project.  If the file cannot be found, the JotFile object is null.  You can use this mechanism to load a .class as well as a .java file, but usually work begins with source.  A more common case (which is covered in the jot samples) gets the file Url from the project itself.  The JotFile object that you have at this point can be accessed for reading or writing.

If you are creating a file, first make a FileNode from the Project.  You can then extract a package manager from there for Jot to begin working.  Don't forget to save the FileNode when you are done.

JBProject proj;  // this is a valid, non-null instance of JBProject
String fullFileName = "c:/dev/test/Unit.java"

java.io.File f = new File(fullFileName);
if(!f.exists()){
  FileNode jfile = proj.getNode(new Url(f));
  // use JOT to write the contents of the file
  jfile.setParent(proj);
  jfile.save();
}

If you are creating a file or adding source to a file you must tell the package manager to commit() those changes.  When you are done working with a file in JOT, you need to tell the package manager to release() the file.  This frees the file for use by other subsystems in JBuilder.

How JOT sees a class

JOT views a class in a manner similar to how the java.lang.reflect package does. A representation of a Class exists (either from source or from a compiled unit). This Class is composed of member items: fields and methods. In many cases, you'll find working with these higher-level items familiar if you've previously worked with the Reflection API. Reflection provides access to the methods, constructors, and fields of a class. You can glean information on the class itself by using java.lang.Class directly.

Using JOT, you can do all that and go much deeper than using reflection alone. To allow it to parse Java code, JOT has representation for all of the Java constructs you might see in a source file. It also adds representation for the file that contains the class. These detail items follow the language guidelines laid out on the Java Language Specification (JLS); the JLS sections are cited in the Javadoc for the JOT classes:

JOT continues where reflection stops by allowing you to work with the individual statements or expressions that make up the heart of most Java code.

Using JOT to read Java

JOT can parse Java classes and Java source files. The methods specified by JOT for parsing Java use the naming convention get<XYZ> to extract whatever XYZ represents. These getter methods are generally used to parse the nested nature of Java code and usually return another item that is represented by a JOT. If you are parsing from Java source, you can extract a little more information than from a Java class. This is because some items represented in the source (such as comments) are stripped out at compile time and aren't represented within the class file.

During parsing, everything works from the outside in. At the outermost level (a JotSourceFile), the parsing procedure is simple. A JotSourceFile contains one or more classes (represented by a JotClass or a JotClassSource), and each of those classes is composed of variables and methods. Usually you have a loop that iterates through all the classes in the file. From each of those classes you can get an array of methods defined in that class and loop through each of these. The implementation of these methods makes things a bit more complicated. For example, a class can have many methods and each method can be composed of several statements, and each of those statements can be a method that is composed of many statements, and so on. At this level, recursion is common.

JOT allows you to parse down to the atomic level. Consider a simple example of a for loop:

public void forLoop(){

 // a bunch of statements in the method body...

    for(int i = 0; i < 5; i++){
        System.out.println("Hello Jot!");
    }

 // even more statements in the method body...
}

The for statement is defined in section 14.12 of the Java Language Specification, which states:

In our example, the initialization code is: int i = 0.  The expression is: i < 5. The statement that is executed is the println inside the code block of the for loop.  The update code is i++.

As the Java Language Specification states, a for loop is a type of statement. This means, at a high level, the for loop in the code would be parsed as being of type JotStatement. This is very generic as many things are JotStatements, so if you are working with a bunch of statements with JOT, you would determine if the type of statement you had encountered was really an instance of JotFor using instanceof.

JotFor breaks the for statement down even further. The expression portion of a for statement describes the condition that causes the loop to terminate. Any valid Java expression can be used here, ranging from a simple value check to a complex expression. JotFor.getCondition() returns a JotExpression object that you could further interrogate to determine what type of an expression you had and what it was composed of, and so on.

Use the following code to see how a simple class is broken down and parsed with JOT:

package com.borland.samples.jot;

import java.awt.*;

  public class JotExample extends Component{
    static final int PI = 4;

    public JotExample(){
      // xtor implementation goes here
    }

    public boolean doesItRock(int x){
      int ctr = x;
      for(int i = 0; i < ctr; i++){
         System.out.println("JBuilder Rocks!");
         System.out.println("Oh yes, it does");
      }
      return true;
    }
  }
For this example we have an instance of JotSourceFile named jsf that represents this file. To get the package statement use this code:

String pkg = jsf.getPackage();

To get the import statements from the file, use this code and iterate through the array:

JotImport [] imports = jsf.getImports();

To get the class(es) from the file, use this code:

JotClass [] classes = jsf.getClasses();
In this example there's only one class file represented so you know that your class is the first element in the array of classes returned by JotSourceFile.getClasses(). Therefore, you can use this code to get the specific instance of JotClass:
JotClass jc = classes[0];

From the JotClass you can extract information about the class definition as well as its member fields and methods. The basic pattern remains the same; that is, a getter returns a value or object that you can interrogate further for more information.

// info about the class declaration
int mod = jc.getModifiers();
JotType super = jc.getSuperclass();
JotType[] imps = jc.getInterfaces();
These methods allow you to find out the modifiers for the class, the super class and an array of interfaces implemented by the class. You could get more information about the superclass by using the methods in JotType to get a JotClass or JotClassSource that represents the superclass of the original object parsed.

Throughout JOT there are methods declared to return arrays of objects. If there is no value to be returned JOT returns an array of length 0. In our example, JotClass.getInterfaces() returns a zero length array of JotType. To avoid errors, check the length of any array that is returned before trying to use the objects.

Where modifiers are used in Java, as, for example a public class or a static final variable, JOT allows access through a getModifiers() method. getModifiers() returns an int value that you deconstruct by checking against the values in java.lang.reflect.Modifier. You can do this work yourself or use one of the getters in com.borland.jbuilder.jot.util.JotModifers or you can use the methods built into java.lang.reflect.Modifier.  A note when working with the modifers for classes.  The bit that represents 'synchronized' is always set to true for a class.  To eliminate any confusion when generating a String representation for the modifier, be sure to mask out this bit.  For example:

JotClass firstClass; // this is a valid, non-null instance of JotClass
String c_mods = Modifier.toString(firstClass.getModifiers() & ~Modifier.SYNCHRONIZED);

If you continue with the parsing of the class, you can get a list of fields and a list of methods contained in the class. Again, you can further parse the results to get to the atomic portions of the fields and methods:

// details on the contents of the class
JotField[] fields = jc.getFields();
JotMethod[] methods = jc.getMethods();
Taking this one level further demonstrates another important item in JOT: the need for casting. JOT uses interfaces to describe the aspects of Java. The implementation isn't available when using JOT, so you must write all your code against types that are really interfaces and not concrete instances of the objects where all the work is done.

When you use JotClass.getMethods() you get an array of JotMethod objects. The JotMethod interface describes the components that make up a method, such as the modifiers, the method name, any exceptions that it throws, and so on. It doesn't give you access to the contents of the method body itself. JotMethod has a child interface, JotMethodSource, that provides the access to the method body elements. You should be aware that generally methods will return the more generic (higher up in the inheritance tree) Interface type. This makes it easier when you are parsing or when you need to pass items as parameters to methods, but it also makes for a fair amount of casting when you need to get to specific details. To get the statements that make up the method in this sample, use this code:

JotMethodSource jms = (JotMethodSource)methods[0];
JotStatement[] jsa = jms.getCodeBlock().getStatements();
Or you could combine it all into one step and use this code:
JotStatement[] jsa =
   ((JotMethodSource)methods[0]).getCodeBlock().getStatements();
Of course, this extracts just the statements for the first method. Usually you would loop through the methods, and so on.

For a more complete example of using JOT to parse source see the sample  com.borland.samples.opentools.jot.read.ReadingSource.  This sample takes the selected file and gathers information about classes and methods using JOT.  This information is sent to the message window inside the IDE.  Another example of using JOT to parse source and/or classes is the sample  com.borland.samples.opentools.jot.packagetree.PackageTree.  This sample takes the selected node in the IDE, determines it's package and then builds a hierarchy tree for all the classes in that package.

Note:  When you are finished reading the file, remember to tell the package manager to release() the file you were using.

Using JOT to write Java source

JOT can be used to write source as well.  When writing source, you deal with much higher level items like methods or statements instead of the very fine-grained items available to you when reading source or class files with JOT.  The methods specified by JOT for writing Java code use the naming convention add<XYZ> and set<XYZ> where XYZ represents the item you are creating or modifying.  For example, JotClassSource.addMethod() adds a method to a given class.  JotMethodSource.setModifiers() sets the modifer bits for that method.

Almost all items that you read and write with JOT implement the JotMarker interface.  JotMarker is used to control placement of the items you are writing relative to other items already in code.  When you are writing code with JOT, almost all of the methods will take a JotMarker as the first parameter followed by a boolean value followed by whatever data is needed for the item you are writing out (usually a String of some sort).  The JotMarker is the item relative to which you are adding code.  The boolean value controls the position of the added code.  If it is true, the code is added before the JotMarker.  If it is false, it is added after the JotMarker.  The samples mentioned at the end of this section demonstrate this.  Note:  If you are writing code sequentially, as is the case with many Wizards, you can just use null as the value for JotMarker.  The example code in this section does this.

Note: JOT is unforgiving of being told to write bad code and will throw a JotParseException once it has tried to read back anything you have written.  If you are using JOT to generate code and your code doesn't appear, go back and make sure you are creating legal code.

As an example of writing code using JOT, let's create the same class that was used in the parsing section, but we'll build it up in smaller chunks. This simple "starting point" of the class looks like:

package com.borland.samples.jot;

import java.awt.*;

  public class JotExample extends Component{
  }
Again, we begin with the outermost item, the source file itself. For JOT, the source file is represented by a JotSourceFile that encapsulates the package statements, the imports and the class declaration.

Assuming you have an instance of a JotSourceFile named jsf, use the following methods to create the class:

jsf.setPackage("com.borland.samples.jot");
JotImport ji = jsf.addImport("java.awt");
JotClassSource jcs = jsf.addClass(null, false, "JotExample", false);
jcs.setSuperClass("Component");
jcs.setModifiers(Modifier.PUBLIC);
As is the case with getModifiers() discussed in the section on parsing, setModifiers() works with int values. These values are taken from the java.lang.reflect.Modifier class. For this example, assume that there is an import statement for java.lang.reflect.Modifer in place, which allows the use of the short name of the class in the code.

To finish, you need to add the rest of the content (a member variable, a constructor and a method) to the class. This is what the code that is generated will look like:

package com.borland.samples.jot;
import java.awt.*;

public class JotExample extends Component{
  static final int PI = 4;
  public JotExample(){
      // xtor implementation goes here
    }
  public boolean doesItRock(int x){
      int ctr = x;

      for(int i = 0; i < ctr; i++){
        System.out.println("JBuilder   Rocks!");
        System.out.println("Oh yes, it does");
      }

    return true;
    }
  }
To add the field, use code like this:
JotFieldDeclaration jf = jcs.addField(null, false, int, "PI");
jf.setInitializer("4");

Java allows for additional modifiers to be used together. You can do that with JOT by OR'ing the modifier values together when you pass them into setModifiers(). To make the field PI both static and final, do this:

jf.setModifiers(Modifier.STATIC | Modifier.FINAL);
To create the constructor and make it public, do this:
JotConstructorSource xtor = jcs.addConstructor(null, false);
xtor.setModifiers(Modifier.PUBLIC);
To set up the "shell" of the method with a parameter x, use this code:
JotMethodSource jms = jcs.addMethod(null, false, "boolean", "doesItRock");
jms.setModifiers(Modifier.PUBLIC);
jms.setParameter(null, false, "int", "x");
Add the local variable ctr and assign it the value of the parameter passed to the method. Because we're going to add many items to the code block of this method, create an instance variable to represent the code block:
JotCodeBlock rockBlock = jms.getCodeBlock();
JotVariableDeclaration jvd;
jvd = rockBlock.addVariableDeclaration(null, false, "ctr", "int");
jvd.setInitializer("x");
Add in the for statement. Because you'll be adding content to the body of the for statement, create an instance variable to represent the code block of the for statement. Note that the entire setup for the control portion of the for statement is passed in as a complete String instead of three separate JOT types.
JotFor jfor;
jfor = rockBlock.addForStatement(null, false, "int i = 0; i < ctr; i++");
JotCodeBlock forBlock = jfor.getCodeBlock();
Create the body of the for statement. In this case, you're just adding two println statements. Note that the statements you are passing in include their semi-colons at the end. This matches up with the third parameter in JotCodeBlock.addStatement() that specifies if a semi-colon is needed. If the value is set to true, JOT adds a semi-colon at the end of the statement for you.
String stmt1 = "System.out.println(\"JBuilder Rocks!\");";
String stmt2 = " System.out.println(\"Oh yes, it does\");";
forBlock.addStatement(null, false, false, stmt1);
forBlock.addStatement(null, false, false, stmt2);

Finally, add the return statement to the method.

rockBlock.addReturnStatement(null, false, "true");
For a more complete example of using JOT to write source see the sample com.borland.samples.opentools.jot.write.WritingSource.  This sample generates a simple classfile.  Another sample of using JOT to read and write is com.borland.samples.opentools.jot.readwrite.Commenter.  This sample takes the selectes source node and adds Javadoc comments for top level classes and methods in classes.  It only adds them if there isn't a comment there already.

When you are done writing your file, be sure to have the package manager commit() and release() the file.  If you were working with a node that you created in the project, it is also a good idea to have the file saved there, as well.