Dealing with exceptions occured while datanucleus enhancement process in Google App Engine+Java+JDO2+Maven style project.

This article is very specific to Google app engine + java + JDO2 + Maven stack legacy application. I am going to talk about few exceptions facing while running/deploying the application of this type. You may skip if this is not applicable scenario to you.  But if you arrived at this page searching for the exceptions facing related to datanucleus enhancement for app engine project than please go ahead.

Why I have started facing the issue ?? 

Previously I was using IDE(Eclipse/Intellij) feature to create Google App Engine project. In this case I was able to run and deploy the code perfectly fine as the class enhancement process was taken care by IDE plugin if you create Google App Engine project.Now due to some deployment and build requirement I want to convert that project to Maven project so that I can build/run/deploy the project without depending on the IDE. I have converted traditional app engine project to traditional maven project and tried running/deploying the project. When project was deployed and I tried to access the project I was getting below exception for one of my Entity/data class:

1) Persistent class "Class com.ali.data.jdo.Student does not seem to have been enhanced.  You may want to rerun the enhancer and check for errors in the output." has no table in the database, but the operation requires it. Please check the specification of the MetaData for this class.

So this one is clear that you need to perform enhancement on your Data/Entity classes. Below link has more details why byte code enhancement needed after compilation and what are different ways to achieve that.
http://www.datanucleus.org/products/datanucleus/jdo/enhancer.html

Lets say you have selected one of the way from the above URL for enhancement and you are still facing one of the below exception in that process.

2) "org.datanucleus.enhancer" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL 
"file:/C:/Users/<username>/.gradle/caches/modules-2/files-2.1/org.datanucleus/datanucleus-enhancer/3.1.1/b141c67d55cc19f14639f091b84e692e2198dc50/datanucleus-enhancer-3.1.1.jar" 
is already registered, and you are trying to register an identical plugin located at URL 
"file:/C:/Users/<username>/.gradle/appengine-sdk/appengine-java-sdk-1.9.3/lib/opt/tools/datanucleus

This error will come due to multiple datanucleus-core.jar file presence in classpath. You need to remove it from one place depending on the way you are compiling the classes.

3) datanucleus enhancer failing command too long error
commandline too long) when starting the enhancer in forked mode

I got this error while I was using Maven plugin way to enhance the classes.
http://www.datanucleus.org/servlet/jira/browse/NUCMAVEN-47
http://www.datanucleus.org/servlet/jira/browse/NUCMAVEN-44
Here you can see they are claiming that it has been fixed in the newer version of the plugin so I tried it and got rid from this error at least. After that I have started getting below exception.


4) org.datanucleus.exceptions.NucleusException: Error reading the Meta-Data input "Content is not allowed in prolog.
I am not able to find the solution for this exception but at least I understood that it is coming because while enhancing it is trying to read some other format files (e.g: .xml,.md,.properties etc) from classpath. So try to provide path/packages list in plugin or by some other way to instruct which classes to enhance.

5) You have selected to use ClassEnhancer "ASM" yet the JAR for that enhancer does not seem to be in the CLASSPATH!
This one is very nasty and believe me even though you will do all the possible things to fix this stubborn bug it will not leave you. The datanucleous enhancer needs ASM jar in the classpath. Add the right version (may be 5) and see if you can get rid of it. Also check for presence of older version ASM jar in classpath and remove it. I googled a lot for this bug but nothing worked our for me. Comment down in case something will work for you than.

Final Hybrid Solution which I have derived for my requirement:

I have tried Maven datanucleous plugin way but it was not working for me.

Finally I have decided to go with "Manual invocation at the command line"

But as I said earlier I want to automate the build/run/deploy process automated.  So I achieved that by below 2 steps:

1) Created a script containing the list of commands to perform manual enhancement on the selected data classes.

As I am on windows platform I have decided to put my manual enhancement java commands in one of the .bat file which I will integrate in maven build flow later in the step 2.

My enhance.bat script will look like:

 java -cp target\hello-world-1.0\WEB-INF\classes;scripts\required-lib\* org.datanucleus.enhancer.DataNucleusEnhancer target\hello-world-1.0\WEB-INF\classes\com\ali\data\jdo\*.class  
 java -cp target\hello-world-1.0\WEB-INF\classes;scripts\required-lib\* org.datanucleus.enhancer.DataNucleusEnhancer target\hello-world-1.0\WEB-INF\classes\com\ali\common\jdo\*.class  

Here we are trying to run the DataNucleusEnhancer class on some selected compiled packges.

• target\hello-world-1.0\WEB-INF\classes  : The intermediate classes generated by maven which is must to include in classpath to run this command
• scripts\required-lib\* : I have created this folder structure in the root directory of project and put all the dependency jars you have used for project in this folder and also add "datanucleus-enhancer-1.1.4" jar file which has the main class DataNucleusEnhancer. 
• org.datanucleus.enhancer.DataNucleusEnhancer : The main class which is entry point for performing the enhancement.
• target\hello-world-1.0\WEB-INF\classes\com\ali\data\jdo\*.class : The list of packages containing the data/entity classes which you want to enhance.

2) Added a plugin in maven pom.xml file which calls that script in "process-classes" phase which is after the compilation and class files generated.

Plugin:

http://www.mojohaus.org/exec-maven-plugin/

http://www.mojohaus.org/exec-maven-plugin/usage.html

This is how I am calling the .bat script created in the first step from maven.

 <plugin>  
         <artifactId>exec-maven-plugin</artifactId>  
         <groupId>org.codehaus.mojo</groupId>  
         <executions>  
           <execution>  
             <id>enhancer</id>  
             <phase>process-classes</phase>  
             <goals>  
               <goal>exec</goal>  
             </goals>  
             <configuration>  
               <executable>${project.basedir}/scripts/enhance.bat</executable>  
             </configuration>  
           </execution>  
         </executions>  
       </plugin>  

This will process the compiled classes and then prepare WAR file from the enhanced classes.

For JDO2+ Google App Engine combination you must use below versioned jar files only:
asm-5.0.4.jar
datanucleus-core-1.1.5.jar
datanucleus-enhancer-1.1.4.jar
datanucleus-jpa-1.1.5.jar
datanucleus-appengine-1.0.10.jar
jdo2-api-2.3-eb.jar


Let me know for any queries or questions. Comment down in case you are stuck in similar scenario and want a detailed solution for the approach I have used.