Debugging Maven plugin in IntelliJ
Do you use Maven to manage your Java project and came across a really cool Maven plugin but realized it doesn’t work perfectly for your project? Are you new to Maven and want to understand it better? Are you interested in Open Source Software Development, and maybe thought about contributing to one?
You are not alone. Three months ago, I started working in Maven and came across a variety of dependency management issues in my Google Cloud Client library project so I started exploring ways to improve the way we manage dependencies.
I met with Ray Tsang at Google to discuss some of the issues we’ve been facing and suggestions he might have. After a few fruitful conversations, we decided to use the Flatten plugin to flatten the client library project POM in order to eliminate transitive dependencies. This is enabled by the new configuration, <flattenDependencyMode>all</flattenDependencyMode>that Ray recently contributed to the Flatten plugin.
It is a powerful new feature that could help us address the problem of the loss of transitive dependency version when it is managed by the <dependencyManagement> section and allow us to provide client library consumers with a cleaner and fully resolved dependency tree.
I was very eager to implement it in my project, BigQueryStorage Java client. However, after some testing, we quickly discovered two issues with Ray’s original contribution: 1) the flat order was in reverse; and 2) a dependency (gax-grpc) was missing from the flattened POM.
This article captures what I learned from the process of enhancing and contributing to a Maven Plugin.
Getting started
First, configure the plugin as recommended in the project’s parent POM, and run mvn verifyto get the .flattened.pom in the project’s root directory.
To get started with debugging the plugin, fork and clone the source code of flatten-maven-plugin.
After forking and cloning the project locally, make sure to first run mvn verify to see if the project would compile and build. It should without any errors. Then it is time to inspect the structure of the project to find out where the code lives and where the tests live.
Understanding the code at a high-level
The project structure seems pretty clear and straightforward. The src/it directory contains all the integration tests. It is further broken down to mrm and project directories where mrm provides all the test dependencies and project provides all the integration tests that make use of the test dependencies.
If you inspect a couple poms in the integration tests in project you will see that they are using the test dependencies created in mrm. This is a great start!
Debugging
Next, how do we actually get started with the actual debugging? Well, one thing that we do know is the bugs — somewhere in the code, the logic is broken which resulted in the bugs. Therefore, if we can reproduce the bugs in the form of an integration test then maybe we can put breakpoints and debug the code to find out which method and then which line is causing it. Sounds like a plan!
Step 1: Start with the easiest bug
Always start with the easier problem to solve. It will not only get you exposure to the codebase quickly but will also allow you to build up confidence to solve harder and more complex problems later on.
Out of our two problems, the dependency resolution in reverse order bug seemed like a low-hanging fruit. Our initial guess was that somewhere in the code, stack instead of queue was being used so that when the result data structure was being constructed, objects were popped in LIFO order instead of polled in FIFO order, which resulted in the reverse order. All we need to do is to find out where this result data structure is being constructed in the code and put a fix there.
Step 2: Write an integration test that breaks
Start by writing an integration test of the expected behavior that would fail. This gives us an entry point to the codebase which then allows us to locate the problematic logic by putting breakpoints and inspecting statements returned by the debugger.
To know how to write a proper integration test, refer to some other integration tests already written in the src/it directory. Once you’re comfortable, write a simple but comprehensive unit test:
This test written here, for example, simply validates that the resolution order of the flattened pom should be in the natural order where dependency with artifactId “dep” is resolved before dependency with artifactId “test”.
To run integration tests, there are a couple options. You can:
- Run a single test:
mvn verify -Dinvoker.test=flatten-dependency-all-resolution-order
2. Run multiple tests that share a common string in the name:
mvn verify -Dinvoker.test=flatten-dependency-all-*
3. Run all integration tests:
mvn verify
Continue by running the unit test of the expected behavior. For example, running flatten-dependency-all-resolution-order gave us a failure where “test” was resolved first. This is exactly what we want so we’re good to continue.
Step 3: Configure Debugger
Next, we need to identify which line of code is causing the problem. To do so, we need to first set up the debugger, put some breakpoints, and then trigger the test again to see what’s being returned on each line.
To set up the debugger, you simply need to go to “Edit Configuration” in the top menu bar and select “Remote” from the “+” drop-down menu. Give the debugger a name and then update port to 8000. Select the appropriate JDK version base on what you are developing in.
Once the debugger is set up, you can run the test in debugger mode:
mvn verify -Dinvoker.test=flatten-dependency-all-resolution-order -Dinvoker.mavenExecutable=mvnDebug
This will stop and hang at each integration test to listen for a debugger connection — click on the green bug icon in the top menu bar in order to start the debugger.
Without any breakpoints, the debugger will run, pass/fail, and exit. This is when it’s very helpful to put breakpoints at methods you suspect to be causing the problems and inspect what gets returned at each line.
Step 4: Add breakpoints
Inspect the configuration you are using with the plugin and read the code. Make an educated guess on which method is related to the feature you are using and put a couple breakpoints to see if running the integration test hits anything.
In our case, we quickly figured out that most methods we were interested in live in FlattenMojo.java. We were able to place breakpoints to narrow down problematic areas and eventually identified the exact method that was causing the problem.
Once you locate the problematic logic, it is relatively straightforward to put a fix. You would just need to read through the code, understand what’s going on, look at the debugger output, and then make a code change. Once you make the change, run the integration test again to see the new output and whether the integration test passes. Go through this process iteratively and make small, incremental changes.
Step 5: Run all the integration tests
Once you are confident about your change, make sure to run mvn verify before you commit any code changes. This ensures that your fix is not breaking anything else.
Here is the fix to both bugs if you are interested in learning more.
Final Thoughts
Test, test, and test — make sure you add enough unit tests to cover your bases and any other scenarios that may be directly or indirectly related to your fix. In our case, for instance, we were fixing dependency resolution order so we also decided to write additional tests for poms with complicated transitive dependencies. This is to ensure that we’re not introducing new bugs while fixing existing ones. It is really something easier said than done. Do take the time to think through and add test cases if needed.
Special thanks to Ray Tsang for the contribution! Check out Surviving Dependency Hell talk by Ray and Robert Scholte, Apache Maven Chair, if you want to learn more about Maven dependency management strategies.
Lastly, have fun debugging and enjoy learning! 🙂
Follow me on Twitter Stephanie Wang for the latest updates.
