Upgrading Java 8 to 11 is broken

People oftentimes say that you should always keep up with new releases of technologies. Say, you should use PHP 7 or 8 when they come out, instead of having your project be stuck on PHP 5. The same is said about abandoning JDK 8 in favor of newer releases and upgrading the projects that were built for the older ones. And yet, in the industry, JDK 8 is still in widespread usage:

Why is that? I'd say - because many corporations simply don't have the resources to port over huge projects to a set of technologies that have a large number of breaking changes. And when they do, they often underestimate how hard that could be and what the implications on system stability could be.

My experience

I'm currently upgrading a ~1M SLoC Java enterprise app (with regular Spring, Jetty in there, scheduled processes, PrimeFaces for web UI, REST API services, SOAP services, the whole shebang) from Java 8 to Java 11 (since 17 wasn't out when that change was approved) and it's largely proving to be a pain. Since the version of Spring is ancient and changes over to Spring Boot were also approved, now have to rewrite parts of it and also get rid of the XML configuration and other old approaches which are simply no longer compatible with this upgraded tech stack. It feels like something that perhaps a team should do, instead of one dev over a month or so, but scope creep and estimates for something like that are nigh impossible, so we'll see.

My point is that migrating to new releases isn't always trivial, especially the more complicated and complex a project gets. If i knew that i'll run if compiled successfully, then it wouldn't be too bad, but with the amount of reflection, dynamic class loading etc. that frameworks like Spring favor, my workflow to date has been fixing a bug, building and running, something else breaking, fixing that bug, building and running, something else breaking, realizing that i cannot fix this because upgrades to the logic would be inherently "lossy" due to a mismatch of what the new framework versions provide, making it so that breakages that aren't covered by tests will also be created and so on ad infinitum.

Sometimes it makes me wonder why JDK 9 onward just didn't have a compatibility module: "Here, install this to have all of the old Java classes that were removed from the standard library after JDK 8 and retain that old functionality for software projects that would otherwise be stuck in development hell short of a full rewrite." Or something like that for Spring Boot, that lets you use web.xml instead of having to use hacks to load its contents and register all of the servlets, with half of them not working anyways, because some class names were changed along the way.

Software doesn't always age beautifully.

Furthermore, it feels like new runtime and package versions are made in ways that accidentally break backwards compatibility with no clear ways to avoid this.

What broke

So, let's also talk a bit more about the actual things that broke in my attempts. I sadly didn't write everything down, or didn't keep references to the GitHub issues, but here's a few of the things that i ran into.

Here's a list of the things that were available in JDK 8 and no longer are. When you stumble upon something like that, at best you can import a Maven dependency with a version that has what you need, at worst you need to rewrite the code to use another library or set of libraries.

If you have any low level logic like custom class loaders written against older JDK versions (think before 8), then they'll be forwards compatible until 8 for the most part, but will break afterwards. Coincidentally, reading code that deals with low level logic is also not easy to do, especially if it's not commented well.

If you rely upon reflection, or use advanced language features (like the JasperReports framework for generating PDFs, which also has a build step for building the reports), in some cases things might compile but not work at runtime due to class mismatches.

Many frameworks need new major versions to support newer releases than JDK 8, for example Spring Boot 1.5 needs to be upgraded, so you're also dealing with all the changes that are encapsulated by your dependencies. In another project that i also migrated, needed to rewrite a lot of web initialization code for Spring Boot 2.X.

Not only that, but with those framework changes, certain things can break in the actual environments. For example, if you package your app as a far .jar, then you'll no longer be able to serve JSP files out of it. It makes no sense, but packaging it as a .war which can be executed with "java -jar your-app.war" will work for some reason.

I some other libraries, method names remain the same, but signatures change, or sometimes things just get deprecated and removed - you have to deal with all of that, which is especially unfun in code that isn't commented but that the business depends on to work correctly. Throw in external factors such as insufficient coverage of tests and you're in for an interesting time. I'm not saying that it's a type of environment that should be condoned, but it's the objective reality in many software projects.

Oh and i hope that you're also okay with updating all of your app servers (like Tomcat) or JDK installs on the server as well, especially if you depend on some of the Tomcat libraries to be provided and for your frameworks/libraries that depend on them being present at runtime to accept those new versions effortlessly. It all feels very brittle at times.

This is especially a nightmare if your servers were configured manually - personally i'm introducing Ansible and containers to at least isolate the damage of Ops rot, but it's been an uphill battle since day 1.

Here's an exceedingly stupid one: sometimes there are checks in code to make sure that you're using the right version of JDK (or even the Oracle JDK), which get confused with the larger versions. It's easy to fix when it's your code, but really annoying when it's external tools - personal nitpick.

Addendum: here's something that i expected to break, but didn't. We use myBatis as an ORM. It has XML mapper files that define how to map entities and construct queries against the DB based on that. Also, it uses Java interfaces and dynamically calls the underlying XML code as necessary. So essentially you have an interface, which has a corresponding XML file in which you have quasi-Java code (e.g. checking what parameters are passed in to the query) that's used alongside a number of tags to dynamically build SQL queries. Here's an example.

Instead of something breaking in myBatis, what broke actually was Hibernate in the newer versions of Spring. Oh, and Flyway for DB migrations simply refused to work with a particular Oracle DB version as well.

So, what to do about it?

Personally, i'd never let a single monolith grow that far. Having a problem component or two that could be left on JDK 8 until they die and are rewritten (which would actually be possible with systems that have clearly defined boundaries and smaller scope), while migrating everything else.

Sadly, i don't get that choice, nor do i get the choice to make the judgement call to leave the monolith on JDK 8 in the name of stability. But hey, at least i'm paid a bit of money for it, so i have some motivation to work on all of it to the best of my abilities, learn a bit more about the JDK internals, have a word or two to tell others about my experience before i inevitably burn out from the churn.

Personally, i do still think that Java is pretty good when it comes to stability and backwards and forwards compatibility, especially since tooling like Maven is actually pretty good when compared to the alternatives (well, the parent POM functionality is confusing sometimes, but oh well, it has its use cases). That said, JDK 8 to newer releases was indeed a generational shift (probably one for the better) and you see similar things with Spring Boot 1.5 to Spring Boot 2.X, those breaking releases are inevitable.

I only wish that the things blocking people from writing more modular systems would disappear over time, so that huge monoliths that are incredibly hard to work with wouldn't be such an issue. I'm not necessarily advocating for microservices here, since people tend to go straight from one ditch into the opposite one (multiple services per person, nightmarish service mesh, needlessly large % of the code being for shuffling data around, suddenly building a distributed system even within a single domain), but at the very least it would be really nice to have someone look at it and say:

• "Hey, this PDF generation logic looks really brittle and perhaps should be a service of its own."
• or maybe: "Oh, hey, we're serving our RESTful API calls and front end resources from the same application, maybe we should have a separate front end app, served through a regular web server?"

Then again, if nothing else, i always have the choice to choose where i'm employed, even though i don't feel like letting my coworkers down at the moment either.