Spring-boot: Support java.nio.file Paths and FileSystems with fat jars

Created on 13 Oct 2016 · 24Comments · Source: spring-projects/spring-boot

In Spring Boot 1.3.x this code works if "/mydir" is in the parent archive (i.e. src/main/resources in the project that creates the jar):

Resource resource = new ClassPathResource("/mydir");
Paths.get(resource.getURI());

In 1.4.x it throws FileSystemNotFoundException which isn't even an IOException, so it breaks existing apps.

enhancement

Source

dsyer

👍10 😕2 ❤1

Most helpful comment

Encountered this problem also.

edwardsre on 1 Aug 2017

👍6

All 24 comments

Some code: https://github.com/spring-projects/spring-boot-issues/tree/master/gh-7161

dsyer on 13 Oct 2016

I get a FileSystemNotFoundException with 1.3.8 as well:

jar:file:/Users/awilkinson/dev/spring/spring-boot-issues/gh-7161/target/demo-0.0.1-SNAPSHOT.jar!/mydir
Exception in thread "main" java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:54)
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:104)
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:61)
    at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:52)
Caused by: java.nio.file.FileSystemNotFoundException
    at com.sun.nio.zipfs.ZipFileSystemProvider.getFileSystem(ZipFileSystemProvider.java:171)
    at com.sun.nio.zipfs.ZipFileSystemProvider.getPath(ZipFileSystemProvider.java:157)
    at java.nio.file.Paths.get(Paths.java:143)
    at com.example.SimpleApplication.main(SimpleApplication.java:17)
    ... 8 more
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:62)
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:104)
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:61)
    at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:52)
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:54)
    ... 3 more
Caused by: java.nio.file.FileSystemNotFoundException
    at com.sun.nio.zipfs.ZipFileSystemProvider.getFileSystem(ZipFileSystemProvider.java:171)
    at com.sun.nio.zipfs.ZipFileSystemProvider.getPath(ZipFileSystemProvider.java:157)
    at java.nio.file.Paths.get(Paths.java:143)
    at com.example.SimpleApplication.main(SimpleApplication.java:17)
    ... 8 more

wilkinsona on 13 Oct 2016

Hmm. Me too, but I thought I tried a "real" use case and 1.3.8 fixed it. Odd. Does that mean we can't fix it in Boot? It's to do with the URI that comes back from Resource.getURI().

dsyer on 13 Oct 2016

The stack trace is slightly different in 1.4. Maybe that was enough to fix my real use case:

Exception in thread "main" java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
    at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:58)
Caused by: java.nio.file.FileSystemNotFoundException
    at com.sun.nio.zipfs.ZipFileSystemProvider.getFileSystem(ZipFileSystemProvider.java:171)
    at com.sun.nio.zipfs.ZipFileSystemProvider.getPath(ZipFileSystemProvider.java:157)
    at java.nio.file.Paths.get(Paths.java:143)
    at com.example.SimpleApplication.main(SimpleApplication.java:18)
    ... 8 more

dsyer on 13 Oct 2016

ZipFileSystemProvider is used because it handles URLs with a jar scheme. It turns the URI (jar:file:/Users/awilkinson/dev/spring/spring-boot-issues/gh-7161/target/demo-0.0.1-SNAPSHOT.jar!/mydir) into a Path (/Users/awilkinson/dev/spring/spring-boot-issues/gh-7161/target/demo-0.0.1-SNAPSHOT.jar) which is used to look up a FileSystem in a Map. There's no FileSystem for the Path so the FileSystemNotFoundException is thrown.

If you create a new FileSystem for the URI before trying to get a path, it works with both 1.3.8 and 1.4:

package com.example;

import java.io.IOException;
import java.net.URI;
import java.nio.file.FileSystems;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Collections;

import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.core.io.ClassPathResource;
import org.springframework.core.io.Resource;

@SpringBootApplication
public class SimpleApplication {

    public static void main(String[] args) throws IOException {
        Resource resource = new ClassPathResource("/mydir");
        URI uri = resource.getURI();
        System.out.println(uri);
        FileSystems.newFileSystem(uri, Collections.emptyMap());
        Path path = Paths.get(uri);
        System.out.println(path);
    }

}

wilkinsona on 13 Oct 2016

Interesting. It doesn't fix my legacy app yet though, because it still expects either an IOException, or a valid file system.

dsyer on 13 Oct 2016

@dsyer Can you share your legacy app, or something that reproduces its behaviour?

wilkinsona on 14 Oct 2016

I updated the sample adding some features. It now fails on startup because a ZipFileSystem doesn't support watches. I'm quite happy to fix the "legacy" code and make it less sensitive to exceptions. But it feels to me like using Paths with ueberjars is not a daft thing to do, and it's hard, so maybe we can make it easier if we are creative.

dsyer on 14 Oct 2016

I'd rather consider this issue a bug then an enhancement. Especially since at least 2 further issues are related to this one.

The workaround I use is quite ugly by converting the URI to a string, replace a character sequence and creating a new URI with the patched string.

166MMX on 5 Apr 2017

Here's an example program that gets an exception with the Files.walk method:

package com.example;

import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.core.io.ClassPathResource;
import org.springframework.core.io.Resource;

import java.io.IOException;
import java.net.URI;
import java.nio.file.*;
import java.util.Collections;

@SpringBootApplication
public class SimpleApplication {

    public static void main(String[] args) throws IOException {
        Resource resource = new ClassPathResource("/mydir");
        URI uri = resource.getURI();
        System.out.println(uri);
        FileSystem fs = FileSystems.newFileSystem(uri, Collections.emptyMap());
        Path path = Paths.get(uri);
        System.out.println(path);
        if (Files.isDirectory(path)) {
            System.out.println("file type is directory");
        } else if (Files.isRegularFile(path)) {
            System.out.println("file type is regular file");
        } else {
            System.out.println("file is neither directory nor regular");
        }
        Files.walk(path).forEach((Path child) -> {
            if (Files.isRegularFile(child)) {
                System.out.print("Regular file: " + child);
            } else {
                System.out.print("Directory: " + child);
            }
        });

    }

}

And I added some directories and files to the jar file (contents not important for example):

        0  2017-04-24 16:28   BOOT-INF/classes/mydir/
        0  2017-04-24 16:28   BOOT-INF/classes/mydir/mySubDir/
       13  2017-04-24 16:28   BOOT-INF/classes/mydir/mySubDir/Hello.txt

bdnett on 25 Apr 2017

👍4

Also encountered this problem, continued attention.

Caratacus on 20 May 2017

Encountered this problem also.

edwardsre on 1 Aug 2017

👍6

I too am encountering this issue. I am trying to get the file path to build up a command line command to execute and when run as a fat jar I am getting the FileSystemNotFoundException.

KingstonDuo on 17 Aug 2017

👍2

Same issue here :(

neterium on 28 Aug 2017

👍5

I also ran into this. My workaround is to explicitly prefix the path with BOOT-INF/classes like so:

  String resourceDirectory = "/my/resource/dir";
  URI uri = getClass().getResource(resourceDirectory).toURI();
  Path path;

  if (uri.getScheme().equals("jar")) {
    FileSystem fileSystem = FileSystems.newFileSystem(uri, Collections.emptyMap());
    path = fileSystem.getPath("/BOOT-INF/classes" + resourceDirectory);
  } else {
    // Not running in a jar, so just use a regular filesystem path
    path = Paths.get(uri);
  }

  Files.walk(path).forEach(...);

That works but I really don't like having an implementation detail of Spring Boot's executable jar layout hardwired into my code.

sgrimm-sg on 29 Dec 2017

👍5

I use springboot 1.5.9.RELEASE and i still get this exception. I am trying to read a resource which is packaged in BOOT-INF/classes/someDir/file. I have the following code

final URI resourceURI = new ClassPathResource("/my/resource");
FileSystems.newFileSystem(resourceURI, Collections.emptyMap());
final byte[] bytes = Files.readAllBytes(resourceURI);

It resolves the URI to BOOT-INF/classes!/my/resource but it throws a java.nio.file.NoSuchFileException. I checked and the file is very much there.

I am a bit surprised that this issue has been open since 2016. Any resolutions?

Best Regards,
Madhav

unmarshall on 25 May 2018

👍3

The example above doesn't compile (ClassPathResource is not a URI), so it's probably not a real use case?

There is no need to use Files here. You could, for instance, just do this:

final byte[] bytes = StreamUtils.copyToByteArray(new ClassPathResource("/my/resource").getInputStream());

dsyer on 25 May 2018

👍2

@dsyer This is the only way that worked for me

osamaabdulsattar on 20 Jun 2018

@dsyer Over a year ago I was repacking jRuby and needed to test whether a given resource is a regular file or a directory or exists at all. I documented everything over #8822 and created a pull request https://github.com/spring-projects/spring-boot-issues/pull/66 . Hopefully that use case is real enough to work for you.

166MMX on 20 Jun 2018

Hey folks any guidance on this question

_:: Spring Boot :: (v2.0.3.RELEASE)_

application.properties:

spring.cloud.gcp.credentials.location=classpath:ArpanShoppingApp-863d536d1f93.json

and running jar file gives exception

java -jar CloudSQLConnect-1.0.jar

2018-06-22 10:46:38.393  INFO 1172 --- [           main] o.s.c.g.s.a.GcpCloudSqlAutoConfiguration : Default MYSQL JdbcUrl provider. Connecting to jdbc:mysql://google/google_sql?cloudSqlInstance=mindful-highway-207309:asia-south1:shopping-db&socketFactory=com.google.cloud.sql.mysql.SocketFactory&useSSL=false with driver com.mysql.jdbc.Driver
2018-06-22 10:46:38.401  INFO 1172 --- [           main] o.s.c.g.s.a.GcpCloudSqlAutoConfiguration : Error reading Cloud SQL credentials file.

java.io.FileNotFoundException: class path resource [ArpanShoppingApp-863d536d1f93.json] cannot be resolved to absolute file path because it does not reside in the file system: jar:file:/Users/arpan/Documents/workspace-sts-3.8.4.RELEASE/CloudSQLConnect/target/CloudSQLConnect-1.0.jar!/BOOT-INF/classes!/ArpanShoppingApp-863d536d1f93.json
at org.springframework.util.ResourceUtils.getFile(ResourceUtils.java:217) ~[spring-core-5.0.7.RELEASE.jar!/:5.0.7.RELEASE]
at org.springframework.core.io.AbstractFileResolvingResource.getFile(AbstractFileResolvingResource.java:133) ~[spring-core-5.0.7.RELEASE.jar!/:5.0.7.RELEASE]
at org.springframework.cloud.gcp.sql.autoconfig.GcpCloudSqlAutoConfiguration.setCredentialsProperty(GcpCloudSqlAutoConfiguration.java:167) [spring-cloud-gcp-starter-sql-1.0.0.M1.jar!/:1.0.0.M1]
at org.springframework.cloud.gcp.sql.autoconfig.GcpCloudSqlAutoConfiguration.defaultJdbcInfoProvider(GcpCloudSqlAutoConfiguration.java:107) [spring-cloud-gcp-starter-sql-1.0.0.M1.jar!/:1.0.0.M1]
at org.springframework.cloud.gcp.sql.autoconfig.GcpCloudSqlAutoConfiguration$$EnhancerBySpringCGLIB$$edf77794.CGLIB$defaultJdbcInfoProvider$1(<generated>) [spring-cloud-gcp-starter-sql-1.0.0.M1.jar!/:1.0.0.M1]

arpan2501 on 22 Jun 2018

@arpan2501 That doesn't appear to be related to this issue as the code in the stack isn't using Paths or FileSystem. If you're looking for some help about Spring Cloud GCP (which is where the problem appears to be), please ask a question on Stack Overflow.

wilkinsona on 22 Jun 2018

@wilkinsona I thought it might be related to Paths because I am able to connect to Cloud SQL perfectly when running the Spring Boot App. The issue comes only when I build and try to run the Jar and it complains FileNotFoundException with this ! in path target/CloudSQLConnect-1.0.jar!/BOOT-INF/classes!/ArpanShoppingApp-863d536d1f93.json

Sorry to bother you here.. Raised the same in stackoverflow!!!

arpan2501 on 22 Jun 2018

I did some work on a similar issue to this here: https://github.com/magneticflux-/classpath-resource-extractor/issues/2 and https://github.com/magneticflux-/classpath-resource-extractor/pull/3

It has a method that visits a URI (that may be nested (including Spring's broken URIs)) as a Path.

magneticflux- on 10 Jun 2020

Related issue appear when @ConfigurationProperties class with java.nio.file.Path property is used.

When such application run from Intellj all works OK, because PathEditor uses ClassLoaders$AppClassLoader however when run from fat-jar which uses LaunchedURLClassLoader it fails.

Some more information at https://stackoverflow.com/questions/64768787/spring-boot-property-of-type-path-read-from-application-yaml

Spring Boot version: 2.3.5.RELEASE