Wiremock: ClasspathFileSource does not work with spring boot fat jar

[mdomasevicius]

The case is that I need to fire up spring boot app with some external mocks using wiremock (duh). The ClasspathFileSource finds the path in the jar but it fails to load the stub mappings.

return FluentIterable.from(toIterable(zipFile.entries())).filter(new Predicate<ZipEntry>() {
            public boolean apply(ZipEntry jarEntry) {
                return !jarEntry.isDirectory() && jarEntry.getName().startsWith(path);
            }
        }).transform(new Function<ZipEntry, TextFile>() {
            public TextFile apply(ZipEntry jarEntry) {
                return new TextFile(getUriFor(jarEntry));
            }
        }).toList();

I suspect that this line: return !jarEntry.isDirectory() && jarEntry.getName().startsWith(path); is problematic since spring boot fat JAR entries do not start with the $path they do start with BOOT-INF/classes/$path/.... - so in this case ClasspathFileSource returns empty list.

[timpeeters]

As a temporary workaround I used:

.fileSource(new ClasspathFileSource("BOOT-INF/classes/wiremock"))

[bdemers]

I just ran into the same problem. I'm guessing the actual usage of the ZipFile is only needed to list the files (even with the current implementation) 🤔

I had to do something slightly uglier so I could support running on a _normal_ classpath too

URL stubsURL = currentThread().getContextClassLoader().getResource("stubs");
String stubsPath = stubsURL.toString().contains("BOOT-INF/classes") ? "BOOT-INF/classes/stubs" : "stubs";

...

.fileSource(new ClasspathFileSource(stubsPath))

I don't think this problem is limited to Spring Boot's uber jar. There are other classpath scenarios that would trigger this same bug. Maybe title the issue ClasspathFileSource does not work with complex classloaders?

[tomakehurst]

Is the problem fundamentally that when you're running in your IDE, the stubs path is always relative to the root of the classpath, but when you've built the Spring JAR they're moved to a sub-path?

Perhaps a good general solution would be to specify multiple paths so that you can provide both the dev time and deploy time variants?

[bdemers]

@tomakehurst for me this isn't related to a IDE use case (I'm packaging tests to be run later for non-jvm based applications).

In both cases the result of the classpath is the same, the issue is the assumptions the ClasspathFileSource makes. For example:

            if (asList("jar", "war", "ear", "zip").contains(pathUri.getScheme())) {
                String jarFileUri = pathUri.getSchemeSpecificPart().split("!")[0];
                String jarFilePath = jarFileUri.replace("file:", "");
                File file = new File(jarFilePath);
                zipFile = new ZipFile(file);
            } else if (pathUri.getScheme().equals("file")) {
                rootDirectory = new File(pathUri);
            } else {
                throw new RuntimeException("ClasspathFileSource can't handle paths of type " + pathUri.getScheme());
            }

I'm guessing the same problem would happen with a self executing war file, were the resources are contained with a jar that is contained within the war. It looks like that pathUri.getSchemeSpecificPart().split("!")[0] is always going to grap the first zip container, it should probably be the last (and not using a ZipFile, but a stream approach instead)

[tomakehurst]

Can you elaborate why not ZipFile?

[tomakehurst]

Also, do you have a project you could share that replicates this problem?

[bdemers]

Classpath items are not guaranteed to be available via a file protocol, for example a nested jar would likely need to be accessed via a stream vs file io.

I'll likely have public example this week, otherwise i'll hack up a PR with a solution (assuming there are existing tests to work from)

[tomakehurst]

OK, that'd be much appreciated.

I've started knocking up an example app, but you seem to have a better idea than me of where the edge cases are, so some test cases that exercise those would be very useful.

BTW, you're correct in your earlier comment that ZipFile is only used as a means of listing files. I couldn't find another way of doing it at the time, but if you can think of another way to do this that avoids some of the current pitfalls, it'd definitely be worth investigating.

[birnbuazn]

I had a similar problem because my resource files were located within a nested jar within the Spring Boot 2 UberJar.

I wrote a simple FileSource implementation, which uses Spring to load the resources, which works perfectly, because Spring nows how to load resources from within nested jars:

package my.example;

import com.github.tomakehurst.wiremock.common.BinaryFile;
import com.github.tomakehurst.wiremock.common.FileSource;
import com.github.tomakehurst.wiremock.common.TextFile;
import org.springframework.core.io.ClassPathResource;

import java.io.IOException;
import java.net.URI;
import java.util.ArrayList;
import java.util.List;

class SpringClasspathResourceFileSource implements FileSource {


    private final String path;

    SpringClasspathResourceFileSource(final String path) {
        this.path = path;
    }

    @Override
    public URI getUri() {
        return getURI(path);
    }

    @Override
    public FileSource child(final String subDirectoryName) {
        return new SpringClasspathResourceFileSource(appendToPath(subDirectoryName));
    }

    @Override
    public BinaryFile getBinaryFileNamed(final String name) {
        return new BinaryFile(getURI(appendToPath(name)));
    }

    @Override
    public TextFile getTextFileNamed(final String name) {
        return new TextFile(getURI(appendToPath(name)));
    }

    private URI getURI(final String name) {
        try {
            return new ClassPathResource(name).getURI();
        } catch (IOException root) {
            throw new RuntimeException(root);
        }
    }

    private String appendToPath(final String name) {
        return path + "/" + name;
    }

    @Override
    public void createIfNecessary() {
    }

    @Override
    public String getPath() {
        return path;
    }

    @Override
    public List<TextFile> listFilesRecursively() {
        return new ArrayList<>();
    }

    @Override
    public void writeTextFile(final String name, final String contents) {
    }

    @Override
    public void writeBinaryFile(final String name, final byte[] contents) {
    }

    @Override
    public boolean exists() {
        return true;
    }

    @Override
    public void deleteFile(final String name) {
    }
}

Just use this FileSource when configuring WireMock and you are good to go:

return new WireMockServer(
    options()
        .port(port)
        .fileSource(new SpringClasspathResourceFileSource("classpath/to/your/resources"))
);

[x418]

We have a similar issue wherein our wiremock mappings are packaged in a nested jar (under WEB-INF/lib) in the Spring Boot war. The following is how we solved it

```package com.example;

import com.github.tomakehurst.wiremock.common.BinaryFile;
import com.github.tomakehurst.wiremock.common.FileSource;
import com.github.tomakehurst.wiremock.common.TextFile;
import lombok.extern.slf4j.Slf4j;
import org.springframework.core.io.ClassPathResource;

import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.net.URI;
import java.util.ArrayList;
import java.util.List;
import java.util.stream.Collectors;
import java.util.zip.ZipEntry;
import java.util.zip.ZipFile;
import java.util.zip.ZipInputStream;

import static com.github.tomakehurst.wiremock.common.Exceptions.throwUnchecked;
import static com.google.common.collect.Lists.newArrayList;
import static java.util.Arrays.asList;

@Slf4j
class JarReadingClasspathFileSource implements FileSource {

private final String path;
private URI pathUri;
private ZipFile warFile;
private String jarFileName;
private File rootDirectory;

JarReadingClasspathFileSource(String path) {
this.path = path;
try {
pathUri = new ClassPathResource(path).getURI();
if (asList("jar", "war", "ear", "zip").contains(pathUri.getScheme())) {
log.info("Path URI is : [{}]", pathUri);
String[] pathElements = pathUri.getSchemeSpecificPart().split("!", 3);
String warFileUri = pathElements[0];
String warFilePath = warFileUri.replace("file:", "");
warFile = new ZipFile(new File(warFilePath));
jarFileName = pathElements[1].substring(1);
} else if (pathUri.getScheme().equals("file")) {
rootDirectory = new File(pathUri);
} else {
throw new RuntimeException("ClasspathFileSource can't handle paths of type " + pathUri.getScheme());
}
} catch (Exception e) {
throwUnchecked(e);
}
}

@Override
public BinaryFile getBinaryFileNamed(final String name) {
if (isFileSystem()) {
return new BinaryFile(new File(rootDirectory, name).toURI());
}
return new BinaryFile(getUri(name));
}

private boolean isFileSystem() {
return rootDirectory != null;
}

@Override
public TextFile getTextFileNamed(String name) {
if (isFileSystem()) {
return new TextFile(new File(rootDirectory, name).toURI());
}
return new TextFile(getUri(name));
}

private URI getUri(final String name) {
try {
return new ClassPathResource(path + "/" + name).getURI();
} catch (IOException e) {
return throwUnchecked(e, URI.class);
}
}

@Override
public void createIfNecessary() {
}

@Override
public FileSource child(String subDirectoryName) {
return new JarReadingClasspathFileSource(path + "/" + subDirectoryName);
}

@Override
public String getPath() {
return path;
}

@Override
public URI getUri() {
return pathUri;
}

@Override
public List listFilesRecursively() {
if (isFileSystem()) {
assertExistsAndIsDirectory();
List fileList = newArrayList();
recursivelyAddFilesToList(rootDirectory, fileList);
return toTextFileList(fileList);
}
List files = new ArrayList<>();
ZipEntry zipEntry = warFile.getEntry(jarFileName);
try (final InputStream zipFileStream = warFile.getInputStream(zipEntry)) {
final ZipInputStream zipInputStream = new ZipInputStream(zipFileStream);
ZipEntry nestedZipEntry;
while ((nestedZipEntry = zipInputStream.getNextEntry()) != null) {
if (!nestedZipEntry.isDirectory() && nestedZipEntry.getName().startsWith(path)) {
files.add(new TextFile(new ClassPathResource(nestedZipEntry.getName()).getURI()));
}
}
} catch (IOException e) {
log.error("Unable to read war file", e);
}

return files;

}

@Override
public void writeTextFile(String name, String contents) {
}

@Override
public void writeBinaryFile(String name, byte[] contents) {
}

@Override
public boolean exists() {
return !isFileSystem() || rootDirectory.exists();
}

@Override
public void deleteFile(String name) {
}

private List toTextFileList(List fileList) {
return fileList.stream()
.map(input -> new TextFile(input.toURI()))
.collect(Collectors.toList());
}

private void recursivelyAddFilesToList(File root, List fileList) {
File[] files = root.listFiles();
for (File file : files) {
if (file.isDirectory()) {
recursivelyAddFilesToList(file, fileList);
} else {
fileList.add(file);
}
}
}

private void assertExistsAndIsDirectory() {
if (rootDirectory.exists() && !rootDirectory.isDirectory()) {
throw new RuntimeException(rootDirectory + " is not a directory");
} else if (!rootDirectory.exists()) {
throw new RuntimeException(rootDirectory + " does not exist");
}
}
}
```