Aws-sdk-java: S3 listBuckets failed to parse XML document

Created on 9 Jul 2015  路  22Comments  路  Source: aws/aws-sdk-java

I keep getting the following XML parse error when trying to perform listBuckets() on the S3 client.

Exception in thread "main" com.amazonaws.AmazonClientException: Failed to parse XML document with handler class com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser$ListAllMyBucketsHandler at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseXmlInputStream(XmlResponsesSaxParser.java:164) at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseListMyBucketsResponse(XmlResponsesSaxParser.java:331) at com.amazonaws.services.s3.model.transform.Unmarshallers$ListBucketsUnmarshaller.unmarshall(Unmarshallers.java:54) at com.amazonaws.services.s3.model.transform.Unmarshallers$ListBucketsUnmarshaller.unmarshall(Unmarshallers.java:51) at com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle(S3XmlResponseHandler.java:62) at com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle(S3XmlResponseHandler.java:31) at com.amazonaws.http.AmazonHttpClient.handleResponse(AmazonHttpClient.java:1050) at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:724) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:467) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:302) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3672) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3621) at com.amazonaws.services.s3.AmazonS3Client.listBuckets(AmazonS3Client.java:690) at com.amazonaws.services.s3.AmazonS3Client.listBuckets(AmazonS3Client.java:698) at esp.topologies.SignatureTopology.main(SignatureTopology.java:55) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140) Caused by: org.xml.sax.SAXParseException; lineNumber: 160; columnNumber: 91; The entity "ntilde" was referenced, but not declared. at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:203) at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:177) at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:441) at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:368) at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1436) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEntityReference(XMLDocumentFragmentScannerImpl.java:1902) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:3067) at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606) at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:117) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510) at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848) at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777) at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141) at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213) at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseXmlInputStream(XmlResponsesSaxParser.java:151)```

response-requested
Source
picture amedeiros
👍2

Most helpful comment

I encountered this and worked around it. I read this article here.

They key appears to set the encoding type to _url_.

I was able to do this and get around that XML error.

ListObjectsRequest listObjectsRequest = new ListObjectsRequest()
                .withBucketName("mytoplevelbucket")
                .withPrefix("myfolderinthebucket")
                .withEncodingType("url");

        ObjectListing objectListing = amazonS3.listObjects(listObjectsRequest);
picture brainopener on 15 Jul 2016
👍4 🎉1

All 22 comments

Hi ! Did you find any solution by any chance ?

FrancisToth picture FrancisToth on 23 Sep 2015

Can you provide a repro test case for this ?

manikandanrs picture manikandanrs on 28 Nov 2015

I get the same error with this simple test:

import com.amazonaws.regions.Region;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3Client;

public class AmazonS3Test {
    public static void main(String[] args) {
        AmazonS3 s3 = new AmazonS3Client();
        s3.setRegion(Region.getRegion(Regions.US_WEST_2));

        s3.listBuckets().stream().forEach(System.out::println);
    }
}

This is using version 1.10.30

wimdeblauwe picture wimdeblauwe on 4 Dec 2015
👍1

I've tried the code you provided on my side, but I don't see any exception. Could you provide more context for this?

zhangzhx picture zhangzhx on 7 Mar 2016

Maybe I'm saying something that is obvious but the test case should create the resource that the list response is failing to parse, i.e., a bucket with a "帽" character in the name, but AFAIK you can't create a bucket with the "帽" character in the name...

samuelgmartinez picture samuelgmartinez on 15 Mar 2016

I still get this error but I actually get it for several S3 api calls heres another example.

com.amazonaws.AmazonClientException路Failed to parse XML document with handler class com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser$AccessControlListHandler
XmlResponsesSaxParser.java:159com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseXmlInputStream   
XmlResponsesSaxParser.java:353com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseAccessControlListResponse    
Unmarshallers.java:115com.amazonaws.services.s3.model.transform.Unmarshallers$AccessControlListUnmarshaller.unmarshall  
Unmarshallers.java:112com.amazonaws.services.s3.model.transform.Unmarshallers$AccessControlListUnmarshaller.unmarshall  
S3XmlResponseHandler.java:62com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle  
S3XmlResponseHandler.java:31com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle  
AmazonHttpClient.java:1153com.amazonaws.http.AmazonHttpClient.handleResponse    
AmazonHttpClient.java:853com.amazonaws.http.AmazonHttpClient.executeOneRequest  
AmazonHttpClient.java:576com.amazonaws.http.AmazonHttpClient.executeHelper  
AmazonHttpClient.java:362com.amazonaws.http.AmazonHttpClient.doExecute  
AmazonHttpClient.java:328com.amazonaws.http.AmazonHttpClient.executeWithTimer   
AmazonHttpClient.java:307com.amazonaws.http.AmazonHttpClient.execute    
AmazonS3Client.java:3659com.amazonaws.services.s3.AmazonS3Client.invoke 
AmazonS3Client.java:3612com.amazonaws.services.s3.AmazonS3Client.invoke 
AmazonS3Client.java:2932com.amazonaws.services.s3.AmazonS3Client.getAcl 
AmazonS3Client.java:956com.amazonaws.services.s3.AmazonS3Client.getBucketAcl    
AmazonS3Client.java:947com.amazonaws.services.s3.AmazonS3Client.getBucketAcl
amedeiros picture amedeiros on 22 Mar 2016

I've encountered this problem when the SDK can't locate AWS credentials. The request is redirected to an HTML page, hence the parsing failure.

You can supply credentials using one of the methods described at http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/DefaultAWSCredentialsProviderChain.html

athempel picture athempel on 23 Mar 2016

@athempel We use session credentials for everything. All our requests are third party access so we have to use session credentials. So it is not an issue of not using credentials. The problem is simply an xml parse error from the sdk.

Let me see if I can gather more information.

amedeiros picture amedeiros on 23 Mar 2016

I encountered this and worked around it. I read this article here.

They key appears to set the encoding type to _url_.

I was able to do this and get around that XML error.

ListObjectsRequest listObjectsRequest = new ListObjectsRequest()
                .withBucketName("mytoplevelbucket")
                .withPrefix("myfolderinthebucket")
                .withEncodingType("url");

        ObjectListing objectListing = amazonS3.listObjects(listObjectsRequest);
brainopener picture brainopener on 15 Jul 2016
👍4 🎉1

URL encoding is enabled by default in the new versions of the SDK. Please try with the latest version of the SDK. Feel free to reopen if the issue still exists.

manikandanrs picture manikandanrs on 17 Aug 2016

@manikandanrs can you please specify from which version of the SDK the encoding type has been added by default?

hiyer picture hiyer on 1 Mar 2017

From the Git history, I can see this change was released as part of 1.10.45

manikandanrs picture manikandanrs on 1 Mar 2017
👍1

@manikandanrs thank you for the update!

hiyer picture hiyer on 1 Mar 2017

I still see this error. I am using sdk version: 1.11.337

binshi picture binshi on 11 Jun 2018

I also experience this error on sdk version 1.11.308

bobthebuilderberlin picture bobthebuilderberlin on 12 Jun 2018
👍1

getting this in 1.11.355

com.amazonaws.SdkClientException: Failed to parse XML document with handler class com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser$ListBucketHandler
  at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseXmlInputStream(XmlResponsesSaxParser.java:162) ~[aws-java-sdk-s3-1.11.355.jar!/:na]
  at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseListBucketObjectsResponse(XmlResponsesSaxParser.java:318) ~[aws-java-sdk-s3-1.11.355.jar!/:na]
  at com.amazonaws.services.s3.model.transform.Unmarshallers$ListObjectsUnmarshaller.unmarshall(Unmarshallers.java:70) ~[aws-java-sdk-s3-1.11.355.jar!/:na]
  at com.amazonaws.services.s3.model.transform.Unmarshallers$ListObjectsUnmarshaller.unmarshall(Unmarshallers.java:59) ~[aws-java-sdk-s3-1.11.355.jar!/:na]
  at com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle(S3XmlResponseHandler.java:62) ~[aws-java-sdk-s3-1.11.355.jar!/:na]
  at com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle(S3XmlResponseHandler.java:31) ~[aws-java-sdk-s3-1.11.355.jar!/:na]
  at com.amazonaws.http.response.AwsResponseHandlerAdapter.handle(AwsResponseHandlerAdapter.java:70) ~[aws-java-sdk-core-1.11.355.jar!/:na]
  at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleResponse(AmazonHttpClient.java:1555) ~[aws-java-sdk-core-1.11.355.jar!/:na]
  at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1272) ~[aws-java-sdk-core-1.11.355.jar!/:na]
  at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1058) ~[aws-java-sdk-core-1.11.355.jar!/:na]
  at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743) ~[aws-java-sdk-core-1.11.355.jar!/:na]
  at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717) ~[aws-java-sdk-core-1.11.355.jar!/:na]
  at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699) ~[aws-java-sdk-core-1.11.355.jar!/:na]
  at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667) ~[aws-java-sdk-core-1.11.355.jar!/:na]
  at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649) ~[aws-java-sdk-core-1.11.355.jar!/:na]
  at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513) ~[aws-java-sdk-core-1.11.355.jar!/:na]
  at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4365) ~[aws-java-sdk-s3-1.11.355.jar!/:na]
  at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4312) ~[aws-java-sdk-s3-1.11.355.jar!/:na]
  at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4306) ~[aws-java-sdk-s3-1.11.355.jar!/:na]
  at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:840) ~[aws-java-sdk-s3-1.11.355.jar!/:na]
  at com.amazonaws.services.s3.AmazonS3Client.listNextBatchOfObjects(AmazonS3Client.java:911) ~[aws-java-sdk-s3-1.11.355.jar!/:na]
  at com.amazonaws.services.s3.AmazonS3Client.listNextBatchOfObjects(AmazonS3Client.java:888) ~[aws-java-sdk-s3-1.11.355.jar!/:na]
...
Caused by: org.xml.sax.SAXParseException: XML document structures must start and end within the same entity.
  at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:203) ~[na:1.8.0_171]
...
  at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseXmlInputStream(XmlResponsesSaxParser.java:148) ~[aws-java-sdk-s3-1.11.355.jar!/:na]
MastaP picture MastaP on 29 Jun 2018
👍1

Try to get listObjects:

 ListObjectsRequest listObjectsRequest = new ListObjectsRequest()
                    .withBucketName(outputBucketName)
                    .withPrefix("folderName/")
                    .withEncodingType("url");
            ObjectListing objects = s3Client.listObjects(listObjectsRequest);

and getting the same error:

Request processing failed; nested exception is com.amazonaws.SdkClientException: Failed to parse XML document with handler class com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser$ListBucketHandler] with root cause

org.xml.sax.SAXParseException: Premature end of file.
at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source) ~[xercesImpl-2.9.1.jar:na]
at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source) ~[xercesImpl-2.9.1.jar:na]
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source) ~[xercesImpl-2.9.1.jar:na]
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source) ~[xercesImpl-2.9.1.jar:na]
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source) ~[xercesImpl-2.9.1.jar:na]
at org.apache.xerces.impl.XMLVersionDetector.determineDocVersion(Unknown Source) ~[xercesImpl-2.9.1.jar:na]
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) ~[xercesImpl-2.9.1.jar:na]
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) ~[xercesImpl-2.9.1.jar:na]
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source) ~[xercesImpl-2.9.1.jar:na]
at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source) ~[xercesImpl-2.9.1.jar:na]
at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseXmlInputStream(XmlResponsesSaxParser.java:142) ~[aws-java-sdk-s3-1.11.125.jar:na]
at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseListBucketObjectsResponse(XmlResponsesSaxParser.java:298) ~[aws-java-sdk-s3-1.11.125.jar:na]
at com.amazonaws.services.s3.model.transform.Unmarshallers$ListObjectsUnmarshaller.unmarshall(Unmarshallers.java:70) ~[aws-java-sdk-s3-1.11.125.jar:na]
at com.amazonaws.services.s3.model.transform.Unmarshallers$ListObjectsUnmarshaller.unmarshall(Unmarshallers.java:59) ~[aws-java-sdk-s3-1.11.125.jar:na]
at com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle(S3XmlResponseHandler.java:62) ~[aws-java-sdk-s3-1.11.125.jar:na]
at com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle(S3XmlResponseHandler.java:31) ~[aws-java-sdk-s3-1.11.125.jar:na]
at com.amazonaws.http.response.AwsResponseHandlerAdapter.handle(AwsResponseHandlerAdapter.java:70) ~[aws-java-sdk-core-1.11.125.jar:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleResponse(AmazonHttpClient.java:1503) ~[aws-java-sdk-core-1.11.125.jar:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1226) ~[aws-java-sdk-core-1.11.125.jar:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1030) ~[aws-java-sdk-core-1.11.125.jar:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:742) ~[aws-java-sdk-core-1.11.125.jar:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:716) ~[aws-java-sdk-core-1.11.125.jar:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699) ~[aws-java-sdk-core-1.11.125.jar:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667) ~[aws-java-sdk-core-1.11.125.jar:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649) ~[aws-java-sdk-core-1.11.125.jar:na]
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513) ~[aws-java-sdk-core-1.11.125.jar:na]
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4169) ~[aws-java-sdk-s3-1.11.125.jar:na]
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4116) ~[aws-java-sdk-s3-1.11.125.jar:na]
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4110) ~[aws-java-sdk-s3-1.11.125.jar:na]
at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:819) ~[aws-java-sdk-s3-1.11.125.jar:na]

I added policy for outputBucket for listObjects and also for user policy to get listObjects.

nikolaynest picture nikolaynest on 3 Jul 2018

So, is there a solution to this issue?

glaksh100 picture glaksh100 on 23 Jul 2018

I had the same issue with listing objects or reading anything from the bucket. Changing the S3 bucket permissions solved the problem. Just attaching S3FullAccessControl IAM Policy to the user wont do. Give the necessary permissions to access the bucket else it will keep throwing different errors.

ChiroBan picture ChiroBan on 29 Aug 2018
👎2

I encountered this and worked around it. I read this article here.

They key appears to set the encoding type to _url_.

I was able to do this and get around that XML error.

ListObjectsRequest listObjectsRequest = new ListObjectsRequest()
                .withBucketName("mytoplevelbucket")
                .withPrefix("myfolderinthebucket")
                .withEncodingType("url");

        ObjectListing objectListing = amazonS3.listObjects(listObjectsRequest);

January 2019, I found that this is still the solution to the problem. Other parts of the API just get a full bucket + folder, but the list objects v2 needs the folder as a separate field.

cclark31415 picture cclark31415 on 8 Jan 2019 
ListObjectsRequest listObjectsRequest = new ListObjectsRequest()
                ....

January 2019, I found that this is still the solution to the problem.

Not to discount this reply as a solution to a ListObjects problem, but this issue is for ListBuckets, not ListObjects. I just had this (ListBuckets) problem after authenticating through an AWS credentials file (~/.aws/credentials) from the SSO "Command Line or Programmatic Access" dialog instructions. The ListBuckets call fails with "failed to parse XML document". My solution was to not use the default authentication chain and instead use BasicSessionCredentials and AWSStaticCredentialsProvider:

            BasicSessionCredentials basicSessionCredentials =
                    new BasicSessionCredentials(
                            accessId,
                            accessKey,
                            sessionToken);

            AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
                    .withCredentials(new AWSStaticCredentialsProvider(
                            basicSessionCredentials))
                    .build();

            return s3Client.listBuckets();
twelveeighty picture twelveeighty on 21 Jan 2020

I encountered this and worked around it. I read this article here.

They key appears to set the encoding type to _url_.

I was able to do this and get around that XML error.

ListObjectsRequest listObjectsRequest = new ListObjectsRequest()
                .withBucketName("mytoplevelbucket")
                .withPrefix("myfolderinthebucket")
                .withEncodingType("url");

        ObjectListing objectListing = amazonS3.listObjects(listObjectsRequest);

June, 2020. aws-java-sdk-s3:1.11.415
Same error. And helps only the hack above

kgusman picture kgusman on 5 Jun 2020
👍2
Was this page helpful?
0 / 5 - 0 ratings

Related issues

umangs9 picture umangs9  路  3Comments
gribbet picture gribbet  路  5Comments
DaichiUeura picture DaichiUeura  路  3Comments
dsilvasc picture dsilvasc  路  5Comments
martinth picture martinth  路  4Comments