exist 🚀 - Client unable to upload large XML files (4.1.0)

How in the client are you uploading the file?

On Mon, 14 May 2018, 06:15 thammr, notifications@github.com wrote:

When I try to upload a large XML file (>512k) using the client, I see the
following error in the Java console:

Exception in thread "Thread-23" java.lang.NullPointerException
at
org.exist.xmldb.RemoteCollection.uploadAndStore(RemoteCollection.java:606)
at
org.exist.xmldb.RemoteCollection.storeResource(RemoteCollection.java:507)
at
org.exist.xmldb.RemoteCollection.storeResource(RemoteCollection.java:476)
at org.exist.client.InteractiveClient.store(InteractiveClient.java:1763)
at org.exist.client.InteractiveClient.parse(InteractiveClient.java:1673)
at org.exist.client.ClientFrame$2.run(ClientFrame.java:980)

I tried to reconstruct the code to understand what the problem is.

It appears the res.getContent() returns a String (not a File or
InputStream). Consequently the InputStream instance "is" is never
instantiated and generates a null pointer exception.

This bug also causes restore to skip large files.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/eXist-db/exist/issues/1874, or mute the thread
https://github.com/notifications/unsubscribe-auth/ABNJuQc5FFLFQW-Ff8rsduGayEO4w0lbks5tyNOogaJpZM4T9BnS
.

adamretter on 14 May 2018

Selecting the Add Document button on the toolbar gives the above error if the file is large. The upload dialog simply hangs.
Restoring a backup gives a "null" for each large file in the progress dialog. Each failed file is listed in the error dialog at the completion. The stack dumps in the java console indicate the same error location ie 606 in RemoteCollection.

thammr on 14 May 2018

Please fill in the Issue template as a lot of essential information is missing here, making this a potential long ping-pong thread with a lot of followup questions. For next ticket please use the template otherwise we'll not be able to pickup the issues reported effectively.

dizzzz on 14 May 2018

Where do I find the Issue Template?

thammr on 15 May 2018

You’ll see it whenever you create a new issue. It’s supplied from https://github.com/eXist-db/exist/blob/develop/.github/ISSUE_TEMPLATE.md.

joewiz on 15 May 2018

Problem

Client cannot upload large XML files. If I upload such a file by clicking on the add resource button, the upload fails to complete. The java console shows the stack-dump, I initially reported.
More significantly, when restoring a backup, large XML files are skipped. The java console displays a stack-dump for each failed file. This stack-dump indicates the same error location. The error dialog displayed at the end of the backup apparently tries to list the skipped files but this dialog has no scroll-bars and is generally scrambled.
Looking at the code, it appears that any XML file larger than 512k will cause this error.
I can find no relevant errors in the eXist logs.
All these files can be successfully uploaded using the Collection App.

Expected

I expect the client to upload large XML files without error. As far as I know, the client provides the only convenient way of creating and restoring partial backups. This is useful when upgrading eXist.

Reproduce

In my experience, any attempt to upload a large XML file using the client fails.

Context

eXist 4.1.0
Not sure about git revision hash but version.txt says scm.revision 919514c68
OpenJDK 1.8.0_131 on CentOS 7-4 64 bit
jdk 1.8.0_151-b12 on CentOS 6.5 64 bit
jdk1.8.0_152-b16 on Windows 7 32 bit
No custom changes

thammr on 16 May 2018

Hmm I was trying to reproduce this, and found that on macOS 10.13.4 i cannot use the java web client at all see java bug. Somehow when trying to start the jnlp it cannot detect java. So not much help, i did restore a backup with some large file (10mb and up) without problems.

duncdrum on 17 May 2018

We are also experiencing similar issue. When we are trying to upload large file using Java web client, it just continuous forever without any error. We are also seeing this error while doing the database restore then we get following error message,

Exception: ----------------------------------------
Problems occurred found during restore:
WARN: Failed to restore resource 'somename.xml'
from file 'c:...\somename.xml'.
Reason: null

We had very simple xml file with simple structure but with high repetition of nodes shown in below example.
<results> <errorCodes> <errorCode family="FAMILY_1"> <component>COMPONENT_1</component> <code>0000</code> </errorCode> <errorCode family="FAMILY_2"> <component>COMPONENT_2</component> <code>0001</code> </errorCode> ... ... ... ... </errorCodes> </results>

So the main issue is not because of large file (the above file fails with just less than 1 MB size) but with high repetition of xml nodes, like in the above example if <errorCode>...</errorCode> repeated for large number of (lets say 10000) times then it fails.

iampopuri on 22 May 2018

@iampopuri what version of eXist-db ? I am pretty sure this has been fixed ( i had similar issues, like you describe).

this ticket is about a different problem.

dizzzz on 22 May 2018

4.1.0
I think the actual root cause of above issue mentioned by @thammr is because of reason that I have provided in my previous post.

iampopuri on 22 May 2018

Hi all,

We have the same error with files larger than 512Mb.

We have installed eXist-db 4.1 (project.built=201804161617 scm.revision=919514c68) on a server and our java application is writing some xml files in the database. To do it, we use the method "storeResource" in the class org.exist-db.xmldb.

If the files length is greater than 512K, we got the same error as mentionned by @thammr above:

Caused by: java.lang.NullPointerException
at org.exist.xmldb.RemoteCollection.uploadAndStore(RemoteCollection.java:606) ~[classes/:?]
at org.exist.xmldb.RemoteCollection.storeResource(RemoteCollection.java:507) ~[classes/:?]
at org.exist.xmldb.RemoteCollection.storeResource(RemoteCollection.java:476) ~[classes/:?]
at com.lswe.irisportail.irisweb.existdb.DbHandler.addOrReplaceResourceToCol(DbHandler.java:371) ~[classes/:?]
at com.lswe.irisportail.irisweb.existdb.DbHandler.addOrReplaceResourceToCol(DbHandler.java:411) ~[classes/:?]
at com.lswe.irisportail.irisweb.existdb.DbHandler.addOrReplaceResourceToCol(DbHandler.java:417) ~[classes/:?]
at sun.reflect.GeneratedMethodAccessor123.invoke(Unknown Source) ~[?:?]

It seems that the "res.getContent" within the uploadStore method returns a byte[] and not a file or an input stream.

algrw on 23 May 2018

I can confirm the same issue :-( with the webstart client uploading a large file fails for v4.1 :-(

client side: (log)

serverside:
nothing

Code:

@adamretter

dizzzz on 24 May 2018

😕1

👍 One of our clients just encountered the same problem using Oxygen 20.0 to upload files to an Exist 4.1 server.

raducoravu on 24 May 2018

Yes, it was me. Also, the problem I'm having concerns any file size (for example an .xml file weighing 57KB).

This is the error I get when I drag and drop a file to my eXist-db 4.1.0 collection from oXygen 20:

```The application detected an error that is not automatically handled. It is possible that this error is not critical and the application may continue to work normally but our recommendation is to report the error and restart the application. If the application has become unstable and cannot be closed normally, you can use the Force Quit button from this dialog. Be aware that by doing this you will lose all unsaved changes in the opened documents.

Here are the error details:

[ CopyMove_DND_Thread ] - java.lang.NullPointerException
java.lang.NullPointerException
at org.exist.xmldb.RemoteCollection.uploadAndStore(RemoteCollection.java:606)
at org.exist.xmldb.RemoteCollection.storeResource(RemoteCollection.java:507)
at org.exist.xmldb.RemoteCollection.storeResource(RemoteCollection.java:476)
at ro.sync.db.nxd.exist.i$_b.g(Unknown Source)
at ro.sync.db.nxd.d.mi(Unknown Source)
at ro.sync.db.nxd.d.oi(Unknown Source)
at ro.sync.db.nxd.d.ni(Unknown Source)
at ro.sync.db.f$1.pfe(Unknown Source)
at ro.sync.ui.application.nb.run(Unknown Source)
at ro.sync.ui.application.nb.start(Unknown Source)
at ro.sync.db.f.kig(Unknown Source)
at ro.sync.db.f.ivf(Unknown Source)
at ro.sync.db.f.lig(Unknown Source)
at ro.sync.db.f.urlsDropped(Unknown Source)
at ro.sync.exml.editor.se.ggg(Unknown Source)
at ro.sync.exml.editor.se.kfg(Unknown Source)
at ro.sync.ui.k.b.b(Unknown Source)
at ro.sync.ui.k.b.drop(Unknown Source)
at ro.sync.ui.k.b.d$_b.drop(Unknown Source)
at sun.awt.dnd.SunDropTargetContextPeer.processDropMessage(SunDropTargetContextPeer.java:538)
at sun.lwawt.macosx.CDropTargetContextPeer.processDropMessage(CDropTargetContextPeer.java:143)
at sun.awt.dnd.SunDropTargetContextPeer$EventDispatcher.dispatchDropEvent(SunDropTargetContextPeer.java:852)
at sun.awt.dnd.SunDropTargetContextPeer$EventDispatcher.dispatchEvent(SunDropTargetContextPeer.java:776)
at sun.awt.dnd.SunDropTargetEvent.dispatch(SunDropTargetEvent.java:48)
at java.awt.Component.dispatchEventImpl(Component.java:4744)
at java.awt.Container.dispatchEventImpl(Container.java:2295)
at java.awt.Component.dispatchEvent(Component.java:4711)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4889)
at java.awt.LightweightDispatcher.processDropTargetEvent(Container.java:4600)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4462)
at java.awt.Container.dispatchEventImpl(Container.java:2281)
at java.awt.Window.dispatchEventImpl(Window.java:2746)
at java.awt.Component.dispatchEvent(Component.java:4711)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:760)
at java.awt.EventQueue.access$500(EventQueue.java:97)
at java.awt.EventQueue$3.run(EventQueue.java:709)
at java.awt.EventQueue$3.run(EventQueue.java:703)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:80)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:90)
at java.awt.EventQueue$4.run(EventQueue.java:733)
at java.awt.EventQueue$4.run(EventQueue.java:731)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:80)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:730)
at ro.sync.ui.application.ApplicationLauncher$_b$1.dispatchEvent(Unknown Source)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
````

lguariento on 24 May 2018

👍 And yet another Oxygen XML Editor + Exist 4.1 client encountered this problem.

raducoravu on 29 May 2018

Hello, I'm not sure this is related to the same issue: if I edit an existing file in oXygen and hit 'save', I get:

Cannot save the file: java.io.IOException null

java.io.IOException
    at ro.sync.db.nxd.exist.ExistSession$_b.close(Unknown Source)
    at sun.nio.cs.StreamEncoder.implClose(StreamEncoder.java:320)
    at sun.nio.cs.StreamEncoder.close(StreamEncoder.java:149)
    at java.io.OutputStreamWriter.close(OutputStreamWriter.java:233)
    at java.io.FilterWriter.close(FilterWriter.java:104)
    at java.io.BufferedWriter.close(BufferedWriter.java:266)
    at ro.sync.io.i.f(Unknown Source)
    at ro.sync.exml.editor.vc.saveInWriter(Unknown Source)
    at ro.sync.exml.editor.xmleditor.sb.saveInWriter(Unknown Source)
    at ro.sync.exml.editor.le.s(Unknown Source)
    at ro.sync.exml.editor.le.m(Unknown Source)
    at ro.sync.exml.editor.vc.saveDocument(Unknown Source)
    at ro.sync.exml.editor.t.e.mkp(Unknown Source)
    at ro.sync.exml.editor.t.e$1.actionPerformed(Unknown Source)
    at ro.sync.ui.application.action.x$_b$1.pfe(Unknown Source)
    at ro.sync.ui.application.nb.run(Unknown Source)
Caused by: java.lang.NullPointerException
    at org.exist.xmldb.RemoteCollection.uploadAndStore(RemoteCollection.java:606)
    at org.exist.xmldb.RemoteCollection.storeResource(RemoteCollection.java:507)
    at org.exist.xmldb.RemoteCollection.storeResource(RemoteCollection.java:476)
    ... 16 more

eXist-db 4.1.0, oXygen 20

lguariento on 31 May 2018

All traces show an NPE with line 606 of RemoteCollection#uploadAndStore(), so yes, I expect so.

dizzzz on 31 May 2018

@dizzzz Any hint about when you plan to release Exist 4.1.1?

raducoravu on 31 May 2018

According to https://github.com/eXist-db/exist/milestones it is in 2 weeks and will be v4.2.0. @adamretter

dizzzz on 31 May 2018

@raducoravu eXist-db 4.2.0 will be released today/tomorrow with the fix. Look out for the release announcement...

adamretter on 5 Jun 2018

👍2

Great, @lguariento if at some point you install the Exist 4.2 server and try to save to it from Oxygen, please tell us if this worked for you.

raducoravu on 6 Jun 2018

@raducoravu @adamretter I have upgraded to eXist-db 4.2.0, removed the oXygen database connection, recreated, and I get

Check datasource configuration. Could not instantiate: ro.sync.db.nxd.exist.ExistSession due to: java.lang.NoClassDefFoundError: Could not initialize class org.apache.logging.log4j.util.PropertiesUtil

opon opening the connection itself in oXygen database browser. What should I double-check?

lguariento on 6 Jun 2018

@lguariento Possible you need extra JAR libraries loaded by the data source configured on the client side.
Can you try to create a new data source using the Preferences->"Data Sources" -> "Create eXist-db XML Connection" link?

raducoravu on 6 Jun 2018

@raducoravu That's what I always use.

lguariento on 6 Jun 2018

@raducoravu Oddly (at least for me), after closing and reopening oXygen the connection opens correctly, and I can, at last, save and upload* documents without issues now. Problem solved, as far as I am concerned! Thanks to all.

*I spoke too soon (see below) :(

lguariento on 6 Jun 2018

👍 Great, Oxygen has some internal class loader caches and this is probably why reopening it solved the problem.

raducoravu on 6 Jun 2018

copy
@raducoravu @adamretter OK, so, now I can edit files already in my eXist-db folder and I can delete them. Nebertheless, the copy of a file from my local folder to my /db/apps/etc folder, hangs. Even just dragging and dropping a single .xml file results in an 'operation in progresss' window which remains there forever. If I click 'cancel', the 'cancel' button greys out and the whole oXygen is unusable. The only way to have it usable again is a 'force quit' of oXygen and reopening it (with no .xml copied anyway :/).

lguariento on 6 Jun 2018

Does oxygen have the latest eXist jars?

adamretter on 6 Jun 2018

@adamretter Is it somethign I should check? That's what I see from my end.
jar

lguariento on 6 Jun 2018

@lguariento @adamretter I will try to install Exist 4.2 on my side and reproduce the hang, then add some logging and see where it hangs.

raducoravu on 6 Jun 2018

👍1

Thanks @raducoravu

adamretter on 6 Jun 2018

👍1

We are uploading the resource on a separate thread (because we want to show the progress). I reproduced the problem and connected to Oxygen using jconsole, the thread is active (it's not blocked) and it seems to keep saving the resource over and over again. The thread stack trace at a certain moment in time looks something like this:

        Name: CopyMove_DND_Thread
        State: RUNNABLE
        Total blocked: 0  Total waited: 0

        Stack trace: 
         java.net.SocketInputStream.socketRead0(Native Method)
        java.net.SocketInputStream.socketRead(Unknown Source)
        java.net.SocketInputStream.read(Unknown Source)
        java.net.SocketInputStream.read(Unknown Source)
        org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139)
        org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155)
        org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284)
        org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
        org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
        org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
        org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165)
        org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:167)
        org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
        org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
        org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
        org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
        org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
        org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
        org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
        org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        ro.sync.net.protocol.http.hc4.WebdavMethodExecHC4.executeMethod(WebdavMethodExecHC4.java:347)
        ro.sync.net.protocol.http.abstraction.AbstractWebdavMethodExec.executeMethod(AbstractWebdavMethodExec.java:194)
        ro.sync.net.protocol.http.abstraction.AbstractWebdavMethodExec.executeMethod(AbstractWebdavMethodExec.java:162)
        ro.sync.net.protocol.http.WebdavHttpURLConnection$WebdavOutputStream.close(WebdavHttpURLConnection.java:757)
        java.util.zip.DeflaterOutputStream.close(Unknown Source)
        org.apache.xmlrpc.client.XmlRpcStreamTransport$ReqWriterImpl.write(XmlRpcStreamTransport.java:71)
        org.apache.xmlrpc.client.XmlRpcStreamTransport$GzipReqWriter.write(XmlRpcStreamTransport.java:88)
        org.apache.xmlrpc.client.XmlRpcSunHttpTransport.writeRequest(XmlRpcSunHttpTransport.java:104)
        org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151)
        org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143)
        org.apache.xmlrpc.client.XmlRpcSunHttpTransport.sendRequest(XmlRpcSunHttpTransport.java:69)
        org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56)
        org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167)
        org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:158)
        org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:147)
        org.exist.xmldb.RemoteCollection.uploadAndStore(RemoteCollection.java:625)
        org.exist.xmldb.RemoteCollection.storeResource(RemoteCollection.java:500)
        org.exist.xmldb.RemoteCollection.storeResource(RemoteCollection.java:469)
        ro.sync.db.nxd.exist.ExistContainer$ExistImportCollection.createResource(ExistContainer.java:661)
        ro.sync.db.nxd.NXDContainer.importUnit(NXDContainer.java:369)
           - locked ro.sync.db.nxd.exist.ExistContainer@49cb21ae
        ro.sync.db.nxd.NXDContainer.importUnits(NXDContainer.java:464)
        ro.sync.db.nxd.NXDContainer.internalImport(NXDContainer.java:434)
        ro.sync.db.DBDnDTreeHandler$1.appRun(DBDnDTreeHandler.java:421)
        ro.sync.ui.application.ApplicationThread.run(ApplicationThread.java:135)

Oxygen calls "RemoteCollection.storeResource" only once (I added some logging to make sure of this) but somehow the Exist code seems to repeat numerous times either the writing of the resource or the closing of the output stream (which triggers the HTTP PUT to be called in Oxygen's custom HTTP url connection implementation).

raducoravu on 6 Jun 2018

Thanks @raducoravu . Since this has proven to be (still) a problem, can someone re-open the issue? Or shall we open a new issue?

lguariento on 6 Jun 2018

@dizzzz I think it was you that submitted a patch to fix this - if you have a moment, could you take a look? I won't be able to get to this just yet

adamretter on 6 Jun 2018

OK...

dizzzz on 7 Jun 2018

reference: https://github.com/eXist-db/exist/pull/1883

dizzzz on 7 Jun 2018

@dizzzz Looking over your commits, a fix like this:

  is = new ByteArrayInputStream(((String) content).getBytes());

is in my opinion not good because it breaks the original encoding of the string. On Windows for example the chars will be converted to bytes using the Cp1252 encoding which is very restrictive. Ideally you could detect the encoding from the XML string by looking at the XML declaration PI, if you cannot detect it then fallback to "UTF8" and do something like ".getBytes("UTF8")"

About the current problem with the XML content being continuously POST-ed by Oxygen to the Exist server, where can I find the latest code for the "RemoteCollection.java"? Is it this one:

https://github.com/eXist-db/exist/blob/develop/src/org/exist/xmldb/RemoteCollection.java

? Because the Java code does not seem to fit with the line/column information I obtained on the stack trace I logged above (RemoteCollection.uploadAndStore(RemoteCollection.java:625)).
Somehow I suspect that the method RemoteCollection.uploadAndStore has that while loop:

        while ((len = is.read(chunk)) > -1) {

and that particular FastByteArrayInputStream input stream implementation might start returning 0 when the end of stream is reached instead of -1 and remain in the is.read loop.

raducoravu on 7 Jun 2018

Found the problem, if you log the "chunk.length" before the while, it's "0", so you will keep on reading 0 bytes from the stream and sending it to the server.
A possible fix could look like this:

        int chunkLength = -1;
        if (res instanceof ExtendedResource) {
            if(res instanceof AbstractRemoteResource) {
                chunkLength = (int)Math.min(((AbstractRemoteResource)res).getContentLength(), MAX_UPLOAD_CHUNK);
            } else {
                chunkLength = (int)Math.min(((ExtendedResource)res).getStreamLength(), MAX_UPLOAD_CHUNK);
            }
        } else {
            chunkLength = MAX_UPLOAD_CHUNK;
        }
        if(chunkLength <= 0) {
          chunkLength = MAX_UPLOAD_CHUNK;
        }
        chunk = new byte[chunkLength];

or you may investigate further into why the "getStreamLength()" and "getContentLength()" may return "0".

raducoravu on 7 Jun 2018

@dizzzz @raducoravu we are also getting below error due to following code. See the exception log below

is = new ByteArrayInputStream(((String) content).getBytes());

Buildfile: C:\Localdata\eXist-db-4.2.0\samples\ant\build.xml
[condition] ERROR StatusLogger No Log4j 2 configuration file found. Using default configuration (logging only errors to the console), or user programmatically provided configurations. Set system property 'log4j2.debug' to show Log4j 2 internal initialization logging. See https://logging.apache.org/log4j/2.x/manual/configuration.html for instructions on how to configure Log4j 2
[http://exist-db.org/ant:exist] Database driver registered.
[http://exist-db.org/ant:exist] Checking collection: xmldb:exist://localhost:8080/exist/xmlrpc/db/test
[http://exist-db.org/ant:exist] Checking collection: xmldb:exist://localhost:8080/exist/xmlrpc/db/test

restore:

[xdb:restore] Restoring from C:\Users\spopuri\Documents\My Received Files\db1
[xdb:restore] Starting restore of backup...
[xdb:restore] Processing backup: C:\Users\spopuri\Documents\My Received Files\db1\__contents__.xml
[xdb:restore] Creating collection /db1
[xdb:restore] Processing backup: C:\Users\spopuri\Documents\My Received Files\db1\Testdata\__contents__.xml
[xdb:restore] Creating collection /db1/Testdata
[xdb:restore] Processing backup: C:\Users\spopuri\Documents\My Received Files\db1\Testdata\Test\__contents__.xml
[xdb:restore] Creating collection /db1/Testdata/Test
[xdb:restore] Failed to restore resource '5468979.xml'
[xdb:restore] from file 'C:\Users\spopuri\Documents\My Received Files\db1\Testdata\Test\5468979.xml'.
[xdb:restore] Reason: networking error
[xdb:restore] 14:37:04.950 [main] ERROR org.exist.backup.restore.RestoreHandler - networking error
[xdb:restore] org.xmldb.api.base.XMLDBException: networking error
[xdb:restore]   at org.exist.xmldb.RemoteCollection.uploadAndStore(RemoteCollection.java:663) ~[exist.jar:4.2.0-SNAPSHOT]
[xdb:restore]   at org.exist.xmldb.RemoteCollection.storeResource(RemoteCollection.java:500) ~[exist.jar:4.2.0-SNAPSHOT]
[xdb:restore]   at org.exist.backup.restore.RestoreHandler.restoreResourceEntry(RestoreHandler.java:392) [exist.jar:4.2.0-SNAPSHOT]
[xdb:restore]   at org.exist.backup.restore.RestoreHandler.startElement(RestoreHandler.java:122) [exist.jar:4.2.0-SNAPSHOT]
[xdb:restore]   at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:509) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:182) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:351) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2784) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643) [?:1.8.0_151]
[xdb:restore]   at org.exist.backup.restore.RestoreHandler.restoreSubCollectionEntry(RestoreHandler.java:255) [exist.jar:4.2.0-SNAPSHOT]
[xdb:restore]   at org.exist.backup.restore.RestoreHandler.startElement(RestoreHandler.java:128) [exist.jar:4.2.0-SNAPSHOT]
[xdb:restore]   at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:509) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:182) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:351) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2784) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643) [?:1.8.0_151]
[xdb:restore]   at org.exist.backup.restore.RestoreHandler.restoreSubCollectionEntry(RestoreHandler.java:255) [exist.jar:4.2.0-SNAPSHOT]
[xdb:restore]   at org.exist.backup.restore.RestoreHandler.startElement(RestoreHandler.java:128) [exist.jar:4.2.0-SNAPSHOT]
[xdb:restore]   at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:509) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:182) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:351) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2784) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213) [?:1.8.0_151]
[xdb:restore]   at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643) [?:1.8.0_151]
[xdb:restore]   at org.exist.backup.Restore.restore(Restore.java:87) [exist.jar:4.2.0-SNAPSHOT]
[xdb:restore]   at org.exist.ant.RestoreTask.execute(RestoreTask.java:92) [exist-optional.jar:4.2.0-SNAPSHOT]
[xdb:restore]   at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292) [ant.jar:1.10.3]
[xdb:restore]   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_151]
[xdb:restore]   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_151]
[xdb:restore]   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_151]
[xdb:restore]   at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_151]
[xdb:restore]   at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106) [ant.jar:?]
[xdb:restore]   at org.apache.tools.ant.Task.perform(Task.java:350) [ant.jar:1.10.3]
[xdb:restore]   at org.apache.tools.ant.Target.execute(Target.java:448) [ant.jar:1.10.3]
[xdb:restore]   at org.apache.tools.ant.Target.performTasks(Target.java:469) [ant.jar:1.10.3]
[xdb:restore]   at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399) [ant.jar:1.10.3]
[xdb:restore]   at org.apache.tools.ant.Project.executeTarget(Project.java:1370) [ant.jar:1.10.3]
[xdb:restore]   at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41) [ant.jar:?]
[xdb:restore]   at org.apache.tools.ant.Project.executeTargets(Project.java:1260) [ant.jar:1.10.3]
[xdb:restore]   at org.apache.tools.ant.Main.runBuild(Main.java:849) [ant.jar:1.10.3]
[xdb:restore]   at org.apache.tools.ant.Main.startAnt(Main.java:228) [ant.jar:1.10.3]
[xdb:restore]   at org.apache.tools.ant.launch.Launcher.run(Launcher.java:283) [ant-launcher.jar:?]
[xdb:restore]   at org.apache.tools.ant.launch.Launcher.main(Launcher.java:101) [ant-launcher.jar:?]
[xdb:restore] Caused by: org.apache.xmlrpc.XmlRpcException: Failed to invoke method parseLocalExt in class org.exist.xmlrpc.RpcConnection: org.xml.sax.SAXException: The XML parser reported a problem: fatal error at (1329,18) : Invalid byte 1 of 1-byte UTF-8 sequence.
[xdb:restore] org.xml.sax.SAXException: fatal error at (1329,18) : Invalid byte 1 of 1-byte UTF-8 sequence.
[xdb:restore] org.xml.sax.SAXParseException; systemId: file:///C:/Users/spopuri/AppData/Local/Temp/exist-db-temp-file-manager-8798511224214355788/exist-db-temp-4769417831116400227.tmp; lineNumber: 1329; columnNumber: 18; Invalid byte 1 of 1-byte UTF-8 sequence.
[xdb:restore]   at org.apache.xmlrpc.client.XmlRpcStreamTransport.readResponse(XmlRpcStreamTransport.java:197) ~[xmlrpc-client-3.1.3.jar:3.1.3]
[xdb:restore]   at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:156) ~[xmlrpc-client-3.1.3.jar:3.1.3]
[xdb:restore]   at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) ~[xmlrpc-client-3.1.3.jar:3.1.3]
[xdb:restore]   at org.apache.xmlrpc.client.XmlRpcSunHttpTransport.sendRequest(XmlRpcSunHttpTransport.java:69) ~[xmlrpc-client-3.1.3.jar:3.1.3]
[xdb:restore]   at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) ~[xmlrpc-client-3.1.3.jar:3.1.3]
[xdb:restore]   at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) ~[xmlrpc-client-3.1.3.jar:3.1.3]
[xdb:restore]   at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:158) ~[xmlrpc-client-3.1.3.jar:3.1.3]
[xdb:restore]   at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:147) ~[xmlrpc-client-3.1.3.jar:3.1.3]
[xdb:restore]   at org.exist.xmldb.RemoteCollection.uploadAndStore(RemoteCollection.java:650) ~[exist.jar:4.2.0-SNAPSHOT]
[xdb:restore]   ... 62 more

iampopuri on 7 Jun 2018

@adamretter As the fix is only client-side any chance for me and @lguariento to get our hands on an "exist.jar" with the modification incorporated? So that we can test and verify this fix with Oxygen before the next Exist release...

raducoravu on 8 Jun 2018

👍1

@raducoravu The latest builds are always here - http://static.adamretter.org.uk/exist-nightly/ however these will be in the 5.0.0-SNAPSHOT line now. Not sure if that is what you want? We just released 4.2.0 and 5.0.0-RC1. There will also be a 4.2.1 shortly

adamretter on 12 Jun 2018

❤1

Thanks @adamretter, then we'll wait for the 4.2.1 release.

raducoravu on 12 Jun 2018

@raducoravu I just installed eXist-db 4.2.1, recreated the oXygen connection, and I can now upload .xml files from local to the server via oXygen, and also save large files without getting weird errors. All solved for me! :)

lguariento on 14 Jun 2018

🎉2

@lguariento Cool, thanks for updating the thread.

raducoravu on 14 Jun 2018

thnx guys

dizzzz on 15 Jun 2018

🎉1

Exist: Client unable to upload large XML files (4.1.0)

Most helpful comment

All 45 comments

Related issues