Dataverse: Publish Dataset - Fails when metadata contains HTML entities w/special characters such as  

[rmo-cdsp]

I added datasets in a Dataverse using Datacite for DOI's. Those datasets were added with python scripts, using the dataverse python api to first create a "simple" (only the required elements) dataset, then update its metadatas with a json (made by extracting datas from xlsx documents).

The problem here seems to come when I try to publish the dataset. The publish fails and there is the error message : "Error – This dataset may not be published because the DataCite Service is currently inaccessible. Please try again. Does the issue continue to persist? Please contact Dataverse Support for assistance. "

Logs are here :dataverse_event_published_error_log.txt

We can see a message : [xml] xml error: The entity "nbsp" was referenced, but not declared.]
So I tried to use the datacite api to try with "custom" xml files, and there was something: if I had a "&nbsp" element, I receive the same error. When I remove it, everything is fine. I tried by adding an entity element ( ]>), (it's a &# and 160 between the "") and the nbsp was replaced and everything worked like a charm. Except that it's by using the api. The problem here is that dataverse doesn't handle nbsp thing (coming from my imports maybe).

Here is a json example asked for doing test on a test server using datacite :
json.txt

[rmo-cdsp]

By looking more precisely to my datas, some utf-8 "No-Break Space", aka \xa0, aka & nbsp, were present. So I purged them, and the error disappeared. I had a second error after that, an argument, optionnal for Dataverse, but mandatory for Datacite it seems:
Response code: 400, [xml] xml error: cvc-minLength-valid: Value '' with length = '0' is not facet-valid with respect to minLength '1' for type '#AnonType_contributorNamecontributorcontributorsresource'.

Here, the contributorName is not part of the contributor field of a Dataset on Dataverse, but it concerns the dataset contact Name, that is optionnal for a dataset, but create an error if you use Datacite, because it's mandatory but not asked anywhere (except in the error message).

After adding a Name to my datasetContact field, I could finally properly Publish my imported Dataset :)

So to resume, there is 1 major problem, and 1 optionnal problem:

• Major : datasetContactName IS mandatory for Datacite publishing
• Optionnal : handle nbsp items entity in the xml sent to Datacite for publishing, or indicate an import problem, or even print in the dataset publishing error message ?

[pdurbin]

See also details at http://irclog.iq.harvard.edu/dataverse/2016-09-02#i_40373

[djbrooke]

We've had another report of the "optional" item from this issue regarding “<em>” tags in the metadata. These can be populated through the Dataverse UI, but they do not play well with Datacite.

[pdurbin]

@djbrooke basic includes: a, b, blockquote, br, cite, code, dd, dl, dt, em according to https://github.com/IQSS/dataverse/blob/v4.5.1/src/main/java/edu/harvard/iq/dataverse/util/MarkupChecker.java#L32 . It sounds like for now we should blacklist em.

[pdurbin]

I believe there's another case of this problem reported here today: https://groups.google.com/d/msg/dataverse-community/v6jgyewGqlI/peTMCoW-DAAJ

[pdurbin]

In pull request #4075 for issue #3845 I worked on the problem of how invalid XML was being sent to DataCite. I'm going to put this is QA so we can at least talk about it at standup.

[djbrooke]

We talked about this in standup. The expectation is that code changes that were already made had fixed this, so we should retest.

[kcondon]

So, to test this I need to:

1. Configure dataverse to use DataCite
2. Identify which metadata fields are sent to DataCite
   Title, Author, Description allow HTML
3. Identify which html tags are unsupported by DataCite
   DataCite doesn't like any tags (<br>) or entities (&,  ) so try each of these.
4. Create a dataset (via API?) which uses unsupported html tags in fields sent to DataCite
   API not needed. Previous code still renders HTML on dataverse but strips tags, characters from metadata sent to DataCite, allowing publication

[pdurbin]

Yeah, HTML content such as   as mentioned above.

[kcondon]

Is there a known list or pattern to unsupported tags or would just trying a known case be enough?

[kcondon]

I see, is this issue essentially a duplicate of https://github.com/IQSS/dataverse/issues/3845?

[kcondon]

OK, still fails:
Caused by: java.lang.RuntimeException: Response code: 400, [xml] xml error: The entity "nbsp" was referenced, but not declared.

This works:
Test DataCite Metadata again & <br>again</br>

This fails:
Test DataCite Metadata again & <br>again</br> &nbsp;

[kcondon]

[2017-09-14T17:19:50.302-0400] [glassfish 4.1] [SEVERE] [] [edu.harvard.iq.dataverse.DataCiteRE
STfullClient] [tid: _ThreadID=56 _ThreadName=jk-connector(4)] [timeMillis: 1505423990302] [leve
lValue: 1000] [[
Response code: 400, [xml] xml error: The entity "nbsp" was referenced, but not declared.]]

[2017-09-14T17:19:50.304-0400] [glassfish 4.1] [WARNING] [AS-EJB-00056] [javax.enterprise.ejb.c
ontainer] [tid: _ThreadID=56 _ThreadName=jk-connector(4)] [timeMillis: 1505423990304] [levelVal
ue: 900] [[
A system exception occurred during an invocation on EJB DOIDataCiteRegisterService, method: p
ublic java.lang.String edu.harvard.iq.dataverse.DOIDataCiteRegisterService.createIdentifier(jav
a.lang.String,java.util.HashMap,edu.harvard.iq.dataverse.Dataset) throws java.io.IOException]]

[2017-09-14T17:19:50.304-0400] [glassfish 4.1] [WARNING] [] [javax.enterprise.ejb.container] [t
id: _ThreadID=56 _ThreadName=jk-connector(4)] [timeMillis: 1505423990304] [levelValue: 900] [[

javax.ejb.TransactionRolledbackLocalException: Exception thrown from bean
at com.sun.ejb.containers.EJBContainerTransactionManager.checkExceptionClientTx(EJBCont
ainerTransactionManager.java:662)
at com.sun.ejb.containers.EJBContainerTransactionManager.postInvokeTx(EJBContainerTrans
actionManager.java:507)
at com.sun.ejb.containers.BaseContainer.postInvokeTx(BaseContainer.java:4566)
at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:2074)
at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:2044)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocati
onHandler.java:220)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObject
InvocationHandlerDelegate.java:88)
at com.sun.proxy.$Proxy348.createIdentifier(Unknown Source)
at edu.harvard.iq.dataverse.__EJB31_Generated__DOIDataCiteRegisterService__Intf____Bean
__.createIdentifier(Unknown Source)
at edu.harvard.iq.dataverse.DOIDataCiteServiceBean.updateIdentifierStatus(DOIDataCiteSe
rviceBean.java:233)
at edu.harvard.iq.dataverse.DOIDataCiteServiceBean.publicizeIdentifier(DOIDataCiteServi
ceBean.java:223)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4786)
at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:656)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:64)
at org.jboss.weld.ejb.SessionBeanInterceptor.aroundInvoke(SessionBeanInterceptor.java:52)
at sun.reflect.GeneratedMethodAccessor118.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
at sun.reflect.GeneratedMethodAccessor120.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4758)
at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4746)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
at com.sun.proxy.$Proxy349.publicizeIdentifier(Unknown Source)
at edu.harvard.iq.dataverse.__EJB31_Generated__DOIDataCiteServiceBean__Intf____Bean__.publicizeIdentifier(Unknown Source)
at edu.harvard.iq.dataverse.engine.command.impl.FinalizeDatasetPublicationCommand.publicizeExternalIdentifier(FinalizeDatasetPublicationCommand.java:166)
at edu.harvard.iq.dataverse.engine.command.impl.FinalizeDatasetPublicationCommand.execute(FinalizeDatasetPublicationCommand.java:100)
at edu.harvard.iq.dataverse.engine.command.impl.FinalizeDatasetPublicationCommand.execute(FinalizeDatasetPublicationCommand.java:42)
at edu.harvard.iq.dataverse.EjbDataverseEngine.submit(EjbDataverseEngine.java:214)
at edu.harvard.iq.dataverse.EjbDataverseEngine$1$1.submit(EjbDataverseEngine.java:385)
at edu.harvard.iq.dataverse.engine.command.impl.PublishDatasetCommand.execute(PublishDatasetCommand.java:86)
at edu.harvard.iq.dataverse.engine.command.impl.PublishDatasetCommand.execute(PublishDatasetCommand.java:37)
at edu.harvard.iq.dataverse.EjbDataverseEngine.submit(EjbDataverseEngine.java:214)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4786)
at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:656)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:64)
at org.jboss.weld.ejb.SessionBeanInterceptor.aroundInvoke(SessionBeanInterceptor.java:52)
at sun.reflect.GeneratedMethodAccessor118.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
at sun.reflect.GeneratedMethodAccessor120.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4758)
at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4746)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
at com.sun.proxy.$Proxy267.submit(Unknown Source)
at edu.harvard.iq.dataverse.__EJB31_Generated__EjbDataverseEngine__Intf____Bean__.submit(Unknown Source)
at edu.harvard.iq.dataverse.DatasetPage.releaseDataset(DatasetPage.java:1812)
at edu.harvard.iq.dataverse.DatasetPage.releaseDataset(DatasetPage.java:1704)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.el.parser.AstValue.invoke(AstValue.java:289)
at com.sun.el.MethodExpressionImpl.invoke(MethodExpressionImpl.java:304)
at org.jboss.weld.util.el.ForwardingMethodExpression.invoke(ForwardingMethodExpression.java:40)
at org.jboss.weld.el.WeldMethodExpression.invoke(WeldMethodExpression.java:50)
at com.sun.faces.facelets.el.TagMethodExpression.invoke(TagMethodExpression.java:105)
at javax.faces.component.MethodBindingMethodExpressionAdapter.invoke(MethodBindingMethodExpressionAdapter.java:87)
at com.sun.faces.application.ActionListenerImpl.processAction(ActionListenerImpl.java:102)
at javax.faces.component.UICommand.broadcast(UICommand.java:315)
at javax.faces.component.UIViewRoot.broadcastEvents(UIViewRoot.java:790)
at javax.faces.component.UIViewRoot.processApplication(UIViewRoot.java:1282)
at com.sun.faces.lifecycle.InvokeApplicationPhase.execute(InvokeApplicationPhase.java:81)
at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:198)
at javax.faces.webapp.FacesServlet.service(FacesServlet.java:646)
at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1682)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:344)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
at org.glassfish.tyrus.servlet.TyrusServletFilter.doFilter(TyrusServletFilter.java:295)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
at org.ocpsoft.rewrite.servlet.RewriteFilter.doFilter(RewriteFilter.java:205)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:316)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:160)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:734)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:673)
at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:99)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:174)
at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:415)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:282)
at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:459)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:167)
at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:201)
at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:175)
at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:235)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:284)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:201)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:133)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:112)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:561)
at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:112)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:117)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:56)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(Wor
kerThreadIOStrategy.java:137)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool
.java:565)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.ja
va:545)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Response code: 400, [xml] xml error: The entity "nbsp" w
as referenced, but not declared.
at edu.harvard.iq.dataverse.DataCiteRESTfullClient.postMetadata(DataCiteRESTfullClient.
java:183)
at edu.harvard.iq.dataverse.DOIDataCiteRegisterService.createIdentifier(DOIDataCiteRegi
sterService.java:93)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43
)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManag
er.java:1081)
at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.
java:1153)
at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4786)
at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:656)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorMana
ger.java:822)
at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(Abstrac
tEJBRequestScopeActivationInterceptor.java:64)
at org.jboss.weld.ejb.SessionBeanInterceptor.aroundInvoke(SessionBeanInterceptor.java:5
2)
at sun.reflect.GeneratedMethodAccessor118.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43
)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorMan
ager.java:883)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
at sun.reflect.GeneratedMethodAccessor120.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4758)
at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4746)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
... 133 more
]]

[2017-09-14T17:19:50.307-0400] [glassfish 4.1] [WARNING] [] [edu.harvard.iq.dataverse.DOIDataCiteServiceBean] [tid: _ThreadID=56 _ThreadName=jk-connector(4)] [timeMillis: 1505423990307] [levelValue: 900] [[
modifyMetadata failed]]

[pdurbin]

Bummer. Thanks for checking @kcondon

The reason this got on my radar is that @amberleahey linked to it from the "Dev Efforts by the Dataverse Community" spreadsheet at https://docs.google.com/spreadsheets/d/1pl9U0_CtWQ3oz6ZllvSHeyB0EG1M_vZEC_aZ7hREnhE/edit?usp=sharing and indicated the @kevinworthington put a fix in the production installation for Scholars Portal. She called it "DataCite metadata validation (strip out HTML encoding)". For #3845 we stripped out HTML tags but we didn't look at entities like  .

[kcondon]

So if we didn't look at entities, why did we bother testing it?

[pdurbin]

It's good to know where the bugs are. All Dataverse installations using EZID will need to move to DataCite: http://www.cdlib.org/cdlinfo/2017/08/28/ezid-service-update-august-2017/

[kcondon]

I know but if we did not specifically fix entities why would we expect them to work? This does not require an answer, just something to think about going forward.

[pdurbin]

This issue wasn't on my radar when I worked on #3845. In the next retrospective I'll mention that we need a better way to group together related issues so we can fix and QA them in a single pull request.

[kcondon]

I guess my point isn't that you didn't fix it or that it wasn't organized in such a way, it was that we knew it wasn't fixed but we tested it rather than just acknowledging that it was something we have still to do. That does not make sense to me. That's all. We can continue off line in stand up.

[pdurbin]

@kcondon @djbrooke I attempted to talk about this during retrospective yesterday but I don't think I worded it well. I'll try again in the future. Thanks.

[pdurbin]

Unfortunately @philippconzett is suffering from this bug in production as originally reported at https://groups.google.com/d/msg/dataverse-community/kVktk1nzBG8/4wsIsZlBBgAJ

He's running 4.7.1 but the bug is in the "develop" branch as well. In the stack trace below that Philipp sent via https://help.hmdc.harvard.edu/Ticket/Display.html?id=254989 the RuntimeException is being thrown because Dataverse is sending invalid XML to DataCite. A workaround could be to edit the dataset to remove the   and trying to re-publish. See #4228 for an example of how a   got into a dataset title.

Here's the code in question that should validate the XML before it's sent to DataCite:

    httpPost.setHeader("Content-Type", "application/xml;charset=UTF-8");
    httpPost.setEntity(new StringEntity(metadata, "utf-8"));
    try {
        HttpResponse response = httpClient.execute(httpPost,context);

        String data = EntityUtils.toString(response.getEntity(), encoding);
        if (response.getStatusLine().getStatusCode() != 201) {
            String errMsg = "Response code: " + response.getStatusLine().getStatusCode() + ", " + data;
            logger.log(Level.SEVERE, errMsg);
            throw new RuntimeException(errMsg);

https://github.com/IQSS/dataverse/blob/5efe63b5a36b00c799cbb02c2549ba11201d40df/src/main/java/edu/harvard/iq/dataverse/DataCiteRESTfullClient.java#L183

Here's the stack trace showing line the RuntimeException from 183 above:

[2017-11-02T19:58:24.209+0100] [glassfish 4.1] [SEVERE] [] [edu.harvard.iq.dataverse.DataCiteRESTfullClient] [tid: _ThreadID=50 _ThreadName=jk-connector(5)] [timeMillis: 1509649104209] [levelValue: 1000] [[
  Response code: 400, [xml] xml error: The entity "nbsp" was referenced, but not declared.]]

[2017-11-02T19:58:24.210+0100] [glassfish 4.1] [WARNING] [AS-EJB-00056] [javax.enterprise.ejb.container] [tid: _ThreadID=50 _ThreadName=jk-connector(5)] [timeMillis: 1509649104210] [levelValue: 900] [[
  A system exception occurred during an invocation on EJB DOIDataCiteRegisterService, method: public java.lang.String edu.harvard.iq.dataverse.DOIDataCiteRegisterService.createIdentifier(java.lang.String,java.util.HashMap,edu.harvard.iq.dataverse.Dataset) throws java.io.IOException]]

[2017-11-02T19:58:24.210+0100] [glassfish 4.1] [WARNING] [] [javax.enterprise.ejb.container] [tid: _ThreadID=50 _ThreadName=jk-connector(5)] [timeMillis: 1509649104210] [levelValue: 900] [[

javax.ejb.TransactionRolledbackLocalException: Exception thrown from bean
    at com.sun.ejb.containers.EJBContainerTransactionManager.checkExceptionClientTx(EJBContainerTransactionManager.java:662)
    at com.sun.ejb.containers.EJBContainerTransactionManager.postInvokeTx(EJBContainerTransactionManager.java:507)
    at com.sun.ejb.containers.BaseContainer.postInvokeTx(BaseContainer.java:4566)
    at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:2074)
    at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:2044)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:220)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
    at com.sun.proxy.$Proxy327.createIdentifier(Unknown Source)
    at edu.harvard.iq.dataverse.__EJB31_Generated__DOIDataCiteRegisterService__Intf____Bean__.createIdentifier(Unknown Source)
    at edu.harvard.iq.dataverse.DOIDataCiteServiceBean.updateIdentifierStatus(DOIDataCiteServiceBean.java:231)
    at edu.harvard.iq.dataverse.DOIDataCiteServiceBean.publicizeIdentifier(DOIDataCiteServiceBean.java:221)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
    at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
    at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4786)
    at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:656)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:64)
    at org.jboss.weld.ejb.SessionBeanInterceptor.aroundInvoke(SessionBeanInterceptor.java:52)
    at sun.reflect.GeneratedMethodAccessor98.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
    at sun.reflect.GeneratedMethodAccessor97.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
    at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
    at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4758)
    at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4746)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
    at com.sun.proxy.$Proxy303.publicizeIdentifier(Unknown Source)
    at edu.harvard.iq.dataverse.__EJB31_Generated__DOIDataCiteServiceBean__Intf____Bean__.publicizeIdentifier(Unknown Source)
    at edu.harvard.iq.dataverse.engine.command.impl.PublishDatasetCommand.execute(PublishDatasetCommand.java:293)
    at edu.harvard.iq.dataverse.engine.command.impl.PublishDatasetCommand.execute(PublishDatasetCommand.java:42)
    at edu.harvard.iq.dataverse.EjbDataverseEngine.submit(EjbDataverseEngine.java:211)
    at sun.reflect.GeneratedMethodAccessor989.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
    at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
    at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4786)
    at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:656)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:64)
    at org.jboss.weld.ejb.SessionBeanInterceptor.aroundInvoke(SessionBeanInterceptor.java:52)
    at sun.reflect.GeneratedMethodAccessor98.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
    at sun.reflect.GeneratedMethodAccessor97.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
    at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
    at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4758)
    at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4746)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
    at com.sun.proxy.$Proxy330.submit(Unknown Source)
    at edu.harvard.iq.dataverse.__EJB31_Generated__EjbDataverseEngine__Intf____Bean__.submit(Unknown Source)
    at edu.harvard.iq.dataverse.DatasetPage.releaseDataset(DatasetPage.java:1685)
    at edu.harvard.iq.dataverse.DatasetPage.releaseDataset(DatasetPage.java:1577)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.el.parser.AstValue.invoke(AstValue.java:289)
    at com.sun.el.MethodExpressionImpl.invoke(MethodExpressionImpl.java:304)
    at org.jboss.weld.util.el.ForwardingMethodExpression.invoke(ForwardingMethodExpression.java:40)
    at org.jboss.weld.el.WeldMethodExpression.invoke(WeldMethodExpression.java:50)
    at com.sun.faces.facelets.el.TagMethodExpression.invoke(TagMethodExpression.java:105)
    at javax.faces.component.MethodBindingMethodExpressionAdapter.invoke(MethodBindingMethodExpressionAdapter.java:87)
    at com.sun.faces.application.ActionListenerImpl.processAction(ActionListenerImpl.java:102)
    at javax.faces.component.UICommand.broadcast(UICommand.java:315)
    at javax.faces.component.UIViewRoot.broadcastEvents(UIViewRoot.java:790)
    at javax.faces.component.UIViewRoot.processApplication(UIViewRoot.java:1282)
    at com.sun.faces.lifecycle.InvokeApplicationPhase.execute(InvokeApplicationPhase.java:81)
    at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
    at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:198)
    at javax.faces.webapp.FacesServlet.service(FacesServlet.java:646)
    at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1682)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:344)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
    at org.glassfish.tyrus.servlet.TyrusServletFilter.doFilter(TyrusServletFilter.java:295)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
    at org.ocpsoft.rewrite.servlet.RewriteFilter.doFilter(RewriteFilter.java:205)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:316)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:160)
    at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:734)
    at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:673)
    at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:99)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:174)
    at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:415)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:282)
    at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:459)
    at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:167)
    at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:201)
    at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:175)
    at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:235)
    at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:284)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:201)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:133)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:112)
    at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
    at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:561)
    at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:112)
    at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:117)
    at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:56)
    at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:137)
    at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:565)
    at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:545)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Response code: 400, [xml] xml error: The entity "nbsp" was referenced, but not declared.
    at edu.harvard.iq.dataverse.DataCiteRESTfullClient.postMetadata(DataCiteRESTfullClient.java:183)
    at edu.harvard.iq.dataverse.DOIDataCiteRegisterService.createIdentifier(DOIDataCiteRegisterService.java:89)
    at sun.reflect.GeneratedMethodAccessor7485.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
    at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
    at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4786)
    at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:656)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:64)
    at org.jboss.weld.ejb.SessionBeanInterceptor.aroundInvoke(SessionBeanInterceptor.java:52)
    at sun.reflect.GeneratedMethodAccessor98.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
    at sun.reflect.GeneratedMethodAccessor97.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
    at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
    at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4758)
    at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4746)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
    ... 127 more
]]

[2017-11-02T19:58:24.212+0100] [glassfish 4.1] [WARNING] [] [edu.harvard.iq.dataverse.DOIDataCiteServiceBean] [tid: _ThreadID=50 _ThreadName=jk-connector(5)] [timeMillis: 1509649104212] [levelValue: 900] [[
  modifyMetadata failed]]

[2017-11-02T19:58:24.212+0100] [glassfish 4.1] [WARNING] [] [edu.harvard.iq.dataverse.DOIDataCiteServiceBean] [tid: _ThreadID=50 _ThreadName=jk-connector(5)] [timeMillis: 1509649104212] [levelValue: 900] [[
  String javax.ejb.EJBTransactionRolledbackException]]

[2017-11-02T19:58:24.212+0100] [glassfish 4.1] [WARNING] [] [edu.harvard.iq.dataverse.DOIDataCiteServiceBean] [tid: _ThreadID=50 _ThreadName=jk-connector(5)] [timeMillis: 1509649104212] [levelValue: 900] [[
  localized message null]]

[2017-11-02T19:58:24.213+0100] [glassfish 4.1] [WARNING] [] [edu.harvard.iq.dataverse.DOIDataCiteServiceBean] [tid: _ThreadID=50 _ThreadName=jk-connector(5)] [timeMillis: 1509649104213] [levelValue: 900] [[
  cause
javax.ejb.TransactionRolledbackLocalException: Exception thrown from bean
    at com.sun.ejb.containers.EJBContainerTransactionManager.checkExceptionClientTx(EJBContainerTransactionManager.java:662)
    at com.sun.ejb.containers.EJBContainerTransactionManager.postInvokeTx(EJBContainerTransactionManager.java:507)
    at com.sun.ejb.containers.BaseContainer.postInvokeTx(BaseContainer.java:4566)
    at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:2074)
    at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:2044)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:220)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
    at com.sun.proxy.$Proxy327.createIdentifier(Unknown Source)
    at edu.harvard.iq.dataverse.__EJB31_Generated__DOIDataCiteRegisterService__Intf____Bean__.createIdentifier(Unknown Source)
    at edu.harvard.iq.dataverse.DOIDataCiteServiceBean.updateIdentifierStatus(DOIDataCiteServiceBean.java:231)
    at edu.harvard.iq.dataverse.DOIDataCiteServiceBean.publicizeIdentifier(DOIDataCiteServiceBean.java:221)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
    at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
    at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4786)
    at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:656)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:64)
    at org.jboss.weld.ejb.SessionBeanInterceptor.aroundInvoke(SessionBeanInterceptor.java:52)
    at sun.reflect.GeneratedMethodAccessor98.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
    at sun.reflect.GeneratedMethodAccessor97.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
    at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
    at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4758)
    at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4746)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
    at com.sun.proxy.$Proxy303.publicizeIdentifier(Unknown Source)
    at edu.harvard.iq.dataverse.__EJB31_Generated__DOIDataCiteServiceBean__Intf____Bean__.publicizeIdentifier(Unknown Source)
    at edu.harvard.iq.dataverse.engine.command.impl.PublishDatasetCommand.execute(PublishDatasetCommand.java:293)
    at edu.harvard.iq.dataverse.engine.command.impl.PublishDatasetCommand.execute(PublishDatasetCommand.java:42)
    at edu.harvard.iq.dataverse.EjbDataverseEngine.submit(EjbDataverseEngine.java:211)
    at sun.reflect.GeneratedMethodAccessor989.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
    at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
    at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4786)
    at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:656)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:64)
    at org.jboss.weld.ejb.SessionBeanInterceptor.aroundInvoke(SessionBeanInterceptor.java:52)
    at sun.reflect.GeneratedMethodAccessor98.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
    at sun.reflect.GeneratedMethodAccessor97.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
    at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
    at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4758)
    at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4746)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
    at com.sun.proxy.$Proxy330.submit(Unknown Source)
    at edu.harvard.iq.dataverse.__EJB31_Generated__EjbDataverseEngine__Intf____Bean__.submit(Unknown Source)
    at edu.harvard.iq.dataverse.DatasetPage.releaseDataset(DatasetPage.java:1685)
    at edu.harvard.iq.dataverse.DatasetPage.releaseDataset(DatasetPage.java:1577)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.el.parser.AstValue.invoke(AstValue.java:289)
    at com.sun.el.MethodExpressionImpl.invoke(MethodExpressionImpl.java:304)
    at org.jboss.weld.util.el.ForwardingMethodExpression.invoke(ForwardingMethodExpression.java:40)
    at org.jboss.weld.el.WeldMethodExpression.invoke(WeldMethodExpression.java:50)
    at com.sun.faces.facelets.el.TagMethodExpression.invoke(TagMethodExpression.java:105)
    at javax.faces.component.MethodBindingMethodExpressionAdapter.invoke(MethodBindingMethodExpressionAdapter.java:87)
    at com.sun.faces.application.ActionListenerImpl.processAction(ActionListenerImpl.java:102)
    at javax.faces.component.UICommand.broadcast(UICommand.java:315)
    at javax.faces.component.UIViewRoot.broadcastEvents(UIViewRoot.java:790)
    at javax.faces.component.UIViewRoot.processApplication(UIViewRoot.java:1282)
    at com.sun.faces.lifecycle.InvokeApplicationPhase.execute(InvokeApplicationPhase.java:81)
    at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
    at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:198)
    at javax.faces.webapp.FacesServlet.service(FacesServlet.java:646)
    at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1682)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:344)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
    at org.glassfish.tyrus.servlet.TyrusServletFilter.doFilter(TyrusServletFilter.java:295)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
    at org.ocpsoft.rewrite.servlet.RewriteFilter.doFilter(RewriteFilter.java:205)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:316)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:160)
    at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:734)
    at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:673)
    at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:99)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:174)
    at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:415)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:282)
    at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:459)
    at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:167)
    at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:201)
    at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:175)
    at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:235)
    at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:284)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:201)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:133)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:112)
    at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
    at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:561)
    at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:112)
    at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:117)
    at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:56)
    at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:137)
    at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:565)
    at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:545)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Response code: 400, [xml] xml error: The entity "nbsp" was referenced, but not declared.
    at edu.harvard.iq.dataverse.DataCiteRESTfullClient.postMetadata(DataCiteRESTfullClient.java:183)
    at edu.harvard.iq.dataverse.DOIDataCiteRegisterService.createIdentifier(DOIDataCiteRegisterService.java:89)
    at sun.reflect.GeneratedMethodAccessor7485.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
    at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
    at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4786)
    at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:656)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:64)
    at org.jboss.weld.ejb.SessionBeanInterceptor.aroundInvoke(SessionBeanInterceptor.java:52)
    at sun.reflect.GeneratedMethodAccessor98.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
    at sun.reflect.GeneratedMethodAccessor97.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
    at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
    at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4758)
    at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4746)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
    ... 127 more
]]

[2017-11-02T19:58:24.213+0100] [glassfish 4.1] [WARNING] [] [edu.harvard.iq.dataverse.DOIDataCiteServiceBean] [tid: _ThreadID=50 _ThreadName=jk-connector(5)] [timeMillis: 1509649104213] [levelValue: 900] [[
  message null]]

[2017-11-02T19:58:24.214+0100] [glassfish 4.1] [WARNING] [AS-EJB-00056] [javax.enterprise.ejb.container] [tid: _ThreadID=50 _ThreadName=jk-connector(5)] [timeMillis: 1509649104214] [levelValue: 900] [[
  A system exception occurred during an invocation on EJB PermissionServiceBean, method: public java.util.Set edu.harvard.iq.dataverse.PermissionServiceBean.permissionsFor(edu.harvard.iq.dataverse.engine.command.DataverseRequest,edu.harvard.iq.dataverse.DvObject)]]

[2017-11-02T19:58:24.214+0100] [glassfish 4.1] [WARNING] [] [javax.enterprise.ejb.container] [tid: _ThreadID=50 _ThreadName=jk-connector(5)] [timeMillis: 1509649104214] [levelValue: 900] [[

javax.ejb.TransactionRolledbackLocalException: Client's transaction aborted
    at com.sun.ejb.containers.EJBContainerTransactionManager.useClientTx(EJBContainerTransactionManager.java:357)
    at com.sun.ejb.containers.EJBContainerTransactionManager.preInvokeTx(EJBContainerTransactionManager.java:251)
    at com.sun.ejb.containers.BaseContainer.preInvokeTx(BaseContainer.java:4524)
    at com.sun.ejb.containers.BaseContainer.preInvoke(BaseContainer.java:1986)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:210)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
    at com.sun.proxy.$Proxy287.permissionsFor(Unknown Source)
    at edu.harvard.iq.dataverse.__EJB31_Generated__PermissionServiceBean__Intf____Bean__.permissionsFor(Unknown Source)
    at edu.harvard.iq.dataverse.EjbDataverseEngine.submit(EjbDataverseEngine.java:190)
    at edu.harvard.iq.dataverse.EjbDataverseEngine$1$1.submit(EjbDataverseEngine.java:382)
    at edu.harvard.iq.dataverse.engine.command.impl.PublishDatasetCommand.execute(PublishDatasetCommand.java:297)
    at edu.harvard.iq.dataverse.engine.command.impl.PublishDatasetCommand.execute(PublishDatasetCommand.java:42)
    at edu.harvard.iq.dataverse.EjbDataverseEngine.submit(EjbDataverseEngine.java:211)
    at sun.reflect.GeneratedMethodAccessor989.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
    at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
    at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4786)
    at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:656)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:64)
    at org.jboss.weld.ejb.SessionBeanInterceptor.aroundInvoke(SessionBeanInterceptor.java:52)
    at sun.reflect.GeneratedMethodAccessor98.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
    at sun.reflect.GeneratedMethodAccessor97.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
    at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
    at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4758)
    at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4746)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
    at com.sun.proxy.$Proxy330.submit(Unknown Source)
    at edu.harvard.iq.dataverse.__EJB31_Generated__EjbDataverseEngine__Intf____Bean__.submit(Unknown Source)
    at edu.harvard.iq.dataverse.DatasetPage.releaseDataset(DatasetPage.java:1685)
    at edu.harvard.iq.dataverse.DatasetPage.releaseDataset(DatasetPage.java:1577)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.el.parser.AstValue.invoke(AstValue.java:289)
    at com.sun.el.MethodExpressionImpl.invoke(MethodExpressionImpl.java:304)
    at org.jboss.weld.util.el.ForwardingMethodExpression.invoke(ForwardingMethodExpression.java:40)
    at org.jboss.weld.el.WeldMethodExpression.invoke(WeldMethodExpression.java:50)
    at com.sun.faces.facelets.el.TagMethodExpression.invoke(TagMethodExpression.java:105)
    at javax.faces.component.MethodBindingMethodExpressionAdapter.invoke(MethodBindingMethodExpressionAdapter.java:87)
    at com.sun.faces.application.ActionListenerImpl.processAction(ActionListenerImpl.java:102)
    at javax.faces.component.UICommand.broadcast(UICommand.java:315)
    at javax.faces.component.UIViewRoot.broadcastEvents(UIViewRoot.java:790)
    at javax.faces.component.UIViewRoot.processApplication(UIViewRoot.java:1282)
    at com.sun.faces.lifecycle.InvokeApplicationPhase.execute(InvokeApplicationPhase.java:81)
    at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
    at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:198)
    at javax.faces.webapp.FacesServlet.service(FacesServlet.java:646)
    at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1682)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:344)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
    at org.glassfish.tyrus.servlet.TyrusServletFilter.doFilter(TyrusServletFilter.java:295)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
    at org.ocpsoft.rewrite.servlet.RewriteFilter.doFilter(RewriteFilter.java:205)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:316)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:160)
    at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:734)
    at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:673)
    at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:99)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:174)
    at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:415)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:282)
    at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:459)
    at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:167)
    at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:201)
    at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:175)
    at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:235)
    at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:284)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:201)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:133)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:112)
    at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
    at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:561)
    at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:112)
    at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:117)
    at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:56)
    at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:137)
    at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:565)
    at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:545)
    at java.lang.Thread.run(Thread.java:748)
]]

[2017-11-02T19:58:24.241+0100] [glassfish 4.1] [SEVERE] [] [edu.harvard.iq.dataverse.DatasetPage] [tid: _ThreadID=50 _ThreadName=jk-connector(5)] [timeMillis: 1509649104241] [levelValue: 1000] [[
  Command edu.harvard.iq.dataverse.engine.command.impl.PublishDatasetCommand@2e107f1d failed: null]]

[pdurbin]

See the note by @philippconzett at https://groups.google.com/d/msg/dataverse-community/kVktk1nzBG8/bhFkuO9cBwAJ for his workaround:

Finally, I figured out what caused the error message: The author had used [ and ] in the description field, rendering the following html code:

[article abstract] Phonological variation ...

I simply used the source code of the dataset page to search for "nbsp" :)

[pdurbin]

I just re-tested this bug on the phoenix server as of 8cfca31 now that pull request #4807 has been merged for issue #4819 and I'm happy to report that while the end user doesn't get a specific error. Instead of some weird error about "FinalizeDatasetPublicationCommand" and "null", the user gets a more appropriate error saying "This dataset may not be published due to an error when contacting the DataCite Service" like this:

Somewhat oddly, the error in Glassfish's server.log changed. I was expecting to see something like Response code: 400, [xml] xml error: The entity "nbsp" was referenced, but not declared as I had seen before (and as shown above) but this time it said Response code: 400, xmlns namespace not found. Weird.

[djbrooke]

Hi @mfenner! We discussed this in our backlog grooming today and we're looking at sanitizing on Dataset create/edit or just on output to Datacite. Any opinions on which approach we should take?

[mheppler]

@pdurbin I saw that you retested this last July. I gave it a whirl locally after #4077 was merged and got a successful publish with ampersand & & & nonbreaking space     for the metadata.

[pdurbin]

@mheppler cool but are you using the "FAKE" DOI provider locally? We need to test with real DataCite.

[mheppler]

Ah, yes, I am set up locally with the FAKE DOI. Apologies for missing that.

[4tikhonov]

We would like to have a line break in the Description field but it seems it's removed on the landing page. However, \n is available in the metadata editor:

The final result should look like on the screenshot, with line breaks:

@pdurbin @djbrooke

[mfenner]

FWIW, DataCite supports only a few HTML tags in the DOI description field, but that includes </br>. The complete list of tags is strong em b i code pre sub sup br.

[pdurbin]

@mfenner thanks!

@4tikhonov you'll need to add the </br> tags manually as part of your migration process. The description field supports HTML and HTML ignores whitespace, generally.

[4tikhonov]

@pdurbin, it makes no sense to add
tags to metadata, it can bring unexpected behavior to the machine-to-machine interaction. At the moment we've modified Dataverse application on frontend but would like to get more sustainable solution.

[pdurbin]

@4tikhonov ok, what would be a more sustainable solution? Markdown support?

[4tikhonov]

@pdurbin, probably it would be better to replace '\n' with <br /> in metadata when rendering on the dataset landing page.

[mfenner]

The subset that we picked (strong em b i code pre sub sup br) is what is typically used in forums and other places that allow "unsafe" input. We are of course maybe too careful, as we also don't support for example links or lists. But I think this is what you typically see with indexing services.

Something that we "unofficially" support is Latex, which for example the folks at CERN make heavy use of and that we support via Mathjax (but that is tricky).

[scolapasta]

@4tikhonov one challenge with that is for those that have used <br/> you will get additional line breaks that may not have been intended. For machine to machine interaction, I think we may strip out the html, or if we don't maybe we should. (we do strip out, for example, when we store the values in solr)