Dataverse: Not able to publish DOI on dataverse

[flaviofrancisco]

I am using datacite.org to generate our DOIs. We are using a test doi and it is draft mode. But when I try this post method: https://{our domain}/api/datasets/461/actions/:publish?type=major&key={our key}

I have the following error:
Command edu.harvard.iq.dataverse.engine.command.impl.FinalizeDatasetPublicationCommand@1681ef35 failed: null

server.log detail:
[2018-10-26T07:22:02.711+0000] [glassfish 4.1] [SEVERE] [] [edu.harvard.iq.dataverse.api.AbstractApiBean] [tid: _ThreadID=27 _ThreadName=http-listener-1(2)] [timeMillis: 1540538522711] [levelValue: 1000] [[
Error while executing command edu.harvard.iq.dataverse.engine.command.impl.PublishDatasetCommand@ee8e163
edu.harvard.iq.dataverse.engine.command.exception.CommandException: Command edu.harvard.iq.dataverse.engine.command.impl.FinalizeDatasetPublicationCommand@1681ef35 failed: null
at edu.harvard.iq.dataverse.EjbDataverseEngine.submit(EjbDataverseEngine.java:226)
at edu.harvard.iq.dataverse.EjbDataverseEngine$1$1.submit(EjbDataverseEngine.java:405)
at edu.harvard.iq.dataverse.engine.command.impl.PublishDatasetCommand.execute(PublishDatasetCommand.java:92)
at edu.harvard.iq.dataverse.engine.command.impl.PublishDatasetCommand.execute(PublishDatasetCommand.java:31)
at edu.harvard.iq.dataverse.EjbDataverseEngine.submit(EjbDataverseEngine.java:223)
at sun.reflect.GeneratedMethodAccessor1390.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4786)
at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:656)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:64)
at org.jboss.weld.ejb.SessionBeanInterceptor.aroundInvoke(SessionBeanInterceptor.java:52)
at sun.reflect.GeneratedMethodAccessor111.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
at sun.reflect.GeneratedMethodAccessor110.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4758)
at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4746)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
at com.sun.proxy.$Proxy350.submit(Unknown Source)
at edu.harvard.iq.dataverse.__EJB31_Generated__EjbDataverseEngine__Intf____Bean__.submit(Unknown Source)
at edu.harvard.iq.dataverse.api.AbstractApiBean.execCommand(AbstractApiBean.java:554)
at edu.harvard.iq.dataverse.api.Datasets.publishDataset(Datasets.java:443)
at sun.reflect.GeneratedMethodAccessor5587.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:171)
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:152)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:387)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:331)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:103)
at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:271)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:297)
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:254)
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1028)
at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:372)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:381)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:344)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:221)
at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1682)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:344)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
at org.ocpsoft.rewrite.servlet.RewriteFilter.doFilter(RewriteFilter.java:226)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
at edu.harvard.iq.dataverse.api.ApiBlockingFilter.doFilter(ApiBlockingFilter.java:162)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
at edu.harvard.iq.dataverse.api.ApiRouter.doFilter(ApiRouter.java:30)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
at org.apache.catalina.core.ApplicationDispatcher.doInvoke(ApplicationDispatcher.java:873)
at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:739)
at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:575)
at org.apache.catalina.core.ApplicationDispatcher.doDispatch(ApplicationDispatcher.java:546)
at org.apache.catalina.core.ApplicationDispatcher.dispatch(ApplicationDispatcher.java:428)
at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:378)
at edu.harvard.iq.dataverse.api.ApiRouter.doFilter(ApiRouter.java:34)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:316)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:160)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:734)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:673)
at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:99)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:174)
at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:415)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:282)
at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:459)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:167)
at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:201)
at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:175)
at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:235)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:284)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:201)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:133)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:112)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:561)
at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:112)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:117)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:56)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:137)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:565)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:545)
at java.lang.Thread.run(Thread.java:748)
Caused by: javax.ejb.TransactionRolledbackLocalException: Client's transaction aborted
at com.sun.ejb.containers.EJBContainerTransactionManager.useClientTx(EJBContainerTransactionManager.java:357)
at com.sun.ejb.containers.EJBContainerTransactionManager.preInvokeTx(EJBContainerTransactionManager.java:251)
at com.sun.ejb.containers.BaseContainer.preInvokeTx(BaseContainer.java:4524)
at com.sun.ejb.containers.BaseContainer.preInvoke(BaseContainer.java:1986)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:210)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
at com.sun.proxy.$Proxy310.getProviderInformation(Unknown Source)
at edu.harvard.iq.dataverse.__EJB31_Generated__DOIEZIdServiceBean__Intf____Bean__.getProviderInformation(Unknown Source)
at edu.harvard.iq.dataverse.engine.command.impl.FinalizeDatasetPublicationCommand.publicizeExternalIdentifier(FinalizeDatasetPublicationCommand.java:189)
at edu.harvard.iq.dataverse.engine.command.impl.FinalizeDatasetPublicationCommand.execute(FinalizeDatasetPublicationCommand.java:93)
at edu.harvard.iq.dataverse.engine.command.impl.FinalizeDatasetPublicationCommand.execute(FinalizeDatasetPublicationCommand.java:40)
at edu.harvard.iq.dataverse.EjbDataverseEngine.submit(EjbDataverseEngine.java:223)
... 109 more
]]

Any guess?
Thanks in advance;
Flavio.

[pdurbin]

@flaviofrancisco hi. Thanks for the bug report and the stack trace. The line numbers are more useful if we know what version of Dataverse you're running. Is it 4.9.4?

[flaviofrancisco]

Hi @pdurbin, thanks for your message here is our version: v. 4.9.1 build 876-7237517

[pdurbin]

@flaviofrancisco thanks at https://github.com/IQSS/dataverse/blob/v4.9.1/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/FinalizeDatasetPublicationCommand.java#L189 I see a comment that says this:

//This exception winds up nowhere that the end user can see
//maybe add notification?

Which is unfortunate. At the very least we should add more logging. From a quick look at 4.9.4 I'm not sure the logging is any better.

Does publishing fail for all datasets? Does dataset 461 have HTML entities in it? If so, you might be suffering from #3328.

It would probably be helpful to see the entire server.log file so please consider emailing it to [email protected]

[flaviofrancisco]

I will check. We will try to create a new dataset and we will let you know either if works or not.
Thanks.

[flaviofrancisco]

Hi @pdurbin I tried to create a new dataset using: https://{our host}/?mode=migrate&key={our key}&id=b107bbae-cd88-447d-89f1-9acdd1de4417
I have as result an 202 Accepted code but no dataset is created. I send the server.log file for the e-mail that you gave to me.
Thanks;
Flavio.

[pdurbin]

@flaviofrancisco thanks I see your log in https://help.hmdc.harvard.edu/Ticket/Display.html?id=268652 but instead of using the "migrate" API (#5138 is related) can you please create a dataset using the GUI and see if publishing works?

[flaviofrancisco]

Using the GUI it works.

[pdurbin]

@flaviofrancisco great! I assume there's a reason you're trying to use the "migrate" API rather than creating new datasets in the GUI. That "migrate" API was designed to migrate from DVN 3.x to Dataverse 4. Support for DDI import is being actively worked on in #4593. There's a new "Import a Dataset into a Dataverse" API endpoint that accepts JSON: http://guides.dataverse.org/en/4.9.4/api/native-api.html

Can you please tell us more about what you are trying to do? Which platform you're exporting your datasets from? Which format? Thanks!

[flaviofrancisco]

I am a software developer at a Sea Research Institute in Netherlands and we built a internal application for the scientist to expose theirs research data with dois. Our application generates the metadata that is uploaded to dataverse (xml) and datacite (json) and through our application the researchers can update theirs dois.

[pdurbin]

@flaviofrancisco thanks, this helps a lot. It sounds like you're happy to let Dataverse mint your DOIs for you.

If you want to create a dataset using XML, I recommend using SWORD: http://guides.dataverse.org/en/4.9.4/api/sword.html

If you want to create a dataset using JSON, I recommend the native API: http://guides.dataverse.org/en/4.9.4/api/native-api.html#create-a-dataset-in-a-dataverse

Does this help?

[flaviofrancisco]

I will take a look and let you know Monday. Thank you.

[pdurbin]

Sure. The other resource I like to point people toward are these two curl commands for creating datasets with XML or JSON: https://github.com/IQSS/dataverse/blob/v4.9.4/scripts/search/tests/create-all-and-test#L8

Please keep us posted. Thanks!

[flaviofrancisco]

Hi @pdurbin where I can find the schema for the ATOM and JSON datasets used by the SWORD api?

[pdurbin]

There's no schema for Dataverse's native JSON format (please feel free to open an issue about that). The script above uses this JSON file as input: https://github.com/IQSS/dataverse/blob/v4.9.4/scripts/search/tests/data/dataset-finch1.json . For a longer example that exercises more fields, please see https://github.com/IQSS/dataverse/blob/v4.9.4/scripts/api/data/dataset-create-new-all-default-fields.json

http://guides.dataverse.org/en/4.9.4/api/sword.html#dublin-core-terms-dc-terms-qualified-mapping-dataverse-db-element-crosswalk shows the available files in XML. The script I pointed to previously uses this XML file as input: https://github.com/IQSS/dataverse/blob/v4.9.4/scripts/search/tests/data/dataset-trees1.xml . You can find out more about the SWORD/ATOM XML schema at http://swordapp.github.io/SWORDv2-Profile/SWORDProfile.html

[flaviofrancisco]

Hi @pdurbin, just to let you know that I am using the native API. https://{our host}/api/v1/dataverses/1/datasets/?key={our key}. Now I am re-mapping my DOI for a JSON compatible with Dataverse.
Thanks for your help.

[pdurbin]

@flaviofrancisco great! You are welcome to close this issue if you want.