Ckan: Error: Process completed but unable to post to result_url

[jcballesteros]

CKAN 2.7.2 with https (letsenctrypt certificate)

After update a CSV resource, the file is updated and datastore content is the same as the file

After update a CSV resource, the file is updated, but the datastore has the old data. In the datastore tab appears this error: Error: Process completed but unable to post to result_url

On the update log, all appear ok:

Fetching from: https://sandbox-ckan.localidata.com/dataset/incidencias-de-trafico/resource/d7ac1cb1-5a46-407a-9282-55a5dd4117ec/download/recurso.csv
hace -1 días Detalles

Determined headers and types: [{'type': u'text', 'id': u'carretera'}, {'type': u'text', 'id': u'causa'}]
hace -1 días Detalles

Saving chunk 0
hace -1 días Detalles

Successfully pushed 2 entries to "d7ac1cb1-5a46-407a-9282-55a5dd4117ec".
hace -1 días Detalles

CKAN 2.7.2 standar with ssl added follow these instructions (with a letscrypt cert ): https://github.com/conwetlab/ckanext-oauth2/wiki/Starting-CKAN-over-HTTPs-using-Apache-and-Nginx

[torfsen]

I just encountered the same error message ("Process completed but unable to post to result_url") on a CKAN 2.7.3 installation when uploading a CSV file to a newly created resource for the first time. In my case, there are no entries in the upload log (the one on the DataStore tab of the "Edit Resource" view).

In my case this seems to be caused by a failed HTTPS certificate check (this was on a test instance for which the production certificate is not valid), and I found corresponding log messages in my DataPusher error logs (/var/log/apache2/datapusher.error.log).

@jcballesteros Do you have any suspicuous entries in your DataPusher logs?

Note that you can disable DataPusher's certificate verification, see the documentation. Obviously this is only recommended for development and testing.

[izhangzhihao]

Same issue here, but I set SSL_VERIFY = False in datapusher_settings.py:

import uuid

DEBUG = False
TESTING = False
SECRET_KEY = 'secret-key'
USERNAME = 'username'
PASSWORD = 'password'

NAME = 'datapusher'

# database

SQLALCHEMY_DATABASE_URI = 'sqlite:////data/job_store.db'

# webserver host and port

HOST = '0.0.0.0'
PORT = 8800

# logging

#FROM_EMAIL = '[email protected]'
#ADMINS = ['[email protected]']  # where to send emails

LOG_FILE = '/tmp/ckan_service.log'
STDERR = True

# http://docs.ckan.org/projects/datapusher/en/latest/using.html#configuring-ssl-verification
SSL_VERIFY = False

# cloud settings
MAX_CONTENT_LENGTH = 73400320

[izhangzhihao]

The latest log in datapusher:

Successfully pushed 12 entries to "04a84ec9-1da3-4af9-bbb8-a33568b402f8".
Fetching from: https://ckan.test.com/dataset/d8f5e36a-f3e2-41f1-bc6c-e47dec8c7f69/resource/33d2a0a3-0510-47fb-8a9c-d9245d8e5eff/download/build.properties
Determined headers and types: []
Successfully pushed 0 entries to "33d2a0a3-0510-47fb-8a9c-d9245d8e5eff".

[Pardhu448]

I have same issue with 'Preview' in 'Explore' button for the new resource/data added to a dataset.
Here is my setup:

• Runnig locally using Docker installation for CKAN 2.8.2
• Data Preview works fine for data uploaded to 'DataStore' using API
  ( curl -X POST http://127.0.0.1:5000/api/3/action/datastore_create -H "Authorization: {YOUR-API-KEY}" -d '{"resource": {"package_id": "{PACKAGE-ID}"}, "fields": [ {"id": "a"}, {"id": "b"} ], "records": [ { "a": 1, "b": "xyz"}, {"a": 2, "b": "zzz"} ]}' )
• But 'Preview' fails for data uploaded through website using 'Add new resource' button to a dataset
• Datapusher error logs are as given below ::

python /usr/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py:573: SAWarning: Unicode type received non-unicodebind param value. param.append(processors[key](compiled_params[key])) Job "push_to_datastore (trigger: RunTriggerNow, run = True, next run at: None)" raised an exception Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/apscheduler/scheduler.py", line 512, in _run_job retval = job.func(*job.args, **job.kwargs) File "/usr/src/app/datapusher/jobs.py", line 222, in push_to_datastore resource = get_resource(resource_id, ckan_url, api_key) File "/usr/src/app/datapusher/jobs.py", line 178, in get_resource 'Authorization': api_key} File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 94, in post return request('post', url, data=data, json=json, **kwargs) File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 49, in request return session.request(method=method, url=url, **kwargs) File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 457, in request resp = self.send(prep, **send_kwargs) File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 569, in send r = adapter.send(request, **kwargs) File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 407, in send raise ConnectionError(err, request=request)

• Error when using 'upload to Datastore' button for the new resource

python Error: Process completed but unable to post to result_url

• Error from Dataproxy:
  Could not load view: DataProxy returned an error (Data transformation failed. error: An error occured while connecting to the server: DNS lookup failed for URL: http://ckan-dev:5000/dataset/e434d948-5048-46c1-a77c-2750e5651502/resource/813c00f6-8ee6-4699-a8a3-1c08c2bce7bd/download/rangareddy_lab_usage_march31st2019.csv)

• I have set SSL_VERIFY = False in datapusher_settings.py

Does it have anything to do with deploying on the local machine. Will this be resolved if deployed on production server.

Thanks in Advance for Help on this !

[nurtayzh]

I am having the same issue as @Pardhu448. Any help would be appreciated.

P.S. CKAN storage directory (containing resource files and uploaded images in resources and storage) has all the files. But CKAN fails to show them in the website.

P.P.S. There is a possible solutions here, but it creates problems during migration (because address has changes from localhost to dockerhost)

[ramnasidharta]

Exactly the same as @Pardhu448. Default installation with docker-compose. Please, any help would be very appreciated!

[heru]

Hi there,

same problem with others. is this has been resolved ? or I missed something ?