Google-cloud-node: Can't connect to local Datastore Emulator

[sebelga]

Hi,
I just updated from 0.29 to 30.3 and I can't connect to the Datastore emulator locally.
This is the steps I do:

Launch datastore with:

gcloud beta emulators datastore start --project=my-project-id --data-dir /path/to/some/dir --host-port localhost:8380

It launches and tells me:

API endpoint: http://localhost:8380/datastore
...
Dev App Server is now running

Init gcloud-node with

var config = {
    projectId: 'my-project-id',
    // keyFilename: '/some/path/to/credential.key.json', (tried both with and without)
};

var gcloud = require('gcloud')(config);
var datastore = gcloud.datastore({
        apiEndpoint: "http://localhost:8380"
});

Create an Entity

var key = datastore.key('BlogPost');
datastore.save({
    key: key,
    data: {
        title: 'Hello world'
    }
}, function(err) {
    console.log(key.path);
});

I receive this in err:
"{"code":503,"metadata":{"_internal_repr":{}}}"

I then changed the apiEndPoint to "http://localhost:8380/datastore" (as indicated when launching the emulator) but then it blocks the request (does not come into the callback) and in the terminal it says:

E0414 09:11:46.140609000 4406423552 resolve_address_posix.c:126] getaddrinfo: nodename nor servname provided, or not know

As I said, I tried both with and withouth credentials without any difference.

Some help would be greatly appreciated! :)
Thanks a lot for this great library.

[Thomas-T]

Hi,

I had the same problem, it's due to the fact that the datastore emulator doesn't implements GRPC yet.
you can see the workaround in the changelog :
https://github.com/GoogleCloudPlatform/gcloud-node/releases/tag/v0.30.0

Additionally, we are now using gRPC to give a performance boost to your Datastore calls. As of now, the gcloud SDK does not have a gRPC emulator, meaning if you wish to use a localhost backend instead of the upstream API, you will have to use the gcd.sh tool directly.

[stephenplusplus]

Thanks for the kind words @sebelga and for the assist @Thomas-T!

Also tagging duplicate: #1220.

[sebelga]

Thanks for fast answer and info @Thomas-T ! , I will try the gcd.sh tool and come back if any problem.
Best

[sebelga]

Ok I got it working with the gcd.sh (had to first use the "create" command on a folder, then the "start" command on that same folder.

Although it works, on each call to the datastore I get (on the node server console):
E0415 17:55:30.698468000 4406132736 tcp_client_posix.c:173] failed to connect to 'ipv6:[::1]:8080': socket error: connection refused

And on the local datastore console:
INFO: Detected HTTP/2 connection

Not a big deal but just checking if it was normal :)

[stephenplusplus]

Thanks for bringing it up!

INFO: Detected HTTP/2 connection

That sounds okay.

E0415 17:55:30.698468000 4406132736 tcp_client_posix.c:173] failed to connect to 'ipv6:[::1]:8080': socket error: connection refused

That's a little more concerning. @pcostell that error is being printed from gRPC. The function calls seem to behave correctly, but that message still prints. Is that anything to be concerned with?

[pcostell]

The INFO message is totally fine, it's just logging that it received an HTTP/2 connection instead of an HTTP/1 connection (we should probably make this FINE messages instead).

The ipv6 error definitely seems more alarming, but unfortunately I don't know much about it. Maybe we should open up a bug over on the grpc github?

[stephenplusplus]

Thanks for checking it out. @murgatroid99 should we open an issue? I thought I'd give you a chance to take a look first.

[sebelga]

One last detail. It seems that when connecting locally to this gcd.sh, the generated ID does not work as I receive a "normal" incremental id (1, 2, 3, ... )

To make sure it was not the gcloud library, I removed the "apiEndpoint" and pointed to Google and the generated ID was like before (16 number figure).

Again, no big deal because it's local but just to let you know..

[zjaml]

Hi, I'm having the same issue.
If the problem is that the emulator does not implement grpc yet, will it eventually implement it? I wish it does.

If the answer is true, should we keep this issue open?

Thanks,

[fortuna]

FYI, I'm getting the same error on a datastore.insert() call:

E0725 18:03:55.390445457  178615 tcp_client_posix.c:173]     failed to connect to 'ipv6:[::1]:8080': socket error: connection refused

This is the output of the emulator:

./gcd/gcd.sh start /tmp/datastore
Jul 25, 2016 6:03:32 PM com.google.appengine.api.datastore.dev.LocalDatastoreService init
INFO: Local Datastore initialized: 
    Type: High Replication
    Storage: /tmp/datastore/WEB-INF/appengine-generated/local_db.bin
Jul 25, 2016 6:03:32 PM io.grpc.internal.ManagedChannelImpl <init>
INFO: [ManagedChannelImpl@1060b431] Created with target localhost:8080
Jul 25, 2016 6:03:32 PM com.google.appengine.api.datastore.dev.LocalDatastoreService load
INFO: The backing store, /tmp/datastore/WEB-INF/appengine-generated/local_db.bin, does not exist. It will be created.
Jul 25, 2016 6:03:32 PM io.gapi.emulators.netty.NettyUtil applyJava7LongHostnameWorkaround
INFO: Applied Java 7 long hostname workaround.
API endpoint: http://localhost:8080/datastore
If you are using a library that supports the DATASTORE_EMULATOR_HOST environment variable, run:

  export DATASTORE_EMULATOR_HOST=localhost:8080

Jul 25, 2016 6:03:55 PM io.gapi.emulators.grpc.GrpcServer$3 operationComplete
INFO: Adding handler(s) to newly registered Channel.
Jul 25, 2016 6:03:55 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
INFO: Detected HTTP/2 connection.
Jul 25, 2016 6:03:55 PM io.gapi.emulators.grpc.GrpcServer$3 operationComplete
INFO: Adding handler(s) to newly registered Channel.
Jul 25, 2016 6:03:55 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
INFO: Detected HTTP/2 connection.

It would be great to reopen this issue. It's a major development hindrance.

[stephenplusplus]

Can someone open an issue on the grpc repo? https://github.com/grpc/grpc. Please link back to this issue so anyone else who might run into this can follow along. You can tag me and @pcostell as well.

[fortuna]

Is this really a gRPC problem? How can you tell?

I was wondering if the error was due to having an empty datastore. Whenever I start the emulator, I get local_db.bin, does not exist. It will be created., which tells me that the datastore is not being created. In fact when I look in the datastore directory, the local_db.bin file is not there. Maybe it's created lazily in the first write?

In any case, I decided to run the tool without using the disk, and it crashed right away:

./gcd/gcd.sh start --store_on_disk=false
Exception in thread "main" java.lang.NullPointerException
    at com.google.cloud.datastore.emulator.CloudDatastore$StartAction.apply(CloudDatastore.java:619)
    at com.google.appengine.tools.util.Parser$ParseResult.applyArgs(Parser.java:48)
    at com.google.cloud.datastore.emulator.CloudDatastore.run(CloudDatastore.java:848)
    at com.google.cloud.datastore.emulator.CloudDatastore.main(CloudDatastore.java:794)

This problem is getting in the way of the Google Cloud Functions Alpha release, as we developers need a way to test things locally.

[stephenplusplus]

I can't tell, but that additional information might be helpful. That sounds like an issue with the emulator.

But, the goal is getting the issue in front of the right eyes. I don't believe this to be a problem with this API client library. It seems to be in either the transport layer (grpc) or Datastore. @pcostell is from the Datastore team and advised opening the issue with gRPC. If you need the quickest response possible, heading over there might be best. And when @pcostell catches all of the name dropping I'm doing, maybe he'll take another look and offer help as well.

[fortuna]

I was able to find a workaround (thanks for a theory from @pcostell that it may be an ipv6 issue).

First, run the emulator with --no-legacy:

gcloud beta emulators datastore start --no-legacy

Then use 127.0.0.1 instead of localhost:

DATASTORE_EMULATOR_HOST=127.0.0.1:<port> node my_app.js

Of course, replace <port> by whatever was output by gcloud (or force a port in the emulator with --host-port)

[pcostell]

It sounds like this may be an ipv4 vs ipv6 issue.

A note if it's helpful, the gcloud emulator supports ipv6 hostnames starting in gcloud 116.0.0:

gcloud beta emulators datastore start --no-legacy --host-port "[::1]:8080"

But it seems like gcloud-node is trying to connect over ipv6 to localhost, whereas the emulator is serving on ipv4.

[stephenplusplus]

Is there anything our library can do?