Chronograf: failed to retrieve default organization

[binary0111]

Bug report

# chronograf -v
2017/12/27 05:27:48 Chronograf 1.4.0.0 (git: e9dfbfcea003ed768a90cef0148a811942653d71)

Details:

Upgraded from chronograf 1.3.10.0 to 1.4.0.0 and started getting the above error. My init script uses the --new-sources= option to specify my influx and kapacitor source. Haven't setup or specified any organizations. I get the same error with a fresh install as well. If I leave out the "--new-sources=" option, every thing runs ok and I can manually add the source from the web ui but the "--new-sources=" option still remains broken.

Logs:

time="2017-12-27T04:03:23Z" level=error msg="failed to retrieve default organization" component=server new-sources=invalid
2017/12/27 04:03:23 failed to retrieve default organization

[nhaugo]

Todo:

• [x] Fix new sources cli
• [x] Normalize stringified sources and kapacitors
• [x] Add cli option for directory selection for .src, .kap, .dashboard, and .org files
• [x] Add an error message to the logs
• [x] List this as deprecated

Hi @binary0111,
Thanks for the report. I am able to replicate this behavior. We will try to make this a bit better of a user experience, but this is a side effect of us moving away from the --new-sources style of source definition.
One thing to note is that --new-sources only adds a source the first time it is run, so once you run it that source is added permanently to the chronograf-v1.db file. Therefore if you have an already initialized db, you can drop the --new-sources flag and the organization migration will occur as expected (if not please let us know).
In 1.4.0.0 we added a new way to configure sources, organizations, and dashboards from the file system, I will follow up with some basic examples of those as reference as we wait for proper documentation, which should be coming early in 2018.

[nhaugo]

To create a new source find the directory specified by the --canned-path command line option (note there will be an override for source, organization and dashboard files in a future release, but for now the are in the same directory as the pre canned dashboards).
In this directory you can create a file called .src, for example a file called example.src.
The file should contain the following information,

{
  "id": "10000",
  "name": "My InfluxDB",
  "username": "test",
  "password": "test",
  "url": "http://localhost:8086",
  "type": "influx",
  "insecureSkipVerify": false,
  "default": true,
  "telegraf": "telegraf",
  "organization": "example_org"
}

The id filed should be a unique stringified non-negative integer, you will probably want to use a 4 or 5 digit number so as to not interfere with existing datasource ids. The name is any string you want to use as the display name of the source. The fields username, password, and url are the username, password and url of the influxdb server. The type field should be influx if you are using the open source influxdb and set to influx-enterprise if you are using the enterprise version. Set insecureSkipVerify to true if you are using a self signed ssl cert on your InfluxDB server. default should be set to true if you want the data source to be the data source that is connected to upon first login. The telegraf option is the name of the telegraf database on your InfluxDB server and organization is the id of the organization you want the data source to be associated with.

To create an organization from the file system you can create a <something>.org file. The contents of this file should look like

{
  "id": "example_org",
  "name": "My Test Organization",
  "defaultRole": "viewer"
}

The id field is a unique string id for the organization, the name is the user facing name of the org, and the defaultRole is the role new users of this org should be given when they first authenticate, the options for this are "admin", "editor", "viewer" and "member".

You can also add dashboards using a <name>.dashboard file. To do so follow these instructions to export a dashboard json file and add the following element "organization": "example_org".

[nhaugo]

Two more things I forgot in my last comment. You can create a kapacitor with a <something>.kap file.

{
  "id": 10000,
  "srcID": 10000,
  "name": "My Kapacitor",
  "url": "http://localhost:9092",
  "active": true,
  "organization": "example_org"
}

Also all of these files can take env vars, so you can define things like export INFLUXDB_URL=http://localhost:8086 then add that to the sources file with a go template variable, e.g.

{
  "id": "10000",
  "name": "My InfluxDB",
  "username": "test",
  "password": "test",
  "url": "{{ .INFLUXDB_URL }}",
  "type": "influx",
  "insecureSkipVerify": false,
  "default": true,
  "telegraf": "telegraf",
  "organization": "example_org"
}

[binary0111]

Hi @nhaugo

Thanks for the prompt and detailed reply. I can confirm that dropping the --new-sources= option with a pre initialised chronograf-v1.db does work.

I tested the file way of adding sources and it worked like a charm. I actually prefer this method over the --new-sources= option as it is much more maintainable. One interesting quirk (which also can be quite confusing) with adding a kapacitor instance is that -

This works:

  "id": 1,
  "srcID": 1,

But unlike adding a source, this doesn't:

  "id": "1",
  "srcID": "1",

[nhaugo]

@binary0111 yeah, side effect of the way the object is stored in boltdb, we will be normalizing this in the future.

[nhaugo]

Todo:

• [x] Fix new sources cli
• [x] Normalize stringified sources and kapacitors
• [x] Add cli option for directory selection for .src, .kap, .dashboard, and .org files
• [x] Add an error message to the logs

[binary0111]

@nhaugo an error message in the logs would be super helpful in debugging if the parsing fails.

[mcfedr]

This seems to be related to the env vars INFLUXDB_URL and KAPACITOR_URL not doing anything anymore