Consul: HTTP PUT to KV store does not retain newline characters

[juliohm1978]

When filing a bug, please include the following:

consul version for both Client and Server

Client: 0.8.1
Server: 0.8.3

consul info for both Client and Server

Client:

root@f2953f3062a1:/# consul info
agent:
    check_monitors = 0
    check_ttls = 0
    checks = 0
    services = 1
build:
    prerelease = 
    revision = 'e9ca44d
    version = 0.8.1
consul:
    known_servers = 3
    server = false
runtime:
    arch = amd64
    cpu_count = 4
    goroutines = 37
    max_procs = 4
    os = linux
    version = go1.8.1
serf_lan:
    encrypted = false
    event_queue = 0
    event_time = 4
    failed = 0
    health_score = 0
    intent_queue = 0
    left = 0
    member_time = 25
    members = 4
    query_queue = 0
    query_time = 1

Server:

/ # consul info
agent:
    check_monitors = 0
    check_ttls = 0
    checks = 0
    services = 1
build:
    prerelease = 
    revision = ea2a82b
    version = 0.8.3
consul:
    bootstrap = false
    known_datacenters = 1
    leader = false
    leader_addr = 11.11.0.4:8300
    server = true
raft:
    applied_index = 11216
    commit_index = 11216
    fsm_pending = 0
    last_contact = 15.756032ms
    last_log_index = 11216
    last_log_term = 2
    last_snapshot_index = 8192
    last_snapshot_term = 2
    latest_configuration = [{Suffrage:Voter ID:11.11.0.4:8300 Address:11.11.0.4:8300} {Suffrage:Voter ID:11.11.0.6:8300 Address:11.11.0.6:8300} {Suffrage:Voter ID:11.11.0.3:8300 Address:11.11.0.3:8300}]
    latest_configuration_index = 1
    num_peers = 2
    protocol_version = 2
    protocol_version_max = 3
    protocol_version_min = 0
    snapshot_version_max = 1
    snapshot_version_min = 0
    state = Follower
    term = 2
runtime:
    arch = amd64
    cpu_count = 4
    goroutines = 79
    max_procs = 4
    os = linux
    version = go1.8.1
serf_lan:
    encrypted = false
    event_queue = 0
    event_time = 4
    failed = 0
    health_score = 0
    intent_queue = 0
    left = 9
    member_time = 25
    members = 13
    query_queue = 0
    query_time = 1
serf_wan:
    encrypted = false
    event_queue = 0
    event_time = 1
    failed = 0
    health_score = 0
    intent_queue = 0
    left = 0
    member_time = 4
    members = 3
    query_queue = 0
    query_time = 1

Operating system and Environment details

We are using Consul in Docker containers. The agent server runs directly from the official image, while the client runs in our custom containers as a client agent. Both clients and server run inside a Docker Swarm, using the same network overlay spanning multiple docker hosts.

Server: Linux 66eea5da5490 4.4.0-53-generic #74-Ubuntu SMP Fri Dec 2 15:59:10 UTC 2016 x86_64 Linux

Clients: Ubuntu 16.04 LTS Linux f2953f3062a1 4.4.0-53-generic #74-Ubuntu SMP Fri Dec 2 15:59:10 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Description of the Issue (and unexpected/desired result)

Regardless of the setup we have for client containers, we have a Consul cluster of three nodes running inside the swarm overlay.

We need to POST key/value data to the Consul server using the HTTP PUT method. Fundamentally, we have a few scripts that use a simple curl line:

curl -v --stderr - --request PUT --data @$FILENAME "http://$CONSUL_ADDR:8500/v1/kv/$KEY"

While $KEY is the name of the key, obtained from the filename, the contents of $FILENAME are used as the value. They need to include newline characeters. Even as curl returns 200 OK and the value is stored, we realized that newline characters were lost. If the file contents are:

   configuration
      that needs
         new lines and indentation

... that becomes:

some   configuration      that needs         new lines and indentation

This does not happen when creating key/value pairs using the web interface (http://consul:8500/ui). Posting data to that HTML form creates the key/value pair correctly, maintaining newline characters.

Reproduction steps

Post data to key value store with newline characters using curl.

Post data to tkey value store using the graphical UI (http://consul:8500/ui).

Am I missing something while using curl?

[juliohm1978]

curl -v --stderr - --request PUT --data-binary @$FILENAME "http://$CONSUL_ADDR:8500/v1/kv/$KEY"

--data-binary resolve the issue.