Aws-sdk-go: Connect to RDS using rdsutils.BuildAuthToken not working

Created on 4 May 2017  路  24Comments  路  Source: aws/aws-sdk-go

Please fill out the sections below to help us address your issue.

Version of AWS SDK for Go?

v1.8.19-6-g7b500fb

Version of Go (go version)?

go 1.8

What issue did you see?

following the doc but unable to make successful DB connection using IAM role from EC2
Also, this is particularly hard to debug because of not actually able to run code as EC2 on dev to test

Steps to reproduce

I follow the below step in https://docs.aws.amazon.com/sdk-for-go/api/service/rds/rdsutils/#BuildAuthToken and use the exact same code snippet but I believe there's some issue with the instruction:

authToken, err := BuildAuthToken(dbEndpoint, awsRegion, dbUser, awsCreds)

// Create the MySQL DNS string for the DB connection
// user:password@protocol(endpoint)/dbname?<params>
dnsStr = fmt.Sprintf("%s:%s@tcp(%s)/%s?tls=true",
   dbUser, authToken, dbEndpoint, dbName,
)

// Use db to perform SQL operations on database
db, err := sql.Open("mysql", dnsStr)

I have tried the above instruction but it was throwing signing errors

below is my latest code snippet, I have tried various different fortmat and this is the one i last end up with:

// c.Hostname = host.xxxx.us-east-1.rds.amazonaws.com
// c.Port = 3306

hostlocation := fmt.Sprintf("https://%v:%v", c.Hostname, c.Port)
token, stdErr := rdsutils.BuildAuthToken(hostlocation, "us-east-1", "appuser", stscreds.NewCredentials(sess, "arn:aws:iam::[AWS ID]:role/SomeRole"))

dnsStr := fmt.Sprintf("[appuser:%s@tcp(%s)]:%v/%s?tls=true", token, c.Hostname, c.Port, "dbname")

// Connect to database
db, stdErr := sql.Open("mysql", dnsStr)

for this one i got:
StdError: invalid DSN: did you forget to escape a param value?

the token returned is actually in the following format:

host.xxxx.us-east-1.rds.amazonaws.com:3306?Action=connect&DBUser=appuser&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAJHJ4MX7HB6STZ5QQ%2F20170504%2Fus-east-1%2Frds-db%2Faws4_request&X-Amz-Date=20170504T164655Z&X-Amz-Expires=900&X-Amz-Security-Token=FQoDYXdzENr%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaDB3PYlunbHE2bh4ylCLWAevRs7cztGGATW3iJm0tpL1J2G%2FsqJjilAlhI2uj6VW%2BWH0txpt2bZ7DgQeZ0lutoJj3rffcznu7o0VIG%2F7L8MXC11BjXOIOEXFwx%2BhEIAqM%2F3v9vpa9Jp1L2xqPBs%2FLuOYmHFxufykYE3D9%2BdoPRp3srEj3AqGbv9Nanw6zRXbsRkAj96VzsAnFTzyTAyOknBUDkWpBzjR%2Fo1Gqdd9gwu6HdJRcp6H%2B9oI0FLrDuQqUfSZez5BUspe4EYDWctSEuNoNREQzzUkkoU2yXB69m4KxA4TyIy4ogLatyAU%3D&X-Amz-SignedHeaders=host&X-Amz-Signature=2b249517f6dc4ad145232cdab3ea59e9b8b20dedad6666b07bbe686e33b6859e

I am not sure if this is the right string that should replace the DSN/DNS password field as indicated in the doc

Can anyone please help me figure out what the actual and correct DSN or DNS will look like? some example will be very helpful as I can just follow the same structure

Thanks

guidance

Most helpful comment

Hi,
I believe I've managed to solve this - it does appear to me that the docs are missing a couple of things.

  1. When passing in endpoint, you must include the port
  2. When creating the db connection, you must register the RDS x509 cert & pass in a specific TLS config

1:

passwd, err := rdsutils.BuildAuthToken(fmt.Sprintf("%s:%d", host, 3306), region, cfg.User, creds)

2:

func RegisterRDSMysqlCerts(c *http.Client) error {
    resp, err := c.Get("https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem")
    if err != nil {
        return errs.Wrap(err)
    }

    defer fileutil.CloseLoggingAnyError(resp.Body)
    pem, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        return errs.Wrap(err)
    }

    rootCertPool := x509.NewCertPool()
    if ok := rootCertPool.AppendCertsFromPEM(pem); !ok {
        return errs.New("couldn't append certs from pem")
    }

    err = mysql.RegisterTLSConfig("rds", &tls.Config{RootCAs: rootCertPool, InsecureSkipVerify: true})
    if err != nil {
        return errs.Wrap(err)
    }
    return nil
}

A full working example (you'll need to fill in the appropriate information for your user/credentials/etc):

package main

import (
    "crypto/tls"
    "crypto/x509"
    "database/sql"
    "fmt"
    "io/ioutil"
    "net/http"

    "github.com/aws/aws-sdk-go/aws/credentials"
    "github.com/aws/aws-sdk-go/service/rds/rdsutils"
    "github.com/go-sql-driver/mysql"
    "github.com/richardwilkes/errs"
    "jaxf-github.fanatics.corp/forge/furnace/fileutil"
)

func RegisterRDSMysqlCerts(c *http.Client) error {
    resp, err := c.Get("https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem")
    if err != nil {
        return errs.Wrap(err)
    }

    defer fileutil.CloseLoggingAnyError(resp.Body)
    pem, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        return errs.Wrap(err)
    }

    rootCertPool := x509.NewCertPool()
    if ok := rootCertPool.AppendCertsFromPEM(pem); !ok {
        return errs.New("couldn't append certs from pem")
    }

    err = mysql.RegisterTLSConfig("rds", &tls.Config{RootCAs: rootCertPool, InsecureSkipVerify: true})
    if err != nil {
        return errs.Wrap(err)
    }
    return nil
}

func main() {
    // FILL THESE OUT:
    host := ""
    user := ""
    dbName := ""
    creds := credentials.NewSharedCredentials("", "")

    region := "us-east-1"

    host = fmt.Sprintf("%s:%d", host, 3306)
    cfg := &mysql.Config{
        User: user,
        Addr: host,
        Net:  "tcp",
        Params: map[string]string{
            "tls": "rds",
        },
        DBName: dbName,
    }
    cfg.AllowCleartextPasswords = true

    var err error
    cfg.Passwd, err = rdsutils.BuildAuthToken(host, region, cfg.User, creds)

    if err != nil {
        panic(err)
    }

    err = RegisterRDSMysqlCerts(http.DefaultClient)
    if err != nil {
        panic(err)
    }

    db, err := sql.Open("mysql", cfg.FormatDSN())
    if err != nil {
        panic(err)
    }

    err = db.Ping()
    if err != nil {
        panic(err)
    }
    fmt.Println("ok")
}

All 24 comments

Hello @jhwang09, thank you for reaching out to us. When the service team tested our implementation, they said it worked. I would ask on their forums to see if they can spot anything that may not look right. In addition, questions like this may be best suited for stackoverflow or gitter. However, if there is an issue with the implementation, please reopen this ticket.

@jhwang09, I am looking at the docs and I see the confusion. Let me try to test this on my end and see if the example is incorrect.

@jhwang09, I have a feeling your DSN string is incorrect. The example does not include [ ] around the user:password.

Why is there [ ] around the DSN string? Just making sure there isn't some information I may be missing.

I have tried the original solution as proposed by the doc but it wasn't working that's why I start trying different variation.

the original doc said that the dbEndpoint should include [scheme]://[host][:port], and the same variable is used for both rdsutils.BuildAuthToken and for the DSN:

dnsStr = fmt.Sprintf("%s:%s@tcp(%s)/%s?tls=true",
   dbUser, authToken, dbEndpoint, dbName,
)

However, I am not sure if the one in DSN actually requires scheme in there

the other thing I find odd is the password used, in this case it's the authToken coming back from rdsutils

and it is of this format
host.xxxx.us-east-1.rds.amazonaws.com:3306?Action=connect&DBUser=appuser&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAJHJ4MX7HB6STZ5QQ%2F20170504%2Fus-east-1%2Frds-db%2Faws4_request&X-Amz-Date=20170504T164655Z&X-Amz-Expires=900&X-Amz-Security-Token=FQoDYXdzENr%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaDB3PYlunbHE2bh4ylCLWAevRs7cztGGATW3iJm0tpL1J2G%2FsqJjilAlhI2uj6VW%2BWH0txpt2bZ7DgQeZ0lutoJj3rffcznu7o0VIG%2F7L8MXC11BjXOIOEXFwx%2BhEIAqM%2F3v9vpa9Jp1L2xqPBs%2FLuOYmHFxufykYE3D9%2BdoPRp3srEj3AqGbv9Nanw6zRXbsRkAj96VzsAnFTzyTAyOknBUDkWpBzjR%2Fo1Gqdd9gwu6HdJRcp6H%2B9oI0FLrDuQqUfSZez5BUspe4EYDWctSEuNoNREQzzUkkoU2yXB69m4KxA4TyIy4ogLatyAU%3D&X-Amz-SignedHeaders=host&X-Amz-Signature=2b249517f6dc4ad145232cdab3ea59e9b8b20dedad6666b07bbe686e33b6859e

does this look correct? this looks more like a host with bunch of query params, not sure if this is supposed to be the authToken/password used in DSN

jhwang09 - What is the error you are seeing when you use that?

I ran into no issues using the example.

see below is the panic

panic: Error | StdError: dial tcp: address https://testprod.xxx.us-east-1.rds.amazonaws.com:3306: too many colons in address | goroutine 1 [running, locked to thread]:
runtime/debug.Stack(0x1086de0, 0xc4201ba3c0, 0x0)
        /var/lib/jenkins/tools/org.jenkinsci.plugins.golang.GolangInstallation/go_1.8/src/runtime/debug/stack.go:24 +0x79
github.com/jhwang09/elmo/errs.NewStdError(0x1086de0, 0xc4201ba3c0, 0xc4201ba3c0, 0x347)
        /var/lib/jenkins/workspace/go/src/github.com/jhwang09/elmo/errs/errs.go:28 +0x26
test/lib/mysql.Config.ConnectViaIAM(0x0, 0x0, 0x0, 0x0, 0xc420013dc0, 0x5, 0xc420013dc5, 0x7, 0xc4200a9520, 0x12, ...)
        /var/lib/jenkins/workspace/go/src/test/lib/mysql/mysql.go:67 +0x29f
test/lib/hdb.init.1()
        /var/lib/jenkins/workspace/go/src/test/lib/hdb/hdb.go:35 +0x37d
test/lib/hdb.init()
        /var/lib/jenkins/workspace/go/src/test/lib/hdb/hdb.go:64 +0x7b
test/model/games.init()
        /var/lib/jenkins/workspace/go/src/test/model/games/game.go:233 +0x5a
test/lib/hutil.init()
        /var/lib/jenkins/workspace/go/src/test/lib/hutil/url.go:124 +0x70
test/middleware.init()
        /var/lib/jenkins/workspace/go/src/test/middleware/userLoginFilter.go:22 +0x6e
test/boot.init()
        /var/lib/jenkins/workspace/go/src/test/boot/boot.go:136 +0x7a
main.init()
        /var/lib/jenkins/workspace/go/src/test/main.go:32 +0x49
 | info:[map[] Time: 2017-05-04 21:55:56.074731368 +0000 UTC

dbEndpoint := fmt.Sprintf("https://%v:%v", c.MySQL.Hostname, c.MySQL.Port)
authToken, err := BuildAuthToken(dbEndpoint, awsRegion, dbUser, awsCreds)

dnsStr = fmt.Sprintf("%s:%s@tcp(%s)/%s?tls=true",
  "someDBUser", authToken, dbEndpoint, "dbName",
)

db, err := sql.Open("mysql", dnsStr)

This is what I currently have, following the doc code snippet

what will be helpful is if you can show me your final dsn, it will be a huge help for me to debug

thanks

@jhwang09 - Please make sure your db endpoint is correct. It should match up with what is in the console. I believe your DSN construction is correct, but your endpoint may be wrong since you are manually constructing it. Can you please verify that it is correct? It should match in the AWS console.

Could you please let me know what the scheme should be? for when we doing the BuildAuthToken? is it https? http? something else?

I tried to use http/https when build authToken, and in the actual TCP dial to db, I remove the scheme, and this is what I got:

panic: Error | StdError: x509: certificate signed by unknown authority | goroutine 1 [running, locked to thread]:
runtime/debug.Stack(0x1087f20, 0xc4200a9520, 0x0)
        /var/lib/jenkins/tools/org.jenkinsci.plugins.golang.GolangInstallation/go_1.8/src/runtime/debug/stack.go:24 +0x79
github.com/jhwang09/elmo/errs.NewStdError(0x1087f20, 0xc4200a9520, 0xc4200a9520, 0x359)
        /var/lib/jenkins/workspace/go/src/github.com/jhwang09/elmo/errs/errs.go:28 +0x26
hoopr/lib/mysql.Config.ConnectViaIAM(0x0, 0x0, 0x0, 0x0, 0xc420013dd0, 0x5, 0xc420013dd5, 0x7, 0xc4200a9540, 0x12, ...)
        /var/lib/jenkins/workspace/go/src/test/lib/mysql/mysql.go:66 +0x35d
test/lib/hdb.init.1()
        /var/lib/jenkins/workspace/go/src/test/lib/hdb/hdb.go:35 +0x380
test/lib/hdb.init()
        /var/lib/jenkins/workspace/go/src/test/lib/hdb/hdb.go:64 +0x7b
test/model/games.init()
        /var/lib/jenkins/workspace/go/src/test/model/games/publishGame.go:233 +0x5a
test/lib/hutil.init()
        /var/lib/jenkins/workspace/go/src/test/lib/hutil/url.go:124 +0x70
test/middleware.init()
        /var/lib/jenkins/workspace/go/src/test/middleware/userLoginFilter.go:22 +0x6e
test/boot.init()
        /var/lib/jenkins/workspace/go/src/test/boot/boot.go:136 +0x7a
main.init()
        /var/lib/jenkins/workspace/go/src/test/main.go:32 +0x49
 | info:[map[] Time: 2017-05-05 12:23:13.715718397 +0000 UTC

this is what the actual DSN looks like:

does this looks correct?

appuser:testdb.xxx.us-east-1.rds.amazonaws.com:3306?Action=connect&DBUser=appuser&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAI3KBUZ6HRLRTS5MQ%2F20170505%2Fus-east-1%2Frds-db%2Faws4_request&X-Amz-Date=20170505T122313Z&X-Amz-Expires=900&X-Amz-Security-Token=FQoDYXdzEO7%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaDJ54WPOcw6PJ4atE6CLWARY%2FB9qZvZ9NI9ZgjJUDbhh%2FwKuvRVLWN5%2F2TgVZ%2F%2F1Fo7Vf3rprSbBeSkR8DIx0YpbkK%2BNrpANfmre6jA0yM13JGvwAJkmD7%2Byt%2Fpjf7wjipBlu%2Fl64rjnqTxcL0%2F0fdic2cjBEk1fJp02%2BOKfRMlJB151A1ckPd9Z1cpJVXf3SraYKoBLu9zQBedwpsUFH%2BpjXiZqUE3H7NbJXvkA%2FjOA9I%2FQWQjkUgLwb5Cf%2F1micbtrIM%2BeoA5zBYkFaT2a1KfiDMQI7dq8IfmOUXwRU%2FsYLtyLZKIUosd2xyAU%3D&X-Amz-SignedHeaders=host&X-Amz-Signature=023c61b185c4c5bb318d277c0a3857e0808344630149d0041cb1ed34727fc88a@tcp(testdb.xxx.us-east-1.rds.amazonaws.com:3306)/dbName?tls=true

to double confirm, my db endpoint is correct, it is consist of [hostname]:[port]

I am suspecting if I should open my RDS up a bit in its security group?

it is only allowing
MYSQL/Aurora
TCP
3306
sg-xxxx (website-sg)

on the console, it has two places it says endpoint, one is with port one is without port... which one is correct?

I would use the port one, because that's what I tested with. And yes, the DSN looks correct. Can I have the errors of when you tried to use the scheme http or https? Also, may want to try static credentials to see if that has anything to do with it.

Please refer to my previous post, If I do http or https, below is the panic:

panic: Error | StdError: dial tcp: address https://testprod.xxx.us-east-1.rds.amazonaws.com:3306: too many colons in address | goroutine 1 [running, locked to thread]:
runtime/debug.Stack(0x1086de0, 0xc4201ba3c0, 0x0)
        /var/lib/jenkins/tools/org.jenkinsci.plugins.golang.GolangInstallation/go_1.8/src/runtime/debug/stack.go:24 +0x79
github.com/jhwang09/elmo/errs.NewStdError(0x1086de0, 0xc4201ba3c0, 0xc4201ba3c0, 0x347)
        /var/lib/jenkins/workspace/go/src/github.com/jhwang09/elmo/errs/errs.go:28 +0x26
test/lib/mysql.Config.ConnectViaIAM(0x0, 0x0, 0x0, 0x0, 0xc420013dc0, 0x5, 0xc420013dc5, 0x7, 0xc4200a9520, 0x12, ...)
        /var/lib/jenkins/workspace/go/src/test/lib/mysql/mysql.go:67 +0x29f
test/lib/hdb.init.1()
        /var/lib/jenkins/workspace/go/src/test/lib/hdb/hdb.go:35 +0x37d
test/lib/hdb.init()
        /var/lib/jenkins/workspace/go/src/test/lib/hdb/hdb.go:64 +0x7b
test/model/games.init()
        /var/lib/jenkins/workspace/go/src/test/model/games/game.go:233 +0x5a
test/lib/hutil.init()
        /var/lib/jenkins/workspace/go/src/test/lib/hutil/url.go:124 +0x70
test/middleware.init()
        /var/lib/jenkins/workspace/go/src/test/middleware/userLoginFilter.go:22 +0x6e
test/boot.init()
        /var/lib/jenkins/workspace/go/src/test/boot/boot.go:136 +0x7a
main.init()
        /var/lib/jenkins/workspace/go/src/test/main.go:32 +0x49
 | info:[map[] Time: 2017-05-04 21:55:56.074731368 +0000 UTC

I tried the static credentials and it works fine. I am almost to a point to give up using the IAMAuth because it's too much pain...

Okay, that is pretty important information. Let me try replicating this with an IAM role. If it succeeds, then it may be something in your middleware. Can you write a simple Go program that eliminates jenkins and all your dependencies and only tests the SDK? Or has that been what you've been doing?

@jhwang09 - I just tried this with STS credentials, and it worked. Is this failing on opening a connection with sql? Or when performing a query?

So, yea, I cannot seem to replicate this. What mysql driver are you using? I am using this one github.com/go-sql-driver/mysql

How do you test STS credentials w/o being on an EC2 host? I tried to assign the role to an user, but it isn't allowing user to assume role for some reason.

This is failing on when we do db.Ping().

My application basically can't start because of unable to do ping on startup

@jhwang09 - you shouldnt need to be on an Amazon EC2 host. What is the error when trying to assume the role? Have you have given permissions the correct permissions to Amazon RDS? What does that look like?

@jhwang09 - I have a PR that contains an example of using the rdsutils #1256. I am going to close this. If you are still having issues, please try stackoverflow or gitter.

Same problem. The documentation truly is a mess.

Same :(

Hello @sujunzhu, what issues are you running into? Can you provide a code sample?

Hi,
I believe I've managed to solve this - it does appear to me that the docs are missing a couple of things.

  1. When passing in endpoint, you must include the port
  2. When creating the db connection, you must register the RDS x509 cert & pass in a specific TLS config

1:

passwd, err := rdsutils.BuildAuthToken(fmt.Sprintf("%s:%d", host, 3306), region, cfg.User, creds)

2:

func RegisterRDSMysqlCerts(c *http.Client) error {
    resp, err := c.Get("https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem")
    if err != nil {
        return errs.Wrap(err)
    }

    defer fileutil.CloseLoggingAnyError(resp.Body)
    pem, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        return errs.Wrap(err)
    }

    rootCertPool := x509.NewCertPool()
    if ok := rootCertPool.AppendCertsFromPEM(pem); !ok {
        return errs.New("couldn't append certs from pem")
    }

    err = mysql.RegisterTLSConfig("rds", &tls.Config{RootCAs: rootCertPool, InsecureSkipVerify: true})
    if err != nil {
        return errs.Wrap(err)
    }
    return nil
}

A full working example (you'll need to fill in the appropriate information for your user/credentials/etc):

package main

import (
    "crypto/tls"
    "crypto/x509"
    "database/sql"
    "fmt"
    "io/ioutil"
    "net/http"

    "github.com/aws/aws-sdk-go/aws/credentials"
    "github.com/aws/aws-sdk-go/service/rds/rdsutils"
    "github.com/go-sql-driver/mysql"
    "github.com/richardwilkes/errs"
    "jaxf-github.fanatics.corp/forge/furnace/fileutil"
)

func RegisterRDSMysqlCerts(c *http.Client) error {
    resp, err := c.Get("https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem")
    if err != nil {
        return errs.Wrap(err)
    }

    defer fileutil.CloseLoggingAnyError(resp.Body)
    pem, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        return errs.Wrap(err)
    }

    rootCertPool := x509.NewCertPool()
    if ok := rootCertPool.AppendCertsFromPEM(pem); !ok {
        return errs.New("couldn't append certs from pem")
    }

    err = mysql.RegisterTLSConfig("rds", &tls.Config{RootCAs: rootCertPool, InsecureSkipVerify: true})
    if err != nil {
        return errs.Wrap(err)
    }
    return nil
}

func main() {
    // FILL THESE OUT:
    host := ""
    user := ""
    dbName := ""
    creds := credentials.NewSharedCredentials("", "")

    region := "us-east-1"

    host = fmt.Sprintf("%s:%d", host, 3306)
    cfg := &mysql.Config{
        User: user,
        Addr: host,
        Net:  "tcp",
        Params: map[string]string{
            "tls": "rds",
        },
        DBName: dbName,
    }
    cfg.AllowCleartextPasswords = true

    var err error
    cfg.Passwd, err = rdsutils.BuildAuthToken(host, region, cfg.User, creds)

    if err != nil {
        panic(err)
    }

    err = RegisterRDSMysqlCerts(http.DefaultClient)
    if err != nil {
        panic(err)
    }

    db, err := sql.Open("mysql", cfg.FormatDSN())
    if err != nil {
        panic(err)
    }

    err = db.Ping()
    if err != nil {
        panic(err)
    }
    fmt.Println("ok")
}

One additional tidbit - if you fail to provide a port in the connection string, you'll encounter this: https://github.com/go-sql-driver/mysql/issues/717

Was this page helpful?
0 / 5 - 0 ratings