Boulder: Publisher ProbeLogs function strips part of URL and connects to the wrong target

[pgporada]

Hi,

I wrote a small function based on https://github.com/letsencrypt/boulder/blob/master/publisher/publisher.go#L414-L445 to understand why a particular log message was coming through to a test CT server. Specifically the url is being overwritten from /${SHARD}/ct/v1/get-sth to /ct/v1/get-sth due to https://github.com/letsencrypt/boulder/blob/master/publisher/publisher.go#L427. As a result, the publisher makes an http connection to an invalid target and returns a 404 instead of a 200 from a known functioning CT log. Apologies for the code quality here.

$ cat test.go 
package main

import (
    "fmt"
    "github.com/google/certificate-transparency-go"
    "net/http"
    "net/url"
)

func main() {
    uri := "https://ct.cloudflare.com/logs/nimbus2019/ct/v1/get-sth"
    url, err := url.Parse(uri)
    fmt.Printf("URL: %s\n", url)

    if err != nil {
        fmt.Printf("failed to parse log URI: %s\n", err)
    }

    fmt.Printf("url.Path: %s\n", url.Path)
    url.Path = ct.GetSTHPath
    fmt.Printf("ct.GetSTHPath: %s\n", ct.GetSTHPath)

    c := http.Client{}
    fmt.Printf("Contacting url: %s\n", url.String())
    resp, err := c.Get(url.String())
    fmt.Printf("resp: %s\n", resp)
}

$ go run test.go 
URL: https://ct.cloudflare.com/logs/nimbus2019/ct/v1/get-sth
url.Path: /logs/nimbus2019/ct/v1/get-sth
ct.GetSTHPath: /ct/v1/get-sth
Contacting url: https://ct.cloudflare.com/ct/v1/get-sth
resp: &{404 Not Found %!s(int=404) HTTP/2.0 %!s(int=2) %!s(int=0) map[Cache-Control:[public, max-age=14400] Cf-Cache-Status:[MISS] Cf-Ray:[4d3e1bee89c9c504-ORD] Content-Length:[14] Content-Type:[text/plain; charset=utf-8] Date:[Wed, 08 May 2019 20:16:02 GMT] Expect-Ct:[max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"] Served-In-Seconds:[0.060] Server:[cloudflare] Set-Cookie:[__cfduid=d96c5142cbfac057ed9f370020b97ba1f1557346562; expires=Thu, 07-May-20 20:16:02 GMT; path=/; domain=.cloudflare.com; HttpOnly] Strict-Transport-Security:[max-age=15780000; includeSubDomains] Vary:[Accept-Encoding]] {%!s(*http.http2clientStream=&{0xc000074a80 0xc0000fc100 <nil> 1 0xc0000663c0 {{0 0} {{} 0xc0000e63e8 {0 0 0 <nil> <nil>} 824634663968} 0xc0000620c0 0xc000080050 <nil> <nil> 0x6323e0} false true 0x632370 {65536 0xc000074ae0} {4194290 0xc000074af0} 14 <nil> 0xc000080940 false 0xc0003ae120 <nil> 0xc0003ae180 true true false 0 map[] 0xc000102108})} %!s(int64=14) [] %!s(bool=false) %!s(bool=false) map[] %!s(*http.Request=&{GET 0xc0000fa080 HTTP/1.1 1 1 map[] <nil> <nil> 0 [] false ct.cloudflare.com map[] map[] <nil> map[]   <nil> <nil> <nil> <nil>}) %!s(*tls.ConnectionState=&{771 true false 49195 h2 true  [0xc00017c000 0xc00017c580 0xc00017cb00] [[0xc00017c000 0xc00017c580 0xc00021f080] [0xc00017c000 0xc00017c580 0xc00017cb00 0xc00017e100]] [] [] 0x5c7da0 [172 212 110 165 145 128 93 150 8 118 178 131]})}

[jsha]

Thanks for catching this! The probe code is obsolete now (we fixed the underlying problem by switching to HTTP/1 in the publisher), so I think the best fix is to remove it entirely.

[cpu]

I think the best fix is to remove it entirely.

Implemented in #4223

[pgporada]

Thanks!