TCP listeners and TCP client connections should both be able to conditionally enable TCP_FASTOPEN. See https://lwn.net/Articles/508865/ This is now available in Linux 3.7 (both client & server). This would be nice to conditionally use in http.DefaultTransport, for GET requests. See also: issue #3097 (adding Timeouts to DialTCP) See also: issue #3610 (DialPlan)
bradfitz
All 47 comments
Probably unrelated, but FYI. ConnectEx and AcceptEx we use to dial / accept tcp connections allow to send / receive some data as part of the same api call. Alex
alexbrainman
on 19 Feb 2013
Re #2: IIUC, that's related, but against spec. TCP permits such a packet, but also says that the recipient isn't supposed to act on it until the connection is verified, since the initial SYN packet could've had its source IP/port spoofed. TCP_FASTOPEN is a formalized version of that which isn't susceptible to spoofing. I never knew that AcceptEx makes that optional. I suppose a Windows webserver could use AcceptEx w/ data and verify first it's a GET request to a public resource and send the response immediately. I remember IE / Microsoft doing this years ago (to win HTTP benchmarks?), but I didn't remember how. Interesting.
bradfitz
on 27 Feb 2013
Now that DialOpt is in (https://code.google.com/p/go/source/detail?r=54731addb542 / docs: http://tip.golang.org/pkg/net/#DialOpt), this is easier. There are two API possibilities. a) lazy. b) explicit. In the lazy API, the connect would be like: conn, err := net.DialOpt("google.com:80", net.FastOpen()) ... and a fake net.Conn is returned immediately. No possibility for timeouts or errors. Not until the first Write does the actual sendto system call (on Linux) get called. This is what Chrome does. With the explicit API alternative (my preference), the API will probably be like: conn, err := net.DialOpt("google.com:80", net.InitialData("GET / HTTP/1.0\r\n\r\n")) Where net.InitialData returns a DialOption (http://tip.golang.org/pkg/net/#DialOption) that enables TFO if the client supports it (on Linux: checking the low bit of /proc/sys/net/ipv6/tcp_fastopen) and otherwise just does a Write after connect. Speaking with Yuchung Cheng, it seems there's enough momentum beh that something like it will be formalized one way or another, it will be formalized one way or another. And Linux 3.8 supporting it (client & server) is promising (and Windows in an alternate form). Thus I'd like to keep "TFO" or "TCPFastOpen" out of the Go API name and make the DialOption be just net.InitialData(...). Up for debate whether that's a []byte, string, or io.Reader. Probably a []byte for consistency with Write.
bradfitz
on 27 Feb 2013
This issue was updated by revision abf5700a157c63f435806da4bfab00b814f5b61.
R=golang-dev, r CC=golang-dev https://golang.org/cl/7369060
bradfitz
on 28 Feb 2013
rsc
on 12 Mar 2013
Published, at last: http://tools.ietf.org/html/rfc7413
mikioh
on 22 Dec 2014
how to make tfo work with http package? http's persist cache depend on established connections.
JunfengJia
on 3 Mar 2015
@JunfengJia you can use syscall to dial a new connection in http.Transport to work with http package.
Here is the tfo library I used:
http://godoc.org/github.com/bronze1man/kmg/kmgNet#TfoLazyDial
bronze1man
on 3 Mar 2015
@JunfengJia,
Still thinking. We need to handle the entire stack, TCP through HTTP 1.x, HTTP/2, or WebSocket w/ or w/o TLS, effectively. Once HTTP/2 support lands, I will revisit this issue.
mikioh
on 3 Mar 2015
@bronze1man,
http will make a read call first, then the connect will fall into 3whs.
JunfengJia
on 3 Mar 2015
@JunfengJia
Thanks.It looks like my library will not work with net/http library right now.
Read or write call first is a complex stuff.I have waited 1ms to make sure write call come first from client with my own defined protocol.
bronze1man
on 4 Mar 2015
FWIW, old changelist 27150044 has moved to https://go-review.googlesource.com/#/c/7121/.
mikioh
on 8 Mar 2015
In addition to TFO, we need to figure out APIs for crypto/tls TLS 1.3 0-RTT stuff. And then the net/http client can use them both.
/cc @agl
bradfitz
on 29 Jul 2015
The change golang.org/cl/7121 is ready to land. However I'm not sure whether crypto/tls and net/http packages really need this feature. Please ping golang.org/cl/7121 when the time comes.
mikioh
on 9 Mar 2016
The change golang.org/cl/7121 is ready to land.
It says "DO NOT REVIEW" so I stopped reviewing.
bradfitz
on 10 Mar 2016
Thanks for the notice, just did s/DO NOT REVIEW/FOR DISCUSSION/
mikioh
on 10 Mar 2016
CL https://golang.org/cl/7121 mentions this issue.
gopherbot
on 10 Mar 2016
@mikioh,
Did this working with http package?
JunfengJia
on 10 Mar 2016
The net/http package raw APIs, http.WriteRequest and http.ReadResponse work fine. Once we agree on supporting TCP fast open option, we need to tweak the existing concurrent IO stuff on http.Transport for well-cooked APIs such as http.Client. It would be another change and be done by net/http package owners.
mikioh
on 11 Mar 2016
Looks like this proposal is going ahead, and that there is a pending CL to implement it in Go 1.8.
adg
on 19 Jul 2016
@mikioh ping
gyf19
on 1 Mar 2017
Sorry, my spare time for Go 1.9 development cycle has been exhausted.
mikioh
on 25 Apr 2017
Hello @mikioh . Just curious on the likelihood if this being in Go 1.10 vs. Go 1.11. Thanks in advance.
ztheory
on 4 Jan 2018
@ztheory,
No, the Go 1.10 release train departed a few months ago.
mikioh
on 5 Jan 2018
Over the past 4 years, the dissemination of fancy newly TCP options including TCP Fast Open option progressed gradually. Linux kernel 3.x or above implements TCP Fast Open option on both active- and passive-open sides by default, and network intermediary manufacturers start to consider dealing with the newly TCP options. Unfortunately, the support of the other host platforms still requires struggling; Darwin kernel focuses on the active-open side only and requires the use of new system calls (though, I guess it'd be a great help when using both MPTCP and TCP Fast Open options), Windows requires the use of specific API such as ConnectEx, and FreeBSD requires to rebuild the kernel. [end of status update]
The good news is that Go 1.11 will probably fix #9661 and that will allow the use of TCP Fast Open option easily, without managing tedious states in between user-surface API and the protocol stack inside the kernel. You may achieve it by using the combo of the Control field of net.Dialer and TCP_FASTOPEN_CONNECT socket option, or the Control field of net.ListenConfig and TCP_FASTOPEN socket option on Linux kernel 4.11 or above.
mikioh
on 5 Jan 2018
Are we going to get this in GO 1.11?
riobard
on 28 Jun 2018
@riobard, no. This bug is still open.
bradfitz
on 28 Jun 2018
Isn't fastopen just a socket option that should be possible in go 1.11 atleast on linux?
For this to be closed, does it need to be solved on all platforms?
AudriusButkevicius
on 10 Jul 2018
Yeah, this bug was about adding nice, portable API for it (ioctls aren't great API), and using it where it makes sense (the top comment mentions net/http for instance)
But it's true that in Go 1.11 you could use https://tip.golang.org/pkg/net/#ListenConfig
bradfitz
on 10 Jul 2018
@AudriusButkevicius
For the server yes. The client needs to send the initial data with sendto(2), continuing with read(2)/write(2). connect(2) triggers a three way handshake instead.
UPDATE: I missed this one: https://github.com/torvalds/linux/commit/19f6d3f3c8422d65b5e3d2162e30ef07c6e21ea2
rumpelsepp
on 10 Jul 2018
Are we going to get this in GO 2?
xqdoo00o
on 29 Sep 2018
This has little if anything to do with Go 2.
bradfitz
on 29 Sep 2018
Any progress?
Or it's freezed for a long time?
qJkee
on 19 Oct 2018
Hello. Is there still interest in this?
Given the flow described in https://github.com/torvalds/linux/commit/19f6d3f3c8422d65b5e3d2162e30ef07c6e21ea2, assuming support for TFO only for Linux 4.11+, the code could be simplified greatly, compared to what CL 7121 does. Adoption in other places in the standard library would also be easier, since caller code would not need to change.
@mikioh, @bradfitz: would it be okay if I tried this from scratch in the upcoming cycle? It looks like https://go-review.googlesource.com/c/go/+/7121/ would need a pretty complicated rebase to bring it into the present.
acln0
on 23 Jan 2019
@acln0, oh, nice! I hadn't seen that. Given that new API, this is basically trivial. In fact, people can do it already today with Go 1.11 and its new:
https://tip.golang.org/pkg/net/#Dialer.Control
Then you can just do the setsockopt(s, IPPROTO_TCP, TCP_FASTOPEN_CONNECT ...) and give that net.Dialer to net/http, etc.
(Of course, if you do so, you should only use that http.Client for GET/HEAD/idempotent requests.)
But sure, you could work on it.
bradfitz
on 23 Jan 2019
@acln0,
If you are talking about stuff in the net package, probably still not the time. If net/http, crypto/tls, or net/http over crypto/tls, it should be another issue and probably it's fine unless you try to introduce a) a quick hack for message boundary handling; especially without considering byte-seq.+partial read/writes vs. datagram/chunks+full reads/writes, b) a quick hack for multipath transport functionality.
mikioh
on 23 Jan 2019
@bradfitz, I had the net/http use case for idempotent requests in mind. I don't know the intricacies of how crypto/tls might make use of TFO in the larger context of TLS 1.3 etc., so I cannot comment on that.
Yes, people can do it at the net.Dialer level themselves today, but I thought there might be some value in adding options to the net package as well. If the standard library is going to use TFO, my thought was the following: instead of making net/http and crypto/tls set socket options themselves, we could try to introduce FastOpen knobs to net.Dialer and net.Listener, to consolidate that code in one well defined place.
Once the new API existed in the net package, both the standard library and other clients could start adopting it, at whichever pace they like. I wrote my initial message in this bug because I was interested in giving the ground work in package net another try.
@mikioh, can you please clarify what you mean about "stuff in the net package, probably still not the time"? Do you mean the possibility of FastOpen knobs on net.Dialer and net.Listener? It seems to me like Linux and FreeBSD support both passive and active sides pretty easily. And OS X supports active side too. Do you think this level of support across operating systems is not wide enough to warrant an option in package net?
acln0
on 23 Jan 2019
@acln0,
Do you mean the possibility of FastOpen knobs on net.Dialer and net.Listener?
Yes.
FreeBSD support both passive and active sides pretty easily
As far as I can see, on FreeBSD, a few GENERIC conf files include TCP_RFC7413 option by default from 12-STABLE, but not all, also unlike passive-open side operation, enabling active-open operation still needs sysctl tweaking.
find freebsd/sys/conf -name GENERIC | xargs grep TCP_RFC7413
amd64/conf/GENERIC:options TCP_RFC7413 # TCP Fast Open
powerpc/conf/GENERIC:options TCP_RFC7413 # TCP Fast Open
arm64/conf/GENERIC:options TCP_RFC7413 # TCP Fast Open
And OS X supports active side too.
As far as I can see, it still supports active-open operation only and requires connectx thru syscall.Syscall9.
this level of support across operating systems is not wide enough to warrant an option in package net?
Still a bit early to me. If it's pretty fine within the real world, it's better to be enabled by default (and we should avoid introducing a control knob each time we need to fix something user uncomfortable on the operation of TCP Fast Open option, including configuring cookie-related life-duration/invalidation/capacity control.)
mikioh
on 23 Jan 2019
@brain
@acln0, oh, nice! I hadn't seen that. Given that new API, this is basically trivial. In fact, people can do it already today with Go 1.11 and its new:
https://tip.golang.org/pkg/net/#Dialer.Control
Then you can just do the
setsockopt(s, IPPROTO_TCP, TCP_FASTOPEN_CONNECT ...)and give that net.Dialer to net/http, etc.(Of course, if you do so, you should only use that http.Client for GET/HEAD/idempotent requests.)
But sure, you could work on it.
Any more complete example?
all setsockoptByte(Inet|ICMP) require fd, but we got only syscall.RawConn, or it can be casted to fd(int)?
qJkee
on 23 Jan 2019
@qJkee, call the Control method on the syscall.RawConn. Its argument is a function of the integer file descriptor. You can call setsockopt in that function.
acln0
on 23 Jan 2019
@acln0 thank you! I will try to implement it asap, and maybe post some bech.
qJkee
on 23 Jan 2019
I'm going to close this. No longer desired.
See:
https://squeeze.isobar.com/2019/04/11/the-sad-story-of-tcp-fast-open/
bradfitz
on 8 Jul 2020
Related issues
bradfitz
路
176Comments
ianlancetaylor
路
519Comments
DemiMarie
路
320Comments
brtzsnr
路
168Comments
derekmarcotte
路
243Comments
Most helpful comment
@acln0, oh, nice! I hadn't seen that. Given that new API, this is basically trivial. In fact, people can do it already today with Go 1.11 and its new:
https://tip.golang.org/pkg/net/#Dialer.Control
Then you can just do the
setsockopt(s, IPPROTO_TCP, TCP_FASTOPEN_CONNECT ...)and give that net.Dialer to net/http, etc.(Of course, if you do so, you should only use that http.Client for GET/HEAD/idempotent requests.)
But sure, you could work on it.