Kops: Inappropriate use of RFC6598 addresses

Created on 9 Mar 2017  路  10Comments  路  Source: kubernetes/kops

I'm not sure if you consider this a bug or not - but I was very surprised when I saw the IP range we got by default with a new kops deployed cluster.

https://github.com/kubernetes/kops/blob/2c93bff6d73b1f3e7e0d0a422a7710cc3777b3b9/upup/pkg/fi/cloudup/defaults.go#L70 defaults the IP range used for the pods and cluster addresses to 100.64 - but this range is specifically reserved for CGN <-> CP interconnect.

At least as I read https://tools.ietf.org/html/rfc6598#section-4 kops fails per " Because CGN service requires non-overlapping address space on each
side of the home NAT and CGN, entities using Shared Address Space for
purposes other than for CGN service, as described in this document,
are likely to experience problems implementing or connecting to CGN
service at such time as they exhaust their supply of public IPv4
addresses."

It would be better - as well as delivering more IP space - to use RFC1918 addresses such as 10/8. I've been trying to think of a technical reason you might prefer RFC6598 addresses, but I'm drawing a blank :). If this isn't actually a strategic choice, I'd be happy to put forward a patch fixing the default.

aredocumentation lifecyclrotten
Source
picture rbtcollins

Most helpful comment

10/8 is a conflict with AWS classic, so breaks ClassicLink.

The thinking behind 100.64:

  • 100.64 is unused in AWS unless you specify it
  • 100.64 won't escape the VPC, so can't conflict with anything outside AWS
  • 100.64 is not very widely used, so most people won't specify it - unlike 172 or 192.168, where we're likely to break VPC peering.
picture justinsb on 9 Mar 2017
👍4

All 10 comments

10/8 is a conflict with AWS classic, so breaks ClassicLink.

The thinking behind 100.64:

  • 100.64 is unused in AWS unless you specify it
  • 100.64 won't escape the VPC, so can't conflict with anything outside AWS
  • 100.64 is not very widely used, so most people won't specify it - unlike 172 or 192.168, where we're likely to break VPC peering.
justinsb picture justinsb on 9 Mar 2017
👍4

PS and you can change it if you disagree - just kops edit cluster before your first create. But 10/8 as a default causes people problems, 100.64 seems not to :-)

justinsb picture justinsb on 9 Mar 2017

Ah thats interesting about classic - I guess some folk still use it :) That said AWS could go crazy and decide to use 100.64 for interconnects to their nat gw's ;).

W.r.t. VPC conflicts - yes, that can and will happen, but once you're running multiple VPC's you are expecting to do IP range design : its not that I discount it as being important, its that it becomes a fact of life - the VPC CIDR for k8s itself also needs to be non-overlapping.

Perhaps a note in the docs about the reason for this choice would reduce surprise for other folk?

FWIW the reason we pay attention to this is we're working on cross k8s cluster gossip clusters - where we need pod to pod connectivity, so we're doing IP range allocations somewhat carefully :).

rbtcollins picture rbtcollins on 9 Mar 2017

Marking this as a docs issue - I think this is just a matter of grabbing @justinsb's two comments and throwing them in docs.

yissachar picture yissachar on 15 Mar 2017
👍1

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

fejta-bot picture fejta-bot on 22 Dec 2017

/remove-lifecycle stale

rbtcollins picture rbtcollins on 9 Jan 2018

Interestingly enough, I just got multiple abuse reports from AWS stating some of my K8S node instances were observed "scanning" some external hosts with IPs in the 100.64.0.0/10 block. So either the packets are escaping the VPC or AWS abuse monitoring happens within each VPC.

activeshadow picture activeshadow on 7 Mar 2018

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

fejta-bot picture fejta-bot on 6 Jun 2018

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten
/remove-lifecycle stale

fejta-bot picture fejta-bot on 6 Jul 2018

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

fejta-bot picture fejta-bot on 5 Aug 2018
Was this page helpful?
0 / 5 - 0 ratings

Related issues

owenmorgan picture owenmorgan  路  3Comments
endejoli picture endejoli  路  4Comments
justinsb picture justinsb  路  4Comments
mikejoh picture mikejoh  路  3Comments
minasys picture minasys  路  3Comments