Go: slices and map random results in loop

[oranze]

What version of Go are you using (go version)?

$ go version
1.12.6

Does this issue reproduce with the latest release?

yes on stable

What operating system and processor architecture are you using (go env)?

Windows x64

go env Output

$ go env

set GOARCH=amd64

set GOBIN=

set GOCACHE=C:UsersmatamanskiiAppDataLocalgo-build

set GOEXE=.exe

set GOFLAGS=

set GOHOSTARCH=amd64

set GOHOSTOS=windows

set GOOS=windows

set GOPATH=C:workgoms-go-core;C:workgoms-go-remote_config;

set GOPROXY=

set GORACE=

set GOROOT=c:go

set GOTMPDIR=

set GOTOOLDIR=c:gopkgtoolwindows_amd64

set GCCGO=gccgo

set CC=gcc

set CXX=g++

set CGO_ENABLED=1

set GOMOD=

set CGO_CFLAGS=-g -O2

set CGO_CPPFLAGS=

set CGO_CXXFLAGS=-g -O2

set CGO_FFLAGS=-g -O2

set CGO_LDFLAGS=-g -O2

set PKG_CONFIG=pkg-config

set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:UsersMATAMA~1AppDataLocalTempgo-build585791102=/tmp/go-build -gno-record-gcc-switches

What did you do?


package main

import (
"fmt"
)

func main() {
blogArticleViews := map[string]int{
"unix": 0,
"python": 1,
"go": 2,
"javascript": 3,
"testing": 4,
"philosophy": 5,
"startups": 6,
"productivity": 7,
"hn": 8,
"reddit": 9,
"C++": 10,
}
for key, views := range blogArticleViews {
fmt.Println( "There are", views, "views for", key )
}

labels := map[string]string{
"org.a": "cli-a",
"org.b": "cli-b",
"org.c": "cli-c",
"org.d": "cli-d",
"org.e": "cli-e",
}
for k, v := range labels {
fmt.Println( k, v )
}
}

What did you expect to see?

There are 1 views for python
There are 2 views for go
There are 4 views for testing
There are 6 views for startups
There are 7 views for productivity
There are 8 views for hn
There are 9 views for reddit
There are 10 views for C++
There are 0 views for unix
There are 3 views for javascript
There are 5 views for philosophy

What did you see instead?

Random set trigger on first iteration:

There are 7 views for productivity
There are 9 views for reddit
There are 2 views for go
There are 3 views for javascript
There are 6 views for startups
There are 5 views for philosophy
There are 8 views for hn
There are 10 views for C++
There are 0 views for unix
There are 1 views for python
There are 4 views for testing
...
There are 8 views for hn
There are 0 views for unix
There are 1 views for python
There are 2 views for go
There are 3 views for javascript
There are 4 views for testing
There are 5 views for philosophy
There are 7 views for productivity
There are 6 views for startups
There are 9 views for reddit
There are 10 views for C++
...
There are 3 views for javascript
There are 4 views for testing
There are 7 views for productivity
There are 8 views for hn
There are 9 views for reddit
There are 0 views for unix
There are 1 views for python
There are 2 views for go
There are 10 views for C++
There are 5 views for philosophy
There are 6 views for startups
...
There are 8 views for hn
There are 10 views for C++
There are 0 views for unix
There are 4 views for testing
There are 6 views for startups
There are 5 views for philosophy
There are 7 views for productivity
There are 9 views for reddit
There are 1 views for python
There are 2 views for go
There are 3 views for javascript

[AlexRouSg]

See https://golang.org/ref/spec#For_statements

The iteration order over maps is not specified and is not guaranteed to be the same from one iteration to the next. If a map entry that has not yet been reached is removed during iteration, the corresponding iteration value will not be produced. If a map entry is created during iteration, that entry may be produced during the iteration or may be skipped. The choice may vary for each entry created and from one iteration to the next. If the map is nil, the number of iterations is 0.

[jgoc]

So there is no order in map keys?

[AlexRouSg]

Not when in a range for loop. If you want or need a order, you need to store the keys in a separate slice and sort it in whatever order you want and loop over that.

[oranze]

Not when in a range for loop. If you want or need a order, you need to store the keys in a separate slice and sort it in whatever order you want and loop over that.

I actually yesterday got this "specific output" when create a slice for sorting and try in "foreach way" get sorted values... When compare few results my brain start leaking )))))) output confessed me and this is a pretty nice word ))))))))))

[oranze]

on go version 1.12.5 this "specific output" can't be reproduced... so Golang playground show all fine as everybody expect:
https://play.golang.org/p/CQ0qTBz2dBT

[av86743]

Tried your https://play.golang.org/p/CQ0qTBz2dBT several times and got consistently this output:

There are 2 views for go
There are 3 views for javascript
There are 6 views for startups
There are 8 views for hn
There are 9 views for reddit
There are 10 views for C++
There are 0 views for unix
There are 1 views for python
There are 4 views for testing
There are 5 views for philosophy
There are 7 views for productivity

org.a cli-a
org.b cli-b
org.c cli-c
org.d cli-d
org.e cli-e

Program exited.

However, if source if updated and is recompiled before next run, order of output changes for blogArticleViews.

Does your "all fine" mean "range output is ordered in the order of initializers" or "... in the order of keys"? In either case, it is not always so (as spec states.)

[AlexRouSg]

btw playground output can be cached, if you run locally, output is different.

[oranze]

The iteration order over maps is not specified and is not guaranteed to be the same from one iteration to the next. If a map entry that has not yet been reached is removed during iteration, the corresponding iteration value will not be produced. If a map entry is created during iteration, that entry may be produced during the iteration or may be skipped. The choice may vary for each entry created and from one iteration to the next. If the map is nil, the number of iterations is 0.

Ok. but normally all normal people and normal developers will be expect normal logic with expected output of for loop, if you make experiments from one subversion to another please create some new for like example for_rand or some key... this is absolutely abnormal condition when some production (stable) code can provide unexpected results on flat way. I have found same reports from Docker guys, hi post one year ago about same problems....

Current solution is:
Do not use for if you need exactly expected ordering in output and use some constructions like:
for i := int64(1); i <= int64( len( keys ) ); i++{ val = keys[i] }

[oranze]

btw playground output can be cached, if you run locally, output is different.

Will be know... may be will be good idea to place some info banner about cache output )))) How many people know about cache of output on playground )))

Thanks for this hidden info

[AlexRouSg]

Will be know... may be will be good idea to place some info banner about cache output )))) How many people know about cache of output on playground )))

Please make a separate issue for that if you really want it.

[oranze]

Will be know... may be will be good idea to place some info banner about cache output )))) How many people know about cache of output on playground )))

Please make a separate issue for that if you really want it.

I already know this "magic" )))))))) Better get stable output (in next versions) for for slices/maps )))

[agnivade]

Please see https://blog.golang.org/go-maps-in-action "Iteration order" section. To be more accurate, the iteration order is "not specified", not "random". As you can see from here - https://twitter.com/CAFxX/status/1135190309514620928/photo/1.

[av86743]

I am sure this is a duplicate of an earlier identical issue.

[oranze]

I am sure this is a duplicate of an earlier identical issue.

yes.. upper I wrote what guys from Docker have'd same issue about 1 year ago...

Please see https://blog.golang.org/go-maps-in-action "Iteration order" section. To be more accurate, the iteration order is "not specified", not "random". As you can see from here - https://twitter.com/CAFxX/status/1135190309514620928/photo/1.

this is not answer why we can't normally (as expected) use for for slices/maps and must use twice for with sort in the middle... Without this trick results will be different from version to version.

in any other language For working like everybody expect... but... not in Go... in Go we must use 2 times for and sort ))) But actually this is not a bigger problem, the bigger problem what on different versions we will get different results (absolutely different by logic)

and in the upper I also show my solution for this problem with 1 for and key access in procedure, looks shorter, question only in speed

[av86743]

... question only in speed

Exactly. If, like in Perl, order of enumeration of map keys is not specified, maps can be implemented as faster hash maps.

[oranze]

... question only in speed

Exactly. If, like in Perl, order of enumeration of map keys is not specified, maps can be implemented as faster hash maps.

Perl was so many many years ago )))) but under a hood of topic (different results in different versions) some tryes to find better algorithm, worst only what iut result's different and if... the out data must be exactly in expected order... we get a problem... And Like result the problem is hidden from developers... in some versions working fine and how 99.9% expected, but if DevOps or somebody update Go version and rebuild app... opps.... the data links can be broken, I mean if like a example in API output we have something like [1.00, 2.30, 3.99, 4.12, 5.76, 6.55] and no any info aout a product, only native connection by position in array.

[dominikh]

Iterating over slices has well-defined and expected order; I'm not sure why slices were brought up at all. Iterating over maps has no defined order for reasons already mentioned. This is the case in a great many languages, and is not likely to change in future versions of Go. Additionally, Go intentionally randomizes the iteration order so that tests are more likely to fail if code incorrectly depends on the (non-existent) order of maps. That is, the claim that "output will be fine in one version but not another" is not true. It will very likely differ between multiple executions.

I am going to close this issue as there is no bug, and nothing to do. If you'd like to ask further questions to understand the nature of maps, I recommend that you do so in one of the many forums for asking questions.