NuGet product used (NuGet.exe | VS UI | Package Manager Console | dotnet.exe): VS UI
VS version (if appropriate): 15.3.0 Preview 6 (26725.2.d14rel)
OS version (i.e. win10 v1607 (14393.321)): 10.0.15063

As expected, a few errors are reported:
"The type or namespace name 'System' could not be found (are you missing a using directive or an assembly reference?)"
Packages are not restored, above errors are still displayed.
Package Manager output window claims all is restored:
All packages are already installed and there is nothing to restore.
Time Elapsed: 00:00:00.0067942
========== Finished ==========
Edit [nikolev]
This is the expected behavior of restore, but the message is confusing.
The message should at least be updated.
/cc @srivatsn
cc: @nkolev92 @emgarten
@tmat this option doesn't disable auto-restore instead it disables restore itself, so what ever you're seeing is the right behavior. We don't have any option to only disable auto-restore as of now.
@jainaashish I see. Then either "Restore NuGet packages" should show a message that nuget package restore is disabled, or override the setting.
@tmat
I've updated the original issue/name.
Thanks for creating this.