Serving: tensorflow_model_server ignores visible gpu device

[vikeshkhanna]

I external/org_tensorflow/tensorflow/core/common_runtime/gpu/gpu_device.cc:944] Found device 0 with properties: 
name: GRID K520
major: 3 minor: 0 memoryClockRate (GHz) 0.797
pciBusID 0000:00:03.0
Total memory: 4.00GiB
Free memory: 3.95GiB
I external/org_tensorflow/tensorflow/core/common_runtime/gpu/gpu_device.cc:965] DMA: 0 
I external/org_tensorflow/tensorflow/core/common_runtime/gpu/gpu_device.cc:975] 0:   Y 
**I external/org_tensorflow/tensorflow/core/common_runtime/gpu/gpu_device.cc:1007] Ignoring visible gpu device (device: 0, name: GRID K520, pci bus id: 0000:00:03.0) with Cuda compute capability 3.0. The minimum required Cuda capability is 3.5.**
I external/org_tensorflow/tensorflow/contrib/session_bundle/session_bundle.cc:125] Running restore op for SessionBundle
I external/org_tensorflow/tensorflow/contrib/session_bundle/session_bundle.cc:233] Loading SessionBundle: success. Took 538789 microseconds.

This is happening even though serving/tensorflow was configured with CUDA capablity 3.0. Both serving and tensorflow are at HEAD.

[vikeshkhanna]

I should add that the separately installed (from sources) Tensorflow is able to discover and use the GPU device just fine.

[jemaw]

there seems to be a problem with ./configure at the moment, we had success with exporting the environment variables directly
export TF_NEED_CUDA=1
export TF_CUDA_VERSION=8.0
export CUDA_TOOLKIT_PATH=/opt/cuda
export TF_CUDNN_VERSION=5.1.5
export CUDNN_INSTALL_PATH=/opt/cuda
export TF_CUDA_COMPUTE_CAPABILITIES=3.0
then use bazel clean --expunge and run configure again

[gautamvasudevan]

This might work now - closing due to age of issue and number of changes since.