Tensorboard: Profiler fails to capture GPU / job

[OverLordGoldDragon]

Following the workaround here, I tried profiling via both tensorflow.keras.callbacks.TensorBoard and with tensorflow.python.eager.profiler.Profiler - but the resulting trace only displays /host:CPU (img below); I understand there should also be /job and /gpu sections, which is also evident from the significant gap between model.train_on_batch() iterations.

Any fixes/workarounds?

---

Note: I'm also getting the following in Anaconda Powershell Prompt:

W0215 15:19:47.154924 14124 core_plugin.py:215] Unable to get first event timestamp for run .: 
No event timestamp could be found

---

Note 2: TF 2.0.0 (stable) captures GPU, but I'm still not seeing /job: (also I still had to use the JSON workaround; TensorBoard PROFILE was empty)

[ProNoobLi]

Check your terminal. Does it say anything about libcupti?? If so, you miss it, tensorboard cannot capture gpu trace.

[mathpluscode]

Check https://github.com/tensorflow/tensorflow/issues/35860, if you have errors about libcupti

[tongda]

I have same issue, even after fix the libcupti problem.

My saved_model is exported by tf version 2.0.0, and tensorboard is 2.1.0. Since my profile is captured by tensorflow-serving, the tensorflow-serving version is 2.1.0.