Edgetpu: Coral Accelerator getting disconnected and reconnected.

[jimxxmy]

Coral Accelerator gets disconnected on its own when Interpretor Delegate is called. As for my case, edgetpu.dll was called. Once the execution of code is completed, the devices gets connected once again.

Operating System: Windows
Programming Langugage : Python
IDE : PyCharm

[Namburger]

@jimxxmy Can you provide more details?
Are you getting some type of errors when running the demo? Any logs would helps us debug

[jimxxmy]

@Namburger There are no errors or warnings shown except the ones provided by the tensorflow module, which I think, is for my non gpu system.

The code is as follows :
interpreter = tflite.Interpreter(model_path, experimental_delegates=[tflite.load_delegate('edgetpu.dll')])

Once this code has been executed, the windows sound for detaching and connecting, again for detaching USB is alerted.

[Namburger]

Humn, so let's say when you run this demo:
https://github.com/google-coral/tflite/tree/master/python/examples/classification
does it works? If not, does it outputs anything in the command prompt?

[jimxxmy]

@Namburger As the link you provided above, I also tested one repo for the object detection, which is provided to test the inference. Both ended in the same issue. It gets disconnected.

[Namburger]

@jimxxmy humnn, i know you mentioned that it gets disconnected, but I was hoping that there would be some type of messages that shows up on the command line interface. This message is crucial in getting some type of leads on the issue. I've not used pycharm, but I think you can open the commandline interface like this:
https://stackoverflow.com/a/28570162/6262757

[jimxxmy]

@Namburger I don't have the device with me as of now, cause it's out of working hours, IST, but will be updating you on the logs/error or anything thats possible from my end, tomorrow.

[Namburger]

Awesome, thanks! Tomorrow will be weekend for me but i;ll try to make it in the office :)
FYI, you are running this on window10, correct?

[jimxxmy]

@Namburger Yes, tested the delegate on both Windows 10 Home and Pro. Resulted in the same.

[jimxxmy]

@Namburger Ran the inference code with a pre trained model provided on coral website today, got the following exception:

"C:\Users\XXX\AppData\Local\Programs\Python\Python37\python.exe" "C:/Users/XXXX/PycharmProjects/Image Classification - Copy/inference.py"
W driver/package_registry.cc:67] Minimum runtime version required by package (5) is lower than expected (10).
W driver/package_registry.cc:67] Minimum runtime version required by package (5) is lower than expected (10).

Process finished with exit code 0

Code:

import tflite_runtime.interpreter as tflite

model_path = "C:/Users/XXXX/PycharmProjects/Image Classification - Copy/classifier/inception_v4_299_quant_edgetpu.tflite"
interpreter = tflite.Interpreter(model_path,
                                 experimental_delegates=[tflite.load_delegate('edgetpu.dll')])

interpreter.allocate_tensors()

The connecting ->> disconnecting ->> and after the process finished with exit code 0, reconnected was observed. Can you please provide a hint as to what is happening?

[jimxxmy]

@Namburger,

Second Update: I ran the example from here: Google Coral Classification Example, device connected to both 2.0 and 3.0 USB port, code was executed but the issue persists of getting the device disconnecting and reconnecting!

Inference times :

```python classify_image.py --model models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite --labels models/inat_bird_labels.txt --input images/parrot.jpg

----INFERENCE TIME----
Note: The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory.
20.8ms
8.5ms
6.9ms
7.5ms
6.8ms
-------RESULTS--------
Ara macao (Scarlet Macaw): 0.77734
```

[Namburger]

@jimxxmy
On the first update, it looks like you are running custom code, I wouldn't really be able to tell what's wrong unless I can see it.

Minimum runtime version required by package (5) is lower than expected (10).

Seems to indicates that you're using a different tensorflow runtime? If you link me to your code, I could give more pointers.

On the second update, everything looks fine to me, the example ran inference 5 times over the parrot image and outputs time + probability.

[jimxxmy]

@Namburger I have mentioned the code above. That was the three lines of code, including:

1. loading a pre trained model in .tflite.
2. calling the delegate for the interpreter.
3. allocating the tensor.

All that happens but only after the device gets disconnected. That is the issue. Coral being disconncted isn't odd to you?

[Namburger]

@jimxxmy Sorry, I don't have a window machine so I can't say for sure if the connect/disconnect noise is normal or not. I'll have a call with someone in our team with a window machine to check later today
However, libedgetpu does transfer bytes of models + inputs into the edgetpu and reset the state when allocate tensors is called so I wouldn't be surprised that it triggers some windows mechanism that thinks it being disconnected.
From the ouputs of our demo, everything looks good and as documented.

For your model, can you try running this script?

import numpy as np 
import sys, time
from tflite_runtime.interpreter import Interpreter
from tflite_runtime.interpreter import load_delegate

if len(sys.argv) < 2:
    print('Usage:', sys.argv[0], 'model_path')
    exit()

def main():
    """Runs inference with an input tflite model.""" 
    model_path = str(sys.argv[1])
    if model_path.endswith('edgetpu.tflite'):
        print('initialized for edgetpu')
        delegates = [load_delegate('edgetpu.dll')]                             
        interpreter = Interpreter(model_path, experimental_delegates=delegates)
    else: 
        print('initialized for cpu')
        interpreter = Interpreter(model_path)

    interpreter.allocate_tensors() 
    input_details = interpreter.get_input_details() 
    images = np.zeros(input_details[0]['shape'], input_details[0]['dtype'])
    interpreter.set_tensor(input_details[0]['index'], images)
    output_details = interpreter.get_output_details() 
    outputs = interpreter.get_tensor(output_details[0]['index']) 
    print(outputs)
    print('Success.') 

if __name__== '__main__':
    main()

It basically loads the model + delegate and run 1 inference on random data and print the outputs.

[Namburger]

Also, can you provide the md5sum of edgetpu.dll? Just wanted to make sure that you're on the most updated version

[jimxxmy]

@jimxxmy Sorry, I don't have a window machine so I can't say for sure if the connect/disconnect noise is normal or not. I'll have a call with someone in our team with a window machine to check later today
However, libedgetpu does transfer bytes of models + inputs into the edgetpu and reset the state when allocate tensors is called so I wouldn't be surprised that it triggers some windows mechanism that thinks it being disconnected.
From the ouputs of our demo, everything looks good and as documented.

For your model, can you try running this script?

import numpy as np 
import sys, time
from tflite_runtime.interpreter import Interpreter
from tflite_runtime.interpreter import load_delegate

if len(sys.argv) < 2:
    print('Usage:', sys.argv[0], 'model_path')
    exit()

def main():
    """Runs inference with an input tflite model.""" 
    model_path = str(sys.argv[1])
    if model_path.endswith('edgetpu.tflite'):
        print('initialized for edgetpu')
        delegates = [load_delegate('edgetpu.dll')]                             
        interpreter = Interpreter(model_path, experimental_delegates=delegates)
    else: 
        print('initialized for cpu')
        interpreter = Interpreter(model_path)

    interpreter.allocate_tensors() 
    input_details = interpreter.get_input_details() 
    images = np.zeros(input_details[0]['shape'], input_details[0]['dtype'])
    interpreter.set_tensor(input_details[0]['index'], images)
    output_details = interpreter.get_output_details() 
    outputs = interpreter.get_tensor(output_details[0]['index']) 
    print(outputs)
    print('Success.') 

if __name__== '__main__':
    main()

It basically loads the model + delegate and run 1 inference on random data and print the outputs.

Ran this code of yours, with the command line interface as you suggested: python your_code.py GET /feeds and had this come up in the interface:
python model_to_tflite.py GET /feeds initialized for cpu Traceback (most recent call last): File "model_to_tflite.py", line 33, in <module> main() File "model_to_tflite.py", line 20, in main interpreter = Interpreter(model_path) File "C:\Users\XXX\AppData\Local\Programs\Python\Python37\lib\site-packages\tflite_runtime\interpreter.py", line 205, in __init__ model_path, self._custom_op_registerers)) ValueError: Could not open 'GET'.

When run through the IDE, the scriptes gets executed with this INFO:

C:\Users\XXXX\PycharmProjects\Image Classification - Copy\gui\inference_module\classifier\inception_v4_299_quant_edgetpu.tflite

Process finished with exit code 0

[jimxxmy]

Also, can you provide the md5sum of edgetpu.dll? Just wanted to make sure that you're on the most updated version

For the direct one: 637B4ECE601BF447BFEB9257DC3403CA

For the throttled one: EA58212B2E690DA0DAF6289AC5B1BF08

[Namburger]

python model_to_tflite.py GET /feeds

sorry, but that's incorrect usage... the script expects a model path and get it from sys.argv[1],

if len(sys.argv) < 2:
    print('Usage:', sys.argv[0], 'model_path')
    exit()

Basically

python3 filename.py path/to/model.tflite

The error you are seeing is the interpreter trying to load a model call "GET" which doesn't exist

[jimxxmy]

@Namburger No, in the second one I gave the path to tflite model as you can see where it says process exited with exit code 0 The device did not disconnected, neither did anything happen.

[Namburger]

1) On the cmd you are not giving the model, I suggest you use this method and fix the model arguments
2) on the IDE it looks like you're just running the model, not my script, hence nothing happens.

My apologies, but I really need your info in order to debug, without collaboration on your side I'm not getting the necessary info to give you assistance. Anyhow, to me this still seems like a user errors to me as the demo is working just as expected.

[jimxxmy]

Hi, @Namburger. Executed your script that goes as follows:

import numpy as np
import sys, time
from tflite_runtime.interpreter import Interpreter
from tflite_runtime.interpreter import load_delegate

if len(sys.argv) < 2:
    print('Usage:', sys.argv[0], 'model_path')
    exit()


def main():
    """Runs inference with an input tflite model."""
    model_path = str(sys.argv[1])
    if model_path.endswith('edgetpu.tflite'):
        print('initialized for edgetpu')
        delegates = [load_delegate('edgetpu.dll')]
        interpreter = Interpreter(model_path, experimental_delegates=delegates)
    else:
        print('initialized for cpu')
        interpreter = Interpreter(model_path)

    interpreter.allocate_tensors()
    input_details = interpreter.get_input_details()
    images = np.zeros(input_details[0]['shape'], input_details[0]['dtype'])
    interpreter.set_tensor(input_details[0]['index'], images)
    output_details = interpreter.get_output_details()
    outputs = interpreter.get_tensor(output_details[0]['index'])
    print(outputs)
    print('Success.')


if __name__ == '__main__':
    main()

This is what I got today, with the noise of disconnecting and reconecting as usual. The output info seems to be the same as I posted earlier.

Output : Output:

W driver/package_registry.cc:67] Minimum runtime version required by package (5) is lower than expected (10).
W driver/package_registry.cc:67] Minimum runtime version required by package (5) is lower than expected (10).
[[0 0 0 ... 0 0 0]]
Success.

[Namburger]

This is actually what I was looking for:

[[0 0 0 ... 0 0 0]]
Success.

That means that the model successfully loaded can ran on the edgetpu and produced outputs, this is exactly what is expected for the edgetpu to do.
As for the edgetpu.dll, that looks okay, however only one should be loaded during runtime, could you check again the md5sum for the edgetpu.dll in c:\windows\system32?

[Edit] I checked with my team and it appears that the disconnect - connect sound is expected as the edgetpu.dll will reset the tpu's state when allocate_tensors() is called. We'll try see what we can do about that, but as of now things seems to be working as expected

[jimxxmy]

@Namburger Hi, apologies for the delayed response. Here is the md5sum once again for the edgetpu.dll that I checked in the system32

EA58212B2E690DA0DAF6289AC5B1BF08

Also, since the disconnect - connect sound is normal, I think it should be ignored as for now. : )