Darkflow: How to convert a checkpoint to weight file?

[urbov]

Yolo2 Tiny works great to detect cars using the Udacity training dataset. They have released a tensorflow checkpoint, see https://github.com/udacity/self-driving-car/tree/master/vehicle-detection/darkflow.

My embedded system uses the darknet C-implementation. Is is possible to convert the checkpoint to a weight file?

[lotfielhafi]

I am also very interested in this. In particular, I use a ROS implementation that uses the .cfg and .weights files to detect objects from a camera stream. I thought the main purpose of darkflow was to obtain a new .weights file after training but I could not. Any idea about how to do it?

[zhisong]

I think probably the easiest thing to try is to use sess.run() on the weight variables, converting them to numpy arrays and then do whatever you like on them.

[wflijunnan]

And I am also very interested in this，too，hope someone give us a solution about the weigts and checkpoint。I don‘t know the different of two

[4F2E4A2E]

I am also very interested in this.
We kind of need the weight files only to test or predict, but can I do that with the checkpoints as well?
Or do we have to save the protobuf file using --savepb?

[merryHunter]

I am very interested, can anyone give me a hint how I can convert graph to .pb if I am training a model from scratch?

[thtrieu]

I will probably add this, though really not seeing the appeal of it, especially when there is a new feature of loading from .pb now.

.weights file structure is not intuitive to me, the rules and parameters of it are read from darknet source code and follow verbatim in python. Reverse engineering this process is trivial, the process includes reading from .weights using ./darkflow/utils/loader.py script and reshape/transpose these 1D floats using finalize() methods of ./darkflow/dark/connected.py and ./darkflow/dark/convolution.py

[4F2E4A2E]

@thtrieu thank your a lot for our input. My goal is to train from scratch and to then save the state of the training and be able to test or predict on it.
I will totally go for the protocol buffer file (.pb) if it is the best and compatible fit when using darkflow.
Would this also mean, that if doing so, no more .weight files are necessary?

[thtrieu]

When you train, darkflow will output ckpt files, loading from them is the recommended way, although you can load from .pb too. In short, yes, no more .weights is needed, since we have ckpt and bp now.

[surajpaib]

I'm looking to run inference with the OpenCV dnn module. It requires the weights and cfg file. Any suggestions on how I can convert checkpoints to these?

[EscVM]

I will probably add this, though really not seeing the appeal of it, especially when there is a new feature of loading from .pb now.

.weights file structure is not intuitive to me, the rules and parameters of it are read from darknet source code and follow verbatim in python. Reverse engineering this process is trivial, the process includes reading from .weights using ./darkflow/utils/loader.py script and reshape/transpose these 1D floats using finalize() methods of ./darkflow/dark/connected.py and ./darkflow/dark/convolution.py

Sorry, I didn't get it 🤕

[adrielkuek]

Hi, not too sure if this thread is still active, but I, too, am looking for the same solution for backward compatibility to go back to .weights file from the chkpts generated. Understand that it may be trivial in the reverse engineering process, but the way I see it, it is quite a tedious process of reshaping and transposing from the [3, 3, 3, 16] 4-dim array back to 1-D floats. Anybody has any luck doing it right?

[bhaskardey]

@adrielkuek

You may have noticed have noticed that in checkpoint folder, files having extension .data-XXXXX-of-YYYYY are generated. Just rename the file to .weights and check if it works. The file size is larger compared to original .weights generated by darknet. It worked for me!

[KausthubK]

@bhaskardey
You may have noticed have noticed that in checkpoint folder, files having extension .data-XXXXX-of-YYYYY are generated. Just rename the file to .weights and check if it works. The file size is larger compared to original .weights generated by darknet. It worked for me!

I'm not sure how you managed to get that working - did you have to modify something else as well? I was getting an assertion error of the following style:

AssertionError: expect 202355760 bytes, found 607678472 bytes

Could you share more information please? It's my understanding that these are inherently different files which is why the new .pb APIs are the main way to go (hence no update by @thtrieu to convert to .weights).

I'm trying to integrate with darknet_ros so I might need to go the long way and go from .pb to caffe to weights as mentioned in #811 - jury is still out on that method though...