Centernet: How to deploy to C++ project, thank you!怎么部署到C++工程，谢谢！

[zoufangyu1987]

Now there is a pytorch model. The CenterNet code is python. My project needs c++. How to deploy it?
Note: My Python code ability is weak

现在已经有pytorch的模型，CenterNet代码都是python的，我的工程需要c++，怎么布署呢？
注：本人python代码能力弱

[kakaluote]

一种解决方案是python做服务器，c++通过socket发送检测请求。
还是希望能c++推断

[zoufangyu1987]

Because there are not only detection modules in my project, but also other modules, so I need C++, look forward to guidance!
因为我的工程里不只有检测模块，还有其它模块，所以需要C++，望指导！

[xingyizhou]

I honestly have no experience in c++ deployment ...

[zoufangyu1987]

@xingyizhou
Thank you anyway.

[Markusgami]

Convert model to caffe. And run it in caffe

[zoufangyu1987]

@Markusgami
This is OK, mainly forward-propagating code needs to be converted to c++, is there any available c++ code? Thank you!
这样是可以，主要是前向传播的代码需要转成c++，有可用的c++代码吗？谢谢！

[kunyao2015]

持续关注，有好的解决方案吗， 用libtorch?

[zoufangyu1987]

Wait online, it's urgent!
在线等，挺急的!

[jnulzl]

@zoufangyu1987 @kunyao2015
实现步骤：
1、把模型转成caffemodel;
2、前后处理自己c++实现；
3、Done!
亲测可以，祝你好运！

[zoufangyu1987]

@jnulzl
Can you share your c++ code? Thank you!
可以分享你的c++代码吗？谢谢！

[jnulzl]

@zoufangyu1987
不好意思，暂时不行

[zoufangyu1987]

@jnulzl
I want to cry!
好想哭!

[BokyLiu]

用trace转模型,再用libtorch部署吧,亲测可用

[wangshankun]

@zoufangyu1987 @kunyao2015
实现步骤：
1、把模型转成caffemodel;
2、前后处理自己c++实现；
3、Done!
亲测可以，祝你好运！

转caffe时候的DCNV2怎么办？caffe又不是天然支持

[zoufangyu1987]

我把没有dcn层的dlav0_34的pytorch模型已经转成caffemodel，这两天理了下centernet的demo的前处理和后处理相关python代码，还是蛮繁琐的，有搞好的朋友分享下C++代码啊，万分感谢！
I have converted the pytorch model of dlav0_34 without DCN layer into caffe model. These two days, I have handled the Python code related to the pre-processing and post-processing of the demo in the CenterNet. It is Difficult. Who can share the C++ code well? Thank you very much!

[Fighting-JJ]

用trace转模型,再用libtorch部署吧,亲测可用

trace 可以成功么？你训练的什么arch的模型？
@BokyLiu

[BokyLiu]

用trace转模型,再用libtorch部署吧,亲测可用

trace 可以成功么？你训练的什么arch的模型？
@BokyLiu

res18的

[Fighting-JJ]

@zoufangyu1987 @kunyao2015
实现步骤：
1、把模型转成caffemodel;
2、前后处理自己c++实现；
3、Done!
亲测可以，祝你好运！

转caffe时候的DCNV2怎么办？caffe又不是天然支持

请问，你解决了DCNv2的部署问题么？

[zoufangyu1987]

@Fighting-JJ
没有，没有找到pytorch转caffemodel支持DCNV2层的代码，现在用dlav0_34，放弃dcn，caffemodel已经验证输出的output参数完全一致，不过是在python上验证的，C++还没有搞，工作量有点大，坑有点多，发现有好些朋友已经成功，但不分享源码也没办法，只能一步一步搞
No, I haven't found the code of pytorch to caffemodel to support DCNV2 layer. Now I use dlav0_34, give up dcn, caffemodel to verify that the output parameters of output are exactly the same. However, it's verified on python. C++ hasn't been done yet, the workload is a bit heavy, the pit is a bit too many. I find that some friends have succeeded, but they don't share the source code. There's no way to do it. We can only do it step by step.

[zoufangyu1987]

等我成功在C++上部署，一定分享给大家源码
When I successfully deploy on C++, I will share the source code with you.

[zoufangyu1987]

我已经把pytorch全部剥离，在python上依赖numpy正常跑通了，下一步转C++，发现numpy有C++版本——"numcpp"，搞得身心疲惫，希望后面少点坑！
I've stripped all pytorch and relied on numpy to run normally on python. Next, I turn to C++. I find that numpy has a version of C++ - "numcpp", which makes me tired physically and mentally. I hope there are fewer pits behind it.

[Fighting-JJ]

我已经把pytorch全部剥离，在python上依赖numpy正常跑通了，下一步转C++，发现numpy有C++版本——"numcpp"，搞得身心疲惫，希望后面少点坑！
I've stripped all pytorch and relied on numpy to run normally on python. Next, I turn to C++. I find that numpy has a version of C++ - "numcpp", which makes me tired physically and mentally. I hope there are fewer pits behind it.

You can use jit.trace to trace the model then deploy it with c++ by libtorch which is a c++ library.
then only the post-process is left.

[chenjx1005]

我已经把pytorch全部剥离，在python上依赖numpy正常跑通了，下一步转C++，发现numpy有C++版本——"numcpp"，搞得身心疲惫，希望后面少点坑！
I've stripped all pytorch and relied on numpy to run normally on python. Next, I turn to C++. I find that numpy has a version of C++ - "numcpp", which makes me tired physically and mentally. I hope there are fewer pits behind it.

It's no need to use numcpp. You can read the image to CV::Mat by opencv in C++ version, and convert the Mat to caffe::Blob.

[zoufangyu1987]

@chenjx1005
我也认为仅仅用OpenCV是可以的，不过我已经在用NumCpp和OpenCV结合，我先试试，如果不行再去掉NumCpp
I also think it's possible to use OpenCV only.But I've combined NumCpp with OpenCV. I'll try it first, and if I can't, I'll remove NumCpp.

[hexiangquan]

https://github.com/hexiangquan/CenterNetCPP

[zoufangyu1987]

@hexiangquan
感激不尽
Be deeply grateful

[zoufangyu1987]

基于numcpp的C++版本加载caffemodel已经成功了，结果一致，谢谢大家！

[BokyLiu]

基于numcpp的C++版本加载caffemodel已经成功了，结果一致，谢谢大家！

期待您的分享

[zoufangyu1987]

@BokyLiu
这几天上班事情有点多，这两天空下来我就把整个流程和相关文件整理一下分享给大家
上面@hexiangquan也已经分享预处理、forward、后处理的c++代码，感谢他！

[Fighting-JJ]

我已经把pytorch全部剥离，在python上依赖numpy正常跑通了，下一步转C++，发现numpy有C++版本——"numcpp"，搞得身心疲惫，希望后面少点坑！
I've stripped all pytorch and relied on numpy to run normally on python. Next, I turn to C++. I find that numpy has a version of C++ - "numcpp", which makes me tired physically and mentally. I hope there are fewer pits behind it.
You can use jit.trace to trace the model then deploy it with c++ by libtorch which is a c++ library.
then only the post-process is left.

您trace的时候dcn_v2部分有没有trace成功？ @Fighting-JJ

我没有使用DCN的，用的DLA034或者是resnet

[Fighting-JJ]

架构没有dcn的没有用到的。

发自我的 iPhone

在 2019年8月24日，下午5:30，yanfei96 [e-mail] 写道：

我已经把pytorch全部剥离，在python上依赖numpy正常跑通了，下一步转C++，发现numpy有C++版本——"numcpp"，搞得身心疲惫，希望后面少点坑！
I've stripped all pytorch and relied on numpy to run normally on python. Next, I turn to C++. I find that numpy has a version of C++ - "numcpp", which makes me tired physically and mentally. I hope there are fewer pits behind it.
You can use jit.trace to trace the model then deploy it with c++ by libtorch which is a c++ library.
then only the post-process is left.

您trace的时候dcn_v2部分有没有trace成功？ @Fighting-JJ

我没有使用DCN的，用的DLA034或者是resnet

我也是基于resnet18实现的，但centernet网络上采样的时候使用了dcn_v2中的层。请问您编译dcnV2没？我编译dcnV2的时候有一些问题，后面是基于#118 (comment) 这上面的方法编译的。 @Fighting-JJ

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[zoufangyu1987]

我已经共享在如下：
链接: https://pan.baidu.com/s/1m6zdWSKE8soSMwXRbU1aeg
提取码: yntr

[18022443868]

我已经共享在如下：
链接：https
://pan.baidu.com/s/1m6zdWSKE8soSMwXRbU1aeg提取码：yntr

麻烦发下完整的DLA34 prototxt文件给我下，万分感谢[email protected]

[zoufangyu1987]

@18022443868 你把共享的DLA.prototxt去掉最后我加的pooling层就是完整的，谢谢！公司发邮件不方便。

去掉最后这一层
layer {
name: "max_pool53"
type: "Pooling"
bottom: "conv_blob53"
top: "max_pool53"
pooling_param {
pool: MAX
kernel_size: 3
stride: 1
pad: 1
}
}

我加这一层是为了代替作者python的热图解析类似非极大值抑制操作，decode.py文件下的这部分：

def _nms(heat, kernel=3):
pad = (kernel - 1) // 2
hmax = nn.functional.max_pool2d(heat, (kernel, kernel), stride=1, padding=pad)
keep = (hmax == heat).float()
return heat * keep

[18022443868]

你有尝试将其转化为ncnn来进行再次加速运算吗？

| |
任云
|
|
邮箱：[email protected]
|

签名由 网易邮箱大师 定制

在2019年09月24日 09:56，zoufangyu1987 写道：

@18022443868 你把共享的DLA.prototxt去掉最后我加的pooling层就是完整的，谢谢！公司发邮件不方便。

去掉最后这一层
layer {
name: "max_pool53"
type: "Pooling"
bottom: "conv_blob53"
top: "max_pool53"
pooling_param {
pool: MAX
kernel_size: 3
stride: 1
pad: 1
}
}

我加这一层是为了代替作者python的热图解析类似非极大值抑制操作，decode.py文件下的这部分：

def _nms(heat, kernel=3):
pad = (kernel - 1) // 2
hmax = nn.functional.max_pool2d(heat, (kernel, kernel), stride=1, padding=pad)
keep = (hmax == heat).float()
return heat * keep

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[18022443868]

你好，如果方便麻烦告知下你的联系方式，谢谢，我看了下你的prototxt也是按照ncnn格式去修改的，那么你在ncnn中调用的cpp文件可否共享参考？万分感激！！！

在 2019-09-24 09:56:03，"zoufangyu1987" [e-mail] 写道：

@18022443868 你把共享的DLA.prototxt去掉最后我加的pooling层就是完整的，谢谢！公司发邮件不方便。

去掉最后这一层
layer {
name: "max_pool53"
type: "Pooling"
bottom: "conv_blob53"
top: "max_pool53"
pooling_param {
pool: MAX
kernel_size: 3
stride: 1
pad: 1
}
}

我加这一层是为了代替作者python的热图解析类似非极大值抑制操作，decode.py文件下的这部分：

def _nms(heat, kernel=3):
pad = (kernel - 1) // 2
hmax = nn.functional.max_pool2d(heat, (kernel, kernel), stride=1, padding=pad)
keep = (hmax == heat).float()
return heat * keep

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[YuhaoYeSteve]

我已经共享在如下：
链接: https://pan.baidu.com/s/1m6zdWSKE8soSMwXRbU1aeg
提取码: yntr

大佬可以传到GitHub开个新repo，这样大家可以讨论

[mingx9527]

@all 本人实际应用中，采用的是grpc模式，请测有效；
即：
1）把C++端搞成客户端（c++ grpc），发送图像image
2）将centernet（pytorch+python）加上python grpc外壳，变为服务器；接受image，并前向出结果，打包并返回
3）C++端再接受 返回的结果 继续后续工作

这种模式好处是：不需要转化模型，再遇到其他模型，也可套用

[mingx9527]

@ALL 另一种方式也可尝试：C++直接调用python下的pytorch模型（亲测有效）
参见博客: https://blog.csdn.net/u011681952/article/details/92765549

[foreversky0611]

我已经共享在如下：
链接: https://pan.baidu.com/s/1m6zdWSKE8soSMwXRbU1aeg
提取码: yntr
@zoufangyu1987 你好，此連結已無法訪問，可以方便在分享嗎?

[ivankk2003]

@zoufangyu1987 你好，此連結已無法訪問，可以方便在分享嗎?

Bcoz baidu will block access from somewhere in the world ...
That's why you can't access it

[zoufangyu1987]

@YuhaoYeSteve
鄙人不才，还没有github上传文件开repo的经验，要不就先在这里讨论
@mingx9527
您的第二种方法我完全赞同，第一种本人没有经验我不发表意见，谢谢您的分享！

[zoufangyu1987]

@foreversky0611
刚刚一直在尝试fan_墙链接网盘，非常遗憾没有fan成功，可能是国庆严禁，所以我也不知道这个网盘能不能访问。

[Wenboisstrong]

@zoufangyu1987 感谢您的工作，通过一点更改我成功的将DLA034 的torch模型 转换成 caffemodel 并在c++验证，但是我有一个疑问 ，torch model 运行会占用500M 的显存 但是 caffe model 会占用 2.5G的显存。我认为caffe model 占用的显存太大了 并不适合在工程中使用 因为我可能会同时加载多个模型。您有什么优化的经验么 。谢谢
Thanks for your work, I successfully converted the DRA034's torch model to caffemodel and verified it in C++ with a little change, but I have a question. The torch model will take up 500M of video memory but the caffe model will take up 2.5G of gpu memory. I think the caffe model takes up too much memory and is not suitable for use in engineering because I might load multiple models at the same time. Do you have any optimization experience? Thank you

[SFM2020]

用trace转模型,再用libtorch部署吧,亲测可用

麻烦大佬写个博客详细告知

[SFM2020]

用trace转模型,再用libtorch部署吧,亲测可用

麻烦大佬写个博客详细告知

[ssshcj]

@zoufangyu1987 您好 我直接使用您提供的代碼和prototxt和caffemodel文件 成功運行 但是跟蹤效果和跟蹤精度非常差 完全不是原來的centernet 不知道是什麼原因 請問您有什麼改進建議嗎？

[zoufangyu1987]

@ssshcj
我是基于自己的数据训练的模型，电警相机抓拍的图像，图像中的目标是什么样子您可以网上查一查资料，我检测5类--大车、小车、行人等。如果您检测的图像不是类似的效果不好是肯定的。

里面所有和类别相关的参数都得修改，抱嫌，当时写代码的时候为了尽快调通，没有搞个宏定义一下。不过这个不难，相信您可以自己搞定它。

[zoufangyu1987]

@Wenboisstrong
其实不转成caffemodel也可以的，用torch的模型在libtorch的C++版本前向，这样就可以控制显存了。
后处理都是一样的。
我暂时没做这个工作，非常感谢您的提醒，我后面有时间也会试试在libtorch上向前来降显存（其实我们也是显存资源紧缺，主要是目标检测和分割占显存）

[ssshcj]

@zoufangyu1987
您好 剛接觸這個領域 不太了解 想問您幾個問題：
1：如何選擇網絡架構呢？比如DLA034和resnet，直接在訓練的命令後面加參數，就可以得到想要的架構的文件了嗎？
2 : dlav0_34和dla34有區別嗎，如果有的話，是不是缺少的DCNV2區別呢？
3：要怎麼刪除DCNv2層呢？
4：原作者提供的ctdet_coco_dla_2x.pth是什麼架構的呢？可以直接拿來轉caffemodel嗎？

[zoufangyu1987]

@ssshcj
针对您的问题，回答如下：
1、对的，直接设置选择模型参数就行，在opts.py文件里设置
2、您说的没错
3、dlav0_34就去除了可变卷积dcnv2
4、这个pth我不确定里有还是没有dcnv2的网络，你可以再看看作者说明，如果是没有dcnv2的模型，我转成功 caffemodel，有dcnv2的我没有转成功，谢谢！（pytorch模型都能转caffemodel，只要有支持对应的层的变换）

[WooXinyi]

你把共享的DLA.prototxt去掉最后我加的pooling层就是完整的，谢谢！公司发邮件不方

你好，我把作者原来的ctdet_coco_hg.pth成功转化为caffe模型，获得.caffemodel和.protxt两个文件。我直接在.protxt文件后面加上对应的最大池化层就可以使用了吗？

[WonderAndMaps]

@Fighting-JJ Is it possible for you to share the code for tracing it with jit.trace? Many thanks!

[zoufangyu1987]

@WooXinyi 是的

[ssshcj]

@zoufangyu1987 您好 使用您的算法运行速度比较慢 速度大约在0.147563s 请问您有什么加速改进的建议吗？

[zoufangyu1987]

@ssshcj 加速一方面是后处理的逻辑的本人写的很差，优化空间很大；还有就是预处理可以由CPU放到GPU；然后就是模型优化了，这个就太广了，研究的方向很多；才疏学浅，了解的就这么点。

[121649982]

用trace转模型,再用libtorch部署吧,亲测可用
兄弟，可以告诉你怎么实现 的吗？我QQ：121649982，可以加一下吗？

[VishnuPJ]

@zoufangyu1987 If it is possible could you please share the C++ pre-processing and post processing code? I have managed to make the pt file for hourglass model and was able to load it in windows using libtorch. Only thing left is the pre-processing and post processing steps. I was notable to access the baidu link you shared. Is there any other way you can share the code?

[zoufangyu1987]

@VishnuPJ
公司无法向外发邮件，要不你再试试连百度网盘，网盘可以链接，总有办法的。
The company can't send out email, or you can try to connect Baidu online disk again, online disk can be linked, there is always a way.

[121649982]

@zoufangyu1987 If it is possible could you please share the C++ pre-processing and post processing code? I have managed to make the pt file for hourglass model and was able to load it in windows using libtorch. Only thing left is the pre-processing and post processing steps. I was notable to access the baidu link you shared. Is there any other way you can share the code?
how can you convert your model to pt?

[121649982]

@ALL 另一种方式也可尝试：C++直接调用python下的pytorch模型（亲测有效）
参见博客: https://blog.csdn.net/u011681952/article/details/92765549

这个需要python

[VishnuPJ]

@zoufangyu1987 If it is possible could you please share the C++ pre-processing and post processing code? I have managed to make the pt file for hourglass model and was able to load it in windows using libtorch. Only thing left is the pre-processing and post processing steps. I was notable to access the baidu link you shared. Is there any other way you can share the code?
how can you convert your model to pt?

This worked for me,
https://github.com/xingyizhou/CenterNet/issues/414#issuecomment-549716771

[zoufangyu1987]

@BokyLiu
@Fighting-JJ
为什么我的模型在libtorch比pytorch多很多，pytorch显存占750M，libtorch占1300M，为什么会这样？
Why does my model have a lot more in libtorch than in pytorch? Why does pytorch display 750m memory and libtorch 1300m memory?

[121649982]

libtorch优化不好，而且速度很慢，建议用tensorrt

[Dantju]

@zoufangyu1987 I cannot find file about Alg_VIR, so cannot build centernet c++ project,can u share this file?

[zoufangyu1987]

@Dantju
This file is not needed. You can remove the header file
这个文件不需要，你可以把包涵的头文件去掉

[Dantju]

@zoufangyu1987 但是centernet.cpp中有一些这样的参数定义

[zoufangyu1987]

@Dantju
你不用管它，它只是我项目工程里的一些东西，当时没有删除。

[Dantju]

@zoufangyu1987 sVIRInput 整个结构体在后面的code中用到了，那么整个头文件十是否能分享一下呢

sVIRInput virInput;
//sVIROutput virOutput;
//std::string img_name;
//#undef GPU
// Caffe::set_mode(Caffe::GPU);
// Caffe::SetDevice(0);
//#define GPU

Caffe::set_mode(Caffe::CPU);

const std::string modelPath;
init(modelPath);

if 01

std::vector<std::string> img_name_path;
std::vector<std::string> Img_info;
cv::Mat imgin;
//一次处理一张图片
while(getline(filelist,line))
{
    //获得图片路径和图片信息
    virInput.vInImg.clear();
    img_name_path.clear();

[Dantju]

@zoufangyu1987 这个code是在windows下编译的吗，我编译总是遇到syntax error:missing ';' before '}'

[zoufangyu1987]

@Dantju
linux

[Dantju]

@zoufangyu1987 ok，那么能提供一下sVIRInput 的头文件吗

[zoufangyu1987]

@Dantju
那只是我自己项目工程里定义的结构体，用来加图像的，你把它删除自己创建一个图像就好了，改一改就行，理解一下代码就行了，很容易

[Dantju]

有成功在windows下跑成功的吗，可以分享下code吗

[xiaowk5516]

感谢您分享的代码，想请问一下，您后来有对代码进行优化么？是怎么做的呢？

[zoufangyu1987]

@xiaowk5516
后处理这部分我没有做优化，个人能力有限。
有个排序函数在debug上处理很慢需要50-70ms(和硬件平台有关系)，但在release模式下大约10ms.
对处理速度方面做了一些优化，主要是针对图像处理部分，采用simd图像并行加速库，对resize和copy等的替换。提升速度显著。特别是在图像预处理部分。可参考：https://www.jianshu.com/p/5b272f108ed2

I didn't optimize the post-processing part, and my personal ability is limited.
A sort function is very slow on debug, which takes 50-70 MS (depending on the hardware platform), but about 10 ms in release mode
The speed of processing has been optimized, mainly for the image processing part, the SIMD image parallel acceleration library is used to replace restore and copy. The lifting speed is remarkable. Especially in the image preprocessing part. For reference: https://www.jianshu.com/p/5b272f108ed2

[xiaowk5516]

@xiaowk5516
后处理这部分我没有做优化，个人能力有限。
有个排序函数在debug上处理很慢需要50-70ms(和硬件平台有关系)，但在release模式下大约10ms.
对处理速度方面做了一些优化，主要是针对图像处理部分，采用simd图像并行加速库，对resize和copy等的替换。提升速度显著。特别是在图像预处理部分。可参考：https://www.jianshu.com/p/5b272f108ed2

I didn't optimize the post-processing part, and my personal ability is limited.
A sort function is very slow on debug, which takes 50-70 MS (depending on the hardware platform), but about 10 ms in release mode
The speed of processing has been optimized, mainly for the image processing part, the SIMD image parallel acceleration library is used to replace restore and copy. The lifting speed is remarkable. Especially in the image preprocessing part. For reference: https://www.jianshu.com/p/5b272f108ed2

多谢您的回答，我会去了解一下的，再次感谢。

[leilaShen]

用trace转模型,再用libtorch部署吧,亲测可用

我用trace转成了模型 但是模型的结果在c++中调用产生的结果和python中不同 应该是forward用trace还是不准确。 但是script太难用了 转不了模型