Detectron2: build_backbone(cfg) creates model on CPU, even when cfg.MODEL.DEVICE is 'cuda'

[gsrivas4]

I am trying to use build_backbone() to build a backbone network. I have image tensors in NCHW_images. When I try to run inference on this backbone, it seems the images are on gpu but the weights are still on cpu. I have checked cfg.MODEL.DEVICE and it seems to be set to 'cuda'. Below is the of code I am running, with the error I get:

ipdb> cfg.MODEL.DEVICE
'cuda'
ipdb> from detectron2.modeling import build_backbone
ipdb> backbone = build_backbone(cfg)
ipdb> backbone(NCHW_images)
*** RuntimeError: Input type (torch.cuda.ByteTensor) and weight type (torch.FloatTensor) should be the same

Is there a cfg parameters I should set to move the backbone model to gpu?

[ppwwyyxx]

Is there a cfg parameters I should set to move the backbone model to gpu?

No but you can call backbone.to('cuda'). The functions are meant to build only the architecture and cfg.MODEL.DEVICE is not used. This is indeed a bit confusing and we'll think about how to improve this

[gsrivas4]

I got another error. I have images in the format required my backbone, still it gives me an error:

ipdb> NCHW_images
tensor([[[[127., 123., 116.,  ..., 249., 249., 249.],
          [126., 123., 119.,  ..., 249., 249., 249.],
          [125., 124., 123.,  ..., 249., 249., 249.],
          ...,
          [ 51.,  49.,  47.,  ..., 145., 144., 144.],
          [ 45.,  42.,  39.,  ..., 135., 140., 143.],
          [ 41.,  38.,  34.,  ..., 129., 137., 143.]],

         [[139., 135., 128.,  ..., 255., 255., 255.],
          [138., 135., 131.,  ..., 255., 255., 255.],
          [137., 136., 135.,  ..., 255., 255., 255.],
          ...,
          [ 49.,  49.,  48.,  ..., 153., 152., 152.],
          [ 45.,  43.,  40.,  ..., 143., 148., 152.],
          [ 42.,  39.,  35.,  ..., 137., 146., 152.]],

         [[175., 171., 164.,  ..., 254., 254., 254.],
          [174., 171., 167.,  ..., 254., 254., 254.],
          [173., 172., 171.,  ..., 254., 254., 254.],
          ...,
          [ 41.,  40.,  39.,  ..., 242., 241., 241.],
          [ 36.,  34.,  31.,  ..., 233., 238., 242.],
          [ 33.,  30.,  25.,  ..., 227., 236., 242.]]]], device='cuda:0')
ipdb> backbone(NCHW_images)
*** RuntimeError: The size of tensor a (67) must match the size of tensor b (68) at non-singleton dimension 3

[ppwwyyxx]

there are requirements on input size https://detectron2.readthedocs.io/modules/modeling.html#detectron2.modeling.ResNet

[liuhl-source]

.to('cuda')

thanks! Can you give me some advice for backbone feature extraction with multi-gpu.