Tensorrt: Dynamic inputs can't add optimization profile use Python Api

Created on 5 Sep 2020  路  2Comments  路  Source: NVIDIA/TensorRT

I use python network api build a small engine successfully, but when I try to build a Dynamic Shape model following the instruction of this:
https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#runtime_dimensions.

I got an error says

"[TensorRT] ERROR: Network has dynamic or shape inputs, but no optimization profile has been defined."

I set the log level to Verbose but got no more information, what I miss?

code :

def build_engine(weights):
    # For more information on TRT basics, refer to the introductory samples.
    with trt.Builder(TRT_LOGGER) as builder, builder.create_network(common.EXPLICIT_BATCH) as network:
        builder.max_workspace_size = common.GiB(1)

        # Populate the network using weights from the PyTorch model.
        # populate_network(network, weights)
        # Build and return an engine.
        input_tensor = network.add_input(
        name=ModelData.INPUT_NAME, dtype=ModelData.DTYPE, shape=ModelData.INPUT_SHAPE)

        conv1_w = weights['conv1.weight'].cpu().numpy()

        out_planes = 64

        conv1 = conv3x3(input_tensor, out_planes, network, conv1_w)
        network.mark_output(tensor=conv1.get_output(0))

        config = builder.create_builder_config() 
        profile = builder.create_optimization_profile()
        profile.set_shape("input", (1,64, 1, 1), ( 1,64, 150, 250), (1,64, 200, 300)) 
        idx = config.add_optimization_profile(profile)
        engine = builder.build_cuda_engine(network)

        return engine

env

tensorRT 7.1.3.4
cuda 10.2
python 3.6

Most helpful comment

I think you've almost got, it except that when you build the engine, you need to provide your config which includes the profile(s):

engine = builder.build_engine(network, config=config)

All 2 comments

I think you've almost got, it except that when you build the engine, you need to provide your config which includes the profile(s):

engine = builder.build_engine(network, config=config)

@bing1zhi2 closing based on the above comment. Please respond if this is still a problem.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

yflv-yanxia picture yflv-yanxia  路  3Comments

peijason picture peijason  路  3Comments

dhkim0225 picture dhkim0225  路  6Comments

AlphaJia picture AlphaJia  路  3Comments

WangXuanBT picture WangXuanBT  路  3Comments