|
|
最近在研究TensorRT 反序列化
python 3.12 編譯.engine檔案時
跑出辨識分數居然只有0.4上下
改用 Python 3.10 環境辨識分數0.8或更換 TensorRT/JetPack 環境後(辨識度高的SSD版本是11.4.0.106 ,libnvinfer-bin 10.16.2.10-1+cuda13.2)
Python TensorRT .engine 反序列化
卻無法使用,須重新編譯.engine檔案
不然會出現錯誤
測試.engine檔案是否正常輸出
- python3 -c "
- import tensorrt as trt
- logger = trt.Logger(trt.Logger.WARNING)
- runtime = trt.Runtime(logger)
- with open('yolo11n.engine', 'rb') as f:
- eng = runtime.deserialize_cuda_engine(f.read())
- print('yolo11n engine:', eng)
- "
複製代碼 出現NoneType,代表輸出錯誤,不能用
AttributeError: 'NoneType' object has no attribute 'create_execution_context'
Python TensorRT .engine 反序列化
這時要檢查你的 TensorRT版本(python 3.12與 python 3.10產出檔案不能共用)
主要是Python TensorRT Builder 版本(如:11.4.0.106 對應的工具建置)
解決步驟1.用 Python TensorRT Builder 將 onnx 轉為 engine:直接用目前 venv 的 tensorrt 建置 engine。在啟動 venv 的終端機執行以下腳本,TensorRT 11.x(11.4.0.106)直接建置並立刻測試反序列化: 官方已移除 BuilderFlag.FP16 與 BuilderFlag.INT8 這類精度旗標,精度改由網路張量型態或模型本身決定,因此呼叫 trt.BuilderFlag.FP16 會直接觸發 AttributeError - python3 -c "
- import tensorrt as trt
- logger = trt.Logger(trt.Logger.WARNING)
- builder = trt.Builder(logger)
- network = builder.create_network(0)
- parser = trt.OnnxParser(network, logger)
- with open('w600k_mbf.onnx', 'rb') as f:
- assert parser.parse(f.read()), 'ONNX parse failed'
- inp = network.get_input(0)
- dims = tuple(112 if d == -1 and i >= 2 else (3 if d == -1 and i == 1 else (1 if d == -1 else d)) for i, d in enumerate(inp.shape))
- profile = builder.create_optimization_profile()
- profile.set_shape(inp.name, dims, dims, dims)
- config = builder.create_builder_config()
- config.add_optimization_profile(profile)
- serialized = builder.build_serialized_network(network, config)
- assert serialized is not None, 'build_serialized_network returned None'
- with open('w600k_mbf.engine', 'wb') as f:
- f.write(serialized)
- print('Built w600k_mbf.engine successfully with TRT', trt.__version__)
- "
複製代碼
Verification: 終端機印出 Deserialized engine: <tensorrt.tensorrt.ICudaEngine ...> 且不是 None,也不再出現 878 或 Error Code 6。
2.用相同方式重新產生 w600k_mbf.engine:套用到人臉特徵模型。確認 yolo11n.engine 可以正常反序列化後,把上面腳本中的 yolo11n.onnx 與 yolo11n.engine 改成 w600k_mbf.onnx 與 w600k_mbf.engine 再跑一次。Verification: w600k_mbf.engine 也能順利反序列化且不為 None。
如果出現建置階段的 Error Code 4:Network has dynamic or shape inputs, but no optimization profile has been defined.
因為 w600k_mbf.onnx 的輸入有動態維度(例如 batch 或 shape),用 Python TensorRT API 建置時必須在 config 加入 IOptimizationProfile,指定輸入張量的 min、opt、max 形狀,否則 build_serialized_network 會回傳 None,接著 f.write(serialized) 就會報 TypeError: a bytes-like object is required, not 'NoneType'。 加上 Optimization Profile 重新建置一般人臉特徵模型(如 w600k_mbf)輸入多為 (1, 3, 112, 112),可以先印出網路輸入名稱與維度,再用 profile.set_shape(...) 設好形狀後建置: - python3 -c "
- import tensorrt as trt
- logger = trt.Logger(trt.Logger.WARNING)
- builder = trt.Builder(logger)
- network = builder.create_network(1 << int(trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH))
- parser = trt.OnnxParser(network, logger)
- with open('w600k_mbf.onnx', 'rb') as f:
- assert parser.parse(f.read()), 'ONNX parse failed'
- inp = network.get_input(0)
- print('Input name:', inp.name, 'shape:', inp.shape)
- # 若維度有 -1(動態),將 -1 替換為實際 shape(例如 (1, 3, 112, 112))
- dims = tuple(112 if d == -1 and i >= 2 else (3 if d == -1 and i == 1 else (1 if d == -1 else d)) for i, d in enumerate(inp.shape))
- profile = builder.create_optimization_profile()
- profile.set_shape(inp.name, dims, dims, dims)
- config = builder.create_builder_config()
- config.add_optimization_profile(profile)
- if builder.platform_has_fast_fp16:
- config.set_flag(trt.BuilderFlag.FP16)
- serialized = builder.build_serialized_network(network, config)
- assert serialized is not None, 'build_serialized_network returned None'
- with open('w600k_mbf.engine', 'wb') as f:
- f.write(serialized)
- runtime = trt.Runtime(logger)
- eng = runtime.deserialize_cuda_engine(serialized)
- print('Deserialized engine:', eng)
- "
複製代碼檢查方式:順利印出 Input name、Deserialized engine: <tensorrt.tensorrt.ICudaEngine ...>,且不再出現 Error Code 4 與 NoneType 錯誤。 若 dims 推算出的形狀跟模型實際需求不同:把 dims 改成該模型正確的輸入維度(例如 (1, 3, 112, 112))再跑一次即可;yolo11n.onnx 若也有動態形狀,也可以用同一套加上 profile 的寫法建置。
|
Orin, Nano, Python, TensorRT, Orin, Nano, Python, Orin, Nano, Python, Orin, Nano, Python, Orin, Nano, Python, Orin, Nano, Python, Orin, 21490
|