将PyTorch模型部署到C++上（Ubuntu）
1. 为什么要这么做？
PyTorch作为⼀个开源的Python机器学习库正受到越来越⼴泛的关注和应⽤。然⽽，作为⼀门语⾔，Python并不是在任何场景下都适⽤的。在⽣产以及部署到⾃动驾驶车辆上等场景中，C++常常是更好的选择。因此需要将PyTorch模型部署到C++上。以下是实现的步骤。
2. 将PyTorch模型转化成Torch Script
Torch Script的作⽤是⽤TorchScript编写的任何代码都可以从Python进程中保存并加载到没有Python依赖关系的进程中。有两种⽅法可以实现这⼀步骤。
（1）tracing
在这种机制中，通过使⽤⽰例输⼊对模型进⾏分析，并记录通过模型的这些输⼊流，来捕获模型的结构。这适⽤于控制流使⽤有限的模型。具体做法为，将⽰例输⼊到模型中，并将其作为⼀个例⼦输⼊到ace函数中。这将产⽣⼀个torch.jit.ScriptModule对象，并在模块的forward⽅法中嵌⼊模型跟踪，具体代码如下：
import torch
什么的张望
import torchvision
# An instance of your model.
model = snet18()
# An example input you would normally provide to your model's forward() method.
example = torch.rand(1,3,224,224)
# U ace to generate a torch.jit.ScriptModule via tracing.
traced_script_module = ace(model, example)
（2）使⽤注释来转化
在某些情况下，⽐如模型采⽤了特定形式的控制流，可能希望直接⽤Torch Script编写模型，并相应地对模型进⾏注释。⽐如以下这个模型：
import torch
class Module):
def__init__(lf, N, M):
super(MyModule, lf).__init__()
lf.weight = Parameter(torch.rand(N, M))
def forward(lf,input):
if input.sum()>0:
output = lf.weight.mv(input)
el:
output = lf.weight +input
return output
舞蹈由于该模块的forward⽅法使⽤依赖于输⼊的控制流，因此不适合tracing。相反，我们可以将其转换为ScriptModule。为了将模块转换为ScriptModule，需要使⽤torch.jit.script编译模块，如下所⽰:
class Module):
def__init__(lf, N, M):
super(MyModule, lf).__init__()
lf.weight = Parameter(torch.rand(N, M))
def forward(lf,input):
if input.sum()>0:
output = lf.weight.mv(input)
el:
output = lf.weight +input
return output
my_module = MyModule(10,20)
sm = torch.jit.script(my_module)
3. 将Script Module输出为⽂件
（1） tracing⽅法：
traced_script_module.save("traced_resnet_model.pt")
（2）script⽅法:
sm.save("my_module_model.pt")
到这步我们就已经结束了在Python上的操作，要开始转战C++了。
4. 在C++中载⼊Script Module
要⽤c++加载序列化的PyTorch模型，应⽤程序必须依赖于LibTorch。LibTorch发⾏版包含了⼀组共享库、头⽂件和CMake配置⽂件。虽然CMake不是必须的，但它是推荐的⽅法，并且在将来会得到很好的⽀持。在本教程中，我们将使⽤CMake和LibTorch构建⼀个c++应⽤程序，它只是⽤于加载和执⾏⼀个序列化的PyTorch模型。
根据⾃⼰的需求和电脑的配置下载，本⽂对应的版本是1.8.1CPU版，其他版本的也许本⽂的⽅法不适⽤，请注意！
⾸先，需要编写代码来载⼊模块。
#include<torch/script.h>// One-stop header.
#include<iostream>
#include<memory>
int main(int argc,const char* argv[]){
if(argc !=2){
std::cerr <<"usage: example-app <path-to-exported-script-module>\n";
return-1;
}
torch::jit::script::Module module;
try{
/
/ Derialize the ScriptModule from a file using torch::jit::load().
设置的英语module = torch::jit::load(argv[1]);
}
catch(const c10::Error& e){
std::cerr <<"error loading the model\n";
return-1;
}
std::cout <<"ok\n";
}
接着，创建⼀个， 内容如下：
cmake_minimum_required(VERSION 3.0 FATAL_ERROR)
project(custom_ops)//写项⽬名
//t(Torch_DIR /home/nio/libtorch-1.8.1/libtorch/share/cmake/Torch)
find_package(Torch REQUIRED)
add_executable(example-app example-app.cpp)//⽂件夹名，⽂件名
target_link_libraries(example-app "${TORCH_LIBRARIES}")//⽂件夹名
t_property(TARGET example-app PROPERTY CXX_STANDARD 14)//⽂件夹名
这⾥需要注意，按官⽹教程，是不⽤t⾏的，在很多别⼈的教程⾥也不⽤，但我在实验的时候，没有t⾏设置Torch地址，后续的操作会报错，提⽰找不到Torch，所以看情况⾃⼰加。
两个⽂件创建好之后，放在⼀个⽂件夹下，如：
example-app/
<
example-app.cpp
在该⽂件夹下，打开终端，输⼊以下指令：
mkdir build
手机ppt怎么做
cd build
cmake -DCMAKE_PREFIX_PATH=/path/to/libtorch ..//背后两个⼩点别忘了
⼀开始我没有设置Torch地址，得到的结果为：
-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- Check for working C compiler:/usr/bin/cc
-- Check for working C compiler:/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-
- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler:/usr/bin/c++
-- Check for working CXX compiler:/usr/bin/c++-- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error :4(find_package):
By not providing "ake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "Torch", but
CMake did not find one.
Could not find a package configuration file provided by "Torch" with any of
the following names:
建军节的来历Add the installation prefix of "Torch" to CMAKE_PREFIX_PATH or t
"Torch_DIR" to a directory containing one of the above files. If "Torch"
provides a parate development package or SDK, be sure it has been
难忘的一节课作文installed.
-- Configuring incomplete, errors occurred!
See also "/home/nio/C++FILE/code/vectornet/build/CMakeFiles/CMakeOutput.log".
设置了之后：
-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- Check for working C compiler:/usr/bin/cc
在爱的路上走呀走
-- Check for working C compiler:/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler:/usr/bin/c++
野芋-- Check for working CXX compiler:/usr/bin/c++-- works
-
- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create -not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads -not found
-- Looking for pthread_create in pthread
-
- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found Torch:/home/nio/libtorch-1.8.1/libtorch/lib/libtorch.so
-- Configuring done
-- Generating done
-- Build files have been written to:/home/nio/C++FILE/code/vectornet/build
成功之后，在build⽂件夹下⽤make编译：
(ba) nio@LT5CG052BHT2:~/C++FILE/code/vectornet/build$ make
Scanning dependencies of target vectornet
[50%] Building CXX object CMakeFiles/vectornet.dir/pytoc.cpp.o
[100%] Linking CXX executable vectornet
[100%] Built target vectornet
运⾏pt⽂件
(ba) nio@LT5CG052BHT2:~/C++FILE/code/vectornet/build$ ./vectornet ../traced_vectornet_model.pt ok
最后，执⾏输⼊并验证输出。在之前的cpp⽂件中加⼊：
// Create a vector of inputs.
std::vector<torch::jit::IValue> inputs;
inputs.push_back(torch::ones({1,3,224,224}));
// Execute the model and turn its output into a tensor.
at::Tensor output = module.forward(inputs).toTensor();
std::cout << output.slice(/*dim=*/1,/*start=*/0,/*end=*/5)<<'\n';
再make和执⾏pt⽂件即可得到结果。
