翼度科技»论坛 编程开发 .net 查看内容

【OpenVINO™】在 C# 中使用OpenVINO™ 部署PP-YOLOE实现物体检测

5

主题

5

帖子

15

积分

新手上路

Rank: 1

积分
15
 前言OpenVINO™ C# API 是一个 OpenVINO™ 的 .Net wrapper,应用最新的 OpenVINO™ 库开发,通过 OpenVINO™ C API 实现 .Net 对 OpenVINO™ Runtime 调用,使用习惯与 OpenVINO™ C++ API 一致。OpenVINO™ C# API 由于是基于 OpenVINO™ 开发,所支持的平台与 OpenVINO™ 完全一致,具体信息可以参考 OpenVINO™。通过使用 OpenVINO™ C# API,可以在 .NET、.NET Framework等框架下使用 C# 语言实现深度学习模型在指定平台推理加速。
OpenVINO™ C# API 项目链接为:
  1. https://github.com/guojin-yan/OpenVINO-CSharp-API.git
复制代码
项目源码链接为:
  1. https://github.com/guojin-yan/OpenVINO-CSharp-API-Samples.git
复制代码
1. 简介

    PP-YOLOE是基于PP-YOLOv2的优秀单级无锚模型,超越了各种流行的YOLO模型。PP-YOLOE有一系列型号,命名为s/m/l/x,通过宽度乘数和深度乘数进行配置。PP-YOLOE避免使用特殊的运算符,如可变形卷积或矩阵NMS,以便友好地部署在各种硬件上。 在本文中,我们将使用OpenVINO™ C# API 部署 PP-YOLOE实现物体检测。
2. 项目环境与依赖

     该项目中所需依赖已经支持通过NuGet Package进行安装,在该项目中,需要安装以下NuGet Package:

  • OpenVINO C# API NuGet Package:
  1. OpenVINO.CSharp.API
  2. OpenVINO.runtime.win
  3. OpenVINO.CSharp.API.Extensions
  4. OpenVINO.CSharp.API.Extensions.OpenCvSharp
复制代码

  • OpenCvSharp NuGet Package:
  1. OpenCvSharp4
  2. OpenCvSharp4.Extensions
  3. OpenCvSharp4.runtime.win
复制代码
3. 项目输出

     项目使用的是控制台输出,运行后输出如下所示:
  1. <00:00:00> Sending http request to https://github.com/guojin-yan/OpenVINO-CSharp-API-Samples/releases/download/Model/ppyoloe_plus_crn_l_80e_coco.tar.
  2. <00:00:02> Http Response Accquired.
  3. <00:00:02> Total download length is 199.68 Mb.
  4. <00:00:02> Download Started.
  5. <00:00:02> File created.
  6. <00:02:03> Downloading: [■■■■■■■■■■] 100% <00:02:03 1.81 Mb/s> 199.68 Mb/199.68 Mb downloaded.
  7. <00:02:03> File Downloaded, saved in E:\GitSpace\OpenVINO-CSharp-API-Samples\model_samples\ppyoloe\ppyoloe_opencvsharp\bin\Release\net6.0\model\ppyoloe_plus_crn_l_80e_coco.tar.
  8. <00:00:00> Sending http request to https://github.com/guojin-yan/OpenVINO-CSharp-API-Samples/releases/download/Image/test_det_02.jpg.
  9. <00:00:02> Http Response Accquired.
  10. <00:00:02> Total download length is 0.16 Mb.
  11. <00:00:02> Download Started.
  12. <00:00:02> File created.
  13. <00:00:02> Downloading: [■■■■■■■■■■] 100% <00:00:02 0.06 Mb/s> 0.16 Mb/0.16 Mb downloaded.
  14. <00:00:02> File Downloaded, saved in E:\GitSpace\OpenVINO-CSharp-API-Samples\model_samples\ppyoloe\ppyoloe_opencvsharp\bin\Release\net6.0\model\test_image.jpg.
  15. [ INFO ] Inference device: CPU
  16. [ INFO ] Start RT-DETR model inference.
  17. [ INFO ] 1. Initialize OpenVINO Runtime Core success, time spend: 4.5204ms.
  18. [ INFO ] 2. Read inference model success, time spend: 228.4451ms.
  19. [ INFO ] Inference Model
  20. [ INFO ]   Model name: Model0
  21. [ INFO ]   Input:
  22. [ INFO ]      name: scale_factor
  23. [ INFO ]      type: float
  24. [ INFO ]      shape: Shape : {?,2}
  25. [ INFO ]      name: image
  26. [ INFO ]      type: float
  27. [ INFO ]      shape: Shape : {?,3,640,640}
  28. [ INFO ]   Output:
  29. [ INFO ]      name: multiclass_nms3_0.tmp_0
  30. [ INFO ]      type: float
  31. [ INFO ]      shape: Shape : {?,6}
  32. [ INFO ]      name: multiclass_nms3_0.tmp_2
  33. [ INFO ]      type: int32_t
  34. [ INFO ]      shape: Shape : {?}
  35. [ INFO ] 3. Loading a model to the device success, time spend:501.0716ms.
  36. [ INFO ] 4. Create an infer request success, time spend:0.2663ms.
  37. [ INFO ] 5. Process input images success, time spend:30.1001ms.
  38. [ INFO ] 6. Set up input data success, time spend:2.3631ms.
  39. [ INFO ] 7. Do inference synchronously success, time spend:286.1085ms.
  40. [ INFO ] 8. Get infer result data success, time spend:0.5189ms.
  41. [ INFO ] 9. Process reault  success, time spend:0.4425ms.
  42. [ INFO ] The result save to E:\GitSpace\OpenVINO-CSharp-API-Samples\model_samples\ppyoloe\ppyoloe_opencvsharp\bin\Release\net6.0\model\test_image_result.jpg
复制代码
     图像预测结果如下图所示:

4. 代码展示

     以下为嘛中所使用的命名空间代码:
  1. using OpenCvSharp.Dnn;
  2. using OpenCvSharp;
  3. using OpenVinoSharp;
  4. using OpenVinoSharp.Extensions;
  5. using OpenVinoSharp.Extensions.utility;
  6. using System.Runtime.InteropServices;
  7. using OpenVinoSharp.preprocess;
  8. using OpenVinoSharp.Extensions.model;
  9. using OpenVinoSharp.Extensions.result;
  10. using OpenVinoSharp.Extensions.process;
  11. namespace ppyoloe_opencvsharp
  12. {
  13.     internal class Program
  14.     {  
  15.             ....
  16.     }
  17. }
复制代码
     下面为定义的模型预测代码:

  • 一般预测流程:
  1. static void ppyoloe_det(string model_path, string image_path, string device)
  2. {
  3.     // -------- Step 1. Initialize OpenVINO Runtime Core --------
  4.     Core core = new Core();
  5.     // -------- Step 2. Read inference model --------
  6.     Model model = core.read_model(model_path);
  7.     OvExtensions.printf_model_info(model);
  8.     // -------- Step 3. Loading a model to the device --------
  9.     CompiledModel compiled_model = core.compile_model(model, device);
  10.     // -------- Step 4. Create an infer request --------
  11.     InferRequest infer_request = compiled_model.create_infer_request();
  12.     // -------- Step 5. Process input images --------
  13.     Mat image = new Mat(image_path); // Read image by opencvsharp
  14.     float[] factor = new float[] { 640.0f / (float)image.Rows, 640.0f / (float)image.Cols };
  15.     float[] im_shape = new float[] { 640.0f, 640.0f };
  16.     Mat input_mat = CvDnn.BlobFromImage(image, 1.0 / 255.0, new OpenCvSharp.Size(640, 640), 0, true, false);
  17.     float[] input_data = new float[640 * 640 * 3];
  18.     Marshal.Copy(input_mat.Ptr(0), input_data, 0, input_data.Length);
  19.     // -------- Step 6. Set up input data --------
  20.     Tensor input_tensor_data = infer_request.get_tensor("image");
  21.     input_tensor_data.set_shape(new Shape(1, 3, 640, 640));
  22.     input_tensor_data.set_data<float>(input_data);
  23.     Tensor input_tensor_factor = infer_request.get_tensor("scale_factor");
  24.     input_tensor_factor.set_shape(new Shape(1, 2));
  25.     input_tensor_factor.set_data<float>(factor);
  26.     // -------- Step 7. Do inference synchronously --------
  27.     infer_request.infer();
  28.     // -------- Step 8. Get infer result data --------
  29.     Tensor output_tensor = infer_request.get_output_tensor(0);
  30.     int output_length = (int)output_tensor.get_size();
  31.     float[] output_data = output_tensor.get_data<float>(output_length);
  32.     // -------- Step 9. Process reault  --------
  33.     List<Rect> position_boxes = new List<Rect>();
  34.     List<int> class_ids = new List<int>();
  35.     List<float> confidences = new List<float>();
  36.     for (int i = 0; i < 300; ++i)
  37.     {
  38.         if (output_data[6 * i + 1] > 0.5)
  39.         {
  40.             class_ids.Add((int)output_data[6 * i]);
  41.             confidences.Add(output_data[6 * i + 1]);
  42.             position_boxes.Add(new Rect((int)output_data[6 * i + 2], (int)output_data[6 * i + 3],
  43.                 (int)(output_data[6 * i + 4] - output_data[6 * i + 2]),
  44.                 (int)(output_data[6 * i + 5] - output_data[6 * i + 3])));
  45.         }
  46.     }
  47.     for (int index = 0; index < class_ids.Count; index++)
  48.     {
  49.         Cv2.Rectangle(image, position_boxes[index], new Scalar(0, 0, 255), 2, LineTypes.Link8);
  50.         Cv2.Rectangle(image, new OpenCvSharp.Point(position_boxes[index].TopLeft.X, position_boxes[index].TopLeft.Y + 30),
  51.             new OpenCvSharp.Point(position_boxes[index].BottomRight.X, position_boxes[index].TopLeft.Y), new Scalar(0, 255, 255), -1);
  52.         Cv2.PutText(image, class_ids[index] + "-" + confidences[index].ToString("0.00"),
  53.             new OpenCvSharp.Point(position_boxes[index].X, position_boxes[index].Y + 25),
  54.             HersheyFonts.HersheySimplex, 0.8, new Scalar(0, 0, 0), 2);
  55.     }
  56.     string output_path = Path.Combine(Path.GetDirectoryName(Path.GetFullPath(image_path)),
  57.         Path.GetFileNameWithoutExtension(image_path) + "_result.jpg");
  58.     Cv2.ImWrite(output_path, image);
  59.     Slog.INFO("The result save to " + output_path);
  60.     Cv2.ImShow("Result", image);
  61.     Cv2.WaitKey(0);
  62. }
复制代码

  • 编译预处理步骤到模型方式推理模型:
  1. static void ppyoloe_det_with_process(string model_path, string image_path, string device)
  2. {
  3.     // -------- Step 1. Initialize OpenVINO Runtime Core --------
  4.     Core core = new Core();
  5.     // -------- Step 2. Read inference model --------
  6.     Model model = core.read_model(model_path);
  7.     OvExtensions.printf_model_info(model);
  8.     PrePostProcessor processor = new PrePostProcessor(model);
  9.     Tensor input_tensor_pro = new Tensor(new OvType(ElementType.U8), new Shape(1, 640, 640, 3));
  10.     InputInfo input_info = processor.input("image");
  11.     InputTensorInfo input_tensor_info = input_info.tensor();
  12.     input_tensor_info.set_from(input_tensor_pro).set_layout(new Layout("NHWC")).set_color_format(ColorFormat.BGR);
  13.     PreProcessSteps process_steps = input_info.preprocess();
  14.     process_steps.convert_color(ColorFormat.RGB).resize(ResizeAlgorithm.RESIZE_LINEAR)
  15.         .convert_element_type(new OvType(ElementType.F32)).scale(255.0f).convert_layout(new Layout("NCHW"));
  16.     Model new_model = processor.build();
  17.     // -------- Step 3. Loading a model to the device --------
  18.     CompiledModel compiled_model = core.compile_model(new_model, device);
  19.     // -------- Step 4. Create an infer request --------
  20.     InferRequest infer_request = compiled_model.create_infer_request();
  21.     // -------- Step 5. Process input images --------
  22.     Mat image = new Mat(image_path); // Read image by opencvsharp
  23.     Mat input_image = new Mat();
  24.     Cv2.Resize(image, input_image, new OpenCvSharp.Size(640, 640));
  25.     float[] factor = new float[] { 640.0f / (float)image.Rows, 640.0f / (float)image.Cols };
  26.     float[] im_shape = new float[] { 640.0f, 640.0f };
  27.     // -------- Step 6. Set up input data --------
  28.     Tensor input_tensor_data = infer_request.get_tensor("image");
  29.     byte[] input_data = new byte[3 * 640 * 640];
  30.     Marshal.Copy(input_image.Ptr(0), input_data, 0, input_data.Length);
  31.     IntPtr destination = input_tensor_data.data();
  32.     Marshal.Copy(input_data, 0, destination, input_data.Length);
  33.     Tensor input_tensor_factor = infer_request.get_tensor("scale_factor");
  34.     input_tensor_factor.set_shape(new Shape(1, 2));
  35.     input_tensor_factor.set_data<float>(factor);
  36.     // -------- Step 7. Do inference synchronously --------
  37.     infer_request.infer();
  38.     // -------- Step 8. Get infer result data --------
  39.     Tensor output_tensor = infer_request.get_output_tensor(0);
  40.     int output_length = (int)output_tensor.get_size();
  41.     float[] output_data = output_tensor.get_data<float>(output_length);
  42.     // -------- Step 9. Process reault  --------
  43.     List<Rect> position_boxes = new List<Rect>();
  44.     List<int> class_ids = new List<int>();
  45.     List<float> confidences = new List<float>();
  46.     for (int i = 0; i < 300; ++i)
  47.     {
  48.         if (output_data[6 * i + 1] > 0.5)
  49.         {
  50.             class_ids.Add((int)output_data[6 * i]);
  51.             confidences.Add(output_data[6 * i + 1]);
  52.             position_boxes.Add(new Rect((int)output_data[6 * i + 2], (int)output_data[6 * i + 3],
  53.                 (int)(output_data[6 * i + 4] - output_data[6 * i + 2]),
  54.                 (int)(output_data[6 * i + 5] - output_data[6 * i + 3])));
  55.         }
  56.     }
  57.     for (int index = 0; index < class_ids.Count; index++)
  58.     {
  59.         Cv2.Rectangle(image, position_boxes[index], new Scalar(0, 0, 255), 2, LineTypes.Link8);
  60.         Cv2.Rectangle(image, new OpenCvSharp.Point(position_boxes[index].TopLeft.X, position_boxes[index].TopLeft.Y + 30),
  61.             new OpenCvSharp.Point(position_boxes[index].BottomRight.X, position_boxes[index].TopLeft.Y), new Scalar(0, 255, 255), -1);
  62.         Cv2.PutText(image, class_ids[index] + "-" + confidences[index].ToString("0.00"),
  63.             new OpenCvSharp.Point(position_boxes[index].X, position_boxes[index].Y + 25),
  64.             HersheyFonts.HersheySimplex, 0.8, new Scalar(0, 0, 0), 2);
  65.     }
  66.     string output_path = Path.Combine(Path.GetDirectoryName(Path.GetFullPath(image_path)),
  67.         Path.GetFileNameWithoutExtension(image_path) + "_result.jpg");
  68.     Cv2.ImWrite(output_path, image);
  69.     Slog.INFO("The result save to " + output_path);
  70.     Cv2.ImShow("Result", image);
  71.     Cv2.WaitKey(0);
  72. }
复制代码

  • 使用封装的方法:
  1. static void ppyoloe_det_using_extensions(string model_path, string image_path, string device)
  2. {
  3.     PPYoloeConfig config = new PPYoloeConfig();
  4.     config.set_model(model_path);
  5.     PPYoloeDet det = new PPYoloeDet(config);
  6.     Mat image = Cv2.ImRead(image_path);
  7.     DetResult result = det.predict(image);
  8.     Mat result_im = Visualize.draw_det_result(result, image);
  9.     Cv2.ImShow("Result", result_im);
  10.     Cv2.WaitKey(0);
  11. }
复制代码
     下面为程序运行的主函数代码,该代码会下载转换好的预测模型,并调用预测方法进行预测:
  1. static void Main(string[] args)
  2. {
  3.     string model_path = "";
  4.     string image_path = "";
  5.     string device = "CPU";
  6.     if (args.Length == 0)
  7.     {
  8.         if (!Directory.Exists("./model"))
  9.         {
  10.             Directory.CreateDirectory("./model");
  11.         }
  12.         if (!File.Exists("./model/model.pdiparams")
  13.             && !File.Exists("./model/model.pdmodel"))
  14.         {
  15.             if (!File.Exists("./model/ppyoloe_plus_crn_l_80e_coco.tar"))
  16.             {
  17.                 _ = Download.download_file_async("https://github.com/guojin-yan/OpenVINO-CSharp-API-Samples/releases/download/Model/ppyoloe_plus_crn_l_80e_coco.tar",
  18.                     "./model/ppyoloe_plus_crn_l_80e_coco.tar").Result;
  19.             }
  20.             Download.unzip("./model/ppyoloe_plus_crn_l_80e_coco.tar", "./model/");
  21.         }
  22.         if (!File.Exists("./model/test_image.jpg"))
  23.         {
  24.             _ = Download.download_file_async("https://github.com/guojin-yan/OpenVINO-CSharp-API-Samples/releases/download/Image/test_det_02.jpg",
  25.                 "./model/test_image.jpg").Result;
  26.         }
  27.         model_path = "./model/model.pdmodel";
  28.         image_path = "./model/test_image.jpg";
  29.     }
  30.     else if (args.Length >= 2)
  31.     {
  32.         model_path = args[0];
  33.         image_path = args[1];
  34.         device = args[2];
  35.     }
  36.     else
  37.     {
  38.         Console.WriteLine("Please enter the correct command parameters, for example:");
  39.         Console.WriteLine("> 1. dotnet run");
  40.         Console.WriteLine("> 2. dotnet run <model path> <image path> <device name>");
  41.     }
  42.     // -------- Get OpenVINO runtime version --------
  43.     OpenVinoSharp.Version version = Ov.get_openvino_version();
  44.     Slog.INFO("---- OpenVINO INFO----");
  45.     Slog.INFO("Description : " + version.description);
  46.     Slog.INFO("Build number: " + version.buildNumber);
  47.     Slog.INFO("Predict model files: " + model_path);
  48.     Slog.INFO("Predict image  files: " + image_path);
  49.     Slog.INFO("Inference device: " + device);
  50.     Slog.INFO("Start RT-DETR model inference.");
  51.     ppyoloe_det(model_path, image_path, device);
  52.     //ppyoloe_det_with_process(model_path, image_path, device);
  53.     //ppyoloe_det_using_extensions(model_path, image_path, device);
  54. }
复制代码
5. 总结

     在该项目中,我们结合之前开发的 OpenVINO™ C# API 项目部署PP-YOLOE模型,实现物体检测。

  • 项目完整代码链接为:
  1. https://github.com/guojin-yan/OpenVINO-CSharp-API-Samples/blob/master/model_samples/ppyoloe/ppyoloe_opencvsharp/Program.cs
复制代码

  • 为了方便EmguCV用户使用需求,同时开发了EmguCV版本,项目链接为:
  1. https://github.com/guojin-yan/OpenVINO-CSharp-API-Samples/blob/master/model_samples/ppyoloe/ppyoloe_emgucv/Program.cs
复制代码
最后如果各位开发者在使用中有任何问题,欢迎大家与我联系。

来源:https://www.cnblogs.com/guojin-blogs/p/18188742
免责声明:由于采集信息均来自互联网,如果侵犯了您的权益,请联系我们【E-Mail:cb@itdo.tech】 我们会及时删除侵权内容,谢谢合作!

本帖子中包含更多资源

您需要 登录 才可以下载或查看,没有账号?立即注册

x

举报 回复 使用道具