新特性

集成Intel MKLDNN加速推理 #1264，#1266, #1277
C++ Serving支持HTTP 请求 #1321
C++ Serving支持gPRC 和HTTP + Proto请求 #1345
新增C++ Client SDK #1370

性能优化

C++ Serving优化Pybind传递数据方法 #1268, #1269
C++ Serving增加GPU多流、异步任务队列，删除冗余加锁 #1289
C++ Serving WebServer使用连接池和数据压缩 #1348
C++ Serving框架新增异步批量合并，支持变长LOD输入 #1366
C++ Serving stage并发执行 #1376
C++ Serving增加各阶段处理耗时日志 #1390

功能变更

重写模型保存方案和命名规则，兼容旧版本 #1354，#1358
支持更多数据类型float64，int16, float16, uint16, uint8, int8, bool , complex64 , complex128 #1338
重写GPU id设置device的逻辑 #1303
指定Fetch list返回部分推理结果 #1359
设置XPU ID #1436
服务优雅关闭 #1470
C++ Serving Client端pybind支持uint8、int8数据读写 #1378
C++ Serving Client端pybind支持uint16、int16数据读写 #1420
C++ Serving支持异步参数设置 #1483
Python Pipeline增加While OP控制循环 #1338
Python pipeline之间可使用gRPC交互 #1358
Python Pipeline 支持Proto结构Tensor数据格式交互 #1369， #1384
Python Pipeline仅获取最快的前置OP结果 #1380
Python Pipeline 支持LoD类型输入 #1472
Cube服务新增python http方式请求样例 #1399
Cube服务增加读取RecordFile工具 #1336
Cube-server和Cube-transfer上线部署优化 #1337
删除multi-lang相关代码 #1321

文档和示例变更

修改Doc目录结构，新增子目录 #1473, #1475
迁移Serving/python/examples路径到Serving/examples，重新设计目录 #1487
修改doc文件名称 #1487
新增C++ Serving Benchmark #1176
新增PaddleClas/DarkNet 加密模型部署示例 #1352
新增Model Zoo文档 #1492
新增Install文档 #1473
新增Quick_Start文档 #1473
新增Serving_Configure文档 #1495
新增C++_Serving/Inference_Protocols_CN.md #1500
新增C++_Serving/Introduction_CN.md #1497
新增C++_Serving/Performance_Tuning_CN.md #1497
新增Python_Pipeline/Performance_Tuning_CN.md #1503
更新Java SDK文档 #1357
更新Compile文档 #1502
更新Readme文档 #1473
更新Latest_Package_CN.md #1513
更新Run_On_Kubernetes_CN.md #1520

Bug修复

修复内存池使用问题 #1283
修复多线程中错误加锁问题 #1289
修复C++ Serving多模型组合场景，无法加载第二个模型问题 #1294
修复请求数据大时越界问题 #1308
修复Detection模型结果偏离问题 #1413
修复use_calib设置错误问题 #1414
修复C++ OCR示例结果不正确问题 #1415
修复并行推理出core问题 #1417

For English:

New Features

Integrate Intel MKLDNN #1264，#1266, #1277
C++ Serving supports HTTP requests #1321
C++ Serving supports gPRC and HTTP + Proto requests #1345
Added C++ Client SDK #1370

Performance optimization

C++ Serving optimizes Pybind data transfer method #1268, #1269
C++ Serving adds GPU multi-stream, asynchronous task queue, deletes redundant locks #1289
C++ Serving webserver uses connection pool and data compression #1348
C++ Serving framework adds asynchronous batch merge and supports variable length LOD input #1366
C++ Serving stage concurrent execution #1376
C++ Serving adds time-consuming log processing at each stage #1390

Function changes

Rewrite model saving methods and naming rules, compatible with the old version #1354，#1358
Support more data types float64, int16, float16, uint16, uint8, int8, bool, complex64, complex128 #1338
Rewrite the method of GPU id binding device #1303
Specify Fetch list to return partial inference results #1359
Set XPU ID #1436
Service closed gracefully #1470
C++ Serving Client pybind supports uint8, int8 data #1378
C++ Serving Client pybind supports uint16, int16 data #1420
C++ Serving supports asynchronous parameter setting #1483
Python Pipeline adds While OP control loop #1338
GRPC interaction can be used between Python pipelines #1358
Python Pipeline supports Proto structure Tensor data format interaction #1369， #1384
Python Pipeline only gets the fastest pre-OP results #1380
Python Pipeline supports LoD type input #1472
Cube service adds python http request sample #1399
Cube service adds a tool to read RecordFile #1336
Cube-server and Cube-transfer online deployment optimization #1337
Delete multi-lang related code #1321

Documentation and example changes

Modify the Doc directory structure and add subdirectories #1473, #1475
Move python/examples path to parent directory, and redesign directory #1487
Modify the doc file name #1487
Add C++ Serving Benchmark #1176
Add one PaddleClas/DarkNet encryption model example #1352
Add Model Zoo doc #1492
Add Install doc #1473
Add Quick Start doc #1473
Add Serving Configure doc #1495
Add C++_Serving/Inference_Protocols_CN.md#1500
Add C++_Serving/Introduction_CN.md#1497
Add C++_Serving/Performance_Tuning_CN.md#1497
Add Python_Pipeline/Performance_Tuning_CN.md#1503
Update Java SDK doc #1357
Update Compile doc #1502
Update Readme doc #1473
Update Latest_Package_CN.md#1513
Update Run_On_Kubernetes_CN.md#1520

Bug fix

Fix one memory pool usage problem #1283
Fix the wrong locking problem in multi-threading #1289
Fix the problem of C++ Serving multi-model combination #1294
Fix the problem of out of bounds when the requested data is large #1308
Fix the problem of inaccurate prediction results of the Detection model #1413
Fix the wrong setting of use_calib #1414
Fix the problem of incorrect C++ OCR example results #1415
Fix the core problem of parallel reasoning #1417

性能优化

C++ Serving优化Pybind传递数据方法 #1268, #1269

C++ Serving增加GPU多流、异步任务队列，删除冗余加锁 #1289

C++ Serving WebServer使用连接池和数据压缩 #1348

C++ Serving框架新增异步批量合并，支持变长LOD输入 #1366

C++ Serving stage并发执行 #1376

C++ Serving增加各阶段处理耗时日志 #1390

功能变更

重写模型保存方案和命名规则，兼容旧版本 #1354，#1358

支持更多数据类型float64，int16, float16, uint16, uint8, int8, bool , complex64 , complex128 #1338

重写GPU id设置device的逻辑 #1303

指定Fetch list返回部分推理结果 #1359

设置XPU ID #1436

服务优雅关闭 #1470

C++ Serving Client端pybind支持uint8、int8数据读写 #1378

C++ Serving Client端pybind支持uint16、int16数据读写 #1420

C++ Serving支持异步参数设置 #1483

Python Pipeline增加While OP控制循环 #1338

Python pipeline之间可使用gRPC交互 #1358

Python Pipeline 支持Proto结构Tensor数据格式交互 #1369， #1384

Python Pipeline仅获取最快的前置OP结果 #1380

Python Pipeline 支持LoD类型输入 #1472

Cube服务新增python http方式请求样例 #1399

Cube服务增加读取RecordFile工具 #1336

Cube-server和Cube-transfer上线部署优化 #1337

删除multi-lang相关代码 #1321

文档和示例变更

修改Doc目录结构，新增子目录 #1473, #1475

迁移Serving/python/examples路径到Serving/examples，重新设计目录 #1487

修改doc文件名称 #1487

新增C++ Serving Benchmark #1176

新增PaddleClas/DarkNet 加密模型部署示例 #1352

新增Model Zoo文档 #1492

新增Install文档 #1473

新增Quick_Start文档 #1473

新增Serving_Configure文档 #1495

新增C++_Serving/Inference_Protocols_CN.md #1500

新增C++_Serving/Introduction_CN.md #1497

新增C++_Serving/Performance_Tuning_CN.md #1497

新增Python_Pipeline/Performance_Tuning_CN.md #1503

更新Java SDK文档 #1357

更新Compile文档 #1502

更新Readme文档 #1473

更新Latest_Package_CN.md #1513

更新Run_On_Kubernetes_CN.md #1520

Bug修复

修复内存池使用问题 #1283

修复多线程中错误加锁问题 #1289

修复C++ Serving多模型组合场景，无法加载第二个模型问题 #1294

修复请求数据大时越界问题 #1308

修复Detection模型结果偏离问题 #1413

修复use_calib设置错误问题 #1414

修复C++ OCR示例结果不正确问题 #1415

修复并行推理出core问题 #1417

Performance optimization

C++ Serving optimizes Pybind data transfer method #1268, #1269

C++ Serving adds GPU multi-stream, asynchronous task queue, deletes redundant locks #1289

C++ Serving webserver uses connection pool and data compression #1348

C++ Serving framework adds asynchronous batch merge and supports variable length LOD input #1366

C++ Serving stage concurrent execution #1376

C++ Serving adds time-consuming log processing at each stage #1390

Function changes

Rewrite model saving methods and naming rules, compatible with the old version #1354，#1358

Support more data types float64, int16, float16, uint16, uint8, int8, bool, complex64, complex128 #1338

Rewrite the method of GPU id binding device #1303

Specify Fetch list to return partial inference results #1359

Set XPU ID #1436

Service closed gracefully #1470

C++ Serving Client pybind supports uint8, int8 data #1378

C++ Serving Client pybind supports uint16, int16 data #1420

C++ Serving supports asynchronous parameter setting #1483

Python Pipeline adds While OP control loop #1338

GRPC interaction can be used between Python pipelines #1358

Python Pipeline supports Proto structure Tensor data format interaction #1369， #1384

Python Pipeline only gets the fastest pre-OP results #1380

Python Pipeline supports LoD type input #1472

Cube service adds python http request sample #1399

Cube service adds a tool to read RecordFile #1336

Cube-server and Cube-transfer online deployment optimization #1337

Delete multi-lang related code #1321

Documentation and example changes

Modify the Doc directory structure and add subdirectories #1473, #1475

Move python/examples path to parent directory, and redesign directory #1487

Modify the doc file name #1487

Add C++ Serving Benchmark #1176

Add one PaddleClas/DarkNet encryption model example #1352

Add Model Zoo doc #1492

Add Install doc #1473

Add Quick Start doc #1473

Add Serving Configure doc #1495

Add C++_Serving/Inference_Protocols_CN.md#1500

Add C++_Serving/Introduction_CN.md#1497

Add C++_Serving/Performance_Tuning_CN.md#1497

Add Python_Pipeline/Performance_Tuning_CN.md#1503

Update Java SDK doc #1357

Update Compile doc #1502

Update Readme doc #1473

Update Latest_Package_CN.md#1513

Update Run_On_Kubernetes_CN.md#1520

Bug fix

Fix one memory pool usage problem #1283

Fix the wrong locking problem in multi-threading #1289

Fix the problem of C++ Serving multi-model combination #1294

Fix the problem of out of bounds when the requested data is large #1308

Fix the problem of inaccurate prediction results of the Detection model #1413

Fix the wrong setting of use_calib #1414

Fix the problem of incorrect C++ OCR example results #1415

Fix the core problem of parallel reasoning #1417

Serving

Release v0.7.0

新特性

性能优化

功能变更

文档和示例变更

Bug修复

For English:

New Features

Performance optimization

Function changes

Documentation and example changes

Bug fix

More C++ Projects

tensorflow

electron

llama.cpp

godot

Release v0.7.0

新特性

性能优化

功能变更

文档和示例变更

Bug修复

For English:

New Features

Performance optimization

Function changes

Documentation and example changes

Bug fix

More C++ Projects

tensorflow

electron

llama.cpp

godot