packages/libkperf
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

udpate version to 1.1-2

Browse Source
This commit is contained in:
eho 2024-11-19 10:52:47 +08:00
parent 096b48f2f7
commit 5dacbbf2cd
23 changed files with 2060 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

26
0001-update-python-modules-_libkperf-Symbol.py.patch Normal file
View File

@ -0,0 +1,26 @@
From 3742958aa92ed147e56ea38226ca4ee683f618f0 Mon Sep 17 00:00:00 2001
From: Caohongtao <caohongtao_yewu@cmss.chinamobile.com>
Date: Thu, 11 Jul 2024 01:29:32 +0000
Subject: [PATCH 01/20] update python/modules/_libkperf/Symbol.py.
Signed-off-by: Caohongtao <caohongtao_yewu@cmss.chinamobile.com>
---
python/modules/_libkperf/Symbol.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/python/modules/_libkperf/Symbol.py b/python/modules/_libkperf/Symbol.py
index 6db0f2c..9134c96 100644
--- a/python/modules/_libkperf/Symbol.py
+++ b/python/modules/_libkperf/Symbol.py
@@ -224,7 +224,7 @@ class Stack:
self.__c_stack = CtypesStack(
symbol=symbol.c_sym if symbol else None,
next=next.c_stack if next else None,
- prev=prev.c_stack if next else None,
+ prev=prev.c_stack if prev else None,
count=count
)
--
2.43.0

769
0002-update-docs.patch Normal file
View File

@ -0,0 +1,769 @@
From d22ccd84355100e95b4aafa6ed9753236533dfec Mon Sep 17 00:00:00 2001
From: ganlixiong <ganli2012@gmail.com>
Date: Tue, 24 Sep 2024 15:18:52 +0800
Subject: [PATCH 02/20] update docs.
Update README and add Details.md for detailed usage of libkperf.
---
README.en.md | 178 +++-----------------------------
README.md | 179 ++++----------------------------
docs/Details.md | 264 ++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 298 insertions(+), 323 deletions(-)
create mode 100644 docs/Details.md
diff --git a/README.en.md b/README.en.md
index ff85b3a..40fbc26 100644
--- a/README.en.md
+++ b/README.en.md
@@ -1,58 +1,36 @@
# libkperf
#### Description
-Implement a low overhead pmu collection library, providing abstract interfaces for counting, sampling and symbol resolve.
+libkperf is a lightweight performance collection library on linux, that enables developers to perform performance collection in an API fashion. libkperf provides performance data in memory and allows develops to process data directly, reducing overhead of writing and reading perf.data.
-#### Software Architecture
-This repo includes two modules: pmu collections and symbol resolve.
-
-Pmu collection module is developed on syscall perf_event_open to enable kernel pmu counting and sampling, using -thread or per-core mode depending on user input.
-Pmu data packets are read from ring buffer and are parsed to different structure for counting, sampling and spe sampling.
-For sampling, symbols are resolved according to ips or pc from data packet. Each symbol contains symbol name, address, source file path and line number if possible.
-
-Symbol resolve module is developed on elfin-parser, a library for parsing elf and dwarf. The module manages all symbol data in well-designed data structures for fast query.
-
-#### Download
-
-Git method:
+#### Build
+To build a library with C API:
```shell
git clone --recurse-submodules https://gitee.com/openeuler/libkperf.git
-```
-If you only use
-```shell
-git clone https://gitee.com/openeuler/libkperf.git
-```
-Please continue with the execution
-```shell
cd libkperf
-git submodule update --init --recursive
+bash build.sh install_path=/path/to/install
```
-When unable to use git:
-
-1. Download the libkperf compressed file and decompress it.
-
-2. Go to the third_party directory of libkperf on Gitee, click on the link(as shown in the example elfin-parser@13e57e2 Click on the submit ID after @), to redirect and download the compressed package of the third-party library. After decompression, place it in the third_party directory of the local libkperf project. (elfin Parser is necessary for installation)
-
-#### Installation
-Run bash script:
-
-```sh
-bash build.sh install_path=/home/libkperf
+To build a library with debug version:
+```shell
+bash build.sh install_path=/path/to/install buildType=debug
```
-As mentioned above, the header and library will be installed in the/home/libkperf output directory, and installPath is an optional parameter. If not set, it will be installed in the output directory under libkperf by default.
-If you want to add additional python library support, you can install it as follows:
+To build a python package:
```shell
-bash build.sh python=true
+bash build.sh install_path=/path/to/install python=true
```
-If you need to uninstall the python library after installation, you can run the following command:
+To uninstall python package:
```shell
-python -m pip uninstall -y libkperf
+python3 -m pip uninstall -y libkperf
```
+#### Documents
+Refer to ```docs``` directory for detailed docs:
+- [Detailed usage](./docs/Details.md)
+
#### Instructions
All pmu functions are accomplished by the following interfaces:
* PmuOpen
@@ -60,14 +38,12 @@ All pmu functions are accomplished by the following interfaces:
* PmuEnable
Start collection.
* PmuRead
- Read pmu data and a list is returned.
+ Read collection data.
* PmuDisable
Stop collection.
* PmuClose
Close pmu device.
-Refer to pmu.h for details of interfaces.
-
Here are some examples:
* Get pmu count for a process.
```C
@@ -147,89 +123,16 @@ PmuDataFree(data);
// Like fd, call PmuClose if pd will not be used.
PmuClose(pd);
```
-* config event group function
-```C
-int pidList[1];
-pidList[0] = pid;
-unsigned numEvt = 16;
-char *evtList[numEvt] = {"r3", "r4", "r1", "r14", "r10", "r12", "r5", "r25",
- "r2", "r26", "r2d", "r17", "r8", "r22", "r24", "r11"};
-// initialize event list, the same group id is the same event group.
-// if event grouping is not required, leave the event group_id list blank.
-// In addition, if group_id is -1, the event group function is forcibly disabled.
-PmuAttr attr = {0};
-attr.evtList = evtList;
-attr.numEvt = numEvt;
-attr.pidList = pidList;
-attr.numPid = 1;
-struct EvttAttr groupId[numEvt] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 13, 13, 13};
-attr.evtAttr = groupId;
-// Call PmuOpen and pmu descriptor <pd> is return.
-// <pd> is an identity for current task.
-int pd = PmuOpen(COUNTING, &attr);
-// Start collection.
-PmuEnable(pd);
-// Collect for one second.
-sleep(1);
-// Stop collection.
-PmuDisable(pd);
-PmuData *data = NULL;
-// Read pmu data. You can also read data before PmuDisable.
-int len = PmuRead(pd, &data);
-for (int i = 0; i < len; ++i) {
- ...
-}
-// To free PmuData, call PmuDataFree.
-PmuDataFree(data);
-// Like fd, call PmuClose if pd will not be used.
-PmuClose(pd);
-```
-
-- Counting supports fork thread.
-```C
-int pidList[1];
-pidList[0] = pid;
-unsigned numEvt = 1;
-char *evtList[numEvt] = {"cycles"};
-PmuAttr attr = {0};
-attr.evtList = evtList;
-attr.numEvt = numEvt;
-attr.pidList = pidList;
-attr.numPid = 1;
-// In count mode, enable it you can get the new child thread count, default is disabled.
-attr.includeNewFork = 1;
-// Call PmuOpen and pmu descriptor <pd> is return.
-// <pd> is an identity for current task.
-int pd = PmuOpen(COUNTING, &attr);
-// Start collection.
-PmuEnable(pd);
-// Collect for two second.
-sleep(2);
-// Stop collection.
-PmuDisable(pd);
-PmuData *data = NULL;
-// Read pmu data. You can also read data before PmuDisable.
-int len = PmuRead(pd, &data);
-for (int i = 0; i < len; ++i) {
- ...
-}
-// To free PmuData, call PmuDataFree.
-PmuDataFree(data);
-// Like fd, call PmuClose if pd will not be used.
-PmuClose(pd);
-```
Python examples:
```python
import time
from collections import defaultdict
-import subprocess
import kperf
def Counting():
evtList = ["r11", "cycles"]
- evtAttr = [2, 2] # Event group id list corresponding to the event list. the same group id is the same event group.
pmu_attr = kperf.PmuAttr(evtList=evtList)
pd = kperf.open(kperf.PmuTaskType.COUNTING, pmu_attr)
if pd == -1:
@@ -250,53 +153,4 @@ def Counting():
kperf.disable(pd)
kperf.close(pd)
-
-
-def NewFork():
- # test_new_fork demo in test_perf, you can find test_new_fork.cpp
- p=subprocess.Popen(['test_new_fork']);
- pidList=[p.pid]
- evtList=["cycles"]
- pmu_attr = kperf.PmuAttr(evtList=evtList, includeNewFork=True, pidList=pidList)
- pd = kperf.open(kperf.PmuTaskType.COUNTING, pmu_attr)
- if pd == -1:
- print(kperf.error())
- return
- kperf.enable(pd)
- time.sleep(4)
- pmu_data = kperf.read(pd)
- for data in pmu_data.iter:
- print(f"evt:{data.evt} count:{data.count} tid:{data.tid} pid:{data.pid}")
- kperf.disable(pd)
- kperf.close(pd)
-
-
-def PerfList():
- event_iter = kperf.event_list(kperf.PmuEventType.CORE_EVENT)
- for event in event_iter:
- print(f"event: {event}")
-
-
-if __name__ == '__main__':
- Counting()
- PerfList()
- NewFork()
```
-
-
-#### Contribution
-
-1. Fork the repository
-2. Create Feat_xxx branch
-3. Commit your code
-4. Create Pull Request
-
-
-#### Gitee Feature
-
-1. You can use Readme\_XXX.md to support different languages, such as Readme\_en.md, Readme\_zh.md
-2. Gitee blog [blog.gitee.com](https://blog.gitee.com)
-3. Explore open source project [https://gitee.com/explore](https://gitee.com/explore)
-4. The most valuable open source project [GVP](https://gitee.com/gvp)
-5. The manual of Gitee [https://gitee.com/help](https://gitee.com/help)
-6. The most popular members [https://gitee.com/gitee-stars/](https://gitee.com/gitee-stars/)
diff --git a/README.md b/README.md
index 0763ac8..505fb59 100644
--- a/README.md
+++ b/README.md
@@ -2,54 +2,25 @@
#### 描述
-实现了一个低开销的pmu集合库，为计数、采样和符号解析提供了抽象接口。
+libkperf是一个轻量级linux性能采集库，它能够让开发者以API的方式执行性能采集，包括pmu采样和符号解析。libkperf把采集数据内存化，使开发者能够在内存中直接处理采集数据，避免了读写perf.data带来的开销。
-#### 软件构架
-
-这个存储库包括两个模块：pmu集合和符号解析。
-
-Pmu收集模块是在syscall perf_event_open上开发的，用于启用内核pmu计数和采样，根据用户输入使用-thread或per-core模式。
-从环形缓冲区读取Pmu数据包，并将其解析为不同的结构，进行计数，采样和spe采样。
-对于采样，根据ips或pc从数据包中解析符号。每个符号包含符号名称、地址、源文件路径和行号（如果可能）。
-
-符号解析模块是在elfin-parser上开发的，elfin-parser是一个解析elf和dwarf的库。该模块以设计良好的数据结构管理所有符号数据，以实现快速查询。
-
-#### 下载
-
-git方法:
+#### 编译
+编译生成动态库和C的API：
```shell
git clone --recurse-submodules https://gitee.com/openeuler/libkperf.git
-```
-如果你只使用
-```shell
-git clone https://gitee.com/openeuler/libkperf.git
-```
-请再执行
-```shell
cd libkperf
-git submodule update --init --recursive
+bash build.sh install_path=/path/to/install
```
-无法使用git时:
-
-1.下载libkperf压缩包并解压。
-
-2.进入gitee上的libkperf的third_party目录，点击链接(如elfin-parser@13e57e2，点击@后面的提交ID)，进行跳转并下载第三方库的压缩包，解压后放置于本地的libkperf项目的third_party目录。(elfin-parser对于安装是必须的)
-
-#### 安装
-
-运行bash脚本:
-
+如果想要编译调试版本：
```shell
-bash build.sh install_path=/home/libkperf
+bash build.sh install_path=/path/to/install buildType=debug
```
-如上，头文件和库将安装到/home/libkperf输出目录，installPath是可选参数，若没有设置，则默认安装到libkperf下的output目录。
-
-如果要额外增加python库支持，可以通过如下方式安装
+如果想要编译python包：
```shell
-bash build.sh python=true
+bash build.sh install_path=/path/to/install python=true
```
安装后若需要卸载python库， 可以执行下述命令
@@ -57,22 +28,23 @@ bash build.sh python=true
python3 -m pip uninstall -y libkperf
```
-#### 指令
+#### 文档
+详细文档可以参考docs目录：
+- [详细使用文档](./docs/Details.md)
-所有pmu功能都通过以下接口完成：
+#### 快速使用
+主要有以下几个API：
- PmuOpen
输入pid、core id和event，打开pmu设备。
- PmuEnable
开始收集。
- PmuRead
- 读取pmu数据并返回一个列表。
+ 读取采集数据。
- PmuDisable
停止收集。
- PmuClose
- 关闭PMU装置。
-
-API的详细说明请参考pmu.h。
+ 关闭pmu设备。
以下是一些示例：
@@ -101,7 +73,8 @@ PmuData *data = NULL;
// 读取PmuData，它是一个数组，长度是len。
int len = PmuRead(pd, &data);
for (int i = 0; i < len; ++i) {
- ...
+ PmuData *d = &data[i];
+ ...
}
// 释放PmuData。
PmuDataFree(data);
@@ -150,75 +123,6 @@ PmuDataFree(data);
// 类似fd，当任务结束时调用PmuClose释放资源。
PmuClose(pd);
```
-- 配置事件分组功能
-```C
-int pidList[1];
-pidList[0] = pid;
-unsigned numEvt = 16;
-char *evtList[numEvt] = {"r3", "r4", "r1", "r14", "r10", "r12", "r5", "r25",
- "r2", "r26", "r2d", "r17", "r8", "r22", "r24", "r11"};
-// 初始化事件列表，相同的group_id表示是同一个事件组
-// 如果不使用事件分组功能，需要配置事件组参数列表为空
-// 说明：如果事件的group_id为-1，表示此事件强制不进行分组
-PmuAttr attr = {0};
-attr.evtList = evtList;
-attr.numEvt = numEvt;
-attr.pidList = pidList;
-attr.numPid = 1;
-struct EvttAttr groupId[numEvt] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 13, 13, 13};
-attr.evtAttr = groupId;
-// 调用PmuOpen，返回pd。pd表示该任务的id。
-int pd = PmuOpen(COUNTING, &attr);
-// 开始采集。
-PmuEnable(pd);
-// 采集1秒。
-sleep(1);
-// 停止采集。
-PmuDisable(pd);
-PmuData *data = NULL;
-// 读取PmuData，它是一个数组，长度是len。
-int len = PmuRead(pd, &data);
-for (int i = 0; i < len; ++i) {
- ...
-}
-// 释放PmuData。
-PmuDataFree(data);
-// 类似fd，当任务结束时调用PmuClose释放资源。
-PmuClose(pd);
-```
-
-- Counting模式支持采集fork新生成子进程的能力
-```C
-int pidList[1];
-pidList[0] = pid;
-unsigned numEvt = 1;
-char *evtList[numEvt] = {"cycles"};
-PmuAttr attr = {0};
-attr.evtList = evtList;
-attr.numEvt = numEvt;
-attr.pidList = pidList;
-attr.numPid = 1;
-// 增加参数includeNewFork，为1可获取新生成子进程的数据，默认不获取
-attr.includeNewFork = 1;
-// 调用PmuOpen，返回pd。pd表示该任务的id。
-int pd = PmuOpen(COUNTING, &attr);
-// 开始采集。
-PmuEnable(pd);
-// 采集1秒。
-sleep(1);
-// 停止采集。
-PmuDisable(pd);
-PmuData *data = NULL;
-// 读取PmuData，它是一个数组，长度是len。
-int len = PmuRead(pd, &data);
-for (int i = 0; i < len; ++i) {
- ...
-}
-// 释放PmuData。
-PmuDataFree(data);
-// 类似fd，当任务结束时调用PmuClose释放资源。
-PmuClose(pd);
-```
Python 例子:
```python
@@ -230,8 +134,7 @@ import kperf
def Counting():
evtList = ["r11", "cycles"]
- evtAttr = [2, 2] # 与事件列表对应的事件分组id列表，相同的事件id表示是同一个事件组; 不启用的话，可以不使用这个参数
- pmu_attr = kperf.PmuAttr(evtList=evtList, evtAttr=evtAttr)
+ pmu_attr = kperf.PmuAttr(evtList=evtList)
pd = kperf.open(kperf.PmuTaskType.COUNTING, pmu_attr)
if pd == -1:
print(kperf.errorno())
@@ -252,50 +155,4 @@ def Counting():
kperf.disable(pd)
kperf.close(pd)
-
-def NewFork():
- # test_new_fork demo in test_perf, you can find test_new_fork.cpp
- p=subprocess.Popen(['test_new_fork']);
- pidList=[p.pid]
- evtList=["cycles"]
- pmu_attr = kperf.PmuAttr(evtList=evtList, includeNewFork=True, pidList=pidList)
- pd = kperf.open(kperf.PmuTaskType.COUNTING, pmu_attr)
- if pd == -1:
- print(kperf.error())
- return
- kperf.enable(pd)
- time.sleep(4)
- pmu_data = kperf.read(pd)
- for data in pmu_data.iter:
- print(f"evt:{data.evt} count:{data.count} tid:{data.tid} pid:{data.pid}")
- kperf.disable(pd)
- kperf.close(pd)
-
-
-def PerfList():
- event_iter = kperf.event_list(kperf.PmuEventType.CORE_EVENT)
- for event in event_iter:
- print(f"event: {event}")
-
-
-if __name__ == '__main__':
- Counting()
- PerfList()
```
-
-#### 参与贡献
-
-1. Fork 本仓库
-2. 新建 Feat_xxx 分支
-3. 提交代码
-4. 新建 Pull Request
-
-
-#### 特技
-
-1. 使用 Readme\_XXX.md 来支持不同的语言，例如 Readme\_en.md, Readme\_zh.md
-2. Gitee 官方博客 [blog.gitee.com](https://blog.gitee.com)
-3. 你可以 [https://gitee.com/explore](https://gitee.com/explore) 这个地址来了解 Gitee 上的优秀开源项目
-4. [GVP](https://gitee.com/gvp) 全称是 Gitee 最有价值开源项目，是综合评定出的优秀开源项目
-5. Gitee 官方提供的使用手册 [https://gitee.com/help](https://gitee.com/help)
-6. Gitee 封面人物是一档用来展示 Gitee 会员风采的栏目 [https://gitee.com/gitee-stars/](https://gitee.com/gitee-stars/)
diff --git a/docs/Details.md b/docs/Details.md
new file mode 100644
index 0000000..db0b407
--- /dev/null
+++ b/docs/Details.md
@@ -0,0 +1,264 @@
+Details
+============
+### Counting
+libkperf提供Counting模式，类似于perf stat功能。
+例如，如下perf命令:
+```
+perf stat -e cycles,branch-misses
+```
+该命令是对系统采集cycles和branch-misses这两个事件的计数。
+
+对于libkperf，可以这样来设置PmuAttr：
+```c++
+char *evtList[2];
+evtList[0] = "cycles";
+evtList[1] = "branch-misses";
+PmuAttr attr = {0};
+attr.evtList = evtList;
+attr.numEvt = 2;
+int pd = PmuOpen(COUNTING, &attr);
+```
+通过调用```PmuOpen```初始化了采集任务，并获得了任务的标识符pd。
+然后，可以利用pd来启动采集：
+```c++
+PmuEnable(pd);
+sleep(any_duration);
+PmuDisable(pd);
+```
+不论是否停止了采集，都可以通过```PmuRead```来读取采集数据：
+```c++
+PmuData *data = NULL;
+int len = PmuRead(pd, &data);
+```
+```PmuRead```会返回采集数据的长度。
+如果是对系统采集，那么PmuData的长度等于core的数量乘以事件的数量，PmuData的数据类似如下：
+```
+cpu 0 count 123 evt cycles
+cpu 1 count 1242354 evt cycles
+cpu 2 count 7897234 evt cycles
+...
+cpu 0 count 423423 evt branch-misses
+cpu 1 count 124235 evt branch-misses
+cpu 2 count 789723 evt branch-misses
+...
+```
+如果是对进程采集，那么PmuData的长度等于进程内线程的数量乘以事件的数量，PmuData的数据类似如下：
+```
+pid 4156 tid 4156 count 123 evt cycles
+pid 4156 tid 4157 count 534123 evt cycles
+pid 4156 tid 4158 count 1241244 evt cycles
+...
+pid 4156 tid 4156 count 12414 evt branch-misses
+pid 4156 tid 4157 count 5123 evt branch-misses
+pid 4156 tid 4158 count 64574 evt branch-misses
+...
+```
+
+### Sampling
+libkperf提供Sampling模式，类似于perf record的如下命令：
+```
+perf record -e cycles,branch-misses
+```
+该命令是对系统采样cycles和branch-misses这两个事件。
+
+设置PmuAttr的方式和Counting一样，在调用PmuOpen的时候，把任务类型设置为SAMPLING，并且设置采样频率：
+```c++
+// 采样频率是1000HZ
+attr.freq = 1000;
+attr.useFreq = 1;
+int pd = PmuOpen(SAMPLING, &attr);
+```
+
+启动采集和读取数据的方式和Counting一致。
+如果是对系统采集，PmuData的数据类似如下（长度取决于数据量）：
+```
+cpu 0 pid 3145 tid 3145 period 12314352
+cpu 0 pid 4145 tid 4145 period 12314367
+...
+cpu 1 pid 23423 tid 23423 period 1231241
+...
+...
+```
+如果是对进程采集，PmuData的数据类似如下：
+```
+cpu 32 pid 7878 tid 7878 period 123144
+cpu 32 pid 7878 tid 7879 period 1523342
+cpu 32 pid 7878 tid 7879 period 1234342
+...
+```
+每一条记录还包含触发事件的程序地址和符号信息，关于如何获取符号信息，可以参考[获取符号信息](#获取符号信息)这一章节。
+
+### SPE Sampling
+libkperf提供SPE采样模式，类似于perf record的如下命令：
+```
+perf record -e arm_spe_0/load_filter=1/
+```
+该命令是对系统进行spe采样，关于linux spe采样的详细介绍，可以参考[这里](https://www.man7.org/linux/man-pages/man1/perf-arm-spe.1.html)。
+
+对于libkperf，可以这样设置PmuAttr：
+```c++
+PmuAttr attr = {0};
+// 采样周期是8192
+attr.period = 8192;
+// 设置filter属性为load_filter
+attr.dataFilter = LOAD_FILTER;
+```
+对于spe采样，不需要设置evtList，而是通过设置dataFilter和evFilter来指定需要采集的事件。dataFilter和evFilter的含义仍然可以参考[perf spe的说明文档](https://www.man7.org/linux/man-pages/man1/perf-arm-spe.1.html)。
+
+采样数据PmuData和Sampling模式差不多，差别是：
+- SPE采样的调用栈只有一层，而Sampling可以有多层调用栈。
+- SPE的PmuData提供了额外的数据struct PmuDataExt *ext.
+PmuDataExt包含spe特有的数据：访存的物理地址、虚拟地址和事件bit。
+```c++
+struct PmuDataExt {
+ unsigned long pa; // physical address
+ unsigned long va; // virtual address
+ unsigned long event; // event id, which is a bit map of mixed events, event bit is defined in SPE_EVENTS.
+};
+```
+其中，物理地址pa需要在启用PA_ENABLE的情况下才能采集。
+event是一个bit map，是多个事件的集合，每一个事件占据一个bit，事件对应的bit参考枚举SPE_EVENTS：
+```c++
+enum SPE_EVENTS {
+ SPE_EV_EXCEPT = 1 << 0,
+ SPE_EV_RETIRED = 1 << 1,
+ SPE_EV_L1D_ACCESS = 1 << 2,
+ SPE_EV_L1D_REFILL = 1 << 3,
+ SPE_EV_TLB_ACCESS = 1 << 4,
+ SPE_EV_TLB_WALK = 1 << 5,
+ SPE_EV_NOT_TAKEN = 1 << 6,
+ SPE_EV_MISPRED = 1 << 7,
+ SPE_EV_LLC_ACCESS = 1 << 8,
+ SPE_EV_LLC_MISS = 1 << 9,
+ SPE_EV_REMOTE_ACCESS= 1 << 10,
+ SPE_EV_ALIGNMENT = 1 << 11,
+ SPE_EV_PARTIAL_PRED = 1 << 17,
+ SPE_EV_EMPTY_PRED = 1 << 18,
+};
+```
+
+### 获取符号信息
+结构体PmuData里提供了采样数据的调用栈信息，包含调用栈的地址、符号名称等。
+```c++
+struct Symbol {
+ unsigned long addr;
+ char* module;
+ char* symbolName;
+ char* fileName;
+ unsigned int lineNum;
+ ...
+};
+
+struct Stack {
+ struct Symbol* symbol;
+ struct Stack* next;
+ struct Stack* prev;
+ ...
+} __attribute__((aligned(64)));
+```
+
+Stack是链表结构，每一个元素都是一层调用函数。
+```mermaid
+graph LR
+a(Symbol) --> b(Symbol)
+b --> c(Symbol)
+c --> d(......)
+```
+
+Symbol的字段信息受PmuAttr影响：
+- PmuAttr.callStack会决定Stack是完整的调用栈，还是只有一层调用栈（即Stack链表只有一个元素）。
+- PmuAttr.symbolMode如果等于NO_SYMBOL_RESOLVE，那么PmuData的stack是空指针。
+- PmuAttr.symbolMode如果等于RESOLVE_ELF，那么Symbol的fileName和lineNum没有数据，都等于0，因为没有解析dwarf信息。
+- PmuAttr.symbolMode如果等于RESOLVE_ELF_DWARF，那么Symbol的所有信息都有效。
+
+### 采集uncore事件
+libkperf支持uncore事件的采集，只有Counting模式支持uncore事件的采集（和perf一致）。
+可以像这样设置PmuAttr：
+```c++
+char *evtList[1];
+evtList[0] = "hisi_sccl1_ddrc0/flux_rd/";
+PmuAttr attr = {0};
+attr.evtList = evtList;
+attr.numEvt = 1;
+int pd = PmuOpen(COUNTING, &attr);
+```
+uncore事件的格式为```<device>/<event>/```，上面代码是采集设备hisi_sccl1_ddrc0的flux_rd事件。
+
+也可以把设备索引号省略：
+```c++
+evtList[0] = "hisi_sccl1_ddrc/flux_rd/";
+```
+这里把hisi_sccl1_ddrc0改为了hisi_sccl1_ddrc，这样会采集设备hisi_sccl1_ddrc0、hisi_sccl1_ddrc1、hisi_sccl1_ddrc2...，并且采集数据PmuData是所有设备数据的总和：count = count(hisi_sccl1_ddrc0) + count(hisi_sccl1_ddrc1) + count(hisi_sccl1_ddrc2) + ...
+
+也可以通过```<device>/config=0xxx/```的方式来指定事件名：
+```c++
+evtList[0] = "hisi_sccl1_ddrc0/config=0x1/";
+```
+这样效果是和指定flux_rd是一样的。
+
+### 采集tracepoint
+libkperf支持tracepoint的采集，支持的tracepoint事件可以通过perf list来查看（通常需要root权限）。
+可以这样设置PmuAttr：
+```c++
+char *evtList[1];
+evtList[0] = "sched:sched_switch";
+PmuAttr attr = {0};
+attr.evtList = evtList;
+attr.numEvt = 1;
+```
+
+tracepoint支持Counting和Sampling两种模式，API调用流程和两者相似。
+tracepoint能够获取每个事件特有的数据，比如sched:sched_switch包含的数据有：prev_comm, prev_pid, prev_prio, prev_state, next_comm, next_pid, next_prio.
+想要查询每个事件包含哪些数据，可以查看/sys/kernel/tracing/events下面的文件内容，比如/sys/kernel/tracing/events/sched/sched_switch/format。
+
+libkperf提供了接口PmuGetField来获取tracepoint的数据。比如对于sched:sched_switch，可以这样调用：
+```c++
+int prev_pid;
+PmuGetField(pmuData->rawData, "prev_pid", &prev_pid, sizeof(prev_pid));
+char next_comm[16];
+PmuGetField(pmuData->rawData, "next_comm", &next_comm, sizeof(next_comm));
+```
+这里调用者需要提前了解数据的类型，并且指定数据的大小。数据的类型和大小仍然可以从/sys/kernel/tracing/下每个事件的format文件来得知。
+
+### 事件分组
+libkperf提供了事件分组的能力，能够让多个事件同时处于采集状态。
+该功能类似于perf的如下使用方式：
+```
+perf stat -e "{cycles,branch-loads,branch-load-misses,iTLB-loads}",inst_retired
+```
+
+对于libkperf，可以通过设置PmuAttr的evtAttr字段来设定哪些事件放在一个group内。
+比如，可以这样调用：
+```c
+unsigned numEvt = 5;
+char *evtList[numEvt] = {"cycles","branch-loads","branch-load-misses","iTLB-loads","inst_retired"};
+// 前四个事件是一个分组
+struct EvtAttr groupId[numEvt] = {1,1,1,1,-1};
+PmuAttr attr = {0};
+attr.evtList = evtList;
+attr.numEvt = numEvt;
+attr.evtAttr = groupId;
+```
+上述代码把前四个事件设定为一个分组，groupId都设定为1，最后一个事件不分组，groupId设定为-1。
+事件数组attr.evtList和事件属性数组attr.evtAttr必须一一对应，即长度必须一致。
+或者attr.evtAttr也可以是空指针，那么所有事件都不分组。
+
+事件分组的效果可以从PmuData.countPercent来体现。PmuData.countPercent表示事件实际采集时间除以事件期望采集时间。
+对于同一组的事件，他们的countPercent是相同的。如果一个组的事件过多，超过了硬件计数器的数目，那么这个组的所有事件都不会被采集，countPercent会等于-1.
+
+### 对进程子线程计数采集
+```mermaid
+graph TD
+a(主线程) --perf stat--> b(创建线程)
+b --> c(子线程)
+c --end perf--> d(子线程退出)
+```
+考虑上面的场景：用perf stat对进程采集，之后进程创建了子线程，采集一段事件后，停止perf。
+查看采集结果，perf只会显示主线程的采集结果，而无法看到子线程的结果：count = count(main thread) + count(thread). perf把子线程的数据聚合到了主线程上。
+
+libkperf提供了采集子线程的能力。如果想要在上面场景中获取子线程的计数，可以把PmuAttr.incluceNewFork设置为1.
+```c++
+attr.includeNewFork = 1;
+```
+然后，通过PmuRead获取到的PmuData，便能包含子线程计数信息了。
+注意，该功能是针对Counting模式，因为Sampling和SPE Sampling本身就会采集子线程的数据。
\ No newline at end of file
--
2.43.0

340
0003-fix-after-pmuopen-delete-evtList-exception.patch Normal file
View File

@ -0,0 +1,340 @@
From 3e9e0c7f5d1a54b7688924243cc14707906a14cf Mon Sep 17 00:00:00 2001
From: eho <2220386943@qq.com>
Date: Wed, 25 Sep 2024 16:34:45 +0800
Subject: [PATCH 03/20] fix after pmuopen delete evtList exception if you
delete the pointer pointed to by attr.evtList immediately after pmuuopen is
successful, the event will be garbled.
---
build/common.sh | 4 ++
pmu/pmu.cpp | 108 +++++++++++++++++++++++-----------
pmu/pmu_list.cpp | 26 +++++---
pmu/pmu_list.h | 6 +-
test/test_perf/test_count.cpp | 57 ++++++++++++++++++
5 files changed, 155 insertions(+), 46 deletions(-)
diff --git a/build/common.sh b/build/common.sh
index 9cc16b0..f48b64e 100644
--- a/build/common.sh
+++ b/build/common.sh
@@ -15,6 +15,10 @@ set -e
cpu_core_num=$(($(nproc)-1))
+if [ "$cpu_core_num" -eq 0 ];then
+ cpu_core_num=1
+fi
+
creat_dir(){
local target_dir="$1"
if [ -d "${target_dir}" ];then
diff --git a/pmu/pmu.cpp b/pmu/pmu.cpp
index b7507ec..207d4fb 100644
--- a/pmu/pmu.cpp
+++ b/pmu/pmu.cpp
@@ -160,17 +160,29 @@ static int CheckAttr(enum PmuTaskType collectType, struct PmuAttr *attr)
return LIBPERF_ERR_INVALID_SAMPLE_RATE;
}
- if ((collectType == SAMPLING || collectType == COUNTING) && attr->evtAttr == nullptr) {
- struct EvtAttr *evtAttr = new struct EvtAttr[attr->numEvt];
+ return SUCCESS;
+}
+
+static void CopyAttrData(PmuAttr* newAttr, PmuAttr* inputAttr, enum PmuTaskType collectType)
+{
+ //Coping event data to prevent delete exceptions
+ if (inputAttr->numEvt > 0) {
+ char **newEvtList = new char *[inputAttr->numEvt];
+ for (int i = 0; i < inputAttr->numEvt; ++i) {
+ newEvtList[i] = new char[strlen(inputAttr->evtList[i]) + 1];
+ strcpy(newEvtList[i], inputAttr->evtList[i]);
+ }
+ newAttr->evtList = newEvtList;
+ }
+
+ if ((collectType == SAMPLING || collectType == COUNTING) && inputAttr->evtAttr == nullptr) {
+ struct EvtAttr *evtAttr = new struct EvtAttr[inputAttr->numEvt];
// handle event group id. -1 means that it doesn't run event group feature.
- for (int i = 0; i < attr->numEvt; ++i) {
+ for (int i = 0; i < inputAttr->numEvt; ++i) {
evtAttr[i].group_id = -1;
}
- attr->evtAttr = evtAttr;
-
+ newAttr->evtAttr = evtAttr;
}
-
- return SUCCESS;
}
static bool FreeEvtAttr(struct PmuAttr *attr)
@@ -196,6 +208,21 @@ static bool FreeEvtAttr(struct PmuAttr *attr)
return SUCCESS;
}
+static void FreeEvtList(unsigned evtNum, char** evtList)
+{
+ if (!evtList) {
+ return;
+ }
+ for (int i = 0; i < evtNum; i++) {
+ if (evtList[i]) {
+ delete[] evtList[i];
+ evtList[i] = nullptr;
+ }
+ }
+ delete[] evtList;
+ evtList = nullptr;
+}
+
static bool AppendChildEvents(char* evt, unordered_map<string, char*>& eventSplitMap)
{
string strName(evt);
@@ -304,51 +331,64 @@ static void PmuTaskAttrFree(PmuTaskAttr *taskAttr)
int PmuOpen(enum PmuTaskType collectType, struct PmuAttr *attr)
{
SetWarn(SUCCESS);
+ PmuAttr copiedAttr = {*attr};
+ pair<unsigned, char**> previousEventList = {0, nullptr};
try {
auto err = CheckAttr(collectType, attr);
if (err != SUCCESS) {
return -1;
}
+ CopyAttrData(&copiedAttr, attr, collectType);
+ previousEventList = make_pair(copiedAttr.numEvt, copiedAttr.evtList);
+ int pd = -1;
unordered_map<string, char*> eventSplitMap;
- if (!SplitUncoreEvent(attr, eventSplitMap)) {
- return -1;
- }
- auto previousEventList = make_pair(attr->numEvt, attr->evtList);
- vector<char*> newEvtlist;
- vector<struct EvtAttr> newEvtAttrList;
- auto numEvt = GenerateSplitList(eventSplitMap, newEvtlist, attr, newEvtAttrList);
- FreeEvtAttr(attr);
- attr->numEvt = numEvt;
- attr->evtList = newEvtlist.data();
- attr->evtAttr = newEvtAttrList.data();
-
- auto pTaskAttr = AssignPmuTaskParam(collectType, attr);
- if (pTaskAttr == nullptr) {
- return -1;
- }
- unique_ptr<PmuTaskAttr, void (*)(PmuTaskAttr*)> taskAttr(pTaskAttr, PmuTaskAttrFree);
+ do {
+ if (!SplitUncoreEvent(&copiedAttr, eventSplitMap)) {
+ break;
+ }
+ vector<char *> newEvtlist;
+ vector<struct EvtAttr> newEvtAttrList;
+ auto numEvt = GenerateSplitList(eventSplitMap, newEvtlist, &copiedAttr, newEvtAttrList);
+ FreeEvtAttr(&copiedAttr);
+ copiedAttr.numEvt = numEvt;
+ copiedAttr.evtList = newEvtlist.data();
+ copiedAttr.evtAttr = newEvtAttrList.data();
+
+ auto pTaskAttr = AssignPmuTaskParam(collectType, &copiedAttr);
+ if (pTaskAttr == nullptr) {
+ break;
+ }
+ unique_ptr<PmuTaskAttr, void (*)(PmuTaskAttr *)> taskAttr(pTaskAttr, PmuTaskAttrFree);
+
+ pd = KUNPENG_PMU::PmuList::GetInstance()->NewPd();
+ if (pd == -1) {
+ New(LIBPERF_ERR_NO_AVAIL_PD);
+ break;
+ }
+
+ KUNPENG_PMU::PmuList::GetInstance()->SetSymbolMode(pd, attr->symbolMode);
+ err = KUNPENG_PMU::PmuList::GetInstance()->Register(pd, taskAttr.get());
+ if (err != SUCCESS) {
+ PmuList::GetInstance()->Close(pd);
+ pd = -1;
+ }
+ New(err);
+ } while(false);
- auto pd = KUNPENG_PMU::PmuList::GetInstance()->NewPd();
if (pd == -1) {
- New(LIBPERF_ERR_NO_AVAIL_PD);
+ FreeEvtList(previousEventList.first, previousEventList.second);
return -1;
}
-
- KUNPENG_PMU::PmuList::GetInstance()->SetSymbolMode(pd, attr->symbolMode);
- err = KUNPENG_PMU::PmuList::GetInstance()->Register(pd, taskAttr.get());
- if (err != SUCCESS) {
- PmuList::GetInstance()->Close(pd);
- pd = -1;
- }
// store eventList provided by user and the mapping relationship between the user eventList and the split
// eventList into buff
KUNPENG_PMU::PmuList::GetInstance()->StoreSplitData(pd, previousEventList, eventSplitMap);
- New(err);
return pd;
} catch (std::bad_alloc&) {
+ FreeEvtList(previousEventList.first, previousEventList.second);
New(COMMON_ERR_NOMEM);
return -1;
} catch (exception& ex) {
+ FreeEvtList(previousEventList.first, previousEventList.second);
New(UNKNOWN_ERROR, ex.what());
return -1;
}
diff --git a/pmu/pmu_list.cpp b/pmu/pmu_list.cpp
index 0caba7b..4f293a7 100644
--- a/pmu/pmu_list.cpp
+++ b/pmu/pmu_list.cpp
@@ -283,8 +283,8 @@ namespace KUNPENG_PMU {
return SUCCESS;
}
- void PmuList::StoreSplitData(const unsigned pd, pair<unsigned, char**> previousEventList,
- unordered_map<string, char*> eventSplitMap)
+ void PmuList::StoreSplitData(const unsigned pd, pair<unsigned, char**>& previousEventList,
+ unordered_map<string, char*>& eventSplitMap)
{
lock_guard<mutex> lg(dataParentMtx);
parentEventMap.emplace(pd, move(eventSplitMap));
@@ -303,7 +303,7 @@ namespace KUNPENG_PMU {
RemoveEpollFd(pd);
EraseSpeCpu(pd);
EraseDummyEvent(pd);
- EraseParentEventMap();
+ EraseParentEventMap(pd);
SymResolverDestroy();
PmuEventListFree();
PointerPasser::FreeRawFieldMap();
@@ -409,15 +409,23 @@ namespace KUNPENG_PMU {
return dataEvtGroupList[pd];
}
- void PmuList::EraseParentEventMap()
+ void PmuList::EraseParentEventMap(const unsigned pd)
{
lock_guard<mutex> lg(dataParentMtx);
- for (auto& pair: parentEventMap) {
- auto& innerMap = pair.second;
- innerMap.clear();
+ auto iter = parentEventMap.find(pd);
+ if (iter != parentEventMap.end()) {
+ parentEventMap.at(pd).clear();
+ parentEventMap.erase(iter);
+ }
+ auto preIter = previousEventMap.find(pd);
+ if (preIter != previousEventMap.end()) {
+ auto pair = previousEventMap.at(pd);
+ for (int i = 0; i < pair.first; i++) {
+ delete[] pair.second[i];
+ }
+ delete[] pair.second;
+ previousEventMap.erase(preIter);
}
- parentEventMap.clear();
- previousEventMap.clear();
}
PmuList::EventData& PmuList::GetDataList(const unsigned pd)
diff --git a/pmu/pmu_list.h b/pmu/pmu_list.h
index 80346eb..13724f8 100644
--- a/pmu/pmu_list.h
+++ b/pmu/pmu_list.h
@@ -71,8 +71,8 @@ public:
int NewPd();
int GetHistoryData(const int pd, std::vector<PmuData>& pmuData);
- void StoreSplitData(unsigned pd, std::pair<unsigned, char**> previousEventList,
- std::unordered_map<std::string, char*> eventSplitMap);
+ void StoreSplitData(unsigned pd, std::pair<unsigned, char**>& previousEventList,
+ std::unordered_map<std::string, char*>& eventSplitMap);
bool IsAllPidExit(const unsigned pd);
private:
@@ -95,7 +95,7 @@ private:
void InsertEvtList(const unsigned pd, std::shared_ptr<EvtList> evtList);
std::vector<std::shared_ptr<EvtList>>& GetEvtList(const unsigned pd);
void EraseEvtList(const unsigned pd);
- void EraseParentEventMap();
+ void EraseParentEventMap(const unsigned pd);
int EvtInit(const bool groupEnable, const std::shared_ptr<EvtList> evtLeader, const int pd, const std::shared_ptr<EvtList> &evtList);
int Init(const int pd);
diff --git a/test/test_perf/test_count.cpp b/test/test_perf/test_count.cpp
index 1fda697..1b64497 100644
--- a/test/test_perf/test_count.cpp
+++ b/test/test_perf/test_count.cpp
@@ -13,6 +13,7 @@
* Description: Unit test for counting.
******************************************************************************/
#include "test_common.h"
+#include <dirent.h>
using namespace std;
@@ -307,4 +308,60 @@ TEST_F(TestCount, SimdRatio)
ASSERT_EQ(evtMap.size(), evts.size());
auto simdRatio = (double)evtMap[aseSpec]/evtMap[instSpec];
ASSERT_GT(simdRatio, 0.1);
+}
+
+static std::vector<string> GetHHADirs() {
+ vector<string> hhaEvents;
+ unique_ptr<DIR, decltype(&closedir)> dir(opendir("/sys/devices"), &closedir);
+ if(!dir) {
+ return hhaEvents;
+ }
+
+ struct dirent* dt;
+ while((dt = readdir(dir.get())) != nullptr) {
+ std::string name = dt->d_name;
+ if(name == "." || name == "..") {
+ continue;
+ }
+
+ if(dt->d_type == DT_DIR && strstr(name.c_str(), "hha") != nullptr) {
+ hhaEvents.push_back(name + "/");
+ }
+ }
+ return hhaEvents;
+}
+
+
+TEST_F(TestCount, DeleteEvtAfterOpenPmuu) {
+ struct PmuAttr attr = {nullptr};
+ vector<string> hhaEvents = GetHHADirs();
+ ASSERT_TRUE(hhaEvents.size() > 0);
+ set<string> evtNames;
+ vector<string> eventsStr = {"rx_outer", "rx_sccl", "rx_ops_num"};
+ int evtNum = hhaEvents.size() * eventsStr.size();
+ char **evtList = new char *[evtNum];
+ for (int i = 0; i < hhaEvents.size(); ++i) {
+ for (int j = 0; j < eventsStr.size(); ++j) {
+ int evtLen = hhaEvents[i].size() + eventsStr[j].size() + 2;
+ evtList[i * eventsStr.size() + j] = new char[evtLen];
+ snprintf(evtList[i * eventsStr.size() + j], evtLen, "%s%s/", hhaEvents[i].c_str(), eventsStr[j].c_str());
+ evtNames.emplace(hhaEvents[i] + eventsStr[j] + "/");
+ }
+ }
+ attr.evtList = evtList;
+ attr.numEvt = evtNum;
+ int pd = PmuOpen(COUNTING, &attr);
+ for (int i = 0; i < evtNum; i++) {
+ delete[] evtList[i];
+ }
+ delete[] evtList;
+ PmuEnable(pd);
+ sleep(1);
+ PmuDisable(pd);
+ PmuData* pmuData = nullptr;
+ int len = PmuRead(pd, &pmuData);
+ for(int i = 0; i < len; i++) {
+ ASSERT_TRUE(evtNames.find(pmuData[i].evt) != evtNames.end());
+ }
+ PmuClose(pd);
}
\ No newline at end of file
--
2.43.0

62
0004-Delete-the-repeated-judgement-on-formatPath.patch Normal file
View File

@ -0,0 +1,62 @@
From 109fd9f0140668eac0ff48415ca99c4f82853503 Mon Sep 17 00:00:00 2001
From: eho <2220386943@qq.com>
Date: Tue, 8 Oct 2024 10:31:41 +0800
Subject: [PATCH 04/20] Delete the repeated judgement on formatPath
---
pmu/trace_pointer_parser.cpp | 22 +++++++++++++++-------
1 file changed, 15 insertions(+), 7 deletions(-)
diff --git a/pmu/trace_pointer_parser.cpp b/pmu/trace_pointer_parser.cpp
index 4417494..235808a 100644
--- a/pmu/trace_pointer_parser.cpp
+++ b/pmu/trace_pointer_parser.cpp
@@ -26,22 +26,30 @@ const char *POINTER_OFFSET_REGEX = "%*[^0-9]%i%*[;] %*[^0-9]%i%*[;] %*[^0-9]%i%*
static std::unordered_map<std::string, std::unordered_map<string, Field>> efMap; //the key is event name, value is field and ths field name map.
static std::unordered_map<char *, std::string> dEvtMap; //The key is the data pointer, value is event name.
static std::map<Field, SampleRawField *> fsrMap;
+static std::unordered_map<std::string, std::string> formatMap;
-bool PointerPasser::IsNeedFormat(std::ifstream &file, const std::string &evtName) {
+static std::string GetFormatRealPath(const std::string &evtName) {
auto colonId = evtName.find(':');
if (colonId == string::npos) {
- return false;
+ return {};
}
string eventName = evtName.substr(colonId + 1);
string systemName = evtName.substr(0, colonId);
const string &eventDir = GetTraceEventDir();
if (eventDir.empty()) {
- return false;
+ return {};
}
string formatPath = eventDir + systemName + "/" + eventName + "/format";
- string realPath = GetRealPath(formatPath);
- if (realPath.empty()) {
- return false;
+ return GetRealPath(formatPath);
+}
+
+bool PointerPasser::IsNeedFormat(std::ifstream &file, const std::string &evtName) {
+ std::string realPath;
+ if (formatMap.find(evtName) != formatMap.end()) {
+ realPath = formatMap.at(evtName);
+ } else {
+ realPath = GetFormatRealPath(evtName);
+ formatMap.emplace(evtName, realPath);
}
if (!IsValidPath(realPath)) {
return false;
@@ -88,7 +96,7 @@ void PointerPasser::ParserRawFormatData(struct PmuData *pd, KUNPENG_PMU::PerfRaw
union KUNPENG_PMU::PerfEvent *event,
const std::string &evtName) {
ifstream file;
- if (!IsNeedFormat(file, evtName)) {
+ if (efMap.find(evtName) == efMap.end() && !IsNeedFormat(file, evtName)) {
pd->rawData = nullptr;
return;
}
--
2.43.0

147
0005-pmu-event.patch Normal file
View File

@ -0,0 +1,147 @@
From 4a084d12ceb0129c8b901798282ef89f3df13bf2 Mon Sep 17 00:00:00 2001
From: ganlixiong <ganli2012@gmail.com>
Date: Fri, 11 Oct 2024 10:50:07 +0800
Subject: [PATCH 05/20] =?UTF-8?q?=E4=BF=AE=E5=A4=8D=E5=9C=A8=E6=9F=90?=
=?UTF-8?q?=E4=BA=9B=E7=8E=AF=E5=A2=83=E4=B8=8A=E6=97=A0=E6=B3=95=E8=AF=BB?=
=?UTF-8?q?=E5=8F=96=E5=88=B0pmu=20event=E7=9A=84=E9=97=AE=E9=A2=98?=
=?UTF-8?q?=E3=80=82?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
问题：
在某些环境上调用PmuEventList，返回的事件列表为空。
原因：
该环境上的arm core设备名称为armv8_pmuv3，而在查询事件列表时，搜索了armv8_pmuv3_0设备的事件。
解决方法：
动态寻找arm core设备，而不是用硬编码。参考perf
tool的实现，查询devices下面的所有设备，如果设备下面包含文件cpus，那么认为该设备为arm
core设备。
---
pmu/pfm/core.cpp | 44 +++++++++++++++++++++++++++++++++++++++---
pmu/pfm/core.h | 2 ++
pmu/pmu_event_list.cpp | 10 ++++++++--
3 files changed, 51 insertions(+), 5 deletions(-)
diff --git a/pmu/pfm/core.cpp b/pmu/pfm/core.cpp
index 80d5e0b..0ab4607 100644
--- a/pmu/pfm/core.cpp
+++ b/pmu/pfm/core.cpp
@@ -15,6 +15,7 @@
#include <vector>
#include <unordered_map>
#include <fstream>
+#include <dirent.h>
#include "pmu_event.h"
#include "core.h"
#include "common.h"
@@ -22,7 +23,7 @@
using namespace std;
using PMU_PAIR = std::pair<std::string, KUNPENG_PMU::CoreConfig>;
static CHIP_TYPE g_chipType = UNDEFINED_TYPE;
-
+static string pmuDevice = "";
namespace SOFTWARE_EVENT {
PMU_PAIR ALIGNMENT_FAULTS = {
@@ -853,7 +854,11 @@ static struct PmuEvt* ConstructPmuEvtFromCore(KUNPENG_PMU::CoreConfig config, in
static int64_t GetKernelCoreEventConfig(const string &name)
{
- string eventPath = "/sys/devices/armv8_pmuv3_0/events/" + name;
+ auto pmuDevicePath = GetPmuDevicePath();
+ if (pmuDevicePath.empty()) {
+ return -1;
+ }
+ string eventPath = pmuDevicePath + "/events/" + name;
string realPath = GetRealPath(eventPath);
if (!IsValidPath(realPath)) {
return -1;
@@ -874,7 +879,11 @@ static int64_t GetKernelCoreEventConfig(const string &name)
static int64_t GetKernelCoreEventType()
{
- string eventPath = "/sys/devices/armv8_pmuv3_0/type";
+ auto pmuDevicePath = GetPmuDevicePath();
+ if (pmuDevicePath.empty()) {
+ return -1;
+ }
+ string eventPath = pmuDevicePath + "/type";
string realPath = GetRealPath(eventPath);
if (!IsValidPath(realPath)) {
return -1;
@@ -917,4 +926,33 @@ struct PmuEvt* GetCoreEvent(const char* pmuName, int collectType)
return ConstructPmuEvtFromCore(KUNPENG_PMU::CORE_EVENT_MAP.at(g_chipType).at(pmuName), collectType);
}
return ConstructPmuEvtFromKernel(pmuName, collectType);
+}
+
+std::string GetPmuDevicePath()
+{
+ if (!pmuDevice.empty()) {
+ return pmuDevice;
+ }
+
+ static const string DEVICE_PATH = "/sys/bus/event_source/devices/";
+ DIR *dir = opendir(DEVICE_PATH.c_str());
+ if (dir == nullptr) {
+ return "";
+ }
+ struct dirent *dent;
+ while (dent = readdir(dir)) {
+ if (!strcmp(dent->d_name, ".") || !strcmp(dent->d_name, "..") || !strcmp(dent->d_name, "cpu")) {
+ continue;
+ }
+
+ // look for devices like /sys/bus/event_source/devices/armv8_pmuv3_0/cpus.
+ // Refer to function <is_arm_pmu_core> in kernel.
+ string armPmuPath = DEVICE_PATH + dent->d_name + "/cpus";
+ if (ExistPath(armPmuPath)) {
+ pmuDevice = DEVICE_PATH + dent->d_name;
+ break;
+ }
+ }
+
+ return pmuDevice;
}
\ No newline at end of file
diff --git a/pmu/pfm/core.h b/pmu/pfm/core.h
index 0df6a25..4ecf3fe 100644
--- a/pmu/pfm/core.h
+++ b/pmu/pfm/core.h
@@ -24,5 +24,7 @@ namespace KUNPENG_PMU {
struct PmuEvt* GetCoreEvent(const char* pmuName, int collectType);
+std::string GetPmuDevicePath();
+
#endif
diff --git a/pmu/pmu_event_list.cpp b/pmu/pmu_event_list.cpp
index 2007561..c4cf226 100644
--- a/pmu/pmu_event_list.cpp
+++ b/pmu/pmu_event_list.cpp
@@ -127,10 +127,16 @@ const char** QueryCoreEvent(unsigned *numEvt)
}
DIR* dir;
struct dirent* entry;
- string path = "/sys/devices/armv8_pmuv3_0/events/";
+ auto pmuDevPath = GetPmuDevicePath();
+ if (pmuDevPath.empty()) {
+ *numEvt = coreEventList.size();
+ return coreEventList.data();
+ }
+ string path = pmuDevPath + "/events/";
dir = opendir(path.c_str());
if (dir == nullptr) {
- return nullptr;
+ *numEvt = coreEventList.size();
+ return coreEventList.data();
}
while ((entry = readdir(dir)) != nullptr) {
if (entry->d_type == DT_REG) {
--
2.43.0

110
0006-adapt-events.patch Normal file
View File

@ -0,0 +1,110 @@
From 2e4695bff251032e023098e4ec25ff43d6690faf Mon Sep 17 00:00:00 2001
From: eho <2220386943@qq.com>
Date: Sat, 12 Oct 2024 15:47:28 +0800
Subject: [PATCH 06/20] adapt events
---
pmu/pfm/core.cpp | 40 ++++++++++++++++++++++++++++++++++++++++
pmu/pmu_event_list.cpp | 2 +-
util/cpu_map.cpp | 3 ++-
util/cpu_map.h | 1 +
4 files changed, 44 insertions(+), 2 deletions(-)
diff --git a/pmu/pfm/core.cpp b/pmu/pfm/core.cpp
index 0ab4607..55e5921 100644
--- a/pmu/pfm/core.cpp
+++ b/pmu/pfm/core.cpp
@@ -833,11 +833,51 @@ const std::unordered_map<std::string, KUNPENG_PMU::CoreConfig> HIP_F_CORE_PMU_MA
SOFTWARE_EVENT::TASK_CLOCK,
};
+const std::unordered_map<std::string, KUNPENG_PMU::CoreConfig> HIP_E_CORE_PMU_MAP{
+ HARDWARE_EVENT::BRANCH_MISSES,
+ HARDWARE_EVENT::CACHE_MISSES,
+ HARDWARE_EVENT::CACHE_REFERENCES,
+ HARDWARE_EVENT::CPU_CYCLES,
+ HARDWARE_EVENT::CYCLES,
+ HARDWARE_EVENT::INSTRUCTIONS,
+ HARDWARE_EVENT::STALLED_CYCLES_BACKEND,
+ HARDWARE_EVENT::STALLED_CYCLES_FRONTED,
+ HARDWARE_EVENT::IDLE_CYCLES_BACKEND,
+ HARDWARE_EVENT::IDLE_CYCLES_FRONTED,
+ HW_CACHE_EVENT::L1_DCACHE_LOAD_MISSES,
+ HW_CACHE_EVENT::L1_DCACHE_LOADS,
+ HW_CACHE_EVENT::L1_ICACHE_LOAD_MISSES,
+ HW_CACHE_EVENT::L1_ICACHE_LOADS,
+ HW_CACHE_EVENT::LLC_LOAD_MISSES,
+ HW_CACHE_EVENT::LLC_LOADS,
+ HW_CACHE_EVENT::BRANCH_LOAD_MISSES,
+ HW_CACHE_EVENT::BRANCH_LOADS,
+ HW_CACHE_EVENT::DTLB_LOAD_MISSES,
+ HW_CACHE_EVENT::DTLB_LOADS,
+ HW_CACHE_EVENT::ITLB_LOAD_MISSES,
+ HW_CACHE_EVENT::ITLB_LOADS,
+ SOFTWARE_EVENT::ALIGNMENT_FAULTS,
+ SOFTWARE_EVENT::BPF_OUTPUT,
+ SOFTWARE_EVENT::CONTEXT_SWITCHES,
+ SOFTWARE_EVENT::CS,
+ SOFTWARE_EVENT::CPU_CLOCK,
+ SOFTWARE_EVENT::CPU_MIGRATIONS,
+ SOFTWARE_EVENT::MIGRATIONS,
+ SOFTWARE_EVENT::DUMMY,
+ SOFTWARE_EVENT::EMULATION_FAULTS,
+ SOFTWARE_EVENT::MAJOR_FAULTS,
+ SOFTWARE_EVENT::MINOR_FAULTS,
+ SOFTWARE_EVENT::PAGE_FAULTS,
+ SOFTWARE_EVENT::FAULTS,
+ SOFTWARE_EVENT::TASK_CLOCK,
+};
+
const KUNPENG_PMU::CORE_EVT_MAP KUNPENG_PMU::CORE_EVENT_MAP = {
{CHIP_TYPE::HIPA, HIP_A_CORE_PMU_MAP},
{CHIP_TYPE::HIPB, HIP_B_CORE_PMU_MAP},
{CHIP_TYPE::HIPC, HIP_C_CORE_PMU_MAP},
{CHIP_TYPE::HIPF, HIP_F_CORE_PMU_MAP},
+ {CHIP_TYPE::HIPE, HIP_E_CORE_PMU_MAP},
};
static struct PmuEvt* ConstructPmuEvtFromCore(KUNPENG_PMU::CoreConfig config, int collectType)
diff --git a/pmu/pmu_event_list.cpp b/pmu/pmu_event_list.cpp
index c4cf226..b96e500 100644
--- a/pmu/pmu_event_list.cpp
+++ b/pmu/pmu_event_list.cpp
@@ -37,7 +37,7 @@ static const string EVENT_DIR = "/events/";
static std::mutex pmuEventListMtx;
-static vector<const char*> supportDevPrefixs = {"hisi", "smmuv3"};
+static vector<const char*> supportDevPrefixs = {"hisi", "smmuv3", "hns3"};
static vector<const char*> uncoreEventList;
static vector<const char*> traceEventList;
diff --git a/util/cpu_map.cpp b/util/cpu_map.cpp
index 7ed2cd2..ec8f077 100644
--- a/util/cpu_map.cpp
+++ b/util/cpu_map.cpp
@@ -33,7 +33,8 @@ static CHIP_TYPE g_chipType = CHIP_TYPE::UNDEFINED_TYPE;
static map<string, CHIP_TYPE> chipMap = {{"0x00000000481fd010", HIPA},
{"0x00000000480fd020", HIPB},
{"0x00000000480fd030", HIPC},
- {"0x00000000480fd220", HIPF}};
+ {"0x00000000480fd220", HIPF},
+ {"0x00000000480fd450", HIPE},};
static inline bool ReadCpuPackageId(int coreId, CpuTopology* cpuTopo)
{
diff --git a/util/cpu_map.h b/util/cpu_map.h
index de38af5..58c61b8 100644
--- a/util/cpu_map.h
+++ b/util/cpu_map.h
@@ -26,6 +26,7 @@ enum CHIP_TYPE {
HIPB = 2,
HIPC = 3,
HIPF = 4,
+ HIPE = 5,
};
struct CpuTopology* GetCpuTopology(int coreId);
--
2.43.0

30
0007-update-python-modules-_libkperf-Pmu.py.patch Normal file
View File

@ -0,0 +1,30 @@
From 8d93617bed18193386102587841a0eab2da207d5 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=E8=8F=A0=E8=90=9D=E6=9C=89=E7=82=B9=E9=85=B8?=
<yanxiaoqiang_yewu@cmss.chinamobile.com>
Date: Sat, 12 Oct 2024 08:45:14 +0000
Subject: [PATCH 07/20] update python/modules/_libkperf/Pmu.py.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Signed-off-by: 菠萝有点酸 <yanxiaoqiang_yewu@cmss.chinamobile.com>
---
python/modules/_libkperf/Pmu.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/python/modules/_libkperf/Pmu.py b/python/modules/_libkperf/Pmu.py
index 9c111bf..c743d7d 100644
--- a/python/modules/_libkperf/Pmu.py
+++ b/python/modules/_libkperf/Pmu.py
@@ -639,7 +639,7 @@ class CtypesPmuData(ctypes.Structure):
const char *comm; // process command
uint64_t period; // number of Samples
uint64_t count; // event count. Only available for Counting.
- double countPercent; // event count percent. when count = 0, countPercent = -1; Only avaliable for Counting.
+ double countPercent; // event count percent. when count = 0, countPercent = -1; Only available for Counting.
struct PmuDataExt *ext; // extension. Only available for Spe.
};
"""
--
2.43.0

56
0008-.patch Normal file
View File

@ -0,0 +1,56 @@
From 4e13130c164640cc02470074cb816047159159d2 Mon Sep 17 00:00:00 2001
From: lixiang <lixiang_yewu@cmss.chinamobile.com>
Date: Tue, 15 Oct 2024 09:20:40 +0800
Subject: [PATCH 08/20] =?UTF-8?q?=E6=8F=90=E4=BA=A4=E4=BF=AE=E6=94=B9?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
---
pmu/CMakeLists.txt | 2 +-
pmu/evt_list.h | 2 +-
python/tests/test_api.py | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/pmu/CMakeLists.txt b/pmu/CMakeLists.txt
index 189e43e..c68bfe0 100644
--- a/pmu/CMakeLists.txt
+++ b/pmu/CMakeLists.txt
@@ -25,7 +25,7 @@ include_directories(${PROJECT_TOP_DIR}/include)
include_directories(${PMU_FILE_DIR}/)
include_directories(${PFM_FILE_DIR})
-# directories for ultilities and symbol resolving
+# directories for utilities and symbol resolving
include_directories(${UTIL_FILE_DIR})
include_directories(${SYMBOL_FILE_DIR})
include_directories(${PMU_DECODER_DIR})
diff --git a/pmu/evt_list.h b/pmu/evt_list.h
index 802e7cf..50b888e 100644
--- a/pmu/evt_list.h
+++ b/pmu/evt_list.h
@@ -131,7 +131,7 @@ struct EventGroupInfo {
// store event group child events state flag info
/* event group child state explain:
* Enumeration variable uncoreState has four state, Initialization is the InitState;
- * sacn the event List, if find the uncore event, the uncoreState is config the high bit set to 1;
+ * scan the event List, if found the uncore event, the uncoreState is configured with the high bit set to 1;
* if find the other event, the uncoreState is config the low bit set to 0.
*/
enum class UncoreState uncoreState;
diff --git a/python/tests/test_api.py b/python/tests/test_api.py
index e62294b..7e32437 100644
--- a/python/tests/test_api.py
+++ b/python/tests/test_api.py
@@ -52,7 +52,7 @@ def TestAPI_SpeInitBusy():
print(f"error number: {kperf.errorno()} error message: {kperf.error()}")
badpd = kperf.open(kperf.PmuTaskType.SPE_SAMPLING, pmu_attr)
if badpd == -1:
- print(f"badpd rror number: {kperf.errorno()} badpd error message: {kperf.error()}")
+ print(f"badpd error number: {kperf.errorno()} badpd error message: {kperf.error()}")
kperf.close(badpd)
def TestAPI_OpenInvalidTaskType():
--
2.43.0

25
0009-adapt-gcc-4.8.5.patch Normal file
View File

@ -0,0 +1,25 @@
From 31b4552c07cc0216a0d1834c3c67c645129332ad Mon Sep 17 00:00:00 2001
From: eho <2220386943@qq.com>
Date: Tue, 15 Oct 2024 16:39:20 +0800
Subject: [PATCH 09/20] adapt gcc 4.8.5
---
pmu/pmu.cpp | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/pmu/pmu.cpp b/pmu/pmu.cpp
index 207d4fb..d2f4c9f 100644
--- a/pmu/pmu.cpp
+++ b/pmu/pmu.cpp
@@ -331,7 +331,7 @@ static void PmuTaskAttrFree(PmuTaskAttr *taskAttr)
int PmuOpen(enum PmuTaskType collectType, struct PmuAttr *attr)
{
SetWarn(SUCCESS);
- PmuAttr copiedAttr = {*attr};
+ PmuAttr copiedAttr = *attr;
pair<unsigned, char**> previousEventList = {0, nullptr};
try {
auto err = CheckAttr(collectType, attr);
--
2.43.0

48
0010-fix-spelling-error.patch Normal file
View File

@ -0,0 +1,48 @@
From d0e12d3418b1ba52f0dd5e75e94543b6e7322e91 Mon Sep 17 00:00:00 2001
From: baixu <baixu_yewu@cmss.chinamobile.com>
Date: Tue, 15 Oct 2024 16:47:07 +0800
Subject: [PATCH 10/20] fix spelling error
---
include/pcerrc.h | 4 ++--
python/modules/kperf/perror.py | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/include/pcerrc.h b/include/pcerrc.h
index 13c04e9..c2dd791 100644
--- a/include/pcerrc.h
+++ b/include/pcerrc.h
@@ -82,7 +82,7 @@ extern "C" {
#define UNKNOWN_ERROR 9999
-// warnning code
+// warning code
#define LIBPERF_WARN_CTXID_LOST 1000
#define LIBPERF_WARN_FAIL_GET_PROC 1001
#define LIBPERF_WARN_INVALID_GROUP_HAS_UNCORE 1002
@@ -97,7 +97,7 @@ int Perrorno();
const char* Perror();
/**.
- * @brief Get warnning codes
+ * @brief Get warning codes
*/
int GetWarn();
diff --git a/python/modules/kperf/perror.py b/python/modules/kperf/perror.py
index 6f5cfb3..097f1f6 100644
--- a/python/modules/kperf/perror.py
+++ b/python/modules/kperf/perror.py
@@ -81,7 +81,7 @@ class Error:
UNKNOWN_ERROR = 9999
- # warnning code
+ # warning code
LIBPERF_WARN_CTXID_LOST = 1000
LIBPERF_WARN_FAIL_GET_PROC = 1001
LIBPERF_WARN_INVALID_GROUP_HAS_UNCORE = 1002
--
2.43.0

51
0011-add-perrno-operation-not-supported.patch Normal file
View File

@ -0,0 +1,51 @@
From c9c20f95f9f58c10af07850bad2a0b33d97a71d8 Mon Sep 17 00:00:00 2001
From: eho <2220386943@qq.com>
Date: Thu, 24 Oct 2024 10:35:40 +0800
Subject: [PATCH 11/20] add perrno operation not supported.
---
pmu/pmu_event.cpp | 2 ++
test/test_perf/test_api.cpp | 16 +++++++++++++++-
2 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/pmu/pmu_event.cpp b/pmu/pmu_event.cpp
index 8a0bdc6..71a3616 100644
--- a/pmu/pmu_event.cpp
+++ b/pmu/pmu_event.cpp
@@ -30,6 +30,8 @@ namespace KUNPENG_PMU {
return LIBPERF_ERR_NO_PROC;
case EMFILE:
return LIBPERF_ERR_TOO_MANY_FD;
+ case EOPNOTSUPP:
+ return LIBPERF_ERR_INVALID_EVENT;
default:
return UNKNOWN_ERROR;
}
diff --git a/test/test_perf/test_api.cpp b/test/test_perf/test_api.cpp
index 20f0de2..5cf9213 100644
--- a/test/test_perf/test_api.cpp
+++ b/test/test_perf/test_api.cpp
@@ -601,4 +601,18 @@ TEST_F(TestAPI, TestSPEEventGroup)
attr.evtAttr = groupId;
pd = PmuOpen(SPE_SAMPLING, &attr);
ASSERT_TRUE(pd == -1);
-}
\ No newline at end of file
+}
+
+TEST_F(TestAPI, TestOperationNotSupported)
+{
+ PmuAttr attr = {0};
+ char* evtList[1] = {"hisi_sccl3_ddrc2/flux_rd/"};
+ attr.evtList = evtList;
+ attr.numEvt = 1;
+ attr.freq = 1000;
+ attr.useFreq = 1;
+ attr.symbolMode = RESOLVE_ELF;
+ int pd = PmuOpen(SAMPLING, &attr);
+ ASSERT_EQ(pd, -1);
+ ASSERT_EQ(Perrorno(), LIBPERF_ERR_INVALID_EVENT);
+}
--
2.43.0

44
0012-UT-SpeProcCollectSubProc.patch Normal file
View File

@ -0,0 +1,44 @@
From 065ba5f7493b1d27fd5320c99b597e2d96fb3396 Mon Sep 17 00:00:00 2001
From: glx <ganli2012@gmail.com>
Date: Tue, 29 Oct 2024 11:25:00 +0800
Subject: [PATCH 12/20] =?UTF-8?q?=E4=BF=AE=E5=A4=8DUT=E7=94=A8=E4=BE=8BSpe?=
=?UTF-8?q?ProcCollectSubProc=E3=80=82?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
问题：
UT用例SpeProcCollectSubProc报错。
原因：
用例SpeProcCollectSubProc执行过程：启动应用，开始采集，应用fork进程，校验子进程是否被采集到。
子进程未被采集到，因为UpdateProcMap函数里未找到子进程的父进程，所以子进程没有被添加到procMap。
解决方法：
对于fork子进程的场景，父进程应该是ppid，而不是pid，可以把ppid传入到UpdateProcMap。
---
pmu/spe.cpp | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/pmu/spe.cpp b/pmu/spe.cpp
index 99b7a89..4d964fb 100644
--- a/pmu/spe.cpp
+++ b/pmu/spe.cpp
@@ -372,7 +372,13 @@ void Spe::CoreDummyData(struct SpeCoreContext *context, struct ContextSwitchData
if (header->type == PERF_RECORD_FORK) {
struct PerfRecordFork *sample = (struct PerfRecordFork *)header;
DBG_PRINT("Fork pid: %d tid: %d\n", sample->pid, sample->tid);
- UpdateProcMap(sample->pid, sample->tid);
+ if (sample->pid == sample->tid) {
+ // A new process is forked and the parent pid is ppid.
+ UpdateProcMap(sample->ppid, sample->tid);
+ } else {
+ // A new thread is created and the parent pid is pid(process id).
+ UpdateProcMap(sample->pid, sample->tid);
+ }
dataTail += header->size;
continue;
}
--
2.43.0

29
0013-simplify-CoreSpeData-call-by-removing-redundant-buf-.patch Normal file
View File

@ -0,0 +1,29 @@
From e0142fbd0fe30ae684079342881bfb0e45be51b6 Mon Sep 17 00:00:00 2001
From: gubin <gubin_yewu@cmss.chinamobile.com>
Date: Tue, 29 Oct 2024 13:48:06 +0800
Subject: [PATCH 13/20] simplify CoreSpeData call by removing redundant buf
assignment
Removed unnecessary assignment to buf and directly operated on the provided buffer.
Signed-off-by: gubin <gubin_yewu@cmss.chinamobile.com>
---
pmu/spe.cpp | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/pmu/spe.cpp b/pmu/spe.cpp
index 99b7a89..5083ff0 100644
--- a/pmu/spe.cpp
+++ b/pmu/spe.cpp
@@ -529,7 +529,7 @@ int Spe::SpeReadData(struct SpeContext *context, struct SpeRecord *buf, int size
int remainSize = size;
int dummySize = context->dummyMmapSize;
CoreDummyData(context->coreCtxes, dummyData, dummySize, context->pageSize);
- buf = CoreSpeData(context->coreCtxes, dummyData, buf, &remainSize, context->pageSize, cpu);
+ CoreSpeData(context->coreCtxes, dummyData, buf, &remainSize, context->pageSize, cpu);
return size - remainSize;
}
--
2.43.0

101
0014-Update-Details.md.patch Normal file
View File

@ -0,0 +1,101 @@
From 6309460981655bd0899bc5ca65af123c2e42b638 Mon Sep 17 00:00:00 2001
From: glx <ganli2012@gmail.com>
Date: Tue, 29 Oct 2024 15:19:06 +0800
Subject: [PATCH 14/20] Update Details.md
---
docs/Details.md | 80 ++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 79 insertions(+), 1 deletion(-)
diff --git a/docs/Details.md b/docs/Details.md
index db0b407..14ca694 100644
--- a/docs/Details.md
+++ b/docs/Details.md
@@ -261,4 +261,82 @@ libkperf提供了采集子线程的能力。如果想要在上面场景中获取
attr.includeNewFork = 1;
```
然后，通过PmuRead获取到的PmuData，便能包含子线程计数信息了。
-注意，该功能是针对Counting模式，因为Sampling和SPE Sampling本身就会采集子线程的数据。
\ No newline at end of file
+注意，该功能是针对Counting模式，因为Sampling和SPE Sampling本身就会采集子线程的数据。
+
+### 采集DDRC带宽
+基于uncore事件可以计算DDRC的访存带宽，不同硬件平台有不同的计算方式。
+鲲鹏芯片上的访存带宽公式可以参考openeuler kernel的tools/perf/pmu-events/arch/arm64/hisilicon/hip09/sys/uncore-ddrc.json：
+```json
+ {
+ "MetricExpr": "flux_wr * 32 / duration_time",
+ "BriefDescription": "Average bandwidth of DDRC memory write(Byte/s)",
+ "Compat": "0x00000030",
+ "MetricGroup": "DDRC",
+ "MetricName": "ddrc_bw_write",
+ "Unit": "hisi_sccl,ddrc"
+ },
+ {
+ "MetricExpr": "flux_rd * 32 / duration_time",
+ "BriefDescription": "Average bandwidth of DDRC memory read(Byte/s)",
+ "Compat": "0x00000030",
+ "MetricGroup": "DDRC",
+ "MetricName": "ddrc_bw_read",
+ "Unit": "hisi_sccl,ddrc"
+ },
+```
+
+根据公式，采集flux_wr和flux_rd事件，用于计算带宽：
+```c++
+ // 采集hisi_scclX_ddrc设备下的flux_rd和flux_wr，
+ // 具体设备名称因硬件而异，可以在/sys/devices/下查询。
+ vector<char *> evts = {
+ "hisi_sccl1_ddrc/flux_rd/",
+ "hisi_sccl3_ddrc/flux_rd/",
+ "hisi_sccl5_ddrc/flux_rd/",
+ "hisi_sccl7_ddrc/flux_rd/",
+ "hisi_sccl1_ddrc/flux_wr/",
+ "hisi_sccl3_ddrc/flux_wr/",
+ "hisi_sccl5_ddrc/flux_wr/",
+ "hisi_sccl7_ddrc/flux_wr/"
+ };
+
+ PmuAttr attr = {0};
+ attr.evtList = evts.data();
+ attr.numEvt = evts.size();
+
+ int pd = PmuOpen(COUNTING, &attr);
+ if (pd == -1) {
+ cout << Perror() << "\n";
+ return;
+ }
+
+ PmuEnable(pd);
+ for (int i=0;i<60;++i) {
+ sleep(1);
+ PmuData *data = nullptr;
+ int len = PmuRead(pd, &data);
+ // 有8个uncore事件，所以data的长度等于8.
+ // 前4个是4个numa的read带宽，后4个是4个numa的write带宽。
+ for (int j=0;j<4;++j) {
+ printf("read bandwidth: %f M/s\n", (float)data[j].count*32/1024/1024);
+ }
+ for (int j=4;j<8;++j) {
+ printf("write bandwidth: %f M/s\n", (float)data[j].count*32/1024/1024);
+ }
+ PmuDataFree(data);
+ }
+ PmuDisable(pd);
+ PmuClose(pd);
+```
+
+执行上述代码，输出的结果类似如下：
+```
+read bandwidth: 17.32 M/s
+read bandwidth: 5.43 M/s
+read bandwidth: 2.83 M/s
+read bandwidth: 4.09 M/s
+write bandwidth: 4.35 M/s
+write bandwidth: 2.29 M/s
+write bandwidth: 0.84 M/s
+write bandwidth: 0.97 M/s
+```
--
2.43.0

28
0015-cleanup-remove-unused-variable-childPidList.patch Normal file
View File

@ -0,0 +1,28 @@
From 43baa672413baa5bd19bc95af710e5dc93cdd420 Mon Sep 17 00:00:00 2001
From: gubin <gubin_yewu@cmss.chinamobile.com>
Date: Tue, 29 Oct 2024 23:21:10 +0800
Subject: [PATCH 15/20] cleanup: remove unused variable childPidList
The variable 'childPidList' was declared but never used.
It has been removed to simplify the code and improve readability.
Signed-off-by: gubin <gubin_yewu@cmss.chinamobile.com>
---
pmu/dummy_event.cpp | 1 -
1 file changed, 1 deletion(-)
diff --git a/pmu/dummy_event.cpp b/pmu/dummy_event.cpp
index a058a24..723a753 100644
--- a/pmu/dummy_event.cpp
+++ b/pmu/dummy_event.cpp
@@ -140,7 +140,6 @@ namespace KUNPENG_PMU {
uint8_t* ringBuf = (uint8_t*) (mapPage) + PAGE_SIZE;
uint64_t dataHead = mapPage->data_head;
uint64_t dataTail = mapPage->data_tail;
- std::vector<pid_t> childPidList;
while (dataTail < dataHead) {
uint64_t off = dataTail % mapPage->data_size;
auto* header = (struct perf_event_header*) (ringBuf + off);
--
2.43.0

35
0016-update-include-pmu.h.patch Normal file
View File

@ -0,0 +1,35 @@
From c1c61ed3b437469c6509ac73a02b6f7790ba77d3 Mon Sep 17 00:00:00 2001
From: xuyongliang_01 <xuyongliang_yewu@cmss.chinamobile.com>
Date: Wed, 30 Oct 2024 09:48:57 +0000
Subject: [PATCH 16/20] update include/pmu.h.
Signed-off-by: xuyongliang_01 <xuyongliang_yewu@cmss.chinamobile.com>
---
include/pmu.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/include/pmu.h b/include/pmu.h
index 1e51802..d0b7e43 100644
--- a/include/pmu.h
+++ b/include/pmu.h
@@ -96,7 +96,7 @@ struct PmuAttr {
// event group id
// if not use event group function, this field will be nullptr.
- // if use event group function. please confrim the event group id with eveList is one by one.
+ // if use event group function. please confirm the event group id with eveList is one by one.
// the same group id is the a event group.
// Note: if the group id value is -1, it indicates that the event is not grouped.
struct EvtAttr *evtAttr;
@@ -266,7 +266,7 @@ void PmuStop(int pd);
* If <pmuData> is NULL and the error code is not 0, an error occurs in the collection process and data cannot be read.
* @param pd task id
* @param pmuData pmu data which is a pointer to an array
- * @return lenght of pmu data
+ * @return length of pmu data
*/
int PmuRead(int pd, struct PmuData** pmuData);
--
2.43.0

38
0017-Delete-unused-import-packages-and-fix-spelling-mista.patch Normal file
View File

@ -0,0 +1,38 @@
From 11a153b1d67139d2fb38cc43f818d58688b3fa97 Mon Sep 17 00:00:00 2001
From: Susanooo <zhangchujun_yewu@cmss.chinamobile.com>
Date: Wed, 6 Nov 2024 05:47:55 +0000
Subject: [PATCH 17/20] Delete unused import packages and fix spelling mistake
Signed-off-by: Susanooo <zhangchujun_yewu@cmss.chinamobile.com>
---
python/modules/ksym/symbol.py | 2 +-
python/tests/test_api.py | 3 ---
2 files changed, 1 insertion(+), 4 deletions(-)
diff --git a/python/modules/ksym/symbol.py b/python/modules/ksym/symbol.py
index f47b057..0e7099e 100644
--- a/python/modules/ksym/symbol.py
+++ b/python/modules/ksym/symbol.py
@@ -70,7 +70,7 @@ def record_module(pid: int, dwarf: bool = True) -> None:
def get_stack(pid: int, stacks: List[int]) -> Iterator[Stack]:
"""
- Convert a callstack to a unsigned long long hashid
+ Convert a callstack to an unsigned long long hashid
"""
return _libkperf.StackToHash(pid, stacks)
diff --git a/python/tests/test_api.py b/python/tests/test_api.py
index 7e32437..467700a 100644
--- a/python/tests/test_api.py
+++ b/python/tests/test_api.py
@@ -1,6 +1,3 @@
-import time
-from collections import defaultdict
-
import kperf
def TestAPI_InitCountNullEvt():
--
2.43.0

30
0018-Fixed-memory-allocation-check-in-symbol_resolve.cpp-.patch Normal file
View File

@ -0,0 +1,30 @@
From e228d064beed56b2a96b8f991e439df39922dc20 Mon Sep 17 00:00:00 2001
From: Hou haole <houhaole_yewu@cmss.chinamobile.com>
Date: Wed, 6 Nov 2024 17:10:56 +0800
Subject: [PATCH 18/20] Fixed memory allocation check in symbol_resolve.cpp
before calling memset
---
symbol/symbol_resolve.cpp | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/symbol/symbol_resolve.cpp b/symbol/symbol_resolve.cpp
index 3a54142..7bca356 100644
--- a/symbol/symbol_resolve.cpp
+++ b/symbol/symbol_resolve.cpp
@@ -85,10 +85,11 @@ namespace {
static inline char* InitChar(int len)
{
char* str = new char[len + 1];
- memset(str, 0, len + 1);
+
if (str == nullptr) {
return nullptr;
}
+ memset(str, 0, len + 1);
return str;
}
--
2.43.0

30
0019-update-python-modules-kperf-pmu.py.patch Normal file
View File

@ -0,0 +1,30 @@
From 6fd43f900a661ef8e3beed9976fe70cd4fba69fc Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=E8=8F=A0=E8=90=9D=E6=9C=89=E7=82=B9=E9=85=B8?=
<yanxiaoqiang_yewu@cmss.chinamobile.com>
Date: Fri, 8 Nov 2024 01:52:59 +0000
Subject: [PATCH 19/20] update python/modules/kperf/pmu.py.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Signed-off-by: 菠萝有点酸 <yanxiaoqiang_yewu@cmss.chinamobile.com>
---
python/modules/kperf/pmu.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/python/modules/kperf/pmu.py b/python/modules/kperf/pmu.py
index 10cd960..d74fa8d 100644
--- a/python/modules/kperf/pmu.py
+++ b/python/modules/kperf/pmu.py
@@ -95,7 +95,7 @@ class PmuAttr(_libkperf.PmuAttr):
if both <cpuList> and <pidList> are not NULL, specified processes on specified cores will be monitored.
evtAttr: event group id attributes.
if not use event group function, this field will be NULL.
- if use event group function. please confrim the event group id with eveList is one by one.
+ if use event group function. please confirm the event group id with eveList is one by one.
the same group id is the a event group.
Note: if the group id value is -1, it indicates that the event is not grouped.
sampleRate: sample time enum.
--
2.43.0

30
0020-update-pmu-spe.h.patch Normal file
View File

@ -0,0 +1,30 @@
From 65944528b098c9093124fd06b186260f98448c24 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=E8=8F=A0=E8=90=9D=E6=9C=89=E7=82=B9=E9=85=B8?=
<yanxiaoqiang_yewu@cmss.chinamobile.com>
Date: Fri, 8 Nov 2024 01:53:23 +0000
Subject: [PATCH 20/20] update pmu/spe.h.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Signed-off-by: 菠萝有点酸 <yanxiaoqiang_yewu@cmss.chinamobile.com>
---
pmu/spe.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/pmu/spe.h b/pmu/spe.h
index fc937d1..f2b833a 100644
--- a/pmu/spe.h
+++ b/pmu/spe.h
@@ -150,7 +150,7 @@ public:
/**
* @brief Start collect.
- * @param clearPrevRecords whether clear all records from previos collection.
+ * @param clearPrevRecords whether clear all records from previous collection.
*/
int Enable(bool clearPrevRecords = true);
--
2.43.0

BIN
libkperf-1.1.tar.gz
View File

Binary file not shown.

BIN
libkperf-v1.2.tar.gz Normal file
View File

Binary file not shown.

34
libkperf.spec
View File

@ -1,10 +1,30 @@
Name: libkperf Name: libkperf
Version: 1.1 Version: v1.2
Release: 1 Release: 2
Summary: Accelerated perf acquisition and symbol resolution Summary: Accelerated perf acquisition and symbol resolution
License: Mulan PSL v2 License: Mulan PSL v2
URL: https://gitee.com/openeuler/libkperf URL: https://gitee.com/openeuler/libkperf
Source0: %{name}-%{version}.tar.gz Source0: %{name}-%{version}.tar.gz
Patch1: 0001-update-python-modules-_libkperf-Symbol.py.patch
Patch2: 0002-update-docs.patch
Patch3: 0003-fix-after-pmuopen-delete-evtList-exception.patch
Patch4: 0004-Delete-the-repeated-judgement-on-formatPath.patch
Patch5: 0005-pmu-event.patch
Patch6: 0006-adapt-events.patch
Patch7: 0007-update-python-modules-_libkperf-Pmu.py.patch
Patch8: 0008-.patch
Patch9: 0009-adapt-gcc-4.8.5.patch
Patch10: 0010-fix-spelling-error.patch
Patch11: 0011-add-perrno-operation-not-supported.patch
Patch12: 0012-UT-SpeProcCollectSubProc.patch
Patch13: 0013-simplify-CoreSpeData-call-by-removing-redundant-buf-.patch
Patch14: 0014-Update-Details.md.patch
Patch15: 0015-cleanup-remove-unused-variable-childPidList.patch
Patch16: 0016-update-include-pmu.h.patch
Patch17: 0017-Delete-unused-import-packages-and-fix-spelling-mista.patch
Patch18: 0018-Fixed-memory-allocation-check-in-symbol_resolve.cpp-.patch
Patch19: 0019-update-python-modules-kperf-pmu.py.patch
Patch20: 0020-update-pmu-spe.h.patch
BuildRequires: gcc-c++ make cmake numactl-devel BuildRequires: gcc-c++ make cmake numactl-devel
ExclusiveArch: aarch64 ExclusiveArch: aarch64
%description %description
@ -12,7 +32,7 @@ Implement a low overhead pmu collection library, providing abstract interfaces f
%package devel %package devel
Summary: Development package for building Applications that use libkperf Summary: Development package for building Applications that use libkperf
License: Mulan PSL v2 License: Mulan PSL v2
Requires: %{name} = %{version}-%{release} Requires: %{name} = %{version}-%{release}
%description devel %description devel
@ -20,6 +40,7 @@ Development package for libkperf library calls
%prep %prep
%setup -q %setup -q
%autosetup -n %{name}-%{version} -p1
wget https://gitee.com/openeuler/elfin-parser/repository/archive/v1.0-h1.tar.gz wget https://gitee.com/openeuler/elfin-parser/repository/archive/v1.0-h1.tar.gz
tar -xvf v1.0-h1.tar.gz tar -xvf v1.0-h1.tar.gz
cp -r elfin-parser-v1.0-h1/* third_party/elfin-parser/ cp -r elfin-parser-v1.0-h1/* third_party/elfin-parser/
@ -68,5 +89,12 @@ mv %{buildroot}/include/pcerrc.h %{buildroot}/%{_includedir}/libkperf/
%{_libdir}/libsym.so %attr(755,root,root) %{_libdir}/libsym.so %attr(755,root,root)
%changelog %changelog
* Tue Nov 19 2024 echo <2220386943@qq.com> - 1.2-2
- update version to 1.2-2
- support new create thread
- description of supplementary documents
- support event grouping
- add the mangleName in symbol.h
* Fri Aug 9 2024 echo <2220386943@qq.com> - 1.1-1 * Fri Aug 9 2024 echo <2220386943@qq.com> - 1.1-1
- Package init - Package init