Compare commits

...

11 Commits

Author SHA1 Message Date
openeuler-ci-bot
8dd8d01040
!116 [sync] PR-114: Fix congest type flags error and replace a corrupt patch
From: @openeuler-sync-bot 
Reviewed-by: @hellotcc 
Signed-off-by: @hellotcc
2023-12-13 01:32:09 +00:00
Ran Zhou
3d6c469d72 Fix congest type flags error and replace a corrupt patch
Currently, there is a repeated judgement in check_qp_congest_type
whenever enable LDCP or HC3, the congest type flags all will be set
on LDCP.

This patch fixes this bug and replace a corrupt patch--0077, which
has a change that directly acts on patch but not code. This act
will disrupt the patch format.

Signed-off-by: Ran Zhou <zhouran10@h-partners.com>
(cherry picked from commit 4a7be9bb67fa9e61cc21625e448369783982e2b7)
2023-12-12 21:07:05 +08:00
openeuler-ci-bot
da46b98cb5
!111 Fix missing DB when compiler does not support SVE
From: @zzry 
Reviewed-by: @li-yangyang20 
Signed-off-by: @li-yangyang20
2023-12-08 09:22:13 +00:00
Ran Zhou
e5fcbc2552 Fix missing DB when compiler does not support SVE
Currently, if compiler does not support SVE, hns_roce_sve_write512() will
be a empty function, which means that this doorbell will be missed when
HNS_ROCE_QP_CAP_SVE_DIRECT_WQE is set in qp flag.

This patch ensures that driver will at least generate the DB regardless
of whether SVE DWQE is supported or not.

Signed-off-by: Chengchang Tang <tangchengchang@huawei.com
Signed-off-by: Ran Zhou <zhouran10@h-partners.com>
2023-12-08 10:00:51 +08:00
openeuler-ci-bot
a3fb4e5760
!108 Bugfix for lock and owner bit
From: @zzry 
Reviewed-by: @hellotcc 
Signed-off-by: @hellotcc
2023-12-07 07:35:25 +00:00
Ran Zhou
794f3792a7 Bugfix for lock and owner bit
Correct the return of error code, add init of pthread spinlock and mutex
judgement, remove a repeated init of pthread lock init, fix owner bit
when SQ wrqps.

Signed-off-by: Wenpeng Liang <liangwenpeng@huawei.com>
Signed-off-by: Chengchang Tang <tangchengchang@huawei.com
Signed-off-by: Ran Zhou <zhouran10@h-partners.com>
2023-12-07 11:09:09 +08:00
openeuler-ci-bot
a31fe2cdc1
!105 Bugfix for wrong timing of modifying ibv_qp state to err
From: @zzry 
Reviewed-by: @hellotcc 
Signed-off-by: @hellotcc
2023-12-01 10:34:53 +00:00
Ran Zhou
e221c2d5c6 Bugfix for wrong timing of modifying ibv_qp state to err
Currently the QPC state in HW is modified inside the critical section of
spinlock but the ibv_qp state is modified outside. There will be a short
period when QPC state has been modified to err with ibv_qp state still
remaining RTS. WQEs during this period will still be post-send by RTS-state
ibv_qp but then dropped by err-state HW with no flush CQEs generated.

To fix this problem, the QPC state in HW and ibv_qp state should be both
modified to err inside the critical section of spinlock.

Signed-off-by: Ran Zhou <zhouran10@h-partners.com>
Signed-off-by: Yangyang Li <liyangyang20@huawei.com>
2023-12-01 17:55:35 +08:00
openeuler-ci-bot
1fc2c147f3
!102 Corrects several minor issues found in review
From: @zzry 
Reviewed-by: @hellotcc 
Signed-off-by: @hellotcc
2023-11-28 02:50:18 +00:00
Ran Zhou
9185253876 Corrects several minor issues found in review
The issues mainly lies in the memory empty check, variable range
inconsistency, parameter verification, and print format.

Signed-off-by: Luoyouming <luoyouming@huawei.com>
Signed-off-by: Chengchang Tang <tangchengchang@huawei.com
Signed-off-by: Ran Zhou <zhouran10@h-partners.com>
2023-11-27 17:17:32 +08:00
openeuler-ci-bot
d662cb8742
!99 libhns: Get dmac from kernel driver
From: @stinft 
Reviewed-by: @hellotcc 
Signed-off-by: @hellotcc
2023-11-23 03:54:37 +00:00
14 changed files with 898 additions and 19 deletions

View File

@ -1,8 +1,7 @@
From 1b3ec79e4d617555c0e12a3fe12db7925d385ef6 Mon Sep 17 00:00:00 2001
From 99e1e64edab954ce1895d83a3d6f4317bc12c444 Mon Sep 17 00:00:00 2001
From: Yixing Liu <liuyixing1@huawei.com>
Date: Wed, 12 Apr 2023 17:01:09 +0800
Subject: [PATCH 2/2] libhns: Support congestion control algorithm
configuration
Subject: [PATCH] libhns: Support congestion control algorithm configuration
driver inclusion
category: feature
@ -67,7 +66,7 @@ index 76c7adb..41e9599 100644
struct verbs_context ibv_ctx;
void *uar;
diff --git a/providers/hns/hns_roce_u_verbs.c b/providers/hns/hns_roce_u_verbs.c
index 282ab74..6c6120c 100644
index 282ab74..499735c 100644
--- a/providers/hns/hns_roce_u_verbs.c
+++ b/providers/hns/hns_roce_u_verbs.c
@@ -89,10 +89,10 @@ int hns_roce_u_query_device(struct ibv_context *context,
@ -140,7 +139,7 @@ index 282ab74..6c6120c 100644
+ cmd_flag->congest_type_flags |= HNS_ROCE_CREATE_QP_FLAGS_LDCP;
+ break;
+ case HNSDV_QP_CREATE_ENABLE_HC3:
+ cmd_flag->congest_type_flags |= HNS_ROCE_CREATE_QP_FLAGS_LDCP;
+ cmd_flag->congest_type_flags |= HNS_ROCE_CREATE_QP_FLAGS_HC3;
+ break;
+ case HNSDV_QP_CREATE_ENABLE_DIP:
+ cmd_flag->congest_type_flags |= HNS_ROCE_CREATE_QP_FLAGS_DIP;

View File

@ -1,7 +1,7 @@
From 6852a8b49a1af09fc41b7b5432d9487323d0f394 Mon Sep 17 00:00:00 2001
From 64933c92842d34190c8cda9a864505d0558d3f5f Mon Sep 17 00:00:00 2001
From: Chengchang Tang <tangchengchang@huawei.com>
Date: Mon, 30 Oct 2023 16:59:22 +0800
Subject: [PATCH 8/8] libhns: Support write with notify
Subject: [PATCH] libhns: Support write with notify
driver inclusion
category: feature
@ -140,28 +140,24 @@ index 2ad9ea0..5e46f89 100644
}
diff --git a/providers/hns/hnsdv.h b/providers/hns/hnsdv.h
index 98c68fd..792a23e 100644
index 98c68fd..7d95db4 100644
--- a/providers/hns/hnsdv.h
+++ b/providers/hns/hnsdv.h
@@ -96,17 +96,28 @@ int hnsdv_query_device(struct ibv_context *ctx_in,
enum hnsdv_cq_init_attr_mask {
HNSDV_CQ_INIT_ATTR_MASK_CREATE_FLAGS = 1 << 0,
+ HNSDV_CQ_CREATE_ENABLE_NOTIFY = 1 << 1,
};
@@ -100,13 +100,24 @@ enum hnsdv_cq_init_attr_mask {
enum hnsdv_cq_create_flags {
HNSDV_CQ_CREATE_ENABLE_POE_MODE = 1 << 0,
};
+ HNSDV_CQ_CREATE_ENABLE_NOTIFY = 1 << 1,
+};
+
+enum hnsdv_cq_notify_mode {
+ HNSDV_CQ_NOTIFY_MODE_64B_ALIGN = 0,
+ HNSDV_CQ_NOTIFY_MODE_4B_ALIGN = 1,
+ HNSDV_CQ_NOTIFY_MODE_DDR_64B_ALIGN = 2,
+ HNSDV_CQ_NOTIFY_MODE_DDR_4B_ALIGN = 3,
+ HNSDV_CQ_NOTIFY_MODE_GUARD = 4, /* Invalid for user */
+};
+
};
struct hnsdv_cq_init_attr {
uint64_t comp_mask; /* Use enum hnsdv_cq_init_attr_mask */
uint64_t create_flags; /* Use enum hnsdv_cq_create_flags */

View File

@ -0,0 +1,88 @@
From 95f3cc1f25c091f97aefceac268fe6435d0861c3 Mon Sep 17 00:00:00 2001
From: Ran Zhou <zhouran10@partner.com>
Date: Mon, 27 Nov 2023 16:39:48 +0800
Subject: [PATCH 71/75] libhns: Corrects several issues with output format and
variable types.
driver inclusion
category: bugfix
bugzilla: https://gitee.com/src-openeuler/rdma-core/issues/I8J2S3?from=project-issue
--------------------------------------------------------------------------
1.Unify the types of two variables to avoid infinite loop.
2.Standardization the output control character.
Signed-off-by: Luoyouming <luoyouming@huawei.com>
---
providers/hns/hns_roce_u.c | 2 +-
providers/hns/hns_roce_u_hw_v2.c | 10 +++++-----
2 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/providers/hns/hns_roce_u.c b/providers/hns/hns_roce_u.c
index 084385b..f30486f 100644
--- a/providers/hns/hns_roce_u.c
+++ b/providers/hns/hns_roce_u.c
@@ -179,7 +179,7 @@ static void set_dca_pool_param(struct hns_roce_context *ctx,
dca_ctx->min_size = HNS_DCA_MAX_MEM_SIZE;
verbs_debug(&ctx->ibv_ctx,
- "Support DCA, unit %d, max %ld, min %ld Bytes.\n",
+ "Support DCA, unit %u, max %lu, min %lu Bytes.\n",
dca_ctx->unit_size, dca_ctx->max_size, dca_ctx->min_size);
}
diff --git a/providers/hns/hns_roce_u_hw_v2.c b/providers/hns/hns_roce_u_hw_v2.c
index 495fbcb..754f918 100644
--- a/providers/hns/hns_roce_u_hw_v2.c
+++ b/providers/hns/hns_roce_u_hw_v2.c
@@ -2680,8 +2680,8 @@ static void set_inline_data_list_rc(struct hns_roce_qp *qp,
{
unsigned int msg_len = qp->sge_info.total_len;
void *dseg;
+ size_t i;
int ret;
- int i;
hr_reg_enable(wqe, RCWQE_INLINE);
@@ -2741,7 +2741,7 @@ static void wr_set_inline_data_list_rc(struct ibv_qp_ex *ibv_qp, size_t num_buf,
{
struct hns_roce_qp *qp = to_hr_qp(&ibv_qp->qp_base);
struct hns_roce_rc_sq_wqe *wqe = qp->cur_wqe;
- int i;
+ size_t i;
if (!wqe)
return;
@@ -2872,7 +2872,7 @@ static void wr_set_sge_list_ud(struct ibv_qp_ex *ibv_qp, size_t num_sge,
}
hr_reg_write(wqe, UDWQE_MSG_START_SGE_IDX, sge_idx & mask);
- for (int i = 0; i < num_sge; i++) {
+ for (size_t i = 0; i < num_sge; i++) {
if (!sg_list[i].length)
continue;
@@ -2899,8 +2899,8 @@ static void set_inline_data_list_ud(struct hns_roce_qp *qp,
uint8_t data[HNS_ROCE_MAX_UD_INL_INN_SZ] = {};
unsigned int msg_len = qp->sge_info.total_len;
void *tmp;
+ size_t i;
int ret;
- int i;
if (!check_inl_data_len(qp, msg_len)) {
qp->err = EINVAL;
@@ -2962,7 +2962,7 @@ static void wr_set_inline_data_list_ud(struct ibv_qp_ex *ibv_qp, size_t num_buf,
{
struct hns_roce_qp *qp = to_hr_qp(&ibv_qp->qp_base);
struct hns_roce_ud_sq_wqe *wqe = qp->cur_wqe;
- int i;
+ size_t i;
if (!wqe)
return;
--
2.25.1

View File

@ -0,0 +1,45 @@
From 884663cd803e3781aec9c45f7fc0145b6a2b4539 Mon Sep 17 00:00:00 2001
From: Luoyouming <luoyouming@huawei.com>
Date: Mon, 27 Nov 2023 16:39:49 +0800
Subject: [PATCH 72/75] libhns: Add input parameter check for
hnsdv_query_device()
driver inclusion
category: cleanup
bugzilla: https://gitee.com/src-openeuler/rdma-core/issues/I8J2W9?from=project-issue
--------------------------------------------------------------------------
Add null pointer check for pointer parameter.
Signed-off-by: Luoyouming <luoyouming@huawei.com>
---
providers/hns/hns_roce_u_verbs.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/providers/hns/hns_roce_u_verbs.c b/providers/hns/hns_roce_u_verbs.c
index c906632..e597e93 100644
--- a/providers/hns/hns_roce_u_verbs.c
+++ b/providers/hns/hns_roce_u_verbs.c
@@ -126,15 +126,16 @@ int hnsdv_query_device(struct ibv_context *context,
struct hnsdv_context *attrs_out)
{
struct hns_roce_context *ctx = context ? to_hr_ctx(context) : NULL;
- struct hns_roce_device *hr_dev = to_hr_dev(context->device);
+ struct hns_roce_device *hr_dev;
if (!ctx || !attrs_out)
return EINVAL;
- if (!hr_dev) {
+ if (!context->device && !is_hns_dev(context->device)) {
verbs_err(verbs_get_ctx(context), "not a HNS RoCE device!\n");
return EOPNOTSUPP;
}
+ hr_dev = to_hr_dev(context->device);
memset(attrs_out, 0, sizeof(*attrs_out));
--
2.25.1

View File

@ -0,0 +1,63 @@
From 57499cbc906ec4065f65bb8dfda56081a799746e Mon Sep 17 00:00:00 2001
From: Chengchang Tang <tangchengchang@huawei.com>
Date: Mon, 27 Nov 2023 16:39:51 +0800
Subject: [PATCH 73/75] libhns: Fix uninitialized qp attr when flush cqe
driver inclusion
category: bugfix
bugzilla: https://gitee.com/src-openeuler/rdma-core/issues/I8J2XW?from=project-issue
--------------------------------------------------------------------------
When flushing cqe, the state of qp needs to be modified. The incoming qp
attr is not initialized, which may lead to undefined behavior.
Signed-off-by: Chengchang Tang <tangchengchang@huawei.com>
---
providers/hns/hns_roce_u_hw_v2.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/providers/hns/hns_roce_u_hw_v2.c b/providers/hns/hns_roce_u_hw_v2.c
index 754f918..b48cabd 100644
--- a/providers/hns/hns_roce_u_hw_v2.c
+++ b/providers/hns/hns_roce_u_hw_v2.c
@@ -405,7 +405,7 @@ static int hns_roce_u_v2_modify_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr,
static int hns_roce_flush_cqe(struct hns_roce_qp *hr_qp, uint8_t status)
{
- struct ibv_qp_attr attr;
+ struct ibv_qp_attr attr = {};
int attr_mask;
if (status != HNS_ROCE_V2_CQE_WR_FLUSH_ERR) {
@@ -1614,8 +1614,8 @@ int hns_roce_u_v2_post_send(struct ibv_qp *ibvqp, struct ibv_send_wr *wr,
struct hns_roce_qp *qp = to_hr_qp(ibvqp);
struct hns_roce_sge_info sge_info = {};
struct hns_roce_rc_sq_wqe *wqe;
+ struct ibv_qp_attr attr = {};
unsigned int wqe_idx, nreq;
- struct ibv_qp_attr attr;
int ret;
ret = check_qp_send(qp, ctx);
@@ -1788,7 +1788,7 @@ static int hns_roce_u_v2_post_recv(struct ibv_qp *ibvqp, struct ibv_recv_wr *wr,
struct hns_roce_context *ctx = to_hr_ctx(ibvqp->context);
struct hns_roce_qp *qp = to_hr_qp(ibvqp);
unsigned int wqe_idx, nreq, max_sge;
- struct ibv_qp_attr attr;
+ struct ibv_qp_attr attr = {};
int ret;
ret = check_qp_recv(qp, ctx);
@@ -3021,7 +3021,7 @@ static int wr_complete(struct ibv_qp_ex *ibv_qp)
struct hns_roce_context *ctx = to_hr_ctx(ibv_qp->qp_base.context);
struct hns_roce_qp *qp = to_hr_qp(&ibv_qp->qp_base);
unsigned int nreq = qp->sq.head - qp->rb_sq_head;
- struct ibv_qp_attr attr;
+ struct ibv_qp_attr attr = {};
int err = qp->err;
if (err) {
--
2.25.1

View File

@ -0,0 +1,64 @@
From 9e3f4aa0a83ea0ff9512678e3932e611186d573e Mon Sep 17 00:00:00 2001
From: Chengchang Tang <tangchengchang@huawei.com>
Date: Mon, 27 Nov 2023 16:39:50 +0800
Subject: [PATCH 74/75] libhns: Fix possible overflow in cq clean
driver inclusion
category: bugfix
bugzilla: https://gitee.com/src-openeuler/rdma-core/issues/I8J2XP?from=project-issue
--------------------------------------------------------------------------
The ci/pi of hns roce cq allows data to be flipped. but in
__hns_roce_v2_cq_clean(), this flip may lead to an wrong number
of loops.
This patch fixes it by extending the data type to avoid data
flipping.
Signed-off-by: Chengchang Tang <tangchengchang@huawei.com>
---
providers/hns/hns_roce_u_hw_v2.c | 25 +++++++++++++------------
1 file changed, 13 insertions(+), 12 deletions(-)
diff --git a/providers/hns/hns_roce_u_hw_v2.c b/providers/hns/hns_roce_u_hw_v2.c
index b48cabd..fc938de 100644
--- a/providers/hns/hns_roce_u_hw_v2.c
+++ b/providers/hns/hns_roce_u_hw_v2.c
@@ -1847,20 +1847,21 @@ out:
static void __hns_roce_v2_cq_clean(struct hns_roce_cq *cq, uint32_t qpn,
struct hns_roce_srq *srq)
{
- int nfreed = 0;
- bool is_recv_cqe;
- uint8_t owner_bit;
- uint16_t wqe_index;
- uint32_t prod_index;
- struct hns_roce_v2_cqe *cqe, *dest;
- struct hns_roce_context *ctx = to_hr_ctx(cq->verbs_cq.cq.context);
-
- for (prod_index = cq->cons_index; get_sw_cqe_v2(cq, prod_index);
- ++prod_index)
- if (prod_index > cq->cons_index + cq->verbs_cq.cq.cqe)
+ struct hns_roce_context *ctx = to_hr_ctx(cq->verbs_cq.cq.context);
+ uint64_t cons_index = cq->cons_index;
+ uint64_t prod_index = cq->cons_index;
+ struct hns_roce_v2_cqe *cqe, *dest;
+ uint16_t wqe_index;
+ uint8_t owner_bit;
+ bool is_recv_cqe;
+ int nfreed = 0;
+
+ for (; get_sw_cqe_v2(cq, prod_index); ++prod_index)
+ if (prod_index > cons_index + cq->verbs_cq.cq.cqe)
break;
- while ((int) --prod_index - (int) cq->cons_index >= 0) {
+ while (prod_index - cons_index > 0) {
+ prod_index--;
cqe = get_cqe_v2(cq, prod_index & cq->verbs_cq.cq.cqe);
if (hr_reg_read(cqe, CQE_LCL_QPN) == qpn) {
is_recv_cqe = hr_reg_read(cqe, CQE_S_R);
--
2.25.1

View File

@ -0,0 +1,36 @@
From c989f93b01c45f8453bc839722f91ddc8b000037 Mon Sep 17 00:00:00 2001
From: Chengchang Tang <tangchengchang@huawei.com>
Date: Mon, 27 Nov 2023 16:39:52 +0800
Subject: [PATCH 75/75] libhns: Fix unnecessary dca memory detach
driver inclusion
category: bugfix
bugzilla: https://gitee.com/src-openeuler/rdma-core/issues/I8J2Y5?from=project-issue
--------------------------------------------------------------------------
If sq is not enabled, it should not detach dca mem. Moreover, under the
current code logic, if detach dca mem, its sq index will be a random
value because it is not initialized.
Signed-off-by: Chengchang Tang <tangchengchang@huawei.com>
---
providers/hns/hns_roce_u_hw_v2.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/providers/hns/hns_roce_u_hw_v2.c b/providers/hns/hns_roce_u_hw_v2.c
index fc938de..2fb738d 100644
--- a/providers/hns/hns_roce_u_hw_v2.c
+++ b/providers/hns/hns_roce_u_hw_v2.c
@@ -696,7 +696,7 @@ static void dca_detach_qp_buf(struct hns_roce_context *ctx,
hns_roce_spin_unlock(&qp->rq.hr_lock);
hns_roce_spin_unlock(&qp->sq.hr_lock);
- if (is_empty)
+ if (is_empty && qp->sq.wqe_cnt > 0)
hns_roce_detach_dca_mem(ctx, qp->verbs_qp.qp.handle, &attr);
}
--
2.25.1

View File

@ -0,0 +1,43 @@
From 324cd24a22256d964689bf528b643ae06d5a4e58 Mon Sep 17 00:00:00 2001
From: Yangyang Li <liyangyang20@huawei.com>
Date: Fri, 1 Dec 2023 10:43:23 +0800
Subject: [PATCH] libhns: Bugfix for wrong timing of modifying ibv_qp state to
err
driver inclusion
category: bugfix
bugzilla: https://gitee.com/src-openeuler/rdma-core/issues/I8L4YU
--------------------------------------------------------------------------
Currently the QPC state in HW is modified inside the critical section of
spinlock but the ibv_qp state is modified outside. There will be a short
period when QPC state has been modified to err with ibv_qp state still
remaining RTS. WQEs during this period will still be post-send by RTS-state
ibv_qp but then dropped by err-state HW with no flush CQEs generated.
To fix this problem, the QPC state in HW and ibv_qp state should be both
modified to err inside the critical section of spinlock.
Fixes: f1a80cc3dfe2 ("libhns: Bugfix for flush cqe in case multi-process")
Signed-off-by: Yangyang Li <liyangyang20@huawei.com>
---
providers/hns/hns_roce_u_hw_v2.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/providers/hns/hns_roce_u_hw_v2.c b/providers/hns/hns_roce_u_hw_v2.c
index 2fb738d..68d7110 100644
--- a/providers/hns/hns_roce_u_hw_v2.c
+++ b/providers/hns/hns_roce_u_hw_v2.c
@@ -1936,6 +1936,8 @@ static int hns_roce_u_v2_modify_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr,
sizeof(resp_ex));
if (flag) {
+ if (!ret)
+ qp->state = IBV_QPS_ERR;
hns_roce_spin_unlock(&hr_qp->sq.hr_lock);
hns_roce_spin_unlock(&hr_qp->rq.hr_lock);
}
--
2.25.1

View File

@ -0,0 +1,72 @@
From 12f8951a6a98f82dd1f70afc23d66e328cf1988b Mon Sep 17 00:00:00 2001
From: Chengchang Tang <tangchengchang@huawei.com>
Date: Thu, 7 Dec 2023 09:47:59 +0800
Subject: [PATCH] libhns: Fix parent domain unsupported comp mask
driver inclusion
category: bugfix
bugzilla: https://gitee.com/src-openeuler/rdma-core/issues/I8MF28
Hns do not support any comp mask for parent domain. Driver returns
EINVAL if any compmask is set.
This patch Replace the inappropriate return value EINVAL with
EOPNOTSUPP.
The error is found by testcase test_mem_align_ud_traffic.
ERROR: test_mem_align_ud_traffic (tests.test_parent_domain.ParentDomain
TrafficTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "./tests/test_parent_domain.py", line 183, in test_mem_align_ud
_traffic
self.create_players(parent_domain_ud_res,
File "./tests/test_parent_domain.py", line 156, in create_players
self.client = resource(**self.dev_info, **resource_arg)
File "./tests/test_parent_domain.py", line 90, in __init__
super().__init__(**kwargs)
File "./tests/base.py", line 617, in __init__
super(RoCETrafficResources, self).__init__(dev_name, ib_port,
gid_index, **kwargs)
File "./tests/base.py", line 503, in __init__
super(TrafficResources, self).__init__(dev_name=dev_name,
File "./tests/base.py", line 477, in __init__
self.create_pd()
File "./tests/test_parent_domain.py", line 95, in create_pd
create_parent_domain_with_allocators(self)
File "./tests/test_parent_domain.py", line 69, in create_parent_
domain_with_allocators
raise ex
File "./tests/test_parent_domain.py", line 65, in create_parent_
domain_with_allocators
res.pd = ParentDomain(res.ctx, attr=pd_attr)
File "pd.pyx", line 261, in pyverbs.pd.ParentDomain.__init__
pyverbs.pyverbs_error.PyverbsRDMAError: Failed to allocate Parent
Domain.Errno: 22, Invalid argument
Fixes: cfe548d4c78e ("libhns: Add support for the thread
domain and the parent domain")
Signed-off-by: Chengchang Tang <tangchengchang@huawei.com>
---
providers/hns/hns_roce_u_verbs.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/providers/hns/hns_roce_u_verbs.c b/providers/hns/hns_roce_u_verbs.c
index 34e8d07..ba3fef6 100644
--- a/providers/hns/hns_roce_u_verbs.c
+++ b/providers/hns/hns_roce_u_verbs.c
@@ -238,7 +238,7 @@ struct ibv_pd *hns_roce_u_alloc_pad(struct ibv_context *context,
return NULL;
if (attr->comp_mask) {
- errno = EINVAL;
+ errno = EOPNOTSUPP;
return NULL;
}
--
2.25.1

View File

@ -0,0 +1,216 @@
From 3a1432cfdb7c696d3acf97025e6d74bbf3e520dc Mon Sep 17 00:00:00 2001
From: Wenpeng Liang <liangwenpeng@huawei.com>
Date: Thu, 7 Dec 2023 09:48:00 +0800
Subject: [PATCH 78/80] libhns: Add
pthread_spin_destroy()/pthread_mutex_destroy()
driver inclusion
category: bugfix
bugzilla: https://gitee.com/src-openeuler/rdma-core/issues/I8MF59
--------------------------------------------------------------------------
The functions pthread_spin_destroy()/pthread_mutex_destroy()
corresponds to pthread_spin_init()/pthread_mutex_init(). The
driver should call pthread_spin_destroy()/pthread_mutex_destroy()
to clean up resources before exiting.
Signed-off-by: Wenpeng Liang <liangwenpeng@huawei.com>
---
providers/hns/hns_roce_u.c | 61 +++++++++++++++++++++++++++-----
providers/hns/hns_roce_u_hw_v2.c | 1 +
providers/hns/hns_roce_u_verbs.c | 17 ++++++---
3 files changed, 67 insertions(+), 12 deletions(-)
diff --git a/providers/hns/hns_roce_u.c b/providers/hns/hns_roce_u.c
index f30486f..dfcd798 100644
--- a/providers/hns/hns_roce_u.c
+++ b/providers/hns/hns_roce_u.c
@@ -346,6 +346,47 @@ static int query_dev_attr(struct hns_roce_context *context,
return 0;
}
+static int hns_roce_init_context_lock(struct hns_roce_context *context)
+{
+ int ret;
+
+ ret = pthread_spin_init(&context->uar_lock, PTHREAD_PROCESS_PRIVATE);
+ if (ret)
+ return ret;
+
+ ret = pthread_mutex_init(&context->qp_table_mutex, NULL);
+ if (ret)
+ goto destroy_uar_lock;
+
+ ret = pthread_mutex_init(&context->srq_table_mutex, NULL);
+ if (ret)
+ goto destroy_qp_mutex;
+
+ ret = pthread_mutex_init(&context->db_list_mutex, NULL);
+ if (ret)
+ goto destroy_srq_mutex;
+
+ return 0;
+
+destroy_srq_mutex:
+ pthread_mutex_destroy(&context->srq_table_mutex);
+
+destroy_qp_mutex:
+ pthread_mutex_destroy(&context->qp_table_mutex);
+
+destroy_uar_lock:
+ pthread_spin_destroy(&context->uar_lock);
+ return ret;
+}
+
+static void hns_roce_destroy_context_lock(struct hns_roce_context *context)
+{
+ pthread_spin_destroy(&context->uar_lock);
+ pthread_mutex_destroy(&context->qp_table_mutex);
+ pthread_mutex_destroy(&context->srq_table_mutex);
+ pthread_mutex_destroy(&context->db_list_mutex);
+}
+
static struct verbs_context *hns_roce_alloc_context(struct ibv_device *ibdev,
int cmd_fd,
void *private_data)
@@ -365,7 +406,10 @@ static struct verbs_context *hns_roce_alloc_context(struct ibv_device *ibdev,
ucontext_set_cmd(&cmd, ctx_attr);
if (ibv_cmd_get_context(&context->ibv_ctx, &cmd.ibv_cmd, sizeof(cmd),
&resp.ibv_resp, sizeof(resp)))
- goto err_free;
+ goto err_ibv_cmd;
+
+ if (hns_roce_init_context_lock(context))
+ goto err_ibv_cmd;
hr_dev->congest_type = resp.congest_type;
@@ -383,23 +427,23 @@ static struct verbs_context *hns_roce_alloc_context(struct ibv_device *ibdev,
context->qp_table_shift = calc_table_shift(resp.qp_tab_size,
HNS_ROCE_QP_TABLE_BITS);
context->qp_table_mask = (1 << context->qp_table_shift) - 1;
- pthread_mutex_init(&context->qp_table_mutex, NULL);
+
for (i = 0; i < HNS_ROCE_QP_TABLE_SIZE; ++i)
context->qp_table[i].refcnt = 0;
context->srq_table_shift = calc_table_shift(resp.srq_tab_size,
HNS_ROCE_SRQ_TABLE_BITS);
context->srq_table_mask = (1 << context->srq_table_shift) - 1;
- pthread_mutex_init(&context->srq_table_mutex, NULL);
+
for (i = 0; i < HNS_ROCE_SRQ_TABLE_SIZE; ++i)
context->srq_table[i].refcnt = 0;
if (query_dev_attr(context, hr_dev, &resp))
- goto err_free;
+ goto err_query_dev;
if (init_dca_context(context, cmd_fd,
&resp, ctx_attr, hr_dev->page_size))
- goto err_free;
+ goto err_query_dev;
if (init_reset_context(context, cmd_fd, &resp, hr_dev->page_size))
goto reset_free;
@@ -407,8 +451,6 @@ static struct verbs_context *hns_roce_alloc_context(struct ibv_device *ibdev,
if (hns_roce_mmap(hr_dev, context, cmd_fd))
goto uar_free;
- pthread_spin_init(&context->uar_lock, PTHREAD_PROCESS_PRIVATE);
-
verbs_set_ops(&context->ibv_ctx, &hns_common_ops);
verbs_set_ops(&context->ibv_ctx, &hr_dev->u_hw->hw_ops);
@@ -419,7 +461,9 @@ uar_free:
munmap(context->reset_state, hr_dev->page_size);
reset_free:
uninit_dca_context(context);
-err_free:
+err_query_dev:
+ hns_roce_destroy_context_lock(context);
+err_ibv_cmd:
verbs_uninit_context(&context->ibv_ctx);
free(context);
return NULL;
@@ -434,6 +478,7 @@ static void hns_roce_free_context(struct ibv_context *ibctx)
if (context->reset_state)
munmap(context->reset_state, hr_dev->page_size);
uninit_dca_context(context);
+ hns_roce_destroy_context_lock(context);
verbs_uninit_context(&context->ibv_ctx);
free(context);
}
diff --git a/providers/hns/hns_roce_u_hw_v2.c b/providers/hns/hns_roce_u_hw_v2.c
index 68d7110..b2a8858 100644
--- a/providers/hns/hns_roce_u_hw_v2.c
+++ b/providers/hns/hns_roce_u_hw_v2.c
@@ -2049,6 +2049,7 @@ static int hns_roce_u_v2_destroy_qp(struct ibv_qp *ibqp)
hns_roce_unlock_cqs(to_hr_cq(ibqp->send_cq), to_hr_cq(ibqp->recv_cq));
hns_roce_free_qp_buf(qp, ctx);
+ hns_roce_qp_spinlock_destroy(qp);
free(qp);
diff --git a/providers/hns/hns_roce_u_verbs.c b/providers/hns/hns_roce_u_verbs.c
index fae6126..ba3fef6 100644
--- a/providers/hns/hns_roce_u_verbs.c
+++ b/providers/hns/hns_roce_u_verbs.c
@@ -803,6 +803,7 @@ int hns_roce_u_modify_cq(struct ibv_cq *cq, struct ibv_modify_cq_attr *attr)
int hns_roce_u_destroy_cq(struct ibv_cq *cq)
{
+ struct hns_roce_cq *hr_cq = to_hr_cq(cq);
int ret;
ret = ibv_cmd_destroy_cq(cq);
@@ -811,10 +812,13 @@ int hns_roce_u_destroy_cq(struct ibv_cq *cq)
hns_roce_uninit_cq_swc(to_hr_cq(cq));
- hns_roce_free_db(to_hr_ctx(cq->context), to_hr_cq(cq)->db,
+ hns_roce_free_db(to_hr_ctx(cq->context), hr_cq->db,
HNS_ROCE_CQ_TYPE_DB);
- hns_roce_free_buf(&to_hr_cq(cq)->buf);
- free(to_hr_cq(cq));
+ hns_roce_free_buf(&hr_cq->buf);
+
+ hns_roce_spinlock_destroy(&hr_cq->hr_lock);
+
+ free(hr_cq);
return ret;
}
@@ -1071,7 +1075,7 @@ static struct ibv_srq *create_srq(struct ibv_context *context,
set_srq_param(context, srq, init_attr);
if (alloc_srq_buf(srq))
- goto err_free_srq;
+ goto err_destroy_lock;
srq->rdb = hns_roce_alloc_db(hr_ctx, HNS_ROCE_SRQ_TYPE_DB);
if (!srq->rdb)
@@ -1102,6 +1106,9 @@ err_srq_db:
err_srq_buf:
free_srq_buf(srq);
+err_destroy_lock:
+ hns_roce_spinlock_destroy(&srq->hr_lock);
+
err_free_srq:
free(srq);
@@ -1191,6 +1198,8 @@ int hns_roce_u_destroy_srq(struct ibv_srq *ibv_srq)
hns_roce_free_db(ctx, srq->rdb, HNS_ROCE_SRQ_TYPE_DB);
free_srq_buf(srq);
+
+ hns_roce_spinlock_destroy(&srq->hr_lock);
free(srq);
return 0;
--
2.25.1

View File

@ -0,0 +1,38 @@
From 8759b0e6ec4e73994743c1ae0d0ecc186688b6d6 Mon Sep 17 00:00:00 2001
From: Ran Zhou <zhouran10@h-partners.com>
Date: Thu, 7 Dec 2023 09:48:01 +0800
Subject: [PATCH 79/80] libhns: Removes a repeated initialization of a spinlock
driver inclusion
category: bugfix
bugzilla: https://gitee.com/src-openeuler/rdma-core/issues/I8MF83
--------------------------------------------------------------------------
The pthread_spin_init() of qp was done in create_qp(). We removed
the spinlock init in this place to avoid initializing twice.
Signed-off-by: Wenpeng Liang <liangwenpeng@huawei.com>
Signed-off-by: Ran Zhou <zhouran10@h-partners.com>
---
providers/hns/hns_roce_u_verbs.c | 4 ----
1 file changed, 4 deletions(-)
diff --git a/providers/hns/hns_roce_u_verbs.c b/providers/hns/hns_roce_u_verbs.c
index ba3fef6..c404948 100644
--- a/providers/hns/hns_roce_u_verbs.c
+++ b/providers/hns/hns_roce_u_verbs.c
@@ -1933,10 +1933,6 @@ static int hns_roce_alloc_qp_buf(struct ibv_qp_init_attr_ex *attr,
{
int ret;
- if (pthread_spin_init(&qp->sq.hr_lock.lock, PTHREAD_PROCESS_PRIVATE) ||
- pthread_spin_init(&qp->rq.hr_lock.lock, PTHREAD_PROCESS_PRIVATE))
- return -ENOMEM;
-
ret = qp_alloc_wqe(attr, hns_attr, qp, ctx);
if (ret)
return ret;
--
2.25.1

View File

@ -0,0 +1,95 @@
From 96d30f16bc03167c7c52e663785192382688f542 Mon Sep 17 00:00:00 2001
From: Chengchang Tang <tangchengchang@huawei.com>
Date: Thu, 7 Dec 2023 09:48:02 +0800
Subject: [PATCH 80/80] libhns: Fix owner bit when SQ wraps around in new IO
driver inclusion
category: bugfix
bugzilla: https://gitee.com/src-openeuler/rdma-core/issues/I8MF9Q
--------------------------------------------------------------------------
The owner bit has been write in init_rc_wqe() or init_ud_wqe()
with a write value. And it will be overwritten by some subsequent
operations. When the SQ wraps around, the overwritten value will be
an incorrect value.
For example, driver will assign the owner bit in the second step,
and overwrite it in the third step.
```c
ibv_wr_start();
ibv_wr_rdma_write();
if (inline)
ibv_wr_set_inline_data_list();
else
ibv_wr_set_sge_list();
ibv_wr_complete();
```
This patch removes the redundant owner bit assignment operations
in new IO.
Fixes: 36446a56eea5 ("libhns: Extended QP supports the new post send mechanism")
Fixes: 163d62ca6196 ("libhns: Fix the owner bit error of sq in new io")
Signed-off-by: Chengchang Tang <tangchengchang@huawei.com>
---
providers/hns/hns_roce_u_hw_v2.c | 7 -------
1 file changed, 7 deletions(-)
diff --git a/providers/hns/hns_roce_u_hw_v2.c b/providers/hns/hns_roce_u_hw_v2.c
index b2a8858..acbc854 100644
--- a/providers/hns/hns_roce_u_hw_v2.c
+++ b/providers/hns/hns_roce_u_hw_v2.c
@@ -2544,8 +2544,6 @@ static void wr_set_sge_list_rc(struct ibv_qp_ex *ibv_qp, size_t num_sge,
wqe->msg_len = htole32(qp->sge_info.total_len);
hr_reg_write(wqe, RCWQE_SGE_NUM, qp->sge_info.valid_num);
-
- enable_wqe(qp, wqe, qp->sq.head);
}
static void wr_send_rc(struct ibv_qp_ex *ibv_qp)
@@ -2737,7 +2735,6 @@ static void wr_set_inline_data_rc(struct ibv_qp_ex *ibv_qp, void *addr,
qp->sge_info.total_len = length;
set_inline_data_list_rc(qp, wqe, 1, &buff);
- enable_wqe(qp, wqe, qp->sq.head);
}
static void wr_set_inline_data_list_rc(struct ibv_qp_ex *ibv_qp, size_t num_buf,
@@ -2755,7 +2752,6 @@ static void wr_set_inline_data_list_rc(struct ibv_qp_ex *ibv_qp, size_t num_buf,
qp->sge_info.total_len += buf_list[i].length;
set_inline_data_list_rc(qp, wqe, num_buf, buf_list);
- enable_wqe(qp, wqe, qp->sq.head);
}
static struct hns_roce_ud_sq_wqe *
@@ -2892,7 +2888,6 @@ static void wr_set_sge_list_ud(struct ibv_qp_ex *ibv_qp, size_t num_sge,
hr_reg_write(wqe, UDWQE_SGE_NUM, cnt);
qp->sge_info.start_idx += cnt;
- enable_wqe(qp, wqe, qp->sq.head);
}
static void set_inline_data_list_ud(struct hns_roce_qp *qp,
@@ -2958,7 +2953,6 @@ static void wr_set_inline_data_ud(struct ibv_qp_ex *ibv_qp, void *addr,
qp->sge_info.total_len = length;
set_inline_data_list_ud(qp, wqe, 1, &buff);
- enable_wqe(qp, wqe, qp->sq.head);
}
static void wr_set_inline_data_list_ud(struct ibv_qp_ex *ibv_qp, size_t num_buf,
@@ -2976,7 +2970,6 @@ static void wr_set_inline_data_list_ud(struct ibv_qp_ex *ibv_qp, size_t num_buf,
qp->sge_info.total_len += buf_list[i].length;
set_inline_data_list_ud(qp, wqe, num_buf, buf_list);
- enable_wqe(qp, wqe, qp->sq.head);
}
static void wr_start(struct ibv_qp_ex *ibv_qp)
--
2.25.1

View File

@ -0,0 +1,83 @@
From cad30f3d98525d14796094b2905de222c894464f Mon Sep 17 00:00:00 2001
From: Chengchang Tang <tangchengchang@huawei.com>
Date: Fri, 8 Dec 2023 09:49:42 +0800
Subject: [PATCH] libhns: Fix missing DB when compiler does not support SVE
driver inclusion
category: bugfix
bugzilla: https://gitee.com/openeuler/kernel/issues/I8MPTX
----------------------------------------------------------------------
Currently, if compiler does not support SVE, hns_roce_sve_write512() will
be a empty function, which means that this doorbell will be missed when
HNS_ROCE_QP_CAP_SVE_DIRECT_WQE is set in qp flag.
This patch ensures that driver will at least generate the DB regardless
of whether SVE DWQE is supported or not.
Fixes: 7b1f5c5654c2 ("libhns: Add support for SVE Direct WQE function")
Signed-off-by: Chengchang Tang <tangchengchang@huawei.com>
Signed-off-by: Ran Zhou <zhouran10@h-partners.com>
---
providers/hns/hns_roce_u_hw_v2.c | 33 +++++++++++++-------------------
1 file changed, 13 insertions(+), 20 deletions(-)
diff --git a/providers/hns/hns_roce_u_hw_v2.c b/providers/hns/hns_roce_u_hw_v2.c
index acbc854..be4c9f3 100644
--- a/providers/hns/hns_roce_u_hw_v2.c
+++ b/providers/hns/hns_roce_u_hw_v2.c
@@ -318,26 +318,22 @@ static void hns_roce_update_sq_db(struct hns_roce_context *ctx,
hns_roce_write64(ctx, qp->sq.db_reg, (__le32 *)&sq_db);
}
-static void hns_roce_write512(uint64_t *dest, uint64_t *val)
+static void hns_roce_qp_write512(struct hns_roce_qp *qp, uint64_t *val)
{
- mmio_memcpy_x64(dest, val, sizeof(struct hns_roce_rc_sq_wqe));
-}
+ uint64_t *dest = qp->sq.db_reg;
#if defined(HNS_SVE)
-static void hns_roce_sve_write512(uint64_t *dest, uint64_t *val)
-{
- asm volatile(
- "ldr z0, [%0]\n"
- "str z0, [%1]\n"
- ::"r" (val), "r"(dest):"cc", "memory"
- );
-}
-#else
-static void hns_roce_sve_write512(uint64_t *dest, uint64_t *val)
-{
- return;
-}
+ if (qp->flags & HNS_ROCE_QP_CAP_SVE_DIRECT_WQE) {
+ asm volatile(
+ "ldr z0, [%0]\n"
+ "str z0, [%1]\n"
+ ::"r" (val), "r"(dest):"cc", "memory"
+ );
+ return;
+ }
#endif
+ mmio_memcpy_x64(dest, val, sizeof(struct hns_roce_rc_sq_wqe));
+}
static void hns_roce_write_dwqe(struct hns_roce_qp *qp, void *wqe)
{
@@ -355,10 +351,7 @@ static void hns_roce_write_dwqe(struct hns_roce_qp *qp, void *wqe)
hr_reg_write(rc_sq_wqe, RCWQE_DB_SL_H, qp->sl >> HNS_ROCE_SL_SHIFT);
hr_reg_write(rc_sq_wqe, RCWQE_WQE_IDX, qp->sq.head);
- if (qp->flags & HNS_ROCE_QP_CAP_SVE_DIRECT_WQE)
- hns_roce_sve_write512(qp->sq.db_reg, wqe);
- else
- hns_roce_write512(qp->sq.db_reg, wqe);
+ hns_roce_qp_write512(qp, wqe);
}
static void update_cq_db(struct hns_roce_context *ctx, struct hns_roce_cq *cq)
--
2.25.1

View File

@ -1,6 +1,6 @@
Name: rdma-core
Version: 41.0
Release: 20
Release: 25
Summary: RDMA core userspace libraries and daemons
License: GPLv2 or BSD
Url: https://github.com/linux-rdma/rdma-core
@ -76,6 +76,17 @@ patch66: 0067-libhns-Support-STARS-mode-QP.patch
patch67: 0068-Update-kernel-headers-for-supporting-write-with-noti.patch
patch68: 0069-libhns-Support-write-with-notify.patch
patch70: 0070-libhns-Get-dmac-from-kernel-driver.patch
patch71: 0071-libhns-Corrects-several-issues-with-output-format-an.patch
patch72: 0072-libhns-Add-input-parameter-check-for-hnsdv_query_dev.patch
patch73: 0073-libhns-Fix-uninitialized-qp-attr-when-flush-cqe.patch
patch74: 0074-libhns-Fix-possible-overflow-in-cq-clean.patch
patch75: 0075-libhns-Fix-unnecessary-dca-memory-detach.patch
patch76: 0076-libhns-Bugfix-for-wrong-timing-of-modifying-ibv_qp-s.patch
patch77: 0077-libhns-Fix-parent-domain-unsupported-comp-mask.patch
patch78: 0078-libhns-Add-pthread_spin_destroy-pthread_mutex_destro.patch
patch79: 0079-libhns-Removes-a-repeated-initialization-of-a-spinlo.patch
patch80: 0080-libhns-Fix-owner-bit-when-SQ-wraps-around-in-new-IO.patch
patch81: 0081-libhns-Fix-missing-DB-when-compiler-does-not-support.patch
BuildRequires: binutils cmake >= 2.8.11 gcc libudev-devel pkgconfig pkgconfig(libnl-3.0)
BuildRequires: pkgconfig(libnl-route-3.0) valgrind-devel systemd systemd-devel
@ -323,6 +334,36 @@ fi
%{_mandir}/*
%changelog
* Tue Dec 12 2023 Ran Zhou <zhouran10@h-partners.com> - 41.0-25
- Type: bugfix
- ID: NA
- SUG: NA
- DESC: Fix congest type flags error and replace a corrupt patch
* Fri Dec 8 2023 Ran Zhou <zhouran10@h-partners.com> - 41.0-24
- Type: bugfix
- ID: NA
- SUG: NA
- DESC: Fix missing DB when compiler does not support SVE
* Thu Dec 7 2023 Ran Zhou <zhouran10@h-partners.com> - 41.0-23
- Type: bugfix
- ID: NA
- SUG: NA
- DESC: Bugfix for lock and owner bit
* Fri Dec 1 2023 Ran Zhou <zhouran10@h-partners.com> - 41.0-22
- Type: bugfix
- ID: NA
- SUG: NA
- DESC: Bugfix for wrong timing of modifying ibv_qp state to err
* Mon Nov 27 2023 Ran Zhou <zhouran10@h-partners.com> - 41.0-21
- Type: bugfix
- ID: NA
- SUG: NA
- DESC: Corrects several minor issues found in review
* Wed Nov 22 2023 Ran Zhou <zhouran10@h-partners.com> - 41.0-20
- Type: bugfix
- ID: NA