【RDMA】ibv_req_notify_cq()

原文：https://www.rdmamojo.com/2013/02/22/ibv_req_notify_cq/

目录

描述

参数

返回值

例子

常见问题

---

描述

ibv_req_notify_cq()在完成队列(CQ)上请求完成通知。

ibv_req_notify_cq()当一个requested type的下一个WC(工作完成)添加到CQ时请求一个通知。调用ibv_req_notify_cq()之前CQ中存在的任何WC(工作完成)都不会导致创建完成通知。完成通知可使用ibv_get_cq_event()读取。

可以请求两种类型的完成事件：

请求的完成事件-当到达的[发送]或[带立即数的RDMA写入消息] 带有“请求的事件”指示符集(即，远程端发布了SR(发送请求)并在send_flags中设置了IBV_SEND_SOLICITED的消息)产生的WC(成功接收or失败)添加到CQ中时 产生的事件。
非请求的完成事件-在将任何WC(工作完成)添加到CQ时发生，无论它是发送还是接收工作完成，以及它是成功还是失败。

如果 一个Request Completion Notification 在pending，即调用了ibv_req_notify_cq()并且 还没Completion Notification发生，则后续在相同CQ上调用ibv_req_notify_cq()请求相同的Completion Event type将无效;

仅会生成一个完成通知。调用ibv_req_notify_cq()只有在完成通知发生后才会起作用(Calling ibv_req_notify_cq() will have an effect only after the Completion Notification will occur.)。

下一个完成事件的请求完成通知(Request Completion Notification)优先于同一CQ的solicited event completion的请求完成通知(Request Completion Notification)。
如果对同一个CQ进行了多次ibv_req_notify_cq()的调用，并且至少有一个请求将类型设置为[下一个完成](next completion)，则将[下一个完成]添加到该CQ时将生成一个完成通知(即不是为next Solicited Completion)。
Completion Notification一旦发生，如果希望获得更多的Completion Notification(完成通知)，则他必须再次调用ibv_req_notify_cq()。

原文：

ibv_req_notify_cq() requests a Completion Notification on a Completion Queue (CQ).

ibv_req_notify_cq() requests a notification when the next Work Completion of a requested type is added to the CQ. Any Work Completions that existed in the CQ before calling ibv_req_notify_cq() will not result in creating a Completion Notification. The Completion Notification will be read using ibv_get_cq_event().

There are two types of Completion Events that can be requested:

• Solicited Completion Event - Occurs when an incoming Send or RDMA Write with Immediate Data message with the Solicited Event indicator set (i.e. the remote side posted a Send Request with IBV_SEND_SOLICITED set in send_flags) causes a generation of a successful Receive Work Completion or any unsuccessful (either Send of Receive) Work Completion to be added to the CQ.
• Unsolicited Completion Event - occurs when any Work Completion is added to the CQ, whether it is a Send or Receive Work Completion and whether is it successful or unsuccessful Work Completion.

If a Request Completion Notification is pending, i.e. ibv_req_notify_cq() was called and no Completion Notification occurred, subsequent calls to ibv_req_notify_cq() with the same CQ requesting the same Completion Event type will have no effect; only one Completion Notification will be generated. Calling ibv_req_notify_cq() will have an effect only after the Completion Notification will occur.

A Request Completion Notification for the next completion event takes precedence over a Request Completion Notification for a solicited event completion for the same CQ.

If multiple calls to ibv_req_notify_cq() have been made for the same CQ and at least one of the requests set the type to the next completion, a Completion Notification will be generated when the next Completion is added to that CQ (i.e. not for the next Solicited Completion).

Once a Completion Notification occurred, if one wishes to get more Completions Notification, he has to call ibv_req_notify_cq() again.

参数

Name

Direction

Description

cq

in

从ibv_create_cq()返回的CQ

solicited_only

in

Type of the requested Completion Notification.

[请求的完成通知]的类型。

0	The next Completion whether it is Solicited or Unsolicited 下一次完成(无论是请求的还是非请求的)
otherwise	the next Completion is Solicited only or for any unsuccessful Work Completion 下一次完成是请求的，或者是任何不成功的工作完成

返回值

Value	Description
0	On success
errno	On failure.
EINVAL	Invalid CQ handle

例子

1) 请求下一次完成的通知：

struct ibv_cq *cq;
 
if (ibv_req_notify_cq(cq, 0)) {
	fprintf(stderr, "Error, ibv_req_notify_cq() failed when requested a "
		"notification for the next completion\n");
	return -1;
}

2) 请求下一次请求的完成的通知(Request a notification for the next Solicited Completion):

struct ibv_cq *cq;
 
if (ibv_req_notify_cq(cq, 1)) {
	fprintf(stderr, "Error, ibv_req_notify_cq() failed when requested a "
		"notification for the next solicited completion\n");
	return -1;
}

常见问题

I can read Work Completions with ibv_poll_cq(), why would I want to use ibv_req_notify_cq()?

Handling Work Completions with events will decrease the CPU consumption of your application; your process will sleep until a new Work Completion will be added to the CQ.

Can I call ibv_req_notify_cq() with every CQ?

Yes. Although keep in mind that requesting for a notification on Solicited Completions has a meaning only for CQ that is associated with a Receive Queue.

I called ibv_req_notify_cq() when there is a Work Completion in the CQ (of the type that I've asked) will I get a Completion Notification?

No, you won't. A Completion Notification will be generated for the *next* Completion of the type that you've asked (Solicited or next) that will be added to the CQ *after* you called ibv_req_notify_cq().

I called ibv_req_notify_cq() several times before I got a Completion Notification, what is the effect of this?

If all calls to ibv_req_notify_cq() were done when requesting the same Completion type, you will get only one Completion Notification, according to the quested type. If at least one of them requested a Completion Notification for the next Completion, you will get a notification for the next Completion.

Can I ask to get a notification after N Completions will be added to the CQ

No, this isn't supported.

Can I ask to get a notification for every Completion that will add to the CQ automatically (instead of calling ibv_req_notify_cq() again each time)?

No, this isn't supported. After you got a notification, you need to call ibv_req_notify_cq() again if you want to get another notification.