【C语言】ipoib驱动 - ipoib_cm_post_receive_srq_rss函数

一、ipoib_cm_post_receive_srq_rss函数定义

static int ipoib_cm_post_receive_srq_rss(struct net_device *dev,
					 int index, int id)
{
	struct ipoib_dev_priv *priv = ipoib_priv(dev);
	struct ipoib_recv_ring *recv_ring = priv->recv_ring + index;
	struct ib_sge *sge;
	struct ib_recv_wr *wr;
	int i, ret;

	sge = recv_ring->cm.rx_sge;
	wr = &recv_ring->cm.rx_wr;

	wr->wr_id = id | IPOIB_OP_CM | IPOIB_OP_RECV;

	for (i = 0; i < priv->cm.num_frags; ++i)
		sge[i].addr = priv->cm.srq_ring[id].mapping[i];

	ret = ib_post_srq_recv(priv->cm.srq, wr, NULL);
	if (unlikely(ret)) {
		ipoib_warn(priv, "post srq failed for buf %d (%d)\n", id, ret);
		ipoib_cm_dma_unmap_rx(priv, priv->cm.num_frags - 1,
				      priv->cm.srq_ring[id].mapping);
		dev_kfree_skb_any(priv->cm.srq_ring[id].skb);
		priv->cm.srq_ring[id].skb = NULL;
	}

	return ret;
}

二、解读

解读1

这段代码是一个C语言函数，主要用于处理InfiniBand（IB）通信中的SRQ（Subscription/Notification Queue）接收操作。函数名为ipoib_cm_post_receive_srq_rss，它接收三个参数：一个网络设备结构体指针dev，一个整数index，和一个整数id。

以下是该函数的详细分析：

1. 函数定义:

   • static int ipoib_cm_post_receive_srq_rss(struct net_device *dev, int index, int id)：这是一个静态函数，返回类型为int。它接收三个参数：一个网络设备结构体指针dev，一个整数index，和一个整数id。

2. 局部变量:

   • struct ipoib_dev_priv *priv = ipoib_priv(dev);：从网络设备结构体中获取私有数据结构体指针。
   • struct ipoib_recv_ring *recv_ring = priv->recv_ring + index;：根据给定的索引从私有数据结构体的接收环中获取接收环结构体指针。
   • struct ib_sge *sge;：定义一个指向InfiniBand SGE（Scatter Gather Element）的指针。
   • struct ib_recv_wr *wr;：定义一个指向InfiniBand接收写请求的指针。
   • int i, ret;：定义整数变量i和ret。

3. 设置写请求:

   • sge = recv_ring->cm.rx_sge;：将SGE指针设置为接收环中的SGE。
   • wr = &recv_ring->cm.rx_wr;：将写请求指针设置为接收环中的写请求。
   • wr->wr_id = id | IPOIB_OP_CM | IPOIB_OP_RECV;：设置写请求的ID，由id、IPOIB_OP_CM和IPOIB_OP_RECV按位或运算得到。

4. 设置SGE:

   • 循环遍历私有数据结构体的碎片数量，并设置每个SGE的地址。

5. 发送接收请求:

   • ret = ib_post_srq_recv(priv->cm.srq, wr, NULL);：使用InfiniBand API发送SRQ接收请求。如果返回值不为0，则进入错误处理部分。

6. 错误处理:

   • 如果返回值ret不为0，则打印警告信息，取消DMA映射，并释放相关缓冲区。最后返回ret。

7. 返回值:

   • 函数返回ret，即SRQ接收请求的返回值。如果成功，则返回0；否则返回非0值。

总体来说，这个函数的主要目的是通过InfiniBand接口发送SRQ接收请求，并处理可能发生的错误情况。

解读2

It seems like you're providing a function definition for a Linux kernel module. The function?ipoib_cm_post_receive_srq_rss?is used to post a receive request to an InfiniBand CM SRQ (Sending Receive Queue) using Receive Side Scaling (RSS). The function is part of the IPoIB (IP over InfiniBand) subsystem in the Linux kernel.

Here's a brief explanation of what the function does:

1. It takes?struct net_device *dev?as the first argument, which represents the network interface device.
2. The?int index?argument specifies the receive ring index.
3. The?int id?argument is the identifier for the receive buffer.

Inside the function, it initializes a?struct ib_sge?array and a?struct ib_recv_wr?structure. The?struct ib_sge?array points to the DMA-mapped addresses of the receive buffers, and the?struct ib_recv_wr?contains the work request information.

The?wr->wr_id?field is set to a combination of the operation type (CM and RECV) and the identifier?id.

The for loop from line 15 to 18 copies the addresses of the receive buffers into the SGE (scatter/gather element) array.

The?ib_post_srq_recv?function is called to actually post the receive request to the SRQ. If this function returns a non-zero value, it indicates an error, and the function logs a warning message.

Finally, if the receive request posting fails, the function frees up the resources associated with the receive buffer (DMA mapping and the SKB) using?ipoib_cm_dma_unmap_rx?and?dev_kfree_skb_any.

Please note that this function is likely to be called from within the kernel context, and it's important to handle errors and resource cleanup properly to avoid memory leaks or other issues.

这段代码是Linux内核中用于处理InfiniBand（IB）网络设备上的接收操作的函数。它属于IPoIB（IP over InfiniBand）协议栈的一部分，用于在IB网络上传输IP数据包。

函数ipoib_cm_post_receive_srq_rss的主要功能是：

1. 获取与给定网络设备相关的私有数据结构ipoib_dev_priv。
2. 获取指定接收环的指针recv_ring。
3. 设置接收工作请求（WR）的标识符wr->wr_id。
4. 使用ib_post_srq_recv函数向SRQ（Sending Receive Queue）提交接收WR。

下面是代码中关键部分的详细解释：

• struct ipoib_dev_priv *priv = ipoib_priv(dev);：获取与网络设备dev相关的私有数据结构。
• struct ipoib_recv_ring *recv_ring = priv->recv_ring + index;：获取指定接收环的指针，该接收环由index索引。
• struct ib_sge *sge;：声明一个指向结构体ib_sge的指针，该结构体用于描述scatter/gather列表，用于DMA操作。
• struct ib_recv_wr *wr;：声明一个指向结构体ib_recv_wr的指针，该结构体用于描述接收WR。
• wr->wr_id = id | IPOIB_OP_CM | IPOIB_OP_RECV;：设置接收WR的标识符。id是接收缓冲区的标识符，IPOIB_OP_CM和IPOIB_OP_RECV是操作码，表示这是一个与CM（Connection Management）相关的接收操作。
• for (i = 0; i < priv->cm.num_frags; ++i) sge[i].addr = priv->cm.srq_ring[id].mapping[i];：为scatter/gather列表中的每个元素设置地址，这些地址指向SRQ ring中的映射区域。
• ret = ib_post_srq_recv(priv->cm.srq, wr, NULL);：使用ib_post_srq_recv函数向SRQ提交接收WR。如果成功，ret将返回0；否则，将返回错误代码。
• if (unlikely(ret)) {：如果提交接收WR失败，执行以下操作：
  • `ipoib_cm_dma_unmap_rx(priv, priv->cm.num_frags - 1, priv->cm.srq_ring

解读3

This code snippet is a function from a Linux kernel network module for InfiniBand devices using IP over InfiniBand (IPoIB) with Connected Mode (CM). The function ipoib_cm_post_receive_srq_rss is intended to post a receive work request (WR) to a Shared Receive Queue (SRQ) with respect to Receive-Side Scaling (RSS), which is a method used to distribute network processing across multiple CPUs. Let's break down what the code appears to be doing:
1. The function takes three parameters:
? ?- *dev: a pointer to a net_device structure, associated with an InfiniBand network device.
? ?- index: an integer index, typically indicating the specific receive ring to which this posted receive work request belongs.
? ?- id: an identifier used to track the specific receive buffer.
2. ipoib_priv(dev) is a macro or inline function to retrieve a pointer to the IPoIB private structure (ipoib_dev_priv) associated with the net_device.
3. The code calculates the address of a ipoib_recv_ring structure by offsetting the recv_ring pointer in the IPoIB private structure by the index.
4. Local pointers to Scatter-Gather Elements (ib_sge) and a Receive Work Request (ib_recv_wr) are retrieved from the recv_ring structure.
5. The wr_id field of the Receive Work Request is set with the supplied id, and it is flagged with IPOIB_OP_CM and IPOIB_OP_RECV to indicate that the operation is related to a Connected Mode receive action.
6. In a loop controlled by priv->cm.num_frags, the addresses (addr) of the ib_sge array are set. These addresses point to memory mappings that are used by the hardware to place incoming packets.
7. The receive work request is posted to the Shared Receive Queue (srq) using ib_post_srq_recv(). This function usually takes the SRQ, a pointer to the WR, and a pointer where it puts the pointer to the next work request (which is not present, so NULL is passed).
8. If ib_post_srq_recv() fails, indicated by a non-zero return value, an error message is logged with ipoib_warn(). DMA unmapping is performed for the used memory mappings (ipoib_cm_dma_unmap_rx()), the associated socket buffer is freed (dev_kfree_skb_any()), and the skb field is set to NULL.
9. Finally, the function returns the status code (ret) from ib_post_srq_recv() which indicates success (0) or the error code.
This code snippet is from a quite low-level Linux kernel driver and assumes familiarity with kernel development, InfiniBand architecture, device drivers, and Direct Memory Access (DMA). As such, it handles network device operations and memory management that are determined by the InfiniBand specifications and the driver's architecture.?

这段代码是一个用于处理基于IP的InfiniBand (IPoIB) 网络设备的Linux内核网络模块的函数，具体是针对使用“连接模式”(Connected Mode, CM)的IPoIB的。函数 ipoib_cm_post_receive_srq_rss 的目的是将接收工作请求（Receive Work Request, WR）发布到共享接收队列（Shared Receive Queue, SRQ）上，并且需要考虑到接收端扩展（Receive-Side Scaling, RSS），这是一种用来在多个CPU之间分配网络处理任务的方法。下面是代码的具体作用：
1. 函数接受三个参数：
? ?- *dev: 指向 net_device 结构体的指针，该结构体与一个InfiniBand网络设备关联。
? ?- index: 一个整数索引，通常表示特定接收环（receive ring）的索引，发布的接收工作请求属于此环。
? ?- id: 用来追踪特定接收缓冲区的标识符。
2. ipoib_priv(dev) 是一个宏或内联函数，用来检索与 net_device 相关联的 IPoIB 私有数据结构（`ipoib_dev_priv`）的指针。
3. 通过将 IPoIB 私有数据结构中的 recv_ring 指针加上 index 偏移，计算 ipoib_recv_ring 结构的地址。
4. 从 recv_ring 结构中检索 Scatter-Gather 元素（`ib_sge`）的本地指针和接收工作请求（`ib_recv_wr`）。
5. 将接收工作请求的 wr_id 字段设置为提供的 id，并且用 IPOIB_OP_CM 和 IPOIB_OP_RECV 标记，表明该操作涉及连接模式接收动作。
6. 在由 priv->cm.num_frags 控制的循环中，设置 ib_sge 数组的地址（`addr`）数组。这些地址指向的内存映射被硬件用于放置进入的数据包。
7. 通过 ib_post_srq_recv() 函数将接收工作请求发布到共享接收队列（`srq`）。这个函数通常需要 SRQ、WR 的指针，以及用于存放指向下一个工作请求的指针（在这里没有使用，所以传入了 NULL）。
8. 如果 ib_post_srq_recv() 失败（通过非零的返回值表示），会使用 ipoib_warn() 记录一个错误消息。之后，为已使用的内存映射执行 DMA 解映射操作（`ipoib_cm_dma_unmap_rx()`），释放相关的套接字缓冲区（`dev_kfree_skb_any()`），并将 skb 字段设为 NULL。
9. 最后，函数返回 ib_post_srq_recv() 的状态码（`ret`），它指示成功（`0`）或错误代码。
这段代码属于比较低层次的Linux内核驱动程序，假设你熟悉内核开发、InfiniBand架构、设备驱动程序和直接内存访问（DMA）。因此，它处理网络设备的操作和内存管理，这些操作和管理都是由InfiniBand规范和驱动程序的架构决定的。