Ndtyjky

Is it possible to map physical address as data fragment in sk_buff?
I am working on Zynq Ultrascale+ platform (FPGA + ARM SOC). I have memory buffer mapped to physical address. The goal is to efficiently send that data over UDP. By efficiently I mean ZEROCOPY. What I am trying to do is to develop linux driver that would map that physical address into kernel memory and append it to sk_buff as fragment.
I started with:

#define PACKET_LEN 1024

struct page *pag;



struct net_device *dev;

struct sk_buff *skb = NULL;

skb = alloc_skb(LL_RESERVED_SPACE(dev) + PACKET_LEN + ip_header_l +

                udp_header_l, GFP_ATOMIC);

udp = skb_push(skb, udp_header_l);



//Fill up udp header

...



ip = skb_push(skb, ip_header_l);



//fill up ip header

...



dev_hard_header(skb, dev, ETH_P_IP, addr, myaddr, dev->addr_len);

skb->dev = dev;

//map page with data as fragment

skb_fill_page_desc(skb, 0, pag, 0, PACKET_LEN);

//send data

dev_queue_xmit(skb);

And as long as page is created by:

pagebuff = vmalloc(PACKET_LEN);

pag = vmalloc_to_page(pagebuff);

It all works fine. Packet gets send. Packet is send by two DMA transactions (Scatter Gather).
Going towards my goal I replaced vmalloced page with:

res = platform_get_resource(pdev, IORESOURCE_MEM, 0);

membase = devm_ioremap_resource(&pdev->dev, res);

pag = virt_to_page(membase);

Physical address is 0xb0000000 and is mapped to virtual address 0xffffff800ad30000 page is at 0xffffffbf0025e280.
After dev_queue_xmit packet goes to network queue and ends up being mapped for DMA.
Problem arises when swiotlb_map_page uses 0x00ad30000 as phys_addr, which is different than original 0xb0000000.
virt_to_phys is used in swiotlb_map_page to calculate physical address and it basically takes lower 32 bits as phys address. Is there a different way to map memory region so it can be used as sk_buff fragment?

As a temporary fix I created fake page like this:

res = platform_get_resource(pdev, IORESOURCE_MEM, 0);

pag = alloc_page(0); //create fake page

memset(pag, 0, sizeof(struct page));

pag->private = res->start;

And patched ethernet driver to use page private data as mapping address:

mapping = skb_frag_page(frag)->private;

if (mapping) {

    //  printk("macb mapping override to %pn",mapping);

}

else {

    mapping = skb_frag_dma_map(&bp->pdev->dev, frag, offset, size, DMA_TO_DEVICE);

    if (dma_mapping_error(&bp->pdev->dev, mapping))

        goto dma_error;

}

With such a hack it all works. Data is filled with contents of 0xb0000000. Although it works fine I really doubt it is the right way to do it. Nevertheless it shows there is no hardware limitation to do it. Does anyone know how to map that memory correctly?

P.S. I also tried to map physical address to fixed virtual address in such manner that swiotlb_map_page would calculate correct address (and virt_to_phys did), but it ended with "Unable to handle kernel paging request at virtual address" error.

membase = phys_to_virt(res->start);

i = ioremap_page_range(membase, membase + resource_size(res),

                       res->start, PAGE_KERNEL);

//I tried both

pag = phys_to_page(res->start);

pag = virt_to_page(membase);

Maybe I am looking for page at wrong address or maybe it is nonexistent.
Can anyone point me in the right direction. Is there a way to accomplish the goal without such a nasty hack?