summaryrefslogtreecommitdiffstats
path: root/drivers/net/ethernet/sfc/rx.c
Commit message (Collapse)AuthorAgeFilesLines
* net: sfc calls skb_set_hashTom Herbert2013-12-181-1/+2
| | | | | | | | Drivers should call skb_set_hash to set the hash and its type in an skbuff. Signed-off-by: Tom Herbert <therbert@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* sfc: Add RX packet timestamping for EF10Jon Cooper2013-12-121-0/+2
| | | | | | | | | | | | | | The EF10 firmware can optionally insert RX timestamps in the packet prefix. These only include the clock minor value. We must also enable periodic time sync events on each event queue which provide the high bits of the clock value. [bwh: Combined and rebased several changes. Added the above description and some sanity checks for inline vs separate timestamps. Changed efx_rx_skb_attach_timestamp() to read the packet prefix from the skb head area.] Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Copy RX prefix into skb head area in efx_rx_mk_skb()Ben Hutchings2013-12-121-3/+7
| | | | | | | | | We can potentially pull the entire packet contents into the head area and then free the page it was in. In order to read an inline timestamp safely, we need to copy the prefix into the head area as well. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Make initial fill of RX descriptors synchronousJon Cooper2013-12-121-4/+5
| | | | Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: RX buffer allocation takes prefix size into account in IP header alignmentAndrew Rybchenko2013-12-061-3/+3
| | | | | | | | | rx_prefix_size is 4-bytes aligned on Falcon/Siena (16 bytes), but it is equal to 14 on EF10. So, it should be taken into account if arch requires IP header to be 4-bytes aligned (via NET_IP_ALIGN). Fixes: 8127d661e77f ('sfc: Add support for Solarflare SFC9100 family') Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Support ARFS for IPv6 flowsBen Hutchings2013-09-201-28/+62
| | | | | | | | Extend efx_filter_rfs() to map TCP/IPv6 and UDP/IPv6 flows into efx_filter_spec. These are only supported on EF10; on Falcon and Siena they will be rejected by efx_farch_filter_from_gen_spec(). Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Update copyright bannersBen Hutchings2013-08-291-2/+2
| | | | | | | Update the dates for files that have been added to in 2012-2013. Drop the 'Solarstorm' brand name that's still lingering here. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Prepare for RX scatter on EF10Jon Cooper2013-08-291-4/+4
| | | | | | | | | | | | RX DMA scatter is always enabled on EF10. Adjust the common RX completion handling to allow for this. RX completion events on EF10 include the length used from a single descriptor, not the cumulative length used. Add a field to struct efx_rx_queue to hold the cumulative length. [bwh: Also fix a related comment] Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Document conditions for multicast replication vs filter replacementBen Hutchings2013-08-291-0/+34
| | | | | | | | | Add the efx_filter_is_mc_recip() function to decide whether a filter is for a multicast recipient and can coexist with other filters with the same match values. Update efx_filter_insert_filter() kernel-doc to explain the conditions for this. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Add support for reading packet length from prefixBen Hutchings2013-08-271-2/+10
| | | | | | | | | Define a flag for struct efx_rx_buffer and efx_rx_packet() that indicates packet length must be read from the prefix. If this is set, read the length in __efx_rx_packet() (when the prefix should have arrived in cache). Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Generalise packet hash lookup to support EF10 RX prefixJon Cooper2013-08-271-8/+7
| | | | | | | | EF10 uses an entirely different RX prefix format from Falcon-arch. Extend struct efx_nic_type to describe this. [bwh: Also replace the magic numbers used for the Falcon-arch RX prefix] Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Make most filter operations NIC-type-specificBen Hutchings2013-08-221-0/+94
| | | | | | | | | | | | Aside from accelerated RFS, there is almost nothing that can be shared between the filter table implementations for the Falcon architecture and EF10. Move the few shared functions into efx.c and rx.c and the rest into farch.c. Introduce efx_nic_type operations for the implementation and inline wrapper functions that call these. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Refactor queue teardown sequence to allow for EF10 flush behaviourBen Hutchings2013-08-211-1/+0
| | | | | | | | | | | | | | Currently efx_stop_datapath() will try to flush our DMA queues (if DMA is enabled), then finalise software and hardware state for each queue. However, for EF10 we must ask the MC to finalise each queue, which implicitly starts flushing it, and then wait for the flush events. We therefore need to delegate more of this to the NIC type. Combine all the hardware operations into a new NIC-type operation efx_nic_type::fini_dmaq, and call this before tearing down the software state and buffers for all the DMA queues. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Stop RX refill before flushing RX queuesBen Hutchings2013-08-211-4/+4
| | | | | | | | rx_queue::enabled guards refill, so rename it to reflect that. Clear it at the start of the queue teardown process rather than waiting for the RX queue to be flushed. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Fix memory leak when discarding scattered packetsBen Hutchings2013-07-051-7/+20
| | | | | | | | | | | | | | | | | Commit 2768935a4660 ('sfc: reuse pages to avoid DMA mapping/unmapping costs') did not fully take account of DMA scattering which was introduced immediately before. If a received packet is invalid and must be discarded, we only drop a reference to the first buffer's page, but we need to drop a reference for each buffer the packet used. I think this bug was missed partly because efx_recycle_rx_buffers() was not renamed and so no longer does what its name says. It does not change the state of buffers, but only prepares the underlying pages for recycling. Rename it accordingly. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* sfc: Improve test for IOMMU in useBen Hutchings2013-06-241-1/+1
| | | | | | | | | The device::iommu_group field may be set even if no IOMMU is in use. iommu_present() is still a better indicator, although it doesn't tell us whether *our* device is affected. Reported-by: Alex Williamson <alex.williamson@redhat.com> Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Do not pass non-TCP packets into GRO codeBen Hutchings2013-06-241-1/+1
| | | | | | | | | | | GRO can handle non-TCP packets and pass them up without coalescing, but it has to do some extra work to parse the packet which we can bypass using the hardware parse result. (This condition yields a false negative for TCP/IPv6 packets received by Falcon, but its performance is already poor in that case due to lack of checksum offload.) Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Increase size of RX SKB header areaJon Cooper2013-06-241-1/+1
| | | | | | This allows the SKB to hold the headers without reallocation more often. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Enable RX checksum offload for packets not handled by GROJon Cooper2013-06-241-0/+2
| | | | Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Reduce RX scatter buffer size, and reduce alignment if appropriateBen Hutchings2013-05-141-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | efx_start_datapath() asserts that we can fit 2 RX scatter buffers plus a software structure, each appropriately aligned, into a single page. Where L1_CACHE_BYTES == 256 and PAGE_SIZE == 4096, which is the case on s390, this assertion fails. The current scatter buffer size is also not a multiple of 64 or 128, which are more common cache line sizes. If we can make both the start and end of a scatter buffer cache-aligned, this will reduce the need for read-modify-write operations on inter- processor links. Fix the alignment by reducing EFX_RX_USR_BUF_SIZE to 2048 - 256 == 1792. (We could use 2048 - L1_CACHE_BYTES, but EFX_RX_USR_BUF_SIZE also affects user-level networking where a larger amount of housekeeping data may be needed. Although this version of the driver does not support user-level networking, I prefer to keep scattering behaviour consistent with the out-of-tree version.) This still doesn't fix the s390 build because like most architectures it has NET_IP_ALIGN == 2. When NET_IP_ALIGN != 0 we cannot achieve cache line alignment at either the start or end of a scatter buffer, so there is actually no point in padding the buffers to a multiple of the cache line size. All we need is 4-byte alignment of the network header, so do that. Adjust the assertions accordingly. Reported-by: Geert Uytterhoeven <geert@linux-m68k.org> Reported-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Ben Hutchings <bhutchings@solarflare.com> Acked-by: Geert Uytterhoeven <geert@linux-m68k.org> Signed-off-by: David S. Miller <davem@davemloft.net>
* sfc: Delete EFX_PAGE_IP_ALIGN, equivalent to NET_IP_ALIGNBen Hutchings2013-05-141-3/+3
| | | | | | | | | The two architectures that define CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS (powerpc and x86) now both define NET_IP_ALIGN as 0, so there is no need for this optimisation any more. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* sfc: make local functions staticstephen hemminger2013-03-171-2/+2
| | | | | | | Trivial sparse detected functions that should be static. Signed-off-by: Stephen Hemminger <stephen@networkplumber.org> Signed-off-by: David S. Miller <davem@davemloft.net>
* sfc: allocate more RX buffers per pageDaniel Pieczko2013-03-071-38/+42
| | | | | | | | | | Allocating 2 buffers per page is insanely inefficient when MTU is 1500 and PAGE_SIZE is 64K (as it usually is on POWER). Allocate as many as we can fit, and choose the refill batch size at run-time so that we still always use a whole page at once. [bwh: Fix loop condition to allow for compound pages; rebase] Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Replace efx_rx_is_last_buffer() with a flagBen Hutchings2013-03-071-11/+6
| | | | | | This condition is brittle and we have lots of flags to spare. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: reuse pages to avoid DMA mapping/unmapping costsDaniel Pieczko2013-03-071-94/+205
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On POWER systems, DMA mapping/unmapping operations are very expensive. These changes reduce these costs by trying to reuse DMA mapped pages. After all the buffers associated with a page have been processed and passed up, the page is placed into a ring (if there is room). For each page that is required for a refill operation, a page in the ring is examined to determine if its page count has fallen to 1, ie. the kernel has released its reference to these packets. If this is the case, the page can be immediately added back into the RX descriptor ring, without having to re-map it for DMA. If the kernel is still holding a reference to this page, it is removed from the ring and unmapped for DMA. Then a new page, which can immediately be used by RX buffers in the descriptor ring, is allocated and DMA mapped. The time a page needs to spend in the recycle ring before the kernel has released its page references is based on the number of buffers that use this page. As large pages can hold more RX buffers, the RX recycle ring can be shorter. This reduces memory usage on POWER systems, while maintaining the performance gain achieved by recycling pages, following the driver change to pack more than two RX buffers into large pages. When an IOMMU is not present, the recycle ring can be small to reduce memory usage, since DMA mapping operations are inexpensive. With a small recycle ring, attempting to refill the descriptor queue with more buffers than the equivalent size of the recycle ring could ultimately lead to memory leaks if page entries in the recycle ring were overwritten. To prevent this, the check to see if the recycle ring is full is changed to check if the next entry to be written is NULL. [bwh: Combine and rebase several commits so this is complete before the following buffer-packing changes. Remove module parameter.] Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Enable RX DMA scattering where possibleBen Hutchings2013-03-071-73/+138
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Enable RX DMA scattering iff an RX buffer large enough for the current MTU will not fit into a single page and the NIC supports DMA scattering for kernel-mode RX queues. On Falcon and Siena, the RX_USR_BUF_SIZE field is used as the DMA limit for both all RX queues with scatter enabled. Set it to 1824, matching what Onload uses now. Maintain a statistic for frames truncated due to lack of descriptors (rx_nodesc_trunc). This is distinct from rx_frm_trunc which may be incremented when scattering is disabled and implies an over-length frame. Whenever an MTU change causes scattering to be turned on or off, update filters that point to the PF queues, but leave others unchanged, as VF drivers assume scattering is off. Add n_frags parameters to various functions, and make them iterate: - efx_rx_packet() - efx_recycle_rx_buffers() - efx_rx_mk_skb() - efx_rx_deliver() Make efx_handle_rx_event() responsible for updating efx_rx_queue::removed_count. Change the RX pipeline state to a starting ring index and number of fragments, and make __efx_rx_packet() responsible for clearing it. Based on earlier versions by David Riddoch and Jon Cooper. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Update RX buffer address together with lengthBen Hutchings2013-03-071-12/+6
| | | | | | | Adjust rx_buf->page_offset when we eat the RX hash prefix. Remove efx_rx_buf_offset(), which is now redundant. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Explicitly prefetch RX hash prefix, not just Ethernet headeBen Hutchings2013-03-071-1/+1
| | | | | | | | | Currently we prefetch from the Ethernet header, but we will also read the hash prefix. In practice they should be in the same cache line and this won't hurt, but it is still pointless to add on the hash prefix size. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Replace efx_rx_buf_eh() with simpler efx_rx_buf_va()Ben Hutchings2013-03-071-4/+4
| | | | | | | efx_rx_buf_va() returns the virtual address of the current start of the buffer. The callers must add the hash prefix size themselves. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Wrap __efx_rx_packet() with efx_rx_flush_packet()Ben Hutchings2013-03-071-2/+1
| | | | | | | The pipeline mechanism will need to change a bit for scattered packets. Add a wrapper to insulate efx_process_channel() from this. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Properly distinguish RX buffer and DMA lengthsBen Hutchings2013-03-071-11/+8
| | | | | | | Replace efx_nic::rx_buffer_len with efx_nic::rx_dma_len, the maximum RX DMA length. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Remove rx_alloc_method SKBAlexandre Rames2013-03-071-235/+95
| | | | | | [bwh: Remove more dead code, and make efx_ptp_rx() pull the data it needs into the header area.] Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Allow efx_channel_type::receive_skb() to reject a packetBen Hutchings2013-03-071-4/+6
| | | | | | | | Instead of having efx_ptp_rx() call netif_receive_skb() for an invalid PTP packet, make it return false for rejected packets and have efx_rx_deliver() pass them up. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Correct efx_rx_buffer::page_offset when EFX_PAGE_IP_ALIGN != 0Ben Hutchings2013-03-061-1/+1
| | | | | | | | | | RX DMA buffers start at an offset of EFX_PAGE_IP_ALIGN bytes from the start of a cache line. This offset obviously needs to be included in the virtual address, but this was missed in commit b590ace09d51 ('sfc: Fix efx_rx_buf_offset() in the presence of swiotlb') since EFX_PAGE_IP_ALIGN is equal to 0 on both x86 and powerpc. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Fix efx_rx_buf_offset() in the presence of swiotlbBen Hutchings2013-02-261-5/+5
| | | | | | | | | | | We assume that the mapping between DMA and virtual addresses is done on whole pages, so we can find the page offset of an RX buffer using the lower bits of the DMA address. However, swiotlb maps in units of 2K, breaking this assumption. Add an explicit page_offset field to struct efx_rx_buffer. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Properly sync RX DMA buffer when it is not the last in the pageBen Hutchings2013-02-261-5/+10
| | | | | | | | | | | | We may currently allocate two RX DMA buffers to a page, and only unmap the page when the second is completed. We do not sync the first RX buffer to be completed; this can result in packet loss or corruption if the last RX buffer completed in a NAPI poll is the first in a page and is not DMA-coherent. (In the middle of a NAPI poll, we will handle the following RX completion and unmap the page *before* looking at the content of the first buffer.) Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Delete redundant page_addr variable from efx_init_rx_buffers_page()Ben Hutchings2012-12-011-5/+1
| | | | Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Add channel specific receive_skb handler and post_remove callbackStuart Hodgson2012-09-071-2/+11
| | | | | | | | | | | Allows an extra channel to override the standard receive_skb handler and also for extra non generic operations to be performed on remove. Also set default rx strategy so only skbs can be delivered to the PTP receive function. Signed-off-by: Stuart Hodgson <smhodgson@solarflare.com> Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Add explicit RX queue flag to channelStuart Hodgson2012-09-071-2/+5
| | | | | | | | | | The PTP channel will have its own RX queue even though it's not a regular traffic channel. Original work by Ben Hutchings <bhutchings@solarflare.com> Signed-off-by: Stuart Hodgson <smhodgson@solarflare.com> Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* Merge branch 'for-davem' of ↵David S. Miller2012-07-181-11/+11
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/bwh/sfc-next Ben Hutchings says: ==================== 1. Fix potential badness when running a self-test with SR-IOV enabled. 2. Fix calculation of some interface statistics that could run backward. 3. Miscellaneous cleanup. ==================== Signed-off-by: David S. Miller <davem@davemloft.net>
| * sfc: Use generic DMA API, not PCI-DMA APIBen Hutchings2012-07-171-11/+11
| | | | | | | | Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* | drivers/net/ethernet: Fix (nearly-)kernel-doc comments for various functionsBen Hutchings2012-07-101-0/+1
|/ | | | | | | | | | Fix incorrect start markers, wrapped summary lines, missing section breaks, incorrect separators, and some name mismatches. Delete a few that are content-free. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com> Acked-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* sfc: By default refill RX rings as soon as space for a batchDavid Riddoch2012-05-101-6/+12
| | | | | | | Previously we refilled with much larger batches, which caused large latency spikes. We now have many more much much smaller spikes! Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Fill RX rings completely full, rather than to 95% fullDavid Riddoch2012-05-101-11/+4
| | | | | | | There was no runtime control of the fast_fill_limit in any case, so purged that field. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* sfc: Update comments on efx_rx_packet_gro()Ben Hutchings2012-03-061-5/+2
| | | | | | | | The in-tree driver has never supported Driverlink. The rest of the comments are rather redundant, but we can usefully state what the requirements are on the buffer state. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/netDavid S. Miller2012-02-261-2/+2
|\ | | | | | | | | | | | | | | | | | | | | Conflicts: drivers/net/ethernet/sfc/rx.c Overlapping changes in drivers/net/ethernet/sfc/rx.c, one to change the rx_buf->is_page boolean into a set of u16 flags, and another to adjust how ->ip_summed is initialized. Signed-off-by: David S. Miller <davem@davemloft.net>
| * sfc: Fix assignment of ip_summed for pre-allocated skbsBen Hutchings2012-02-251-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When pre-allocating skbs for received packets, we set ip_summed = CHECKSUM_UNNCESSARY. We used to change it back to CHECKSUM_NONE when the received packet had an incorrect checksum or unhandled protocol. Commit bc8acf2c8c3e43fcc192762a9f964b3e9a17748b ('drivers/net: avoid some skb->ip_summed initializations') mistakenly replaced the latter assignment with a DEBUG-only assertion that ip_summed == CHECKSUM_NONE. This assertion is always false, but it seems no-one has exercised this code path in a DEBUG build. Fix this by moving our assignment of CHECKSUM_UNNECESSARY into efx_rx_packet_gro(). Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* | sfc: Leave interrupts and event queues enabled whenever we canBen Hutchings2012-02-161-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When SR-IOV is enabled we may receive FLR (Function-Level Reset) events, associated queue flush events and requests from VF drivers at any time. Therefore we need to keep event queues and interrupts enabled whenever possible. Currently we stop interrupt-driven event processing before flushing RX and TX queues; efx_nic_flush_queues() then polls event queues for flush events and discards any others it finds. Change it to work with the regular event handling functions. Currently efx_start_channel() fills RX queues synchronously when a device is brought up. This could now race with NAPI, so change it to send fill events. This was almost entirely written by Steve Hodgson, formerly shodgson@solarflare.com. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* | sfc: Generate RX fill events based on RX queues, not channelsBen Hutchings2012-02-161-2/+1
| | | | | | | | | | | | | | This makes it harder to accidentally send such events to TX-only channels. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
* | sfc: Use a more sensible cast in efx_rx_buf_offset()Ben Hutchings2012-01-301-1/+1
| | | | | | | | | | | | | | | | | | | | | | This function returns the page offset of the buffer, which can be calculated based on either its DMA address or its virtual address. It used to use the virtual address and we would cast that to unsigned long, as anything smaller would result in a compiler warning. Now that it's using the DMA address we should use unsigned int, matching the return type. It is also unnecessary to use __force. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
generated by cgit v1.2.3 (git 2.25.1) at 2024-12-19 12:01:31 +0000