* Re: [patch net v3] net: fec: fix compile with CONFIG_M5272
From: Nikita Yushchenko @ 2016-12-05 18:06 UTC (permalink / raw)
To: Vitaly Wool
Cc: Johannes Berg, Troy Kisky, linux-kernel, Chris Healy,
David S. Miller, Fugang Duan, Eric Nelson, Andrew Lunn,
Fabio Estevam, netdev, Philippe Reynes
In-Reply-To: <CAMJBoFNd==OugwCjwT-q1MkooFr+Ed7AF00ifdimTk_DFVJO9Q@mail.gmail.com>
> +#define FEC_STATS_SIZE (ARRAY_SIZE(fec_stats) * sizeof(u64))
>
>
> Do I take it right this actually translates to (sizeof(fec_stats) /
> sizeof(u64) * sizeof(u64))?
No.
fec_stats is an array of structs, each struct has car arrsy and integer,
and size of that is definitely not bytes.
ARRAY_SIZE(fec_stats) is number of stats, i.e. it is returned from
fec_enet_get_sset_count()
Each stat in blob is u64. Thus (ARRAY_SIZE(fec_stats) * sizeof(u64)) is
size of stats blob.
^ permalink raw reply
* Re: [PATCH v2] net: phy: dp83848: Support ethernet pause frames
From: David Miller @ 2016-12-05 18:14 UTC (permalink / raw)
To: jesper.nilsson; +Cc: f.fainelli, rogerq, afd, dmurphy, netdev, linux-kernel
In-Reply-To: <20161202145748.GA19016@axis.com>
From: Jesper Nilsson <jesper.nilsson@axis.com>
Date: Fri, 2 Dec 2016 15:57:49 +0100
> According to the documentation, the PHYs supported by this driver
> can also support pause frames. Announce this to be so.
> Tested with a TI83822I.
>
> Acked-by: Andrew F. Davis <afd@ti.com>
> Signed-off-by: Jesper Nilsson <jesper.nilsson@axis.com>
Applied to net-next, thanks.
^ permalink raw reply
* Re: [PATCH iproute2/net-next v3 0/3] tc: flower: SCTP and other port fixes
From: Stephen Hemminger @ 2016-12-05 18:14 UTC (permalink / raw)
To: Simon Horman; +Cc: netdev
In-Reply-To: <1480755160-27768-1-git-send-email-simon.horman@netronome.com>
On Sat, 3 Dec 2016 09:52:37 +0100
Simon Horman <simon.horman@netronome.com> wrote:
> Hi Stephen,
>
> this short series:
>
> * Makes some improvements to the documentation of flower;
> A follow-up to recent work by Paul Blakey and myself.
> * Corrects some errors introduced when SCTP port matching support was
> recently added.
>
> Changes v2->v3:
> * Rebase
> * Dropped merged patch
>
> Changes v1->v2:
> * Rebase
>
> Simon Horman (3):
> tc: flower: document SCTP ip_proto
> tc: flower: correct name of ip_proto parameter to flower_parse_port()
> tc: flower: make use of flower_port_attr_type() safe and silent
>
> man/man8/tc-flower.8 | 14 +++++++-------
> tc/f_flower.c | 32 +++++++++++++++++---------------
> 2 files changed, 24 insertions(+), 22 deletions(-)
>
Applied to net-next
^ permalink raw reply
* Re: [net PATCH 0/2] IPv4 FIB suffix length fixes
From: David Miller @ 2016-12-05 18:16 UTC (permalink / raw)
To: alexander.h.duyck; +Cc: netdev, rshearma
In-Reply-To: <20161201121605.15499.13176.stgit@ahduyck-blue-test.jf.intel.com>
From: Alexander Duyck <alexander.h.duyck@intel.com>
Date: Thu, 01 Dec 2016 07:27:47 -0500
> In reviewing the patch from Robert Shearman and looking over the code I
> realized there were a few different bugs we were still carrying in the IPv4
> FIB lookup code.
>
> These two patches are based off of Robert's original patch, but take things
> one step further by splitting them up to address two additional issues I
> found.
>
> So first have Robert's original patch which was addressing the fact that
> us calling update_suffix in resize is expensive when it is called per add.
> To address that I incorporated the core bit of the patch which was us
> dropping the update_suffix call from resize.
>
> The first patch in the series does a rename and fix on the push_suffix and
> pull_suffix code. Specifically we drop the need to pass a leaf and
> secondly we fix things so we pull the suffix as long as the value of the
> suffix in the node is dropping.
>
> The second patch addresses the original issue reported as well as
> optimizing the code for the fact that update_suffix is only really meant to
> go through and clean things up when we are decreasing a suffix. I had
> originally added code for it to somehow cause an increase, but if we push
> the suffix when a new leaf is added we only ever have to handle pulling
> down the suffix with update_suffix so I updated the code to reflect that.
>
> As far as side effects the only ones I think that will be obvious should be
> the fact that some routes may be able to be found earlier since before we
> relied on resize to update the suffix lengths, and now we are updating them
> before we add or remove the leaf.
Series applied and queued up for -stable, thanks Alex.
^ permalink raw reply
* Re: [PATCH 0/8] irda: w83977af_ir: Neatening
From: Joe Perches @ 2016-12-05 18:16 UTC (permalink / raw)
To: netdev, David Miller; +Cc: Arnd Bergmann, Samuel Ortiz, linux-kernel
In-Reply-To: <cover.1480014088.git.joe@perches.com>
On Thu, 2016-11-24 at 11:10 -0800, Joe Perches wrote:
> On top of Arnd's overly long udelay patch because I noticed a
> misindented block.
>
> Even though I haven't turned on the netwinder in a box in in the
> garage in who knows how long, if this device is still used somewhere,
> might as well neaten the code too.
Marked as "changes requested" in patchwork?
I didn't see any change requested.
^ permalink raw reply
* Re: [PATCH] net: ping: check minimum size on ICMP header length
From: David Miller @ 2016-12-05 18:19 UTC (permalink / raw)
To: keescook
Cc: netdev, mchong, i, kuznet, jmorris, yoshfuji, kaber, linux-kernel
In-Reply-To: <20161203005853.GA117599@beast>
From: Kees Cook <keescook@chromium.org>
Date: Fri, 2 Dec 2016 16:58:53 -0800
> diff --git a/net/ipv4/ping.c b/net/ipv4/ping.c
> index 205e2000d395..8257be3f032c 100644
> --- a/net/ipv4/ping.c
> +++ b/net/ipv4/ping.c
> @@ -654,7 +654,7 @@ int ping_common_sendmsg(int family, struct msghdr *msg, size_t len,
> void *user_icmph, size_t icmph_len) {
> u8 type, code;
>
> - if (len > 0xFFFF)
> + if (len > 0xFFFF || len < icmph_len)
> return -EMSGSIZE;
As suggested by Lorenzo, please use -EINVAL here.
Thanks.
^ permalink raw reply
* Re: [PATCH net-next v2 0/4] bnxt_en: Add DCBNL support.
From: David Miller @ 2016-12-05 18:22 UTC (permalink / raw)
To: michael.chan; +Cc: netdev
In-Reply-To: <1480731438-22671-1-git-send-email-michael.chan@broadcom.com>
From: Michael Chan <michael.chan@broadcom.com>
Date: Fri, 2 Dec 2016 21:17:14 -0500
> This series adds DCBNL operations to support host-based IEEE DCBX.
>
> v2: Updated to the latest firmware interface spec.
>
> David, please consider this series for net-next.
Series applied, thanks.
^ permalink raw reply
* Re: [PATCH 2/7] net: ethernet: ti: cpdma: fix desc re-queuing
From: Grygorii Strashko @ 2016-12-05 18:22 UTC (permalink / raw)
To: Ivan Khoronzhuk
Cc: David S. Miller, netdev, Mugunthan V N, Sekhar Nori, linux-kernel,
linux-omap
In-Reply-To: <20161202232820.GA15799@khorivan>
On 12/02/2016 05:28 PM, Ivan Khoronzhuk wrote:
> On Fri, Dec 02, 2016 at 10:45:07AM -0600, Grygorii Strashko wrote:
>>
>>
>> On 12/02/2016 05:03 AM, Ivan Khoronzhuk wrote:
>>> On Thu, Dec 01, 2016 at 05:34:27PM -0600, Grygorii Strashko wrote:
>>>> The currently processing cpdma descriptor with EOQ flag set may
>>>> contain two values in Next Descriptor Pointer field:
>>>> - valid pointer: means CPDMA missed addition of new desc in queue;
>>> It shouldn't happen in normal circumstances, right?
>>
>> it might happen, because desc push compete with desc pop.
>> You can check stats values:
>> chan->stats.misqueued
>> chan->stats.requeue
>> under different types of net-loads.
> I've done this, of-course.
> By whole logic the misqueued counter has to cover all cases.
> But that's not true.
>
>>
>> TRM:
>> "
>> If the pNext pointer is initially NULL, and more packets need to be queued for transmit, the software
>> application may alter this pointer to point to a newly appended descriptor. The EMAC will use the new
>> pointer value and proceed to the next descriptor unless the pNext value has already been read. In this
>> latter case, the transmitter will halt on the transmit channel in question, and the software application may
>> restart it at that time. The software can detect this case by checking for an end of queue (EOQ) condition
>> flag on the updated packet descriptor when it is returned by the EMAC.
>> "
> That's true. No issues in desc.
> In the code no any place to update next_desc except submit function.
>
> And this case is supposed to be caught here:
> For submit:
> cpdma_chan_submit()
> spin_lock_irqsave(&chan->lock);
> ...
> --->__cpdma_chan_submit()
> ...
> ------> desc_write(prev, hw_next, desc_dma); // here next pointer is updated, it can be not in time
> ...
> ------> mode = desc_read(prev, hw_mode); // pay attention, it's read after updating next pointer
> ------> if ((mode & CPDMA_DESC_EOQ) &&
> ------> (chan->state == CPDMA_STATE_ACTIVE)) { // here checked if it was late update
> ---------> chan_write(chan, hdp, desc_dma); // here transmit is restarted, if needed
You've forgot about CPPI hw itself - it's not sync by sw, and so continue run in
background - as result, CPPI might read "next" already, but flags are not updated yet.
>
> For process it only caught the fact of late update, but it has to be caught in
> submit() already:
> __cpdma_chan_process()
> spin_lock_irqsave(&chan->lock);
> ------> if (mode & CPDMA_DESC_EOQ) // here transmit is restarted, if needed
> ---------> chan_write(chan, hdp, desc_dma); // but w/o updating next pointer
>
> Seems there is no place where hw_next is updated w/o updating hdp :-| in case
> of late hw_next set. And that is strange. I know it happens, I've checked it
> before of-course. Then I thought, maybe there is some problem with write order,
> thus out of sync, nothing more.
>
>>
>>
>>> So, why it happens only for egress channels? And Does that mean
>>> there is some resynchronization between submit and process function,
>>> or this is h/w issue?
>>
>> no hw issues. this patch just removes one unnecessary I/O access
> No objections against patch. Anyway it's better then before.
> Just want to know the real reason why it happens, maybe there is smth else.
>
>>
>>>
>>>> - null: no more descriptors in queue.
>>>> In the later case, it's not required to write to HDP register, but now
>>>> CPDMA does it.
>>>>
>>>> Hence, add additional check for Next Descriptor Pointer != null in
>>>> cpdma_chan_process() function before writing in HDP register.
>>>>
>>>> Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com>
>>>> ---
>>>> drivers/net/ethernet/ti/davinci_cpdma.c | 2 +-
>>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/drivers/net/ethernet/ti/davinci_cpdma.c b/drivers/net/ethernet/ti/davinci_cpdma.c
>>>> index 0924014..379314f 100644
>>>> --- a/drivers/net/ethernet/ti/davinci_cpdma.c
>>>> +++ b/drivers/net/ethernet/ti/davinci_cpdma.c
>>>> @@ -1152,7 +1152,7 @@ static int __cpdma_chan_process(struct cpdma_chan *chan)
>>>> chan->count--;
>>>> chan->stats.good_dequeue++;
>>>>
>>>> - if (status & CPDMA_DESC_EOQ) {
>>>> + if ((status & CPDMA_DESC_EOQ) && chan->head) {
>>>> chan->stats.requeue++;
>>>> chan_write(chan, hdp, desc_phys(pool, chan->head));
>>>> }
>>>> --
>>>> 2.10.1
>>>>
>>
>> --
>> regards,
>> -grygorii
--
regards,
-grygorii
^ permalink raw reply
* Re: [PATCH V2 net 02/20] net/ena: fix error handling when probe fails
From: Netanel Belgazal @ 2016-12-05 18:23 UTC (permalink / raw)
To: Matt Wilson
Cc: linux-kernel, davem, netdev, dwmw, zorik, alex, saeed, msw,
aliguori, nafea
In-Reply-To: <20161205040955.GC4310@u54ee753d2d1854bda401.ant.amazon.com>
On 12/05/2016 06:09 AM, Matt Wilson wrote:
> On Sun, Dec 04, 2016 at 03:19:20PM +0200, Netanel Belgazal wrote:
>> When driver fails in probe, it will release all resources, including
>> adapter.
>> In case of probe failure, ena_remove should not try to free the adapter
>> resources.
> Please word wrap your commit message around 75 columns.
OK
>
>> Signed-off-by: Netanel Belgazal <netanel@annapurnalabs.com>
> Reviewed-by: Matt Wilson <msw@amazon.com>
>
>> ---
>> drivers/net/ethernet/amazon/ena/ena_netdev.c | 1 +
>> 1 file changed, 1 insertion(+)
>>
>> diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c
>> index 33a760e..397c9bc 100644
>> --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c
>> +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c
>> @@ -3054,6 +3054,7 @@ static int ena_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
>> err_free_region:
>> ena_release_bars(ena_dev, pdev);
>> err_free_ena_dev:
>> + pci_set_drvdata(pdev, NULL);
>> vfree(ena_dev);
>> err_disable_device:
>> pci_disable_device(pdev);
^ permalink raw reply
* Re: [PATCH V2 net 03/20] net/ena: fix queues number calculation
From: Netanel Belgazal @ 2016-12-05 18:25 UTC (permalink / raw)
To: Matt Wilson
Cc: linux-kernel, davem, netdev, dwmw, zorik, alex, saeed, msw,
aliguori, nafea
In-Reply-To: <20161205041130.GD4310@u54ee753d2d1854bda401.ant.amazon.com>
On 12/05/2016 06:11 AM, Matt Wilson wrote:
> On Sun, Dec 04, 2016 at 03:19:21PM +0200, Netanel Belgazal wrote:
>> The ENA driver tries to open a queue per vCPU.
>> To determine how many vCPUs the instance have it uses num_possible_cpus
>> while it should have use num_online_cpus instead.
> use () when referring to functions: num_possible_cpus(), num_online_cpus().
Ack
>
>> Signed-off-by: Netanel Belgazal <netanel@annapurnalabs.com>
> Reviewed-by: Matt Wilson <msw@amazon.com>
>
>> ---
>> drivers/net/ethernet/amazon/ena/ena_netdev.c | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c
>> index 397c9bc..224302c 100644
>> --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c
>> +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c
>> @@ -2667,7 +2667,7 @@ static int ena_calc_io_queue_num(struct pci_dev *pdev,
>> io_sq_num = get_feat_ctx->max_queues.max_sq_num;
>> }
>>
>> - io_queue_num = min_t(int, num_possible_cpus(), ENA_MAX_NUM_IO_QUEUES);
>> + io_queue_num = min_t(int, num_online_cpus(), ENA_MAX_NUM_IO_QUEUES);
>> io_queue_num = min_t(int, io_queue_num, io_sq_num);
>> io_queue_num = min_t(int, io_queue_num,
>> get_feat_ctx->max_queues.max_cq_num);
^ permalink raw reply
* Re: [PATCH v2 net-next 0/4]: Allow head adjustment in XDP prog
From: Jesper Dangaard Brouer @ 2016-12-05 18:25 UTC (permalink / raw)
To: Jakub Kicinski
Cc: Martin KaFai Lau, netdev, Alexei Starovoitov, Brenden Blanco,
Daniel Borkmann, David Miller, Saeed Mahameed, Tariq Toukan,
Kernel Team, brouer
In-Reply-To: <20161205175302.0d5d4fa7@jkicinski-Precision-T1700>
On Mon, 5 Dec 2016 17:53:02 +0000
Jakub Kicinski <kubakici@wp.pl> wrote:
> On Sat, 3 Dec 2016 19:17:22 -0800, Martin KaFai Lau wrote:
> > This series adds a helper to allow head adjusting in XDP prog. mlx4
> > driver has been modified to support this feature. An example is written
> > to encapsulate a packet with an IPv4/v6 header and then XDP_TX it
> > out.
>
> Can we just add a feature to one of four drivers which support XDP
> today and AFAICT leave it completely broken on remaining tree?
>
> I'm not seeing any way for the drivers to inform the stack about their
> capabilities, would that not be a pre-requisite?
Thank you for bringing this up Jakub, very good point. I do think it
must be a pre-requisite that we have a capabilities negotiation
interface for XDP ... before adding new features.
As I've also documented here, and below:
https://prototype-kernel.readthedocs.io/en/latest/networking/XDP/design/design.html#ref-prog-negotiation
Capabilities negotiation
========================
.. Warning:: This interface is missing in the implementation
XDP has hooks and feature dependencies in the device drivers.
Planning for extendability, not all device drivers will necessarily
support all of the future features of XDP, and new feature adoption
in device drivers will occur at different development rates.
Thus, there is a need for the device driver to express what XDP
capabilities or features it provides.
When attaching/loading an XDP program into the kernel, a feature or
capabilities negotiation should be conducted. This implies that an
XDP program needs to express what features it wants to use.
If an XDP program being loaded requests features that the given device
driver does not support, the program load should simply be rejected.
.. note:: I'm undecided on whether to have an query interface, because
users could just use the regular load-interface to probe for
supported options. The downside of probing is the issues SElinux
runs into, of false alarms, when glibc tries to probe for
capabilities.
Implementation issue
--------------------
The current implementation is missing this interface. Worse, the two
actions :ref:`XDP_DROP` and :ref:`XDP_TX` should have been expressed
as two different capabilities, because XDP_TX requires more changes to
the device driver than a simple drop like XDP_DROP.
One can (easily) imagine that an older driver only wants to implement
the XDP_DROP facility. The reason is that XDP_TX would require
changing too much driver code, which is a concern for an old, stable
and time-proven driver.
--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Principal Kernel Engineer at Red Hat
LinkedIn: http://www.linkedin.com/in/brouer
^ permalink raw reply
* Re: [PATCH 1/6] net: ethernet: ti: netcp: add support of cpts
From: Grygorii Strashko @ 2016-12-05 18:25 UTC (permalink / raw)
To: Rob Herring
Cc: David S. Miller, netdev, Mugunthan V N, Richard Cochran,
Sekhar Nori, linux-kernel, linux-omap, devicetree,
Murali Karicheri, Wingman Kwok
In-Reply-To: <20161205144918.oj4jpj65aha3x5gf@rob-hp-laptop>
On 12/05/2016 08:49 AM, Rob Herring wrote:
> On Mon, Nov 28, 2016 at 05:04:23PM -0600, Grygorii Strashko wrote:
>> From: WingMan Kwok <w-kwok2@ti.com>
>>
>> This patch adds support of the cpts device found in the
>> gbe and 10gbe ethernet switches on the keystone 2 SoCs
>> (66AK2E/L/Hx, 66AK2Gx).
>>
>> Signed-off-by: WingMan Kwok <w-kwok2@ti.com>
>> Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com>
>> ---
>> .../devicetree/bindings/net/keystone-netcp.txt | 9 +
>> drivers/net/ethernet/ti/Kconfig | 7 +-
>> drivers/net/ethernet/ti/netcp.h | 2 +-
>> drivers/net/ethernet/ti/netcp_core.c | 18 +-
>> drivers/net/ethernet/ti/netcp_ethss.c | 437 ++++++++++++++++++++-
>> 5 files changed, 459 insertions(+), 14 deletions(-)
>>
>> diff --git a/Documentation/devicetree/bindings/net/keystone-netcp.txt b/Documentation/devicetree/bindings/net/keystone-netcp.txt
>> index 04ba1dc..c37b54e 100644
>> --- a/Documentation/devicetree/bindings/net/keystone-netcp.txt
>> +++ b/Documentation/devicetree/bindings/net/keystone-netcp.txt
>> @@ -113,6 +113,15 @@ Optional properties:
>> will only initialize these ports and attach PHY
>> driver to them if needed.
>>
>> + Properties related to cpts configurations.
>> + - cpts_clock_mult/cpts_clock_shift:
>
> Needs vendor prefix. Don't use '_'.
This module is used as part of OMAP and Keystone SoCs, so names for
this props is ABI already :(
>
>> + used for converting time counter cycles to ns as in
>> +
>> + ns = (cycles * clock_mult) >> _shift
>> +
>> + Defaults: clock_mult, clock_shift = calculated from
>> + CPTS refclk
>
> What does this mean?
>
I'll add more description here.
>> +
>> NetCP interface properties: Interface specification for NetCP sub-modules.
>> Required properties:
>> - rx-channel: the navigator packet dma channel name for rx.
--
regards,
-grygorii
^ permalink raw reply
* Re: [PATCH V2 net 04/20] net/ena: fix ethtool RSS flow configuration
From: Netanel Belgazal @ 2016-12-05 18:26 UTC (permalink / raw)
To: Matt Wilson
Cc: linux-kernel, davem, netdev, dwmw, zorik, alex, saeed, msw,
aliguori, nafea
In-Reply-To: <20161205041815.GE4310@u54ee753d2d1854bda401.ant.amazon.com>
On 12/05/2016 06:18 AM, Matt Wilson wrote:
> On Sun, Dec 04, 2016 at 03:19:22PM +0200, Netanel Belgazal wrote:
>> ena_flow_data_to_flow_hash and ena_flow_hash_to_flow_type
>> treat the ena_flow_hash_to_flow_type enum as power of two values.
>>
>> Change the values of ena_admin_flow_hash_fields to be power of two values.
> Then I generally prefer BIT(0), BIT(1), BIT(2), etc.
I'll use BIT(x)
>
> Also it would be helpful to include some comments about the
> consequences of the current state of the code.
I'll add explanation.
>
> --msw
>
>> Signed-off-by: Netanel Belgazal <netanel@annapurnalabs.com>
>> ---
>> drivers/net/ethernet/amazon/ena/ena_admin_defs.h | 12 ++++++------
>> 1 file changed, 6 insertions(+), 6 deletions(-)
>>
>> diff --git a/drivers/net/ethernet/amazon/ena/ena_admin_defs.h b/drivers/net/ethernet/amazon/ena/ena_admin_defs.h
>> index a46e749..f48c886 100644
>> --- a/drivers/net/ethernet/amazon/ena/ena_admin_defs.h
>> +++ b/drivers/net/ethernet/amazon/ena/ena_admin_defs.h
>> @@ -631,22 +631,22 @@ enum ena_admin_flow_hash_proto {
>> /* RSS flow hash fields */
>> enum ena_admin_flow_hash_fields {
>> /* Ethernet Dest Addr */
>> - ENA_ADMIN_RSS_L2_DA = 0,
>> + ENA_ADMIN_RSS_L2_DA = 0x1,
>>
>> /* Ethernet Src Addr */
>> - ENA_ADMIN_RSS_L2_SA = 1,
>> + ENA_ADMIN_RSS_L2_SA = 0x2,
>>
>> /* ipv4/6 Dest Addr */
>> - ENA_ADMIN_RSS_L3_DA = 2,
>> + ENA_ADMIN_RSS_L3_DA = 0x4,
>>
>> /* ipv4/6 Src Addr */
>> - ENA_ADMIN_RSS_L3_SA = 5,
>> + ENA_ADMIN_RSS_L3_SA = 0x8,
>>
>> /* tcp/udp Dest Port */
>> - ENA_ADMIN_RSS_L4_DP = 6,
>> + ENA_ADMIN_RSS_L4_DP = 0x10,
>>
>> /* tcp/udp Src Port */
>> - ENA_ADMIN_RSS_L4_SP = 7,
>> + ENA_ADMIN_RSS_L4_SP = 0x20,
>> };
>>
>> struct ena_admin_proto_input {
^ permalink raw reply
* Re: [PATCH 1/1] netdev: broadcom: propagate error code
From: David Miller @ 2016-12-05 18:26 UTC (permalink / raw)
To: bianpan2016
Cc: michael.chan, prashant.sreedharan, sbaddipa, netdev, linux-kernel
In-Reply-To: <1480758977-4234-1-git-send-email-bianpan2016@163.com>
From: Pan Bian <bianpan2016@163.com>
Date: Sat, 3 Dec 2016 17:56:17 +0800
> Function bnxt_hwrm_stat_ctx_alloc() always returns 0, even if the call
> to _hwrm_send_message() fails. It may be better to propagate the errors
> to the caller of bnxt_hwrm_stat_ctx_alloc().
>
> Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=188661
>
> Signed-off-by: Pan Bian <bianpan2016@163.com>
Applied.
^ permalink raw reply
* Re: [PATCH 1/1] net: usb: set error code when usb_alloc_urb fails
From: David Miller @ 2016-12-05 18:27 UTC (permalink / raw)
To: bianpan2016; +Cc: woojung.huh, UNGLinuxDriver, netdev, linux-usb, linux-kernel
In-Reply-To: <1480764288-5448-1-git-send-email-bianpan2016@163.com>
From: Pan Bian <bianpan2016@163.com>
Date: Sat, 3 Dec 2016 19:24:48 +0800
> In function lan78xx_probe(), variable ret takes the errno code on
> failures. However, when the call to usb_alloc_urb() fails, its value
> will keeps 0. 0 indicates success in the context, which is inconsistent
> with the execution result. This patch fixes the bug, assigning
> "-ENOMEM" to ret when usb_alloc_urb() returns a NULL pointer.
>
> Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=188771
>
> Signed-off-by: Pan Bian <bianpan2016@163.com>
Applied.
^ permalink raw reply
* Re: [PATCH 1/1] net: bridge: set error code on failure
From: David Miller @ 2016-12-05 18:27 UTC (permalink / raw)
To: bianpan2016; +Cc: netdev, bridge, linux-kernel
In-Reply-To: <1480764803-5577-1-git-send-email-bianpan2016@163.com>
From: Pan Bian <bianpan2016@163.com>
Date: Sat, 3 Dec 2016 19:33:23 +0800
> Function br_sysfs_addbr() does not set error code when the call
> kobject_create_and_add() returns a NULL pointer. It may be better to
> return "-ENOMEM" when kobject_create_and_add() fails.
>
> Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=188781
>
> Signed-off-by: Pan Bian <bianpan2016@163.com>
Applied.
^ permalink raw reply
* Re: [PATCH 1/1] atm: lanai: set error code when ioremap fails
From: David Miller @ 2016-12-05 18:28 UTC (permalink / raw)
To: bianpan2016; +Cc: 3chas3, linux-atm-general, netdev, linux-kernel
In-Reply-To: <1480767945-5676-1-git-send-email-bianpan2016@163.com>
From: Pan Bian <bianpan2016@163.com>
Date: Sat, 3 Dec 2016 20:25:45 +0800
> In function lanai_dev_open(), when the call to ioremap() fails, the
> value of return variable result is 0. 0 means no error in this context.
> This patch fixes the bug, assigning "-ENOMEM" to result when ioremap()
> returns a NULL pointer.
>
> Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=188791
>
> Signed-off-by: Pan Bian <bianpan2016@163.com>
Applied.
^ permalink raw reply
* Re: [PATCH V2 net 13/20] net/ena: change driver's default timeouts
From: Netanel Belgazal @ 2016-12-05 18:28 UTC (permalink / raw)
To: Matt Wilson
Cc: linux-kernel, davem, netdev, dwmw, zorik, alex, saeed, msw,
aliguori, nafea
In-Reply-To: <20161205043524.GJ4310@u54ee753d2d1854bda401.ant.amazon.com>
On 12/05/2016 06:35 AM, Matt Wilson wrote:
> On Sun, Dec 04, 2016 at 03:19:31PM +0200, Netanel Belgazal wrote:
>
> ... because? (they turned out to be too aggressive, I believe.)
Yes, The timeout were too aggressive on some specific machines.
>> Signed-off-by: Netanel Belgazal <netanel@annapurnalabs.com>
>> ---
>> drivers/net/ethernet/amazon/ena/ena_com.c | 4 ++--
>> drivers/net/ethernet/amazon/ena/ena_netdev.h | 7 ++++---
>> 2 files changed, 6 insertions(+), 5 deletions(-)
>>
>> diff --git a/drivers/net/ethernet/amazon/ena/ena_com.c b/drivers/net/ethernet/amazon/ena/ena_com.c
>> index 4147d6e..a550c8a 100644
>> --- a/drivers/net/ethernet/amazon/ena/ena_com.c
>> +++ b/drivers/net/ethernet/amazon/ena/ena_com.c
>> @@ -36,9 +36,9 @@
>> /*****************************************************************************/
>>
>> /* Timeout in micro-sec */
>> -#define ADMIN_CMD_TIMEOUT_US (1000000)
>> +#define ADMIN_CMD_TIMEOUT_US (3000000)
>>
>> -#define ENA_ASYNC_QUEUE_DEPTH 4
>> +#define ENA_ASYNC_QUEUE_DEPTH 16
> Why is this changed at the same time?
It related to the too aggressive thresholds.
On some heavy loaded system we reached to a state where the AENQ
was full so the driver missed some events.
>
>> #define ENA_ADMIN_QUEUE_DEPTH 32
>>
>> #define MIN_ENA_VER (((ENA_COMMON_SPEC_VERSION_MAJOR) << \
>> diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.h b/drivers/net/ethernet/amazon/ena/ena_netdev.h
>> index c081fd3..ed42e07 100644
>> --- a/drivers/net/ethernet/amazon/ena/ena_netdev.h
>> +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.h
>> @@ -39,6 +39,7 @@
>> #include <linux/interrupt.h>
>> #include <linux/netdevice.h>
>> #include <linux/skbuff.h>
>> +#include <linux/u64_stats_sync.h>
> This change seems unrelated.
Removed to a different patch (will be submitted in a new patchset)
>
>> #include "ena_com.h"
>> #include "ena_eth_com.h"
>> @@ -100,7 +101,7 @@
>> /* Number of queues to check for missing queues per timer service */
>> #define ENA_MONITORED_TX_QUEUES 4
>> /* Max timeout packets before device reset */
>> -#define MAX_NUM_OF_TIMEOUTED_PACKETS 32
>> +#define MAX_NUM_OF_TIMEOUTED_PACKETS 128
>>
>> #define ENA_TX_RING_IDX_NEXT(idx, ring_size) (((idx) + 1) & ((ring_size) - 1))
>>
>> @@ -116,9 +117,9 @@
>> #define ENA_IO_IRQ_IDX(q) (ENA_IO_IRQ_FIRST_IDX + (q))
>>
>> /* ENA device should send keep alive msg every 1 sec.
>> - * We wait for 3 sec just to be on the safe side.
>> + * We wait for 6 sec just to be on the safe side.
>> */
>> -#define ENA_DEVICE_KALIVE_TIMEOUT (3 * HZ)
>> +#define ENA_DEVICE_KALIVE_TIMEOUT (6 * HZ)
>>
>> #define ENA_MMIO_DISABLE_REG_READ BIT(0)
>>
^ permalink raw reply
* Re: [PATCH V2 net 10/20] net/ena: remove redundant logic in napi callback for busy poll mode
From: Netanel Belgazal @ 2016-12-05 18:29 UTC (permalink / raw)
To: Eric Dumazet
Cc: linux-kernel, davem, netdev, dwmw, zorik, alex, saeed, msw,
aliguori, nafea
In-Reply-To: <1480916739.18162.516.camel@edumazet-glaptop3.roam.corp.google.com>
On 12/05/2016 07:45 AM, Eric Dumazet wrote:
> On Sun, 2016-12-04 at 15:19 +0200, Netanel Belgazal wrote:
>> sk_busy_loop can call the napi callback few million times a sec.
>> For each call there is unmask interrupt.
>> We want to reduce the number of unmasks.
>>
>> Add an atomic variable that will tell the napi handler if
>> it was called from irq context or not.
>> Unmask the interrupt only from irq context.
>>
>> A schenario where the driver left with missed unmask isn't feasible.
>> when ena_intr_msix_io is called the driver have 2 options:
>> 1)Before napi completes and call napi_complete_done
>> 2)After calling napi_complete_done
>>
>> In the former case the napi will unmask the interrupt as needed.
>> In the latter case napi_complete_done will remove napi from the schedule
>> list so napi will be rescheduled (by ena_intr_msix_io) and interrupt
>> will be unmasked as desire in the 2nd napi call.
>>
>> Signed-off-by: Netanel Belgazal <netanel@annapurnalabs.com>
>> ---
>
> This looks very complicated to me.
>
> I guess you missed the recent patches that happened on net-next ?
You are correct.
I didn't see the patches.
It is much better to use the napi_complete_done() return value.
I'll rework my patch.
>
> 2e713283751f494596655d9125c168aeb913f71d net/mlx4_en: use napi_complete_done() return value
> 364b6055738b4c752c30ccaaf25c624e69d76195 net: busy-poll: return busypolling status to drivers
> 21cb84c48ca0619181106f0f44f3802a989de024 net: busy-poll: remove need_resched() from sk_can_busy_loop()
> 217f6974368188fd8bd7804bf5a036aa5762c5e4 net: busy-poll: allow preemption in sk_busy_loop()
>
> napi_complete_done() return code can be used by a driver,
> no need to add yet another atomic operation in fast path.
>
> Anyway, this looks wrong :
>
> @@ -1186,6 +1201,7 @@ static irqreturn_t ena_intr_msix_io(int irq, void *data)
> {
> struct ena_napi *ena_napi = data;
>
> + atomic_set(&ena_napi->unmask_interrupt, 1);
> napi_schedule(&ena_napi->napi);
>
> You probably wanted :
>
> if (napi_schedule_prep(n)) {
> atomic_set(&ena_napi->unmask_interrupt, 1);
> __napi_schedule(n);
> }
>
>
>
> Please rework this napi poll using core infrastructure.
>
> busypoll logic should be centralized, not reimplemented in different ways in a driver.
>
> Thanks.
>
>
>
^ permalink raw reply
* Re: [PATCH V2 net 07/20] net/ena: refactor ena_get_stats64 to be atomic context safe
From: Netanel Belgazal @ 2016-12-05 18:29 UTC (permalink / raw)
To: Matt Wilson
Cc: linux-kernel, davem, netdev, dwmw, zorik, alex, saeed, msw,
aliguori, nafea
In-Reply-To: <20161205042435.GG4310@u54ee753d2d1854bda401.ant.amazon.com>
On 12/05/2016 06:24 AM, Matt Wilson wrote:
> On Sun, Dec 04, 2016 at 03:19:25PM +0200, Netanel Belgazal wrote:
>> ndo_get_stat64 can be called from atomic context.
>> However the current implementation sends an admin command to retrieve
>> the statistics from the device.
>> This admin commands uses sleep.
> Suggest some comment edits:
>
> ndo_get_stat64() can be called from atomic context, but the current
> implementation sends an admin command to retrieve the statistics from
> the device. This admin command can sleep.
>
>> Refactor the implementation of ena_get_stats64 to take the
>> {rx,tx}bytes/cnt from the driver's inner counters
>> and to take the rx drops counter
>> from the asynchronous keep alive (heart bit) event.
> This patch re-factors the implementation of ena_get_stats64() to use
> the {rx,tx}bytes/count from the driver's inner counters, and to obtain
> the rx drop counter from the asynchronous keep alive (heart bit)
> event.
Applied
> --msw
>
>> Signed-off-by: Netanel Belgazal <netanel@annapurnalabs.com>
>> ---
>> drivers/net/ethernet/amazon/ena/ena_admin_defs.h | 8 ++++
>> drivers/net/ethernet/amazon/ena/ena_netdev.c | 57 +++++++++++++++++-------
>> drivers/net/ethernet/amazon/ena/ena_netdev.h | 1 +
>> 3 files changed, 51 insertions(+), 15 deletions(-)
>>
>> diff --git a/drivers/net/ethernet/amazon/ena/ena_admin_defs.h b/drivers/net/ethernet/amazon/ena/ena_admin_defs.h
>> index f48c886..6d70bf5 100644
>> --- a/drivers/net/ethernet/amazon/ena/ena_admin_defs.h
>> +++ b/drivers/net/ethernet/amazon/ena/ena_admin_defs.h
>> @@ -873,6 +873,14 @@ struct ena_admin_aenq_link_change_desc {
>> u32 flags;
>> };
>>
>> +struct ena_admin_aenq_keep_alive_desc {
>> + struct ena_admin_aenq_common_desc aenq_common_desc;
>> +
>> + u32 rx_drops_low;
>> +
>> + u32 rx_drops_high;
>> +};
>> +
>> struct ena_admin_ena_mmio_req_read_less_resp {
>> u16 req_id;
>>
>> diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c
>> index ad5f78f..962ffb5 100644
>> --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c
>> +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c
>> @@ -2176,28 +2176,46 @@ static struct rtnl_link_stats64 *ena_get_stats64(struct net_device *netdev,
>> struct rtnl_link_stats64 *stats)
>> {
>> struct ena_adapter *adapter = netdev_priv(netdev);
>> - struct ena_admin_basic_stats ena_stats;
>> - int rc;
>> + struct ena_ring *rx_ring, *tx_ring;
>> + unsigned int start;
>> + u64 rx_drops;
>> + int i;
>>
>> if (!test_bit(ENA_FLAG_DEV_UP, &adapter->flags))
>> return NULL;
>>
>> - rc = ena_com_get_dev_basic_stats(adapter->ena_dev, &ena_stats);
>> - if (rc)
>> - return NULL;
>> + for (i = 0; i < adapter->num_queues; i++) {
>> + u64 bytes, packets;
>> +
>> + tx_ring = &adapter->tx_ring[i];
>> +
>> + do {
>> + start = u64_stats_fetch_begin_irq(&tx_ring->syncp);
>> + packets = tx_ring->tx_stats.cnt;
>> + bytes = tx_ring->tx_stats.bytes;
>> + } while (u64_stats_fetch_retry_irq(&tx_ring->syncp, start));
>> +
>> + stats->tx_packets += packets;
>> + stats->tx_bytes += bytes;
>>
>> - stats->tx_bytes = ((u64)ena_stats.tx_bytes_high << 32) |
>> - ena_stats.tx_bytes_low;
>> - stats->rx_bytes = ((u64)ena_stats.rx_bytes_high << 32) |
>> - ena_stats.rx_bytes_low;
>> + rx_ring = &adapter->rx_ring[i];
>> +
>> + do {
>> + start = u64_stats_fetch_begin_irq(&rx_ring->syncp);
>> + packets = rx_ring->rx_stats.cnt;
>> + bytes = rx_ring->rx_stats.bytes;
>> + } while (u64_stats_fetch_retry_irq(&rx_ring->syncp, start));
>>
>> - stats->rx_packets = ((u64)ena_stats.rx_pkts_high << 32) |
>> - ena_stats.rx_pkts_low;
>> - stats->tx_packets = ((u64)ena_stats.tx_pkts_high << 32) |
>> - ena_stats.tx_pkts_low;
>> + stats->rx_packets += packets;
>> + stats->rx_bytes += bytes;
>> + }
>> +
>> + do {
>> + start = u64_stats_fetch_begin_irq(&adapter->syncp);
>> + rx_drops = adapter->dev_stats.rx_drops;
>> + } while (u64_stats_fetch_retry_irq(&adapter->syncp, start));
>>
>> - stats->rx_dropped = ((u64)ena_stats.rx_drops_high << 32) |
>> - ena_stats.rx_drops_low;
>> + stats->rx_dropped = rx_drops;
>>
>> stats->multicast = 0;
>> stats->collisions = 0;
>> @@ -3221,8 +3239,17 @@ static void ena_keep_alive_wd(void *adapter_data,
>> struct ena_admin_aenq_entry *aenq_e)
>> {
>> struct ena_adapter *adapter = (struct ena_adapter *)adapter_data;
>> + struct ena_admin_aenq_keep_alive_desc *desc;
>> + u64 rx_drops;
>>
>> + desc = (struct ena_admin_aenq_keep_alive_desc *)aenq_e;
>> adapter->last_keep_alive_jiffies = jiffies;
>> +
>> + rx_drops = ((u64)desc->rx_drops_high << 32) | desc->rx_drops_low;
>> +
>> + u64_stats_update_begin(&adapter->syncp);
>> + adapter->dev_stats.rx_drops = rx_drops;
>> + u64_stats_update_end(&adapter->syncp);
>> }
>>
>> static void ena_notification(void *adapter_data,
>> diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.h b/drivers/net/ethernet/amazon/ena/ena_netdev.h
>> index 69d7e9e..f0ddc11 100644
>> --- a/drivers/net/ethernet/amazon/ena/ena_netdev.h
>> +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.h
>> @@ -241,6 +241,7 @@ struct ena_stats_dev {
>> u64 interface_up;
>> u64 interface_down;
>> u64 admin_q_pause;
>> + u64 rx_drops;
>> };
>>
>> enum ena_flags_t {
^ permalink raw reply
* Re: [PATCH V2 net 06/20] net/ena: fix NULL dereference when removing the driver after device reset faild
From: Netanel Belgazal @ 2016-12-05 18:30 UTC (permalink / raw)
To: Matt Wilson
Cc: linux-kernel, davem, netdev, dwmw, zorik, alex, saeed, msw,
aliguori, nafea
In-Reply-To: <20161205042915.GH4310@u54ee753d2d1854bda401.ant.amazon.com>
On 12/05/2016 06:29 AM, Matt Wilson wrote:
> On Sun, Dec 04, 2016 at 03:19:24PM +0200, Netanel Belgazal wrote:
>> If for some reason the device stop responding and the device reset failed
>> to recover the device, the mmio register read datastructure will not be
>> reinitialized.
> If for some reason the device stops responding, and the device reset
> fails to recover the device, the MMIO register read data structure
> will not be reinitialized.
OK
>
>> On driver removal, the driver will also tries to reset the device
>> but this time the mmio data structure will be NULL.
> On driver removal, the driver will also try to reset the device, but
> this time the MMIO data structure will be NULL.
OK
>> To solve this issue perform the device reset in the remove function only if
>> the device is runnig.
> To solve this issue, perform the device reset in the remove function
> only if the device is running.
>
> Do you have an example of the NULL pointer dereference that you can
> paste in? It can be helpful for those searching for a fix for a bug
> they've experienced.
I'll add a crash dump.
> --msw
>
>> Signed-off-by: Netanel Belgazal <netanel@annapurnalabs.com>
>> ---
>> drivers/net/ethernet/amazon/ena/ena_netdev.c | 6 +++++-
>> 1 file changed, 5 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c
>> index 224302c..ad5f78f 100644
>> --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c
>> +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c
>> @@ -2516,6 +2516,8 @@ static void ena_fw_reset_device(struct work_struct *work)
>> err:
>> rtnl_unlock();
>>
>> + clear_bit(ENA_FLAG_DEVICE_RUNNING, &adapter->flags);
>> +
>> dev_err(&pdev->dev,
>> "Reset attempt failed. Can not reset the device\n");
>> }
>> @@ -3126,7 +3128,9 @@ static void ena_remove(struct pci_dev *pdev)
>>
>> cancel_work_sync(&adapter->resume_io_task);
>>
>> - ena_com_dev_reset(ena_dev);
>> + /* Reset the device only if the device is running. */
>> + if (test_bit(ENA_FLAG_DEVICE_RUNNING, &adapter->flags))
>> + ena_com_dev_reset(ena_dev);
>>
>> ena_free_mgmnt_irq(adapter);
>>
^ permalink raw reply
* Re: [PATCH V2 net 08/20] net/ena: add hardware hints capability to the driver
From: Netanel Belgazal @ 2016-12-05 18:31 UTC (permalink / raw)
To: Matt Wilson
Cc: linux-kernel, davem, netdev, dwmw, zorik, alex, saeed, msw,
aliguori, nafea
In-Reply-To: <20161205043113.GI4310@u54ee753d2d1854bda401.ant.amazon.com>
On 12/05/2016 06:31 AM, Matt Wilson wrote:
> On Sun, Dec 04, 2016 at 03:19:26PM +0200, Netanel Belgazal wrote:
>> The ENA device can update the ena driver about the desire timeouts.
>> The hardware hints are transmitted as Asynchronous event to the driver.
> This is really a new feature, not a bugfix - correct? If it is a new
> feature, submit it separately. If the built-in defaults need to be
> changed, submit that as a bugfix.
I'll submit this patch as a new feature.
There is a patch that sets the new defaults.
> --msw
>
>> In case the device does not support this capability, the driver
>> will use its own defines.
>>
>> Signed-off-by: Netanel Belgazal <netanel@annapurnalabs.com>
>> ---
>> drivers/net/ethernet/amazon/ena/ena_admin_defs.h | 31 +++++++++
>> drivers/net/ethernet/amazon/ena/ena_com.c | 41 ++++++++---
>> drivers/net/ethernet/amazon/ena/ena_com.h | 5 ++
>> drivers/net/ethernet/amazon/ena/ena_ethtool.c | 1 -
>> drivers/net/ethernet/amazon/ena/ena_netdev.c | 86 +++++++++++++++++++-----
>> drivers/net/ethernet/amazon/ena/ena_netdev.h | 19 +++++-
>> drivers/net/ethernet/amazon/ena/ena_regs_defs.h | 2 +
>> 7 files changed, 157 insertions(+), 28 deletions(-)
>>
>> diff --git a/drivers/net/ethernet/amazon/ena/ena_admin_defs.h b/drivers/net/ethernet/amazon/ena/ena_admin_defs.h
>> index 6d70bf5..35ae511 100644
>> --- a/drivers/net/ethernet/amazon/ena/ena_admin_defs.h
>> +++ b/drivers/net/ethernet/amazon/ena/ena_admin_defs.h
>> @@ -70,6 +70,8 @@ enum ena_admin_aq_feature_id {
>>
>> ENA_ADMIN_MAX_QUEUES_NUM = 2,
>>
>> + ENA_ADMIN_HW_HINTS = 3,
>> +
>> ENA_ADMIN_RSS_HASH_FUNCTION = 10,
>>
>> ENA_ADMIN_STATELESS_OFFLOAD_CONFIG = 11,
>> @@ -749,6 +751,31 @@ struct ena_admin_feature_rss_ind_table {
>> struct ena_admin_rss_ind_table_entry inline_entry;
>> };
>>
>> +/* When hint value is 0, driver should use it's own predefined value */
>> +struct ena_admin_ena_hw_hints {
>> + /* value in ms */
>> + u16 mmio_read_timeout;
>> +
>> + /* value in ms */
>> + u16 driver_watchdog_timeout;
>> +
>> + /* Per packet tx completion timeout. value in ms */
>> + u16 missing_tx_completion_timeout;
>> +
>> + u16 missed_tx_completion_count_threshold_to_reset;
>> +
>> + /* value in ms */
>> + u16 admin_completion_tx_timeout;
>> +
>> + u16 netdev_wd_timeout;
>> +
>> + u16 max_tx_sgl_size;
>> +
>> + u16 max_rx_sgl_size;
>> +
>> + u16 reserved[8];
>> +};
>> +
>> struct ena_admin_get_feat_cmd {
>> struct ena_admin_aq_common_desc aq_common_descriptor;
>>
>> @@ -782,6 +809,8 @@ struct ena_admin_get_feat_resp {
>> struct ena_admin_feature_rss_ind_table ind_table;
>>
>> struct ena_admin_feature_intr_moder_desc intr_moderation;
>> +
>> + struct ena_admin_ena_hw_hints hw_hints;
>> } u;
>> };
>>
>> @@ -857,6 +886,8 @@ enum ena_admin_aenq_notification_syndrom {
>> ENA_ADMIN_SUSPEND = 0,
>>
>> ENA_ADMIN_RESUME = 1,
>> +
>> + ENA_ADMIN_UPDATE_HINTS = 2,
>> };
>>
>> struct ena_admin_aenq_entry {
>> diff --git a/drivers/net/ethernet/amazon/ena/ena_com.c b/drivers/net/ethernet/amazon/ena/ena_com.c
>> index 46aad3a..f1e4f04 100644
>> --- a/drivers/net/ethernet/amazon/ena/ena_com.c
>> +++ b/drivers/net/ethernet/amazon/ena/ena_com.c
>> @@ -508,15 +508,13 @@ static int ena_com_comp_status_to_errno(u8 comp_status)
>> static int ena_com_wait_and_process_admin_cq_polling(struct ena_comp_ctx *comp_ctx,
>> struct ena_com_admin_queue *admin_queue)
>> {
>> - unsigned long flags;
>> - u32 start_time;
>> + unsigned long flags, timeout;
>> int ret;
>>
>> - start_time = ((u32)jiffies_to_usecs(jiffies));
>> + timeout = jiffies + usecs_to_jiffies(admin_queue->completion_timeout);
>>
>> while (comp_ctx->status == ENA_CMD_SUBMITTED) {
>> - if ((((u32)jiffies_to_usecs(jiffies)) - start_time) >
>> - ADMIN_CMD_TIMEOUT_US) {
>> + if (time_is_before_jiffies(timeout)) {
>> pr_err("Wait for completion (polling) timeout\n");
>> /* ENA didn't have any completion */
>> spin_lock_irqsave(&admin_queue->q_lock, flags);
>> @@ -560,7 +558,8 @@ static int ena_com_wait_and_process_admin_cq_interrupts(struct ena_comp_ctx *com
>> int ret;
>>
>> wait_for_completion_timeout(&comp_ctx->wait_event,
>> - usecs_to_jiffies(ADMIN_CMD_TIMEOUT_US));
>> + usecs_to_jiffies(
>> + admin_queue->completion_timeout));
>>
>> /* In case the command wasn't completed find out the root cause.
>> * There might be 2 kinds of errors
>> @@ -600,12 +599,14 @@ static u32 ena_com_reg_bar_read32(struct ena_com_dev *ena_dev, u16 offset)
>> struct ena_com_mmio_read *mmio_read = &ena_dev->mmio_read;
>> volatile struct ena_admin_ena_mmio_req_read_less_resp *read_resp =
>> mmio_read->read_resp;
>> - u32 mmio_read_reg, ret;
>> + u32 mmio_read_reg, timeout, ret;
>> unsigned long flags;
>> int i;
>>
>> might_sleep();
>>
>> + timeout = mmio_read->reg_read_to ? : ENA_REG_READ_TIMEOUT;
>> +
>> /* If readless is disabled, perform regular read */
>> if (!mmio_read->readless_supported)
>> return readl(ena_dev->reg_bar + offset);
>> @@ -626,14 +627,14 @@ static u32 ena_com_reg_bar_read32(struct ena_com_dev *ena_dev, u16 offset)
>>
>> writel(mmio_read_reg, ena_dev->reg_bar + ENA_REGS_MMIO_REG_READ_OFF);
>>
>> - for (i = 0; i < ENA_REG_READ_TIMEOUT; i++) {
>> + for (i = 0; i < timeout; i++) {
>> if (read_resp->req_id == mmio_read->seq_num)
>> break;
>>
>> udelay(1);
>> }
>>
>> - if (unlikely(i == ENA_REG_READ_TIMEOUT)) {
>> + if (unlikely(i == timeout)) {
>> pr_err("reading reg failed for timeout. expected: req id[%hu] offset[%hu] actual: req id[%hu] offset[%hu]\n",
>> mmio_read->seq_num, offset, read_resp->req_id,
>> read_resp->reg_off);
>> @@ -1717,6 +1718,20 @@ int ena_com_get_dev_attr_feat(struct ena_com_dev *ena_dev,
>> memcpy(&get_feat_ctx->offload, &get_resp.u.offload,
>> sizeof(get_resp.u.offload));
>>
>> + /* Driver hints isn't mandatory admin command. So in case the
>> + * command isn't supported set driver hints to 0
>> + */
>> + rc = ena_com_get_feature(ena_dev, &get_resp, ENA_ADMIN_HW_HINTS);
>> +
>> + if (!rc)
>> + memcpy(&get_feat_ctx->hw_hints, &get_resp.u.hw_hints,
>> + sizeof(get_resp.u.hw_hints));
>> + else if (rc == -EPERM)
>> + memset(&get_feat_ctx->hw_hints, 0x0,
>> + sizeof(get_feat_ctx->hw_hints));
>> + else
>> + return rc;
>> +
>> return 0;
>> }
>>
>> @@ -1842,6 +1857,14 @@ int ena_com_dev_reset(struct ena_com_dev *ena_dev)
>> return rc;
>> }
>>
>> + timeout = (cap & ENA_REGS_CAPS_ADMIN_CMD_TO_MASK) >>
>> + ENA_REGS_CAPS_ADMIN_CMD_TO_SHIFT;
>> + if (timeout)
>> + /* the resolution of timeout reg is 100ms */
>> + ena_dev->admin_queue.completion_timeout = timeout * 100000;
>> + else
>> + ena_dev->admin_queue.completion_timeout = ADMIN_CMD_TIMEOUT_US;
>> +
>> return 0;
>> }
>>
>> diff --git a/drivers/net/ethernet/amazon/ena/ena_com.h b/drivers/net/ethernet/amazon/ena/ena_com.h
>> index 509d7b8..6883ee5 100644
>> --- a/drivers/net/ethernet/amazon/ena/ena_com.h
>> +++ b/drivers/net/ethernet/amazon/ena/ena_com.h
>> @@ -96,6 +96,8 @@
>> #define ENA_INTR_MODER_LEVEL_STRIDE 2
>> #define ENA_INTR_BYTE_COUNT_NOT_SUPPORTED 0xFFFFFF
>>
>> +#define ENA_HW_HINTS_NO_TIMEOUT 0xFFFF
>> +
>> enum ena_intr_moder_level {
>> ENA_INTR_MODER_LOWEST = 0,
>> ENA_INTR_MODER_LOW,
>> @@ -232,6 +234,7 @@ struct ena_com_admin_queue {
>> void *q_dmadev;
>> spinlock_t q_lock; /* spinlock for the admin queue */
>> struct ena_comp_ctx *comp_ctx;
>> + u32 completion_timeout;
>> u16 q_depth;
>> struct ena_com_admin_cq cq;
>> struct ena_com_admin_sq sq;
>> @@ -266,6 +269,7 @@ struct ena_com_aenq {
>> struct ena_com_mmio_read {
>> struct ena_admin_ena_mmio_req_read_less_resp *read_resp;
>> dma_addr_t read_resp_dma_addr;
>> + u32 reg_read_to; /* in us */
>> u16 seq_num;
>> bool readless_supported;
>> /* spin lock to ensure a single outstanding read */
>> @@ -335,6 +339,7 @@ struct ena_com_dev_get_features_ctx {
>> struct ena_admin_device_attr_feature_desc dev_attr;
>> struct ena_admin_feature_aenq_desc aenq;
>> struct ena_admin_feature_offload_desc offload;
>> + struct ena_admin_ena_hw_hints hw_hints;
>> };
>>
>> struct ena_com_create_io_ctx {
>> diff --git a/drivers/net/ethernet/amazon/ena/ena_ethtool.c b/drivers/net/ethernet/amazon/ena/ena_ethtool.c
>> index 67b2338f..a1fbfc2 100644
>> --- a/drivers/net/ethernet/amazon/ena/ena_ethtool.c
>> +++ b/drivers/net/ethernet/amazon/ena/ena_ethtool.c
>> @@ -80,7 +80,6 @@ static const struct ena_stats ena_stats_tx_strings[] = {
>> ENA_STAT_TX_ENTRY(tx_poll),
>> ENA_STAT_TX_ENTRY(doorbells),
>> ENA_STAT_TX_ENTRY(prepare_ctx_err),
>> - ENA_STAT_TX_ENTRY(missing_tx_comp),
>> ENA_STAT_TX_ENTRY(bad_req_id),
>> };
>>
>> diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c
>> index 962ffb5..0b718ee 100644
>> --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c
>> +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c
>> @@ -1981,6 +1981,7 @@ static netdev_tx_t ena_start_xmit(struct sk_buff *skb, struct net_device *dev)
>>
>> tx_info->tx_descs = nb_hw_desc;
>> tx_info->last_jiffies = jiffies;
>> + tx_info->print_once = 0;
>>
>> tx_ring->next_to_use = ENA_TX_RING_IDX_NEXT(next_to_use,
>> tx_ring->ring_size);
>> @@ -2554,33 +2555,34 @@ static void check_for_missing_tx_completions(struct ena_adapter *adapter)
>> if (!test_bit(ENA_FLAG_DEV_UP, &adapter->flags))
>> return;
>>
>> + if (adapter->missing_tx_completion_to == ENA_HW_HINTS_NO_TIMEOUT)
>> + return;
>> +
>> budget = ENA_MONITORED_TX_QUEUES;
>>
>> for (i = adapter->last_monitored_tx_qid; i < adapter->num_queues; i++) {
>> tx_ring = &adapter->tx_ring[i];
>>
>> + missed_tx = 0;
>> +
>> for (j = 0; j < tx_ring->ring_size; j++) {
>> tx_buf = &tx_ring->tx_buffer_info[j];
>> last_jiffies = tx_buf->last_jiffies;
>> - if (unlikely(last_jiffies && time_is_before_jiffies(last_jiffies + TX_TIMEOUT))) {
>> - netif_notice(adapter, tx_err, adapter->netdev,
>> - "Found a Tx that wasn't completed on time, qid %d, index %d.\n",
>> - tx_ring->qid, j);
>> + if (unlikely(last_jiffies && time_is_before_jiffies(last_jiffies + adapter->missing_tx_completion_to))) {
>> + if (!tx_buf->print_once)
>> + netif_notice(adapter, tx_err, adapter->netdev,
>> + "Found a Tx that wasn't completed on time, qid %d, index %d.\n",
>> + tx_ring->qid, j);
>>
>> - u64_stats_update_begin(&tx_ring->syncp);
>> - missed_tx = tx_ring->tx_stats.missing_tx_comp++;
>> - u64_stats_update_end(&tx_ring->syncp);
>> + tx_buf->print_once = 1;
>> + missed_tx++;
>>
>> - /* Clear last jiffies so the lost buffer won't
>> - * be counted twice.
>> - */
>> - tx_buf->last_jiffies = 0;
>> -
>> - if (unlikely(missed_tx > MAX_NUM_OF_TIMEOUTED_PACKETS)) {
>> + if (unlikely(missed_tx > adapter->missing_tx_completion_threshold)) {
>> netif_err(adapter, tx_err, adapter->netdev,
>> "The number of lost tx completion is above the threshold (%d > %d). Reset the device\n",
>> - missed_tx, MAX_NUM_OF_TIMEOUTED_PACKETS);
>> + missed_tx, adapter->missing_tx_completion_threshold);
>> set_bit(ENA_FLAG_TRIGGER_RESET, &adapter->flags);
>> + return;
>> }
>> }
>> }
>> @@ -2601,8 +2603,11 @@ static void check_for_missing_keep_alive(struct ena_adapter *adapter)
>> if (!adapter->wd_state)
>> return;
>>
>> - keep_alive_expired = round_jiffies(adapter->last_keep_alive_jiffies
>> - + ENA_DEVICE_KALIVE_TIMEOUT);
>> + if (adapter->keep_alive_timeout == ENA_HW_HINTS_NO_TIMEOUT)
>> + return;
>> +
>> + keep_alive_expired = round_jiffies(adapter->last_keep_alive_jiffies +
>> + adapter->keep_alive_timeout);
>> if (unlikely(time_is_before_jiffies(keep_alive_expired))) {
>> netif_err(adapter, drv, adapter->netdev,
>> "Keep alive watchdog timeout.\n");
>> @@ -2625,6 +2630,44 @@ static void check_for_admin_com_state(struct ena_adapter *adapter)
>> }
>> }
>>
>> +static void ena_update_hints(struct ena_adapter *adapter,
>> + struct ena_admin_ena_hw_hints *hints)
>> +{
>> + struct net_device *netdev = adapter->netdev;
>> +
>> + if (hints->admin_completion_tx_timeout)
>> + adapter->ena_dev->admin_queue.completion_timeout =
>> + hints->admin_completion_tx_timeout * 1000;
>> +
>> + if (hints->mmio_read_timeout)
>> + /* convert to usec */
>> + adapter->ena_dev->mmio_read.reg_read_to =
>> + hints->mmio_read_timeout * 1000;
>> +
>> + if (hints->missed_tx_completion_count_threshold_to_reset)
>> + adapter->missing_tx_completion_threshold =
>> + hints->missed_tx_completion_count_threshold_to_reset;
>> +
>> + if (hints->missing_tx_completion_timeout) {
>> + if (hints->missing_tx_completion_timeout == ENA_HW_HINTS_NO_TIMEOUT)
>> + adapter->missing_tx_completion_to = ENA_HW_HINTS_NO_TIMEOUT;
>> + else
>> + adapter->missing_tx_completion_to =
>> + msecs_to_jiffies(hints->missing_tx_completion_timeout);
>> + }
>> +
>> + if (hints->netdev_wd_timeout)
>> + netdev->watchdog_timeo = msecs_to_jiffies(hints->netdev_wd_timeout);
>> +
>> + if (hints->driver_watchdog_timeout) {
>> + if (hints->driver_watchdog_timeout == ENA_HW_HINTS_NO_TIMEOUT)
>> + adapter->keep_alive_timeout = ENA_HW_HINTS_NO_TIMEOUT;
>> + else
>> + adapter->keep_alive_timeout =
>> + msecs_to_jiffies(hints->driver_watchdog_timeout);
>> + }
>> +}
>> +
>> static void ena_update_host_info(struct ena_admin_host_info *host_info,
>> struct net_device *netdev)
>> {
>> @@ -3036,6 +3079,11 @@ static int ena_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
>> INIT_WORK(&adapter->reset_task, ena_fw_reset_device);
>>
>> adapter->last_keep_alive_jiffies = jiffies;
>> + adapter->keep_alive_timeout = ENA_DEVICE_KALIVE_TIMEOUT;
>> + adapter->missing_tx_completion_to = TX_TIMEOUT;
>> + adapter->missing_tx_completion_threshold = MAX_NUM_OF_TIMEOUTED_PACKETS;
>> +
>> + ena_update_hints(adapter, &get_feat_ctx.hw_hints);
>>
>> init_timer(&adapter->timer_service);
>> adapter->timer_service.expires = round_jiffies(jiffies + HZ);
>> @@ -3256,6 +3304,7 @@ static void ena_notification(void *adapter_data,
>> struct ena_admin_aenq_entry *aenq_e)
>> {
>> struct ena_adapter *adapter = (struct ena_adapter *)adapter_data;
>> + struct ena_admin_ena_hw_hints *hints;
>>
>> WARN(aenq_e->aenq_common_desc.group != ENA_ADMIN_NOTIFICATION,
>> "Invalid group(%x) expected %x\n",
>> @@ -3273,6 +3322,11 @@ static void ena_notification(void *adapter_data,
>> case ENA_ADMIN_RESUME:
>> queue_work(ena_wq, &adapter->resume_io_task);
>> break;
>> + case ENA_ADMIN_UPDATE_HINTS:
>> + hints = (struct ena_admin_ena_hw_hints *)
>> + (&aenq_e->inline_data_w4);
>> + ena_update_hints(adapter, hints);
>> + break;
>> default:
>> netif_err(adapter, drv, adapter->netdev,
>> "Invalid aenq notification link state %d\n",
>> diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.h b/drivers/net/ethernet/amazon/ena/ena_netdev.h
>> index f0ddc11..2897fab 100644
>> --- a/drivers/net/ethernet/amazon/ena/ena_netdev.h
>> +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.h
>> @@ -146,7 +146,18 @@ struct ena_tx_buffer {
>> u32 tx_descs;
>> /* num of buffers used by this skb */
>> u32 num_of_bufs;
>> - /* Save the last jiffies to detect missing tx packets */
>> +
>> + /* Used for detect missing tx packets to limit the number of prints */
>> + u32 print_once;
>> + /* Save the last jiffies to detect missing tx packets
>> + *
>> + * sets to non zero value on ena_start_xmit and set to zero on
>> + * napi and timer_Service_routine.
>> + *
>> + * while this value is not protected by lock,
>> + * a given packet is not expected to be handled by ena_start_xmit
>> + * and by napi/timer_service at the same time.
>> + */
>> unsigned long last_jiffies;
>> struct ena_com_buf bufs[ENA_PKT_MAX_BUFS];
>> } ____cacheline_aligned;
>> @@ -170,7 +181,6 @@ struct ena_stats_tx {
>> u64 napi_comp;
>> u64 tx_poll;
>> u64 doorbells;
>> - u64 missing_tx_comp;
>> u64 bad_req_id;
>> };
>>
>> @@ -270,6 +280,8 @@ struct ena_adapter {
>> struct msix_entry *msix_entries;
>> int msix_vecs;
>>
>> + u32 missing_tx_completion_threshold;
>> +
>> u32 tx_usecs, rx_usecs; /* interrupt moderation */
>> u32 tx_frames, rx_frames; /* interrupt moderation */
>>
>> @@ -283,6 +295,9 @@ struct ena_adapter {
>>
>> u8 mac_addr[ETH_ALEN];
>>
>> + unsigned long keep_alive_timeout;
>> + unsigned long missing_tx_completion_to;
>> +
>> char name[ENA_NAME_MAX_LEN];
>>
>> unsigned long flags;
>> diff --git a/drivers/net/ethernet/amazon/ena/ena_regs_defs.h b/drivers/net/ethernet/amazon/ena/ena_regs_defs.h
>> index 26097a2..c3891c5 100644
>> --- a/drivers/net/ethernet/amazon/ena/ena_regs_defs.h
>> +++ b/drivers/net/ethernet/amazon/ena/ena_regs_defs.h
>> @@ -78,6 +78,8 @@
>> #define ENA_REGS_CAPS_RESET_TIMEOUT_MASK 0x3e
>> #define ENA_REGS_CAPS_DMA_ADDR_WIDTH_SHIFT 8
>> #define ENA_REGS_CAPS_DMA_ADDR_WIDTH_MASK 0xff00
>> +#define ENA_REGS_CAPS_ADMIN_CMD_TO_SHIFT 16
>> +#define ENA_REGS_CAPS_ADMIN_CMD_TO_MASK 0xf0000
>>
>> /* aq_caps register */
>> #define ENA_REGS_AQ_CAPS_AQ_DEPTH_MASK 0xffff
^ permalink raw reply
* Re: [PATCH v3 00/13] net: ethernet: ti: cpts: update and fixes
From: Grygorii Strashko @ 2016-12-05 18:31 UTC (permalink / raw)
To: Richard Cochran
Cc: David S. Miller, netdev-u79uwXL29TY76Z2rM5mHXA, Mugunthan V N,
Sekhar Nori, linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-omap-u79uwXL29TY76Z2rM5mHXA, Rob Herring,
devicetree-u79uwXL29TY76Z2rM5mHXA, Murali Karicheri, Wingman Kwok,
Thomas Gleixner
In-Reply-To: <20161203092211.GA2762@netboy>
On 12/03/2016 03:22 AM, Richard Cochran wrote:
> On Fri, Dec 02, 2016 at 02:30:10PM -0600, Grygorii Strashko wrote:
>> It is preparation series intended to clean up and optimize TI CPTS driver to
>> facilitate further integration with other TI's SoCs like Keystone 2.
>>
>> Changes in v3:
>> - patches reordered: fixes and small updates moved first
>> - added comments in code about cpts->cc_mult
>> - conversation range (maxsec) limited to 10sec
>
> On net-next:
>
> $ git am ~/grygorii.strashko
> Applying: net: ethernet: ti: cpts: switch to readl/writel_relaxed()
> Applying: net: ethernet: ti: allow cpts to be built separately
> error: patch failed: drivers/net/ethernet/ti/cpsw.c:1963
> error: drivers/net/ethernet/ti/cpsw.c: patch does not apply
> Patch failed at 0002 net: ethernet: ti: allow cpts to be built separately
Sorry for that, also there build error due to patch reordering :(
>
> Also, you have the order of the SOB tags wrong. The author's SOB goes
> first.
Will fix and resend, sorry again.
--
regards,
-grygorii
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH 0/8] irda: w83977af_ir: Neatening
From: David Miller @ 2016-12-05 18:33 UTC (permalink / raw)
To: joe; +Cc: netdev, arnd, samuel, linux-kernel
In-Reply-To: <1480961801.27883.4.camel@perches.com>
From: Joe Perches <joe@perches.com>
Date: Mon, 05 Dec 2016 10:16:41 -0800
> On Thu, 2016-11-24 at 11:10 -0800, Joe Perches wrote:
>> On top of Arnd's overly long udelay patch because I noticed a
>> misindented block.
>>
>> Even though I haven't turned on the netwinder in a box in in the
>> garage in who knows how long, if this device is still used somewhere,
>> might as well neaten the code too.
>
> Marked as "changes requested" in patchwork?
>
> I didn't see any change requested.
Sorry if I mis-marked it, please resubmit the series.
^ permalink raw reply
* Re: [PATCH V2 net 05/20] net/ena: fix RSS default hash configuration
From: Netanel Belgazal @ 2016-12-05 18:32 UTC (permalink / raw)
To: Matt Wilson
Cc: linux-kernel, davem, netdev, dwmw, zorik, alex, saeed, msw,
aliguori, nafea
In-Reply-To: <20161205042028.GF4310@u54ee753d2d1854bda401.ant.amazon.com>
On 12/05/2016 06:20 AM, Matt Wilson wrote:
> On Sun, Dec 04, 2016 at 03:19:23PM +0200, Netanel Belgazal wrote:
>> ENA default hash configure IPv4_frag hash twice instead of
> configure -> configures. You may want to include "erroneously". What
> is the consequence of this bug?
I'll fix and I'll add explain the effect of this bug.
>
>> configure non ip packets.
> configuring non-IP packets.
OK.
>
> --msw
>
>> Signed-off-by: Netanel Belgazal <netanel@annapurnalabs.com>
>> ---
>> drivers/net/ethernet/amazon/ena/ena_com.c | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/drivers/net/ethernet/amazon/ena/ena_com.c b/drivers/net/ethernet/amazon/ena/ena_com.c
>> index 3066d9c..46aad3a 100644
>> --- a/drivers/net/ethernet/amazon/ena/ena_com.c
>> +++ b/drivers/net/ethernet/amazon/ena/ena_com.c
>> @@ -2184,7 +2184,7 @@ int ena_com_set_default_hash_ctrl(struct ena_com_dev *ena_dev)
>> hash_ctrl->selected_fields[ENA_ADMIN_RSS_IP4_FRAG].fields =
>> ENA_ADMIN_RSS_L3_SA | ENA_ADMIN_RSS_L3_DA;
>>
>> - hash_ctrl->selected_fields[ENA_ADMIN_RSS_IP4_FRAG].fields =
>> + hash_ctrl->selected_fields[ENA_ADMIN_RSS_NOT_IP].fields =
>> ENA_ADMIN_RSS_L2_DA | ENA_ADMIN_RSS_L2_SA;
>>
>> for (i = 0; i < ENA_ADMIN_RSS_PROTO_NUM; i++) {
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox