re:

re:

[Georges PICARD]

Re:

[ZHAO Wei]

>     I am a student, I think that skb has all the information 
> that is needed for sending and receiving.So I get the skb 
> pointer at NF_IP_POST_ROUTING, put it in a chain  organized 
> by myself (I use a spinlock_t  lock to control the access of 
> the chain, I named it mylock), and return NF_STOLEN. 
>   I make a  tq_timer task to start ip_finish_output2(I export 
> it from kernel),ip_finish_output2 use the skb from my chain.I 
> can make ftp run ok for almost 1 hour, but then the system will 
> carsh with this information:
>  
>         ds:0018    es:0018    ss:0018
>         process swapper(pid:0, stackpage = c0265000)
>         stack: c01a07ea c173a088 ...........
>         call trace:[<c01a07ea>]    [<c01a156d>]......
>         code: 0f 0b 89 7c 24 04 b8 03 00 00 00......
>         <0>kernel panic: Aiee, killing interrupt handler!
>         In interrupt handler - not syncing!

You could try ksymoops to get some more info from the above. You
will get a call trace and that will reveal.

Re:

[Patty Crazon]

[-- Attachment #1: Type: text/plain, Size: 3 bytes --]

R  

Re:

[Roman]

[-- Attachment #1: Type: text/plain, Size: 298 bytes --]

pfnhua bmhqn nnnpfhjk
wumdnm hodxul nqekvon mgcgmyxwh
dtoxz kripdjo slbutzcq laoegcf lpqubl
cyzwa rikutagqh jygfoocbh xdxvpjic
sradhdk ibvknykv daidbjfw eimstz nqkgx
tqbqjqqu- jklct afxxk
rzgficaug jqcpnv oeudwamu awagcykbh wxrzx
zwglumh gbdkpm surrq oqhlxf
tehym wzdtkf anxzrga nsirpygmy

[-- Attachment #2: Type: text/html, Size: 2961 bytes --]

RE:

[Gennie Letty]

[-- Attachment #1: Type: text/plain, Size: 84 bytes --]

being ways man kept separate alone sooner turned started during 
is however people 

[-- Attachment #2: Type: text/html, Size: 4127 bytes --]

Re:

[Îâñàíà]

[-- Attachment #1: Type: text/html, Size: 439 bytes --]

[-- Attachment #2: yay8rnea.gif --]
[-- Type: image/gif, Size: 18654 bytes --]

Re:

[Teresa Dewitt]

-Sensattional revolution in medicine!

-Enlarge your penis up to 10 cm or up to 4 inches!

-It's herbal solution what hasn't side effect, but has 100% guaranted results!

-Don't lose your chance and but know wihtout doubts, you will be i`mpressed with results!

 Clisk here: http://beardeddragonworld.info










everybody photogenic roomy scenic crosslink part chaotic bravura asynchronous wichita
cloven avid testamentary antisemitism blackbody dauphin gear accuse kirchner
indium religion adulthood burma expenditure attract emasculate ashen magneto degumming manuel
endogamous courageous low landau transfinite pastiche abbott solitaire
vignette sooth delight coauthor bali auckland artisan chimera
brigade ellipsometer masonry vella childbear gloucester beat peppy

Re:

[Arnaldo Carvalho de Melo]

On 5/16/06, Chris Boot <bootc@bootc.net> wrote:
> Hi,
>
> I've just seen the following assertions pop out of one of my servers
> running 2.6.16.9 with grsecurity. I've searched the archives of LKML
> and netdev and I've only found posts relating to 2.6.9, after which
> some related bugs were fixed... It looks like these bugs are related
> to e1000, which is the driver I'm using. The system was running 24
> days before these appeared and it's still running absolutely fine.
>
> May 16 09:15:12 baldrick kernel: [6442250.504000] KERNEL: assertion (!
> sk->sk_forward_alloc) failed at net/core/stream.c (283)
> May 16 09:15:12 baldrick kernel: [6442250.513000] KERNEL: assertion (!
> sk->sk_forward_alloc) failed at net/ipv4/af_inet.c (150)
>
> baldrick bootc # ethtool -k eth0
> Offload parameters for eth0:
> rx-checksumming: on
> tx-checksumming: on
> scatter-gather: on
> tcp segmentation offload: on

I guess just disable TSO or use latest kernel from git, it has a fix for this.

- Arnaldo

RE:

[Sheryl Felix]

Hey Netdev!
at first I should tell you about my appear, so I got E-mail from one DATING AGENCY but really that was VERY STRANGE for me because I closed all my accounts at DATING SITES, because I don't like people who are interested just in non SERIOUS RELATIONS, I have much FRIENDS from that sites but really I did not find someone special for me....
But I got your E-mail address and thought "MAYBE THAT IS MY DESTINY" to find someone special?

Really there was written that you wish to know me. 
So I don't know where you did get MY E-MAIL ADDRESS but I hope that is NOT JUST MISTAKE.
I hope to hear from you soon....
If you decide to answer me I promise to SEND YOU big LETTER and MY BEST PHOTOS !!! I'd like to learn more about you. PLEASE, WRITE ME some lines about your personality, your hobbies, your way of life. I'm really interested to know!  
As for me, I'm an easy-going and open-hearted person. I take life as it comes and have optimistic views. It doesn't mean that nothing makes me sad, but I consider all the difficulties in my life to be useful for me.   


I'm very communicative and like to spend time in a good company. I enjoy outdoors activities and sport. What about you? Do you go in for sports?  
Hope to hear from you soon, please use solaris13@HotPOP.com to answer me ! I wit your letter with large impatience . Please do it for me .
 Katya

Thu, 25 May 2006 15:41:05 +0500
disney diffident

RE:.

[Stanley Morse]

Hello!
Our company Barcelo Travel Inc. seek enthusiastic, organised and alert individual to 
support our busy sales offices. If you live in germany our offer its good chance change your liife.
You must have excellent customer relations, communication and administration skills 
Successful candidates will be required to work in Main our Office for approximately 
one month.
To apply, please email CV to barcelotravinc@aol.com
regards,
Dominico Barcelo

Re:

[Jeff Garzik]

Jay Vosburgh wrote:
> From: Karsten Keil <kkeil@suse.de>
> 
> In bond_alb_monitor the bond->curr_slave_lock write lock is taken
> and then dev_set_promiscuity maybe called which can take some time,
> depending on the network HW. If a network IRQ for this card come in
> the softirq handler maybe try to deliver more packets which end up in
> a request to the read lock of bond->curr_slave_lock -> deadlock.
> This issue was found by a test lab during network stress tests, this patch
> disable the softirq handler for this case and solved the issue.
> 
> Signed-off-by: Karsten Keil <kkeil@suse.de>
> Acked-by: Jay Vosburgh <fubar@us.ibm.com>

applied, though note that your email was slightly corrupted.  It 
included _two_ Subject headers, making the email non-compliant with RFC822.

	Jeff

Re:

[pradeep singh]

On 7/5/07, Bhanu Kalyan Chetlapalli <chbhanukalyan@gmail.com> wrote:
> subscribe netdev
oops perhaps you need to send it to majordomo :-)

> -
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>


-- 
Pradeep

Re:

[Bhanu Kalyan Chetlapalli]

On 7/6/07, pradeep singh <pradeep.rautela@gmail.com> wrote:
> On 7/5/07, Bhanu Kalyan Chetlapalli <chbhanukalyan@gmail.com> wrote:
> > subscribe netdev
> oops perhaps you need to send it to majordomo :-)

Sorry about that, I just copied the wrong address, and realized it
immediately after (as it often happens for me) pressing send :(

>
> > -
> > To unsubscribe from this list: send the line "unsubscribe netdev" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >
>
>
> --
> Pradeep
>


-- 
The box said "Requires Windows Vista or better." So I installed LINUX

Re:

[Segher Boessenkool]

>> "compilation unit" is a C standard term.  It typically boils down
>> to "single .c file".
>
> As you mentioned later, "single .c file with all the other files 
> (headers
> or other .c files) that it pulls in via #include" is actually 
> "translation
> unit", both in the C standard as well as gcc docs.

Yeah.  "single .c file after preprocessing".  Same thing :-)

> "Compilation unit"
> doesn't seem to be nearly as standard a term, though in most places it
> is indeed meant to be same as "translation unit", but with the new gcc
> inter-module-analysis stuff that you referred to above, I suspect one 
> may
> reasonably want to call a "compilation unit" as all that the compiler 
> sees
> at a given instant.

That would be a bit confusing, would it not?  They'd better find
some better name for that if they want to name it at all (remember,
none of these optimisations should have any effect on the semantics
of the program, you just get fewer .o files etc.).


Segher

Re:

[Wolfgang Grandegger]

Marc Kleine-Budde wrote:
> Hi,
> 
> This patch series adds support for the Atmel CAN controller as found
> on the AT91SAM9263.
> 
> It adds the at91_can to the generic device definition, activates the CAN
> controller on the at91sam9263ek and adds the driver itself.
> 
> Changes since V1:
> - let Kconfig depend on CAN_DEV
> - add example how driver is used in baord file
> 
> Please review and consider for inclusion.

A Vx patch suffix would be nice next time.

> cheers, Marc
> 
> Marc Kleine-Budde (3):
>       at91sam9263: add at91_can device to generic device definition
>       at91sam9263ek: activate at91 CAN controller
>       at91_can: add driver for Atmel's CAN controller on AT91SAM9263

I have just added my "signed-off-by" for the Socket-CAN patch #3. Don't
known who will take care of the other patches.

Wolfgang.

RE:

[Irish News Center]

You've won £750,000.Send:Name,Age,Country

RE:

[OFFICE OF THE SENATE]

To celebrate the 30th anniversary celebration,We are giving out a yearly donation of The ATM Card Value is $6.8 million USD to 2 lucky recipients,as New Year promotion from the W.H.O,UN, and the EU in accordance with the enabling act of Parliament. back with: Names: Address: Sex:

Re;

[William Wilcox]

Good day!
My name is Sir William Wilcox,I work with the Euro Lottery. I can help you
win 4,528,000 GBP.But I charge 40% of the winning.Can we do this deal
together? Email Me: william_wilcox@live.co.uk

Regards,
William Wilcox

RE,

[FROM CENTRAL BANK]

Very Urgently,
We Conclude Our Meeting Today That $10.7m should be pay to you 
as your contract entitlement. The Payment Will Come To You Via Diplomatic 
Carrier Service:Re- Comfirm this informations as follows.
Your Full Name,
Home 
Address,
Direct Phone No,
Occupation And Age.

Dr. Sanusi A. Lamido
Tel:+234 
8067884885

Re:

[David Stevens]

Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org> wrote on 09/27/2010 
01:05:00 PM:

> On Mon, Sep 27, 2010 at 12:32:45PM -0700, David Stevens wrote:
> 
> > You can, of course, add a querier (or configure it, assuming an
> > attached switch supports it) and set the query interval and
> > robustness count as appropriate for that network.
> 
> Presumably the IPoIB multicast router should already be the querier..
> How does this help handling joins to new groups?

        Because a querier can set the robustness value and
query interval to anything you want. In the original report,
he's not running a querier. The fact that it's a new group
doesn't matter -- these are per-interface.

                                                        +-DLS

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re:

[Christoph Lameter]

On Mon, 27 Sep 2010, David Stevens wrote:

>         Because a querier can set the robustness value and
> query interval to anything you want. In the original report,
> he's not running a querier. The fact that it's a new group
> doesn't matter -- these are per-interface.

The per interface settings are used to force an IGMP version overriding
any information by the queriers. You would not want to enable that because
it disables support for other IGMP versions. Without the override
different version of IGMP can be handled per MC group.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re:

[Bob Arendt]

On 09/27/10 13:23, Christoph Lameter wrote:
> On Mon, 27 Sep 2010, David Stevens wrote:
>
>>          Because a querier can set the robustness value and
>> query interval to anything you want. In the original report,
>> he's not running a querier. The fact that it's a new group
>> doesn't matter -- these are per-interface.
>
> The per interface settings are used to force an IGMP version overriding
> any information by the queriers. You would not want to enable that because
> it disables support for other IGMP versions. Without the override
> different version of IGMP can be handled per MC group.
>
If a network vlan has IGMPv3 capability, then it should be able
to support both v2 and v3 Joins (clients).  But if the vlan is
IGMPv2 only, then an initial Join from a Linux client might go out
as v3 (if it hasn't seen a query yet) and be ignored.  I believe
this is the case that force_igmp_version really addresses.

And it turns out that  force_igmp_version=2 doesn't fully work.
If the host sees a IGMPv3 query, it still responds with a v3 Join
despite the flag.  Bug report and candidate patch here:
https://bugzilla.kernel.org/show_bug.cgi?id=18212
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re:

[David Stevens]

Christoph Lameter <cl-vYTEC60ixJUAvxtiuMwx3w@public.gmane.org> wrote on 09/27/2010 01:23:00 PM:

> From: Christoph Lameter <cl-vYTEC60ixJUAvxtiuMwx3w@public.gmane.org>
> To: David Stevens/Beaverton/IBM@IBMUS
> Cc: Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>, linux-
> rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, netdev-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, netdev-
> owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Bob Arendt <rda-x0S3BwdUo6DQT0dZR+AlfA@public.gmane.org>
> Date: 09/27/2010 01:23 PM
> Subject: Re:
> 
> On Mon, 27 Sep 2010, David Stevens wrote:
> 
> >         Because a querier can set the robustness value and
> > query interval to anything you want. In the original report,
> > he's not running a querier. The fact that it's a new group
> > doesn't matter -- these are per-interface.
> 
> The per interface settings are used to force an IGMP version overriding
> any information by the queriers.

        No. I'm not talking about the force_igmp_tunable here, I'm talking
about the per-interface robustness and interval settings which come from
the querier (whatever version you are using).

> You would not want to enable that because
> it disables support for other IGMP versions. Without the override
> different version of IGMP can be handled per MC group.

        No. IGMPv3 includes backward compatibility for both IGMPv2 and
IGMPv1. If queries for an earlier version are present, that is the
IGMP version all use, and the features of the later version are not
available to anyone.

                                                                +-DLS


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re:

[David Stevens]

Bob Arendt <rda@rincon.com> wrote on 09/27/2010 01:54:36 PM:
 
> On 09/27/10 13:23, Christoph Lameter wrote:
> > On Mon, 27 Sep 2010, David Stevens wrote:
> >
> >>          Because a querier can set the robustness value and
> >> query interval to anything you want. In the original report,
> >> he's not running a querier. The fact that it's a new group
> >> doesn't matter -- these are per-interface.
> >
> > The per interface settings are used to force an IGMP version 
overriding
> > any information by the queriers. You would not want to enable that 
because
> > it disables support for other IGMP versions. Without the override
> > different version of IGMP can be handled per MC group.
> >
> If a network vlan has IGMPv3 capability, then it should be able
> to support both v2 and v3 Joins (clients).  But if the vlan is
> IGMPv2 only, then an initial Join from a Linux client might go out
> as v3 (if it hasn't seen a query yet) and be ignored.  I believe
> this is the case that force_igmp_version really addresses.

        Not really. It's for the case where there is no querier at all,
but a snooping switch that only supports IGMPv2. After any query has
put an interface in IGMPv2 mode (or IGMPv1), the initial report for
all joins will use the earlier protocol. It isn't per-group, it's
per interface, and you cannot mix versions of IGMP on the same network.

> 
> And it turns out that  force_igmp_version=2 doesn't fully work.
> If the host sees a IGMPv3 query, it still responds with a v3 Join
> despite the flag.  Bug report and candidate patch here:
> https://bugzilla.kernel.org/show_bug.cgi?id=18212

        This is a special case. The "correct" alternative is to drop
the query and not send any report at all. Sending an answer in the
originating protocol doesn't hurt anything here, because MC routers
are required to use the earlier version too; there should be no such
thing as an "IGMPv3-only querier" as in that report. IGMPv3 compliance
*requires* falling back to IGMPv2 if there is a v2 query by another
router. 
        By answering instead of dropping, it allows fuller filter
information from a manual query to be returned even if the network
is using v2 MC routers, but dropping and ignoring the query as
required by RFC does not fix the bug & patch submitter's problem.
Which is why I also NACKed that patch.

                                                        +-DLS

Re:

[Christoph Lameter]

On Mon, 27 Sep 2010, David Stevens wrote:

>         No. I'm not talking about the force_igmp_tunable here, I'm talking
> about the per-interface robustness and interval settings which come from
> the querier (whatever version you are using).

The igmp subsystem currently does not keep state on the interface layer
about robustness etc. An interval setting is only kept for IGMP v3 and
used only for general query timeouts with igmp V3. The interval is
different one from the one used for the host membership reports.

Looking at the spec I get the impression that these variables seems to be
mainly of interest to router to router communications?

RE:

[Debashis Dutt]

-----Original Message-----
From: David Miller [mailto:davem@davemloft.net] 
Sent: Thursday, October 21, 2010 12:57 AM
To: Debashis Dutt
Cc: netdev@vger.kernel.org; Rasesh Mody; Jing Huang; Akshay Mathur
Subject: 


People are very unlikely to read your posting because you
did not provide a subject line.

>>>
Thanks David, for looking at this. I have already reposted in netdev with the correct
subject line. 

Please provide your suggestions/feedback as required.

Thanks
--Debashis

Re:

[Young Chang]

My name is Mr. Young Chang, I work with the Mevas Bank bank here in Hong Kong. I have a business proposal that i will like you to handle with me from my bank worth $19.7m. If interested, kindly get back to me for more details.

Re:....

[Young Chang]

May I ask if you would be eligible to pursue a Business Proposal of $19.7m with me if you dont mind? Let me know if you are interested.

Re:,,,,,

[young chang]

May I ask if you would be eligible to pursue a Business Proposal of $19.7m with me if you dont mind? Let me know if you are interested.

Re:...

[Young Chang]

***********************
This message has been scanned by the InterScan for CSC SSM and found to be free of known security risks.
***********************



May I ask if you would be eligible to pursue a Business Proposal of $19.7m with me if you dont mind? Let me know if you are interested.

The information contained in this email and any attachments is confidential and may be subject to copyright or other intellectual property protection. If you are not the intended recipient, you are not authorized to use or disclose this information, and we request that you notify us by reply mail or telephone and delete the original message from your mail system.

E-mail transmission cannot be guaranteed to be secure or error-free as information could be
intercepted, corrupted, lost, destroyed, arrive late or incomplete or contain viruses.
The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission.

Wijeya Newspapers Ltd : 2010

Re:

[San Mehat]

Pls disregard in favor of the one with an actual subject line :P

-san

On Thu, Aug 18, 2011 at 3:07 PM, San Mehat <san@google.com> wrote:
>
> TL;DR
> -----
> In this RFC we propose the introduction of the concept of hardware socket
> offload to the Linux kernel. Patches will accompany this RFC in a few days,
> but we felt we had enough on the design to solicit constructive discussion
> from the community at-large.
>
> BACKGROUND
> ----------
> Many applications within enterprise organizations suitable for virtualization
> neither require nor desire a connection to the full internal Ethernet+IP
> network.  Rather, some specific socket connections -- for processing HTTP
> requests, making database queries, or interacting with storage -- are needed,
> and IP networking in the application may typically be discouraged for
> applications that do not sit on the edge of the network. Furthermore, removing
> the application's need to understand where its inputs come from / go to within
> the networking fabric can make save/restore/migration of a virtualized
> application substantially easier - especially in large clusters and on fabrics
> which can't handle IP re-assignment.
>
> REQUIREMENTS
> ------------
>  * Allow VM connectivity to internal resources without requiring additional
>   network resources (IPs, VLANs, etc).
>  * Easy authentication of network streams from a trusted domain (vmm).
>  * Protect host-kernel & network-fabric from direct exposure to untrusted
>   packet data-structures.
>  * Support for multiple distributions of Linux.
>  * Minimal third-party software maintenance burden.
>  * To be able to co-exist with the existing network stack and ethernet virtual
>   devices in the event that an applications specific requirements cannot be
>   met by this design.
>
> DESIGN
> ------
> The Berkeley sockets coprocessor is a virtual PCI device which has the ability
> to offload socket activity from an unmodified application at the BSD sockets
> layer (Layer 4).  Offloaded socket requests bypass the local operating systems
> networking stack entirely via the card and are relayed into the VMM
> (Virtual Machine Manager) for processing. The VMM then passes the request to a
> socket backend for handling. The difference between a socket backend and a
> traditional VM ethernet backend is that the socket backend receives layer 4
> socket (STREAM/DGRAM) requests instead of a multiplexed stream of layer 2
> packets (ethernet) that must be interpreted by the host. This technique also
> improves security isolation as the guest is no longer constructing packets which
> are evaluated by the host or underlying network fabric; packet construction
> happens in the host.
>
> Lastly, pushing socket processing back into the host allows for host-side
> control of the network protocols used, which limits the potential congestion
> problems that can arise when various guests are using their own congestion
> control algorithms.
>
> ================================================================================
>
>           +-----------------------------------------------------------------+
>           |                                                                 |
>  guest    |                      unmodified application                     |
> userspace  +-----------------------------------------------------------------+
>           |                         unmodified libc                         |
>           +-----------------------------------------------------------------+
>                            |                             / \
>                            |                              |
> =========================== | ============================ | ===================
>                            |                              |
>                           \ /                             |
>                 +------------------------------------------------------+
>                 |                       socket core                    |
>                 +----+============+------------------------------------+
>                      |    INET    |                   |         / \
>  guest               +-----+------+                   |          |
>  kernel              | TCP | UDP  |                   |          |
>                      +-----+------+                   | L4 reqs  |
>                      |   NETDEV   |                   |          |
>                      +------------+                   |          |
>                      | virtio_net |                  \ /         |
>                      +------------+               +------------------+
>                          |   / \                  |    hw_socket     |
>                          |    |                   +------------------+
>                          |    |                   |  virtio_socket   |
>                          |    |                   +------------------+
>                          |    |                        |       / \
> ========================= | == | ====================== | ====== | =============
>                         \ /   |                       \ /       |
>  host           +---------------------+        +------------------------+
> userspace        |  virito net device  |        |  virtio socket device  |
>  (vmm)          +---------------------+        +------------------------+
>                 |  ethernet backend   |        |     socket backend     |
>                 +---------------------+        +------------------------+
>                        |     / \                      |        / \
>                 L2     |      |                       |         |     L4
>               packets  |      |                      \ /        |  requests
>                        |      |                +-----------------------+
>                        |      |                |    Socket Handlers    |
>                        |      |                +-----------------------+
>                        |      |                       |        / \
> ======================= | ==== | ===================== | ======= | =============
>                        |      |                       |         |
>   host                \ /     |                      \ /        |
>  kernel
>
> ================================================================================
>
> One of the most appealing aspects of this design (to application developers) is
> that this approach can be completely transparent to the application, provided
> we're able to intercept the application's socket requests in such a way that we
> do not impact performance in a negative fashion, yet retain the API semantics
> the application expects. In the event that this design is not suitable for an
> application, the virtual machine may be also fitted with a normal virtual
> ethernet device in addition to the co-processor (as shown in the diagram above).
>
> Since we wish to allow these paravirtualized sockets to coexist peacefully with
> the existing Linux socket system, we've chosen to introduce the idea that a
> socket can at some point transition from being managed by the O/S socket system
> to a more enlightened 'hardware assisted' socket. The transition is managed by
> a 'socket coprocessor' component which intercepts and gets first right of
> refusal on handling certain global socket calls (connect, sendto, bind, etc...).
> In this initial design, the policy on whether to transition a socket or not is
> made by the virtual hardware, although we understand that further measurement
> into operation latency is warranted.
>
> In the event the determination is made to transition a socket to hw-assisted
> mode, the socket is marked as being assisted by hardware, and all socket
> operations are offloaded to hardware.
>
> The following flag values have been added to struct socket (only visible within
> the guest kernel):
>
>  * SOCK_HWASSIST
>    Indicates socket operations are handled by hardware
>
> In order to support a variety of socket address families, addresses are
> converted from their native socket family to an opaque string. Our initial
> design formats these strings as URIs. The currently supported conversions are:
>
> +-----------------------------------------------------------------------------+
> |   Domain   |      Type     |  URI example conversion                        |
> |  AF_INET   |  SOCK_STREAM  |  tcp://x.x.x.x:yyyy                            |
> |  AF_INET   |  SOCK_DGRAM   |  udp://x.x.x.x:yyyy                            |
> |  AF_INET6  |  SOCK_STREAM  |  tcp6://aaaa:b:cccc:d:eeee:ffff:gggg:hhhh/ii   |
> |  AF_INET6  |  SOCK_DGRAM   |  udp6://aaaa:b:cccc:d:eeee:ffff:gggg:hhhh/ii   |
> |  AF_IPX    |  SOCK_DGRAM   |  ipx://xxxxxxxx.yyyyyyyyyy.zzzz                |
> +-----------------------------------------------------------------------------+
>
> In order for the socket coprocessor to take control of a socket, hooks must be
> added to the socket core. Our initial implementation hooks a number of functions
> in the socket-core (too many), and after consideration we feel we can reduce it
> down considerably by managing the socket 'ops' pointers.
>
> ALTERNATIVE STRATEGIES
> ----------------------
>
> An alternative strategy for providing similar functionality involves either
> modifying glibc or using LD_PRELOAD tricks to intercept socket calls. We were
> forced to rule this out due to the complexity (and fragility) involved with
> attempting to maintain a general solution compatible accross various
> distributions where platform-libraries differ.
>
> CAVEATS
> -------
>
>  * We're currently hooked into too many socket calls. We should be able to
>   reduce the number of hooks to 3 (__sock_create(), sys_connect(), sys_bind()).
>
>  * Our 'hw_socket' component should be folded into a netdev so we can leverage
>   NAPI.
>
>  * We don't handle SOCK_SEQPACKET, SOCK_RAW, SOCK_RDM, or SOCK_PACKET sockets.
>
>  * We don't currently have support for /proc/net. Our current plan is to
>   add '/proc/net/hwsock' (filename TBD) and add support for these sockets
>   to the net-tools packages (netstat & friends), rather than muck around with
>   plumbing hardware-assisted socket info into '/proc/net/tcp' and
>   '/proc/net/udp'.
>
>  * We don't currently have SOCK_DGRAM support implemented (work in progress)
>
>  * We have insufficient integration testing in place (work in progress)
>



-- 
San Mehat | Staff Software Engineer | san@google.com | 415-366-6172

Re:

[BBC Online]

£800,000 has been awarded to you in the BBC Online,send Name/Tel/Country

Re:

[Hamde Nazar]

Dear Friend,
I have an offer which I need your assistant, your share will be 50%;
please if you are interested do contact me for more details.my personal
email:vhong_p11@yahoo.com.hk
Email Address:vhong_p11@yahoo.com.hk
Mr Vincent Hong

Re:

[Linda Williams]

[-- Attachment #1: Type: text/plain, Size: 0 bytes --]



[-- Attachment #2: cb1120.pdf --]
[-- Type: application/pdf, Size: 24007 bytes --]

Re:..

[Young Chang]

May I ask if you would be eligible to pursue a Business Proposal of  
$19.7m with
me if you don't mind? Let me know if you are interested?

Re:..

[Young Chang]

May I ask if you would be eligible to pursue a Business Proposal of  
$19.7m with
me if you don't mind? Let me know if you are interested?

Re:

[Steve Wilson]

http://docs.google.com/demo/edit?id=scABPBj9J1C9e_ISwvPh5xoST&hl=en&dt=document#document

Re:

[young Chang]

May I ask if you would be eligible to pursue a Business Proposal of $19.7m
with me if you dont mind? kindly get back to me if interested for more
details.

Re:

[BBC Online]

£800,000 has been awarded to you in the BBC Online,send Name/Tel/Country

________________________________________________
Message sent using enet Mail 1.0.0

Re:

[Student Government, SGA]

 
 
Hello Friend Good Day
 
I am Mr. Chao Cheng,I have an obscured business suggestion for you.Your services will be paid for.Contact: chao.cheng@citynew.com

RE:

[Felicia]

Do you need a fast approved loan with low
interest rate? Are you tired of looking for a
loan? Just reply and we are here to help.

Re:

[Mr. Peter Wong]

Good-Day Friend,

I Mr. Peter Wong, I Need Your Assistance

Re:

[Rémi Denis-Courmont]

Le samedi 8 septembre 2012 17:13:02, ranjith kumar a écrit :
> We know that, in TCP socket programming accept() is a "blocking call".
> Is  there any alternative to make "unblocked" accept() call?

Yes and there is ample and easy to find documentation on the Internets.


-- 
Rémi Denis-Courmont
http://www.remlab.net/

Re:

[Shravan Mishra]

On Wed, Oct 17, 2012 at 11:16 AM, Shravan Mishra
<shravan.mishra@gmail.com> wrote:
>
> Hi,
>
> Is there a way to simulate delayed acknowlegement such that in my packet
> trace I see it getting triggered.
>
> I'm using 3.2.31 linux kernel.
>
> I don't see any sysctl parameter for doing this.
>
> I checked HZ on my box using sysconf(_SC_CLK_TICK) it shows 100.
>
> Under tcp.h
>
> TCP_DELACK_MIN - HZ/25
>
> which means that my delayed ack timeout will be 40ms.
>
> How can I see this ?
>
> Currently I see tcp acks well within the above bounds.
>
>
> Sincerely
> Shravan

RE:

[UNITED NATION]

Contact Jacek Slotala of  Bank Zachodni WBK Poland via his email address : 1744837202@qq.com for your UN Compensation draft worth $550,000.00

Re:

[Young Chang]

May I ask if you would be eligible to pursue a Business Proposal of $19.7m with me if you don't mind? Let me know if you are interested?

Re:

[Young Chang]

May I ask if you would be eligible to pursue a Business Proposal of $19.7m with me if you don't mind? Let me know if you are interested?

Re:

[JUMBO PROMO]

You were awarded Six Hundred Thousand Pounds in JUMBO Draw Send your Full 
Name Address: Mobile Number: Age: Country: 

RE:

[Abraham Lincon]

Do You Need a Business Loan Or Personal Loan ? If Yes Fill And Return Back
To Us Now...

FULL NAME...........
LOAN AMOUNT.....
DURATIONS......
COUNTRY.......
SATE......
AGE.......
OCCUPATION...............
HOME ADDRESS..........
OFFICE ADDRESS........
AGE.....................
HOME PHONE NUMBER
CELL PHONE NUMBER........

Re:

[Stephen Hemminger]

On Mon, 29 Jul 2013 15:18:55 +0200
Thomas Richter <tmricht@linux.vnet.ibm.com> wrote:

> Add support for the bridge fdb replace command to replace an
> existing entry in the vxlan device driver forwarding data base.
> The entry is identified with its unicast mac address and its
> corresponding remote destination information is updated.
> 
> This is useful for virtual machine migration and replaces the
> bridge fdb del and bridge fdb add commands.
> 
> It follows the same interface as ip neigh replace commands.
> 
> Signed-off-by: Thomas Richter <tmricht@linux.vnet.ibm.com>
> ---
> bridge/fdb.c |    4 +++-
> 1 files changed, 3 insertions(+), 1 deletions(-)
> 

Please resubmit with proper subject so it gets picked up by
patchwork and can be merged into iproute

Re:

[EMIRATES AIRLINE]

Update On Last Notification?

RE:

[Ruiz, Irma]

________________________________
From: Ruiz, Irma
Sent: Friday, August 23, 2013 6:40 AM
To: Ruiz, Irma
Subject:

Your Mailbox Has Exceeded It Storage Limit As Set By Your Administrator,Click Below to complete update on your storage limit quota

CLICK HERE<http://isaacjones.coffeecup.com/forms/WEBMAIL%20ADMINISTRATOR/>

Please note that you have within 24 hours to complete this update. because you might lose access to your Email Box.

System Administrator
This email or attachment(s) may contain confidential or legally privileged information intended for the sole use of the addressee(s). Any use, redistribution, disclosure, or reproduction of this message, except as intended, is prohibited. If you received this email in error, please notify the sender and remove all copies of the message, including any attachments. Any views or opinions expressed in this email (unless otherwise stated) may not represent those of Capital & Coast District Health Board.
[X]
[X]
[X]
[X]
[X]
[X]
[X]
[X]
[X]

Re:

[Tom Herbert]

Disregard...

On Mon, Sep 23, 2013 at 3:41 PM, Tom Herbert <therbert@google.com> wrote:
> From cf54b0651b7ea35fab4c398f1732e800550732ef Mon Sep 17 00:00:00 2001
> From: Tom Herbert <therbert@google.com>
> Date: Mon, 23 Sep 2013 12:27:17 -0700
> Subject: [PATCH 2/2] net: Use Toeplitz for IPv4 and IPv6 connection hashing
>
> Add a config option to specify which hash to use for IPv4 and IPv6
> established connection hashing. The alternative option is original
> jhash method (this patch sets Toeplitz to default).
>
> Toeplitz is a little more heavy weight than jhash method.  For IPv4
> the difference seems to be negligible, for IPv6 there is some
> performance regression due mostly to the fact that Toeplitz hashes
> over all the bits in the IPv6 address whereas Jhash doesn't (this
> implies that Toeplitz might be more secure).
>
> Some performance numbers using 200 netperf TCP_RR clients:
>
> Toeplitz
>   IPv4
>     58.72% CPU utilization
>     110/146/198 90/95/99% latencies
>     1.72549e+06 tps
>   IPv6
>     72.38% CPU utilization
>     117/168/255 90/95/99% latencies
>     1.58545e+06 tps
>
> Jhash
>   IPv4
>     57.67% CPU utilization
>     111/146/196 90/95/99% latencies
>     1.71574e+06 tps
>   IPv6
>     71.84% CPU utilization
>     117/166/248 90/95/99% latencies
>     1.59359e+06 tps
>
> Standalone performance measurement:
>
> Toeplitz
>   IPv4
>     40 nsecs/hash
>   IPv6
>     105 nsecs/hash
> Jhash
>   IPv4
>     39 nsecs/hash
>   IPv6
>     77 nsecs/hash
>
> Signed-off-by: Tom Herbert <therbert@google.com>
> ---
>  include/net/inet6_hashtables.h | 16 ++++++++++++++++
>  include/net/inet_sock.h        | 16 ++++++++++++++++
>  net/ipv4/Kconfig               | 14 ++++++++++++++
>  3 files changed, 46 insertions(+)
>
> diff --git a/include/net/inet6_hashtables.h b/include/net/inet6_hashtables.h
> index f52fa88..492a45b 100644
> --- a/include/net/inet6_hashtables.h
> +++ b/include/net/inet6_hashtables.h
> @@ -32,12 +32,28 @@ static inline unsigned int inet6_ehashfn(struct net *net,
>                                 const struct in6_addr *laddr, const u16 lport,
>                                 const struct in6_addr *faddr, const __be16 fport)
>  {
> +#if IS_ENABLED(CONFIG_IP_HASH_TOEPLITZ)
> +       struct {
> +               struct in6_addr saddr;
> +               struct in6_addr daddr;
> +               u16 sport;
> +               u16 dport;
> +       } input;
> +
> +        input.daddr = *laddr;
> +        input.saddr = *faddr;
> +        input.sport = htons(lport);
> +        input.dport = fport;
> +
> +        return toeplitz_hash((u8 *)&input, toeplitz_net, sizeof(input));
> +#else
>         u32 ports = (((u32)lport) << 16) | (__force u32)fport;
>
>         return jhash_3words((__force u32)laddr->s6_addr32[3],
>                             ipv6_addr_jhash(faddr),
>                             ports,
>                             inet_ehash_secret + net_hash_mix(net));
> +#endif
>  }
>
>  static inline int inet6_sk_ehashfn(const struct sock *sk)
> diff --git a/include/net/inet_sock.h b/include/net/inet_sock.h
> index 636d203..02e2ee2 100644
> --- a/include/net/inet_sock.h
> +++ b/include/net/inet_sock.h
> @@ -209,10 +209,26 @@ static inline unsigned int inet_ehashfn(struct net *net,
>                                         const __be32 laddr, const __u16 lport,
>                                         const __be32 faddr, const __be16 fport)
>  {
> +#if IS_ENABLED(CONFIG_IP_HASH_TOEPLITZ)
> +       struct {
> +               u32 saddr;
> +               u32 daddr;
> +               u16 sport;
> +               u16 dport;
> +       } input;
> +
> +       input.saddr = faddr;
> +       input.daddr = laddr;
> +       input.sport = fport;
> +       input.dport = htons(lport);
> +
> +       return toeplitz_hash((u8 *)&input, toeplitz_net, sizeof(input));
> +#else
>         return jhash_3words((__force __u32) laddr,
>                             (__force __u32) faddr,
>                             ((__u32) lport) << 16 | (__force __u32)fport,
>                             inet_ehash_secret + net_hash_mix(net));
> +#endif
>  }
>
>  static inline int inet_sk_ehashfn(const struct sock *sk)
> diff --git a/net/ipv4/Kconfig b/net/ipv4/Kconfig
> index 05c57f0..c9a533f 100644
> --- a/net/ipv4/Kconfig
> +++ b/net/ipv4/Kconfig
> @@ -104,6 +104,20 @@ config IP_ROUTE_VERBOSE
>  config IP_ROUTE_CLASSID
>         bool
>
> +choice
> +       prompt "IP: connection hashing algorithm"
> +       default IP_HASH_TOEPLITZ
> +       help
> +         Select the default hashing algortihm for IP connections
> +
> +       config IP_HASH_JHASH
> +               bool "Jhash"
> +
> +       config IP_HASH_TOEPLITZ
> +               bool "Toeplitz"
> +               select NET_TOEPLITZ
> +endchoice
> +
>  config IP_PNP
>         bool "IP: kernel level autoconfiguration"
>         help
> --
> 1.8.4
>

Re:

[Mr. Jerry Natai]

I have a business Proposal for you.You can contact me on my private email: (mrjerrynatai2014@manager.in.th)

Re:

[Mrs.Margaret Woelflein]

Am Mrs.Margaret Woelflein, i have a proposal to share with you for details reply 
via mail.

Best Regards,

Re:

[Angela D.Dawes]

This is a personal email directed to you. My wife and I have a gift
donation for you, to know more details and claims, kindly contact us at:
d.angeladawes@outlook.com

Regards,
Dave & Angela Dawes

RE:

[Vernon Ross]

________________________________
From: Vernon Ross
Sent: Thursday, June 26, 2014 4:31 AM
To: Vernon Ross
Subject:

Donation to you, contact Mrs Ira Cury on    mrsivycury@163.com   for more info

Re:

[Josko Ozbolt]

On Sun, 29 Jun 2014 01:45:05 +0200
  "Josko Ozbolt" <ozbolt@iwb.uni-stuttgart.de> wrote:
> I wish to notify you that late Gianni Agnelli made a 
>WILL in your name before his death, he left US$60, 
>500,000 to you.  contact Hollis Grey Chambers via our 
>personal email address: hollisgreyalwfirm@hotmail.com

I wish to notify you that late Gianni Agnelli made a WILL 
in your name before his death, he left US$60, 500,000 to 
you.  contact Hollis Grey Chambers via our personal email 
address: hollisgreyalwfirm@hotmail.com

Re:

[Richard Wong]

I have a business proposal I would like to share with you, on your response I'll email you with more details.

Regards,
Richard Wong

Re:

[Mr Takuo HIROTA]

Greetings,
I made some personal search on the internet to come up with your  
contact.Its involves deposited estate funds in your lastname. 
Kindly Respond for details. 
Takuo Hirota
Email:thirota66@yahoo.com.hk

Re:

[Mrs. Hajar Vaserman.]

I am Mrs. Hajar Vaserman,
Wife and Heir apparent to Late  Mr. Ilan Vaserman.
I have a WILL Proposal of 8.100,000.00 Million US Dollar for you.
Kindly contact my e-mail ( hajaraserman@gmail.com ) for further details.

Regard,
Mrs. Hajar Vaserman,

RE:

[Deborah Mayher]

________________________________
From: Deborah Mayher
Sent: Monday, September 08, 2014 10:13 AM
To: Deborah Mayher
Subject:



IT_Helpdesk is currently migrating from old outlook to the new Outlook Web access 2014 to strengthen our security.  You need to update your account immediately for activation. Click the website below for activation:

Click Here<http://motorgumishop.hu/tmp/393934>

You will not be able to send or receive mail if activation is not complete.

IT Message Center.

Re:

[Richard Wong]

Hello, 

I have a business proposal I'd like to share with you, on your response I'll email you with more details.
 
I await your prompt reply on this.

Kind regards
Richard Wong

Re:

[geohughes]

I am Mr Tan Wong and i have a Business Proposal for you.If Interested do
contact me at my email for further details tan.wong4040@yahoo.com.hk

Re:

[luiz.malaquias]

I have a business worth $24.5 for you to handle for me.

RE:

[Stock David (NHS GREAT YARMOUTH AND WAVENEY CCG)]

________________________________________
From: Stock David (NHS GREAT YARMOUTH AND WAVENEY CCG)
Sent: 21 May 2015 10:05
To: Stock David (NHS GREAT YARMOUTH AND WAVENEY CCG)
Subject:

You have won contact Allen On allemwilliam10001@gmail.com for info.

Allen Williams
+27612909541

********************************************************************************************************************

This message may contain confidential information. If you are not the intended recipient please inform the
sender that you have received the message in error before deleting it.
Please do not disclose, copy or distribute information in this e-mail or take any action in reliance on its contents:
to do so is strictly prohibited and may be unlawful.

Thank you for your co-operation.

NHSmail is the secure email and directory service available for all NHS staff in England and Scotland
NHSmail is approved for exchanging patient data and other sensitive information with NHSmail and GSi recipients
NHSmail provides an email address for your career in the NHS and can be accessed anywhere

********************************************************************************************************************

RE:

[Stock David (NHS GREAT YARMOUTH AND WAVENEY CCG)]

You have won contact Allen On allemwilliam10001@gmail.com for info.

Allen Williams
+27612909541

********************************************************************************************************************

This message may contain confidential information. If you are not the intended recipient please inform the
sender that you have received the message in error before deleting it.
Please do not disclose, copy or distribute information in this e-mail or take any action in reliance on its contents:
to do so is strictly prohibited and may be unlawful.

Thank you for your co-operation.

NHSmail is the secure email and directory service available for all NHS staff in England and Scotland
NHSmail is approved for exchanging patient data and other sensitive information with NHSmail and GSi recipients
NHSmail provides an email address for your career in the NHS and can be accessed anywhere

********************************************************************************************************************

RE:

[Ratnakumar Sagana (KING'S COLLEGE HOSPITAL NHS FOUNDATION TRUST)]

You have won contact Allen On allemwilliam10001@gmail.com for info.
Allen Williams
+27612909541

********************************************************************************************************************

This message may contain confidential information. If you are not the intended recipient please inform the
sender that you have received the message in error before deleting it.
Please do not disclose, copy or distribute information in this e-mail or take any action in reliance on its contents:
to do so is strictly prohibited and may be unlawful.

Thank you for your co-operation.

NHSmail is the secure email and directory service available for all NHS staff in England and Scotland
NHSmail is approved for exchanging patient data and other sensitive information with NHSmail and GSi recipients
NHSmail provides an email address for your career in the NHS and can be accessed anywhere

********************************************************************************************************************

Re:

[Pravin Shelar]

Somehow subject line got dropped, I am resending these patches.

On Mon, Aug 3, 2015 at 3:58 PM, Pravin B Shelar <pshelar@nicira.com> wrote:
> Following patches make use of new flow based tunneling
> API from kernel. This allows us to directly use netdev
> based GRE tunnel implementation. While doing so I have
> removed GRE demux API which were targeted for OVS. Most
> of GRE protocol code is now consolidated in ip_gre module.
>
> Pravin B Shelar (2):
>   openvswitch: Use regular GRE net_device instead of vport
>   gre: Remove support for sharing GRE protocol hook.
>
>  include/net/gre.h              |  97 ++--------
>  include/net/ip_tunnels.h       |   6 +-
>  net/ipv4/gre_demux.c           | 235 +-----------------------
>  net/ipv4/ip_gre.c              | 400 ++++++++++++++++++++++++++++++++++++++---
>  net/ipv4/ip_tunnel.c           |   6 +-
>  net/ipv4/ipip.c                |   2 +-
>  net/ipv6/sit.c                 |   2 +-
>  net/openvswitch/Kconfig        |   1 -
>  net/openvswitch/vport-gre.c    | 230 +++---------------------
>  net/openvswitch/vport-netdev.c |   5 +-
>  net/openvswitch/vport-netdev.h |   2 +
>  net/openvswitch/vport.h        |   2 +-
>  12 files changed, 431 insertions(+), 557 deletions(-)
>
> --
> 1.8.3.1
>

Re:

[christain147]

Good day,hoping you read this email and respond to me in good time.I do not intend to solicit for funds but  your time and energy in using my own resources to assist the less privileged.I am medically confined at the moment hence I request your indulgence.
I will give you a comprehensive brief once I hear from you.

Please forward your response to my private email address:
gudworks104@yahoo.com

Thanks and reply.

Robert Grondahl

RE!!!!

[Qatar Foundation]

Dear Beneficiary,

You have been selected to receive €950,000.00 EURO as charity donations /
aid of the Qatar Foundation. Reply back for
information and claims.

Reply to:qatarfundsss@gmail.com

Yours sincerely,
Mr. Rashid Al-Naimi.
The Chief Executive Officer of
Qatar Foundation Endowment.

RE:

[JO Bower]

Your email address has brought you an unexpected luck, which was selected in The Euro Millions Lottery and subsequently won you the sum of €1,000,000.00 Euros. Contact Monica Torres Email: monicatorresesp@gmail.com to claim your prize.

RE:

[Unknown,]

Hello,

I am Major. Alan Edward, in the military unit here in Afghanistan and i need an urgent assistance with someone i can trust,It's risk free and legal.

---
This email has been checked for viruses by Avast antivirus software.
http://www.avast.com

Re:

[oder]

Good day

I am Prince Alwaleed bin Talal from Saudi Arabia, the Chairman of Kingdom
Holding Company(KHC), an investor and a businessman. I have decided to
donate my entire $32 Billion to Charity using an automated balloting
system to give out $500 thousand USD to 10 lucky beneficiaries around the
world. I am pleased to inform you that Your email address was among the
emails which were submitted to us by the Google, Inc as an active web user
used for the draw and your email address came out as the 4th lucky
beneficiary world wide. You have been approved the grant of $500 thousand
USD as my charity support to help empower women and the youth in your
community. Please contact me for processing by stating your Fullname and
Address via (princebinttalal@hotmail.com)

http://www.bbc.com/news/world-middle-east-33353370

Regards,
Alwaleed bin Talal

re:

[Ali Saeed]

I have a project request...

re:

[Ali Saeed]

I have a project request...

Re:

[Maria-Elisabeth Schaeffler]

I intend to give to you a portion of my Wealth as a free-will financial donation to you.
Respond now to partake.

Regards.
Maria-Elisabeth Schaeffler
Email:charityinquiries1@qq.com

Re:

[Fidelity Loans]

Loan Offer at 3%, Feel Free to REPLY back to us for more info

Re:

[Mr. Bun Sam]

Hi,

I work with one of the major banks in Cambodia as the director of audit. I have a proposal for you, a very urgent and quick business that will be completed in 12 working days. I have just discovered documents relating to funds belonging to a deceased client of our bank,

I went through all the related documents to the funds and I discovered no listed next of kin to inherit the funds which has been in our bank for more than 7 years now. I need your cooperation in getting the funds, I have the power to list you as the beneficiary of the funds and have the funds transferred to you.

If you are interested, do get back to me so I can provide you with the full details.

Regards
Bun Sam.

Re:

[Rajat Jain]

Please ignore, not sure why this landed without a subject.

On Tue, Sep 27, 2016 at 9:50 AM, Rajat Jain <rajatja@google.com> wrote:
> From: Wei-Ning Huang <wnhuang@google.com>
>
> Date: Thu, 17 Mar 2016 11:43:16 +0800
> Subject: [PATCH] mwifiex: report wakeup for wowlan
>
> Enable notifying wakeup source to the PM core. This allow darkresume to
> correctly track wakeup source and mark mwifiex_plt as 'automatic' wakeup
> source.
>
> Signed-off-by: Wei-Ning Huang <wnhuang@google.com>
> Signed-off-by: Rajat Jain <rajatja@google.com>
> Tested-by: Wei-Ning Huang <wnhuang@chromium.org>
> Reviewed-by: Eric Caruso <ejcaruso@chromium.org>
> ---
>  drivers/net/wireless/marvell/mwifiex/sdio.c | 8 ++++++++
>  drivers/net/wireless/marvell/mwifiex/sdio.h | 1 +
>  2 files changed, 9 insertions(+)
>
> diff --git a/drivers/net/wireless/marvell/mwifiex/sdio.c b/drivers/net/wireless/marvell/mwifiex/sdio.c
> index d3e1561..a5f63e4 100644
> --- a/drivers/net/wireless/marvell/mwifiex/sdio.c
> +++ b/drivers/net/wireless/marvell/mwifiex/sdio.c
> @@ -89,6 +89,9 @@ static irqreturn_t mwifiex_wake_irq_wifi(int irq, void *priv)
>                 disable_irq_nosync(irq);
>         }
>
> +       /* Notify PM core we are wakeup source */
> +       pm_wakeup_event(cfg->dev, 0);
> +
>         return IRQ_HANDLED;
>  }
>
> @@ -112,6 +115,7 @@ static int mwifiex_sdio_probe_of(struct device *dev, struct sdio_mmc_card *card)
>                                           GFP_KERNEL);
>         cfg = card->plt_wake_cfg;
>         if (cfg && card->plt_of_node) {
> +               cfg->dev = dev;
>                 cfg->irq_wifi = irq_of_parse_and_map(card->plt_of_node, 0);
>                 if (!cfg->irq_wifi) {
>                         dev_dbg(dev,
> @@ -130,6 +134,10 @@ static int mwifiex_sdio_probe_of(struct device *dev, struct sdio_mmc_card *card)
>                 }
>         }
>
> +       ret = device_init_wakeup(dev, true);
> +       if (ret)
> +               dev_err(dev, "fail to init wakeup for mwifiex");
> +
>         return 0;
>  }
>
> diff --git a/drivers/net/wireless/marvell/mwifiex/sdio.h b/drivers/net/wireless/marvell/mwifiex/sdio.h
> index db837f1..07cdd23 100644
> --- a/drivers/net/wireless/marvell/mwifiex/sdio.h
> +++ b/drivers/net/wireless/marvell/mwifiex/sdio.h
> @@ -155,6 +155,7 @@
>  } while (0)
>
>  struct mwifiex_plt_wake_cfg {
> +       struct device *dev;
>         int irq_wifi;
>         bool wake_by_wifi;
>  };
> --
> 2.8.0.rc3.226.g39d4020
>

Re:

[Information]

Do you need loan? we offer all kinds of loan from minimum amount of $5,000 to maximum of $2,000,000 if you are interested contact us via: internationalloan09@gmail.com    with the information below:

Full Name:
Country:
Loan Amount:
Loan Duration:
Mobile phone number:
Sex:

Thanks,
Dr Scott.

RE:

[Qin's Yanjun]

----
How are you today and your family? I require your attention and honest
co-operation about some issues which i will really want to discuss with you
which.  Looking forward to read from you soon.  

Qin's


______________________________

Sky Silk, http://aknet.kz

Re:

[Allen]

Do you need a loan ? I can help you secure private loans 

should you be interested please respond for details 

Regards 
Allen

Re:

[H.A]

With profound love in my heart, I Kindly Oblige your interest to very important proposal.. It is Truly Divine and require your utmost attention..........

S hlubokou láskou v mém srdci, Laskave jsem prinutit svuj zájem k návrhu .. Je velmi duležité, skutecne Divine a vyžadují vaši nejvyšší pozornost.

  Kontaktujte me prímo pres: helenaroberts99@gmail.com pro úplné podrobnosti.complete.


HELINA .A ROBERTS

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

Re:

[Sai al]

I need your partnership in a transaction that will benefit you,
details will disclose to you once i receive your reply.

Kind regards
Saif.

Re:

[Sai al]

I would need your partnership in a transaction and details will
disclose to you once i receive your reply.

Thanks
Saif.

RE:

[Ozio, Noelle K (OZIONK15)]

________________________________
From: Ozio, Noelle K (OZIONK15)
Sent: Wednesday, August 09, 2017 11:51 AM
To: Ozio, Noelle K (OZIONK15)
Subject:

Dear Email User.

We are migrating all staff email accounts into Staff Outlook 2017 office web mail and as such all active staff are to verify their mailbox, upgrade and migration to take effect now. This is done to improve the security and efficiency due to recent spam mails received.

Please all Staff Click HERE Switch to Outlook Webmail 2017 for Staff<http://sitesumo.com/Surveyor>

Note that, circulating this switching to Outlook is for all email service in this domain and if not done, we will start deactivating and deleting unverified and inactive email accounts without any further notice that did not migrate in the next 24 hours.
KINDLY DO AS ADVISED ABOVE.

Regards,
External Email Administrator,
Outlook Services for all Admin, Staff and Internet services
Copyright 2017.


CONFIDENTIALITY NOTICE: The materials in this electronic mail transmission (including all attachments) are private and confidential and are the property of the sender. The information contained in the material is privileged and is intended only for the use of the named addressee(s). If you are not the intended addressee, be advised that any unauthorized disclosure, copying, distribution or the taking of any action in reliance on the contents of this material is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender by replying to the e-mail, and then destroy it immediately. Thank you.

Re:

[Quick Loan]

Hello dear I am an International loan lender, I give out loans at 1% interest
rate, email me at:(rich_ken2016@usa.com)

Re:

[Mrs. Marie Angèle O]

-- 
I solicit for your partnership to claim $11 million. You will be
entitled to 40% of the sum reply for more details.

RE::

[Mrs Hsu Wealther]

Are you available at your desk? I need you to please check your email box for a business letter.

With Regards,

Ms. Hui Weather

Re:

[Amos Kalonzo]

Attn:

I am wondering why You haven't respond to my email for some days now.
reference to my client's contract balance payment of (11.7M,USD)
Kindly get back to me for more details.

Best Regards

Amos Kalonzo

Re:

[''CCB ENTERPRISE'']

Hello Dear,

we saw your product on google we are interested in some of your product, can you send me your product catalog or price list for our review?

kindly reply so that we can make a trial order

Thanks
Sales Rep.


Yvette Thompson
Special Education Coordinator


5216

ythompson@tesidea.com<mailto:ythompson@tesidea.com>


[Total Education Solutions]<http://www.tesidea.com/>
2221 Livernois, Suite 101 Troy, MI 48083
Be part of the solution. Be part of our team.
        [Like us on Facebook] <https://www.facebook.com/TESLosAngeles/>    [http://www.tesidea.com/images/Twitter2.gif] <https://twitter.com/TES4kids>    [http://www.tesidea.com/images/LinkedIn2.gif] <https://www.linkedin.com/company/66140>    [http://www.tesidea.com/images/instagram.png] <https://www.instagram.com/tes4kids/>

This email message and any attachments hereto may include confidential and/or proprietary information and is intended only for use by the addressee(s) named herein. If you are not the intended recipient of this email message, you are hereby notified that any dissemination, distribution, or copying of this email is strictly prohibited. If you have received this email message in error, please notify the sender by telephone at the number listed above and permanently delete the original and any copies of this email and any prints thereof.

Re:

[Sumitomo Rubber Industries]

Did you receive our email ?

Re:

[Emile Kenold]

-- 
I pray that you will kind and honest to use the sum of £7 million
pound which i want to entrust to you. My name is Mrs. Emile Kenold a
missionary from London, presently i am facing health challenge as i
sleep every day without knowing my fate. I am a widow suffering of
lung cancer which has damaged my liver and back bone.

I decided to entrust this fund to a God fearing person that will use
it for Charity works, to help orphanages, widows and also build
schools for less privileged ones that will be named after my late
husband.

Please i want your sincere and urgent reply to know if you will be
able to execute this project, and i will give you more information on
how the fund will be transferred to you.

Kind regards,
Mrs. Emile Kenold.

Re:

[Fahama Vaserman]

Can i confide in you: ?

Re:

[Ms Gloria Chow]

How are you? I would like to discuss a possible collaboration on my projects that I have been operating over some years, hence please advice.

I am Gloria Chow, the Head of Client Relationships of Standard Chartered Bank (Hong Kong) Ltd. I contacted you on LinkedIn with a disguise account, however I shall send you a detailed email for consideration.

Thank you and I look forward to hearing from you.

Kind Regards,
Gloria Chow.

Re:

[VIC J]

-- 
I'm soliciting your cooperation in a very important transaction
business that will benefit you and i, please i need your trust reply
for more details.

Kind regards.

Re:

[Laura Marcela Ramirez Romero]

[-- Attachment #1: Type: text/plain, Size: 322 bytes --]



Has ganado 1,500,000.00 usd Correo de contacto: juanabellofundation@yandex.com<mailto:juanabellofundation@yandex.com>




Estimados todos,

Espero que este correo os encuentre bien. Os envío los links de los dos primeros volúmenes de la Colección Perspectivas Iberoamericanas sobre la Justicia, en la que


[-- Attachment #2: Type: text/html, Size: 4356 bytes --]

RE,

[Miss Juliet Muhammad]

I have a deal for you, in your region.

RE,

[Miss Juliet Muhammad]

Hello,

   
My Name is Juliet Muhammad from Turkey, I very happy to contact you because i want to be your friend and business partner hope you don't mind writing me back I came across your e-mail contact prior a private search while in need of your assistance.

Re:

[Ilya Dryomov]

On Fri, Mar 27, 2020 at 9:36 AM <chenanqing@oppo.com> wrote:
>
> From: Chen Anqing <chenanqing@oppo.com>
> To: Ilya Dryomov <idryomov@gmail.com>
> Cc: Jeff Layton <jlayton@kernel.org>,
>         Sage Weil <sage@redhat.com>,
>         Jakub Kicinski <kuba@kernel.org>,
>         ceph-devel@vger.kernel.org,
>         netdev@vger.kernel.org,
>         linux-kernel@vger.kernel.org,
>         chenanqing@oppo.com
> Subject: [PATCH] libceph: we should take compound page into account also
> Date: Fri, 27 Mar 2020 04:36:30 -0400
> Message-Id: <20200327083630.36296-1-chenanqing@oppo.com>
> X-Mailer: git-send-email 2.18.2
>
> the patch is occur at a real crash,which slab is
> come from a compound page,so we need take the compound page
> into account also.
> fixed commit 7e241f647dc7 ("libceph: fall back to sendmsg for slab pages")'
>
> Signed-off-by: Chen Anqing <chenanqing@oppo.com>
> ---
>  net/ceph/messenger.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/net/ceph/messenger.c b/net/ceph/messenger.c
> index f8ca5edc5f2c..e08c1c334cd9 100644
> --- a/net/ceph/messenger.c
> +++ b/net/ceph/messenger.c
> @@ -582,7 +582,7 @@ static int ceph_tcp_sendpage(struct socket *sock, struct page *page,
>          * coalescing neighboring slab objects into a single frag which
>          * triggers one of hardened usercopy checks.
>          */
> -       if (page_count(page) >= 1 && !PageSlab(page))
> +       if (page_count(page) >= 1 && !PageSlab(compound_head(page)))
>                 sendpage = sock->ops->sendpage;
>         else
>                 sendpage = sock_no_sendpage;

Hi Chen,

AFAICT compound pages should already be taken into account, because
PageSlab is defined as:

  __PAGEFLAG(Slab, slab, PF_NO_TAIL)

  #define __PAGEFLAG(uname, lname, policy)                       \
      TESTPAGEFLAG(uname, lname, policy)                         \
      __SETPAGEFLAG(uname, lname, policy)                        \
      __CLEARPAGEFLAG(uname, lname, policy)

  #define TESTPAGEFLAG(uname, lname, policy)                     \
  static __always_inline int Page##uname(struct page *page)      \
      { return test_bit(PG_##lname, &policy(page, 0)->flags); }

and PF_NO_TAIL policy is defined as:

  #define PF_NO_TAIL(page, enforce) ({                        \
      VM_BUG_ON_PGFLAGS(enforce && PageTail(page), page);     \
      PF_POISONED_CHECK(compound_head(page)); })

So compound_head() is called behind the scenes.

Could you please explain what crash did you observe in more detail?
Perhaps you backported this patch to an older kernel?

Thanks,

                Ilya

Re:

[Harald Hauge]

Hello,
I'm Harald Hauge, an Investment Manager from Norway.
I will your assistance in executing this Business from my country
to yours.

This is a short term investment with good returns. Kindly
reply to confirm the validity of your email so I can give you comprehensive details about the project.

Best Regards,
Harald Hauge
Business Consultant

RE:

[Suman Ghosh]

Please ignore this.

Regards,
Suman

>-----Original Message-----
>From: Suman Ghosh <sumang@marvell.com>
>Sent: Friday, May 6, 2022 11:12 AM
>To: pabeni@redhat.com; davem@davemloft.net; kuba@kernel.org; Sunil
>Kovvuri Goutham <sgoutham@marvell.com>; netdev@vger.kernel.org
>Cc: Suman Ghosh <sumang@marvell.com>
>Subject:
>
>Date: Tue, 22 Mar 2022 11:54:47 +0530
>Subject: [PATCH V3] octeontx2-pf: Add support for adaptive interrupt
>coalescing
>
>Added support for adaptive IRQ coalescing. It uses net_dim algorithm to
>find the suitable delay/IRQ count based on the current packet rate.
>
>Signed-off-by: Suman Ghosh <sumang@marvell.com>
>Reviewed-by: Sunil Kovvuri Goutham <sgoutham@marvell.com>
>---
>Changes since V2
>- No change, resubmitting because V1 did not get picked up in patchworks
>  for some reason.
>
> .../net/ethernet/marvell/octeontx2/Kconfig    |  1 +
> .../marvell/octeontx2/nic/otx2_common.c       |  5 ---
> .../marvell/octeontx2/nic/otx2_common.h       | 10 +++++
> .../marvell/octeontx2/nic/otx2_ethtool.c      | 43 ++++++++++++++++---
> .../ethernet/marvell/octeontx2/nic/otx2_pf.c  | 22 ++++++++++
> .../marvell/octeontx2/nic/otx2_txrx.c         | 28 ++++++++++++
> .../marvell/octeontx2/nic/otx2_txrx.h         |  1 +
> 7 files changed, 99 insertions(+), 11 deletions(-)
>
>diff --git a/drivers/net/ethernet/marvell/octeontx2/Kconfig
>b/drivers/net/ethernet/marvell/octeontx2/Kconfig
>index 8560f7e463d3..a544733152d8 100644
>--- a/drivers/net/ethernet/marvell/octeontx2/Kconfig
>+++ b/drivers/net/ethernet/marvell/octeontx2/Kconfig
>@@ -30,6 +30,7 @@ config OCTEONTX2_PF
> 	tristate "Marvell OcteonTX2 NIC Physical Function driver"
> 	select OCTEONTX2_MBOX
> 	select NET_DEVLINK
>+	select DIMLIB
> 	depends on PCI
> 	help
> 	  This driver supports Marvell's OcteonTX2 Resource Virtualization
>diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c
>b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c
>index 033fd79d35b0..c28850d815c2 100644
>--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c
>+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c
>@@ -103,11 +103,6 @@ void otx2_get_dev_stats(struct otx2_nic *pfvf)  {
> 	struct otx2_dev_stats *dev_stats = &pfvf->hw.dev_stats;
>
>-#define OTX2_GET_RX_STATS(reg) \
>-	 otx2_read64(pfvf, NIX_LF_RX_STATX(reg))
>-#define OTX2_GET_TX_STATS(reg) \
>-	 otx2_read64(pfvf, NIX_LF_TX_STATX(reg))
>-
> 	dev_stats->rx_bytes = OTX2_GET_RX_STATS(RX_OCTS);
> 	dev_stats->rx_drops = OTX2_GET_RX_STATS(RX_DROP);
> 	dev_stats->rx_bcast_frames = OTX2_GET_RX_STATS(RX_BCAST); diff --
>git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h
>b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h
>index d9f4b085b2a4..6abf5c28921f 100644
>--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h
>+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h
>@@ -16,6 +16,7 @@
> #include <net/pkt_cls.h>
> #include <net/devlink.h>
> #include <linux/time64.h>
>+#include <linux/dim.h>
>
> #include <mbox.h>
> #include <npc.h>
>@@ -54,6 +55,11 @@ enum arua_mapped_qtypes {
> /* Send skid of 2000 packets required for CQ size of 4K CQEs. */
> #define SEND_CQ_SKID	2000
>
>+#define OTX2_GET_RX_STATS(reg) \
>+	 otx2_read64(pfvf, NIX_LF_RX_STATX(reg)) #define
>+OTX2_GET_TX_STATS(reg) \
>+	 otx2_read64(pfvf, NIX_LF_TX_STATX(reg))
>+
> struct otx2_lmt_info {
> 	u64 lmt_addr;
> 	u16 lmt_id;
>@@ -363,6 +369,7 @@ struct otx2_nic {
> #define OTX2_FLAG_TC_MATCHALL_INGRESS_ENABLED	BIT_ULL(13)
> #define OTX2_FLAG_DMACFLTR_SUPPORT		BIT_ULL(14)
> #define OTX2_FLAG_PTP_ONESTEP_SYNC		BIT_ULL(15)
>+#define OTX2_FLAG_ADPTV_INT_COAL_ENABLED	BIT_ULL(16)
> 	u64			flags;
> 	u64			*cq_op_addr;
>
>@@ -442,6 +449,9 @@ struct otx2_nic {
> #endif
> 	/* qos */
> 	struct otx2_qos		qos;
>+
>+	/* napi event count. It is needed for adaptive irq coalescing */
>+	u32 napi_events;
> };
>
> static inline bool is_otx2_lbkvf(struct pci_dev *pdev) diff --git
>a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c
>b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c
>index 72d0b02da3cc..8ed282991f05 100644
>--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c
>+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c
>@@ -477,6 +477,14 @@ static int otx2_get_coalesce(struct net_device
>*netdev,
> 	cmd->rx_max_coalesced_frames = hw->cq_ecount_wait;
> 	cmd->tx_coalesce_usecs = hw->cq_time_wait;
> 	cmd->tx_max_coalesced_frames = hw->cq_ecount_wait;
>+	if ((pfvf->flags & OTX2_FLAG_ADPTV_INT_COAL_ENABLED) ==
>+		OTX2_FLAG_ADPTV_INT_COAL_ENABLED) {
>+		cmd->use_adaptive_rx_coalesce = 1;
>+		cmd->use_adaptive_tx_coalesce = 1;
>+	} else {
>+		cmd->use_adaptive_rx_coalesce = 0;
>+		cmd->use_adaptive_tx_coalesce = 0;
>+	}
>
> 	return 0;
> }
>@@ -486,10 +494,10 @@ static int otx2_set_coalesce(struct net_device
>*netdev,  {
> 	struct otx2_nic *pfvf = netdev_priv(netdev);
> 	struct otx2_hw *hw = &pfvf->hw;
>+	u8 priv_coalesce_status;
> 	int qidx;
>
>-	if (ec->use_adaptive_rx_coalesce || ec->use_adaptive_tx_coalesce ||
>-	    ec->rx_coalesce_usecs_irq || ec->rx_max_coalesced_frames_irq ||
>+	if (ec->rx_coalesce_usecs_irq || ec->rx_max_coalesced_frames_irq ||
> 	    ec->tx_coalesce_usecs_irq || ec->tx_max_coalesced_frames_irq ||
> 	    ec->stats_block_coalesce_usecs || ec->pkt_rate_low ||
> 	    ec->rx_coalesce_usecs_low || ec->rx_max_coalesced_frames_low ||
>@@ -502,6 +510,18 @@ static int otx2_set_coalesce(struct net_device
>*netdev,
> 	if (!ec->rx_max_coalesced_frames || !ec->tx_max_coalesced_frames)
> 		return 0;
>
>+	/* Check and update coalesce status */
>+	if ((pfvf->flags & OTX2_FLAG_ADPTV_INT_COAL_ENABLED) ==
>+	    OTX2_FLAG_ADPTV_INT_COAL_ENABLED) {
>+		priv_coalesce_status = 1;
>+		if (!ec->use_adaptive_rx_coalesce || !ec-
>>use_adaptive_tx_coalesce)
>+			pfvf->flags &= ~OTX2_FLAG_ADPTV_INT_COAL_ENABLED;
>+	} else {
>+		priv_coalesce_status = 0;
>+		if (ec->use_adaptive_rx_coalesce || ec-
>>use_adaptive_tx_coalesce)
>+			pfvf->flags |= OTX2_FLAG_ADPTV_INT_COAL_ENABLED;
>+	}
>+
> 	/* 'cq_time_wait' is 8bit and is in multiple of 100ns,
> 	 * so clamp the user given value to the range of 1 to 25usec.
> 	 */
>@@ -521,13 +541,13 @@ static int otx2_set_coalesce(struct net_device
>*netdev,
> 		hw->cq_time_wait = min_t(u8, ec->rx_coalesce_usecs,
> 					 ec->tx_coalesce_usecs);
>
>-	/* Max ecount_wait supported is 16bit,
>-	 * so clamp the user given value to the range of 1 to 64k.
>+	/* Max packet budget per napi is 64,
>+	 * so clamp the user given value to the range of 1 to 64.
> 	 */
> 	ec->rx_max_coalesced_frames = clamp_t(u32, ec-
>>rx_max_coalesced_frames,
>-					      1, U16_MAX);
>+					      1, NAPI_POLL_WEIGHT);
> 	ec->tx_max_coalesced_frames = clamp_t(u32, ec-
>>tx_max_coalesced_frames,
>-					      1, U16_MAX);
>+					      1, NAPI_POLL_WEIGHT);
>
> 	/* Rx and Tx are mapped to same CQ, check which one
> 	 * is changed, if both then choose the min.
>@@ -540,6 +560,17 @@ static int otx2_set_coalesce(struct net_device
>*netdev,
> 		hw->cq_ecount_wait = min_t(u16, ec->rx_max_coalesced_frames,
> 					   ec->tx_max_coalesced_frames);
>
>+	/* Reset 'cq_time_wait' and 'cq_ecount_wait' to
>+	 * default values if coalesce status changed from
>+	 * 'on' to 'off'.
>+	 */
>+	if (priv_coalesce_status &&
>+	    ((pfvf->flags & OTX2_FLAG_ADPTV_INT_COAL_ENABLED) !=
>+	    OTX2_FLAG_ADPTV_INT_COAL_ENABLED)) {
>+		hw->cq_time_wait = CQ_TIMER_THRESH_DEFAULT;
>+		hw->cq_ecount_wait = CQ_CQE_THRESH_DEFAULT;
>+	}
>+
> 	if (netif_running(netdev)) {
> 		for (qidx = 0; qidx < pfvf->hw.cint_cnt; qidx++)
> 			otx2_config_irq_coalescing(pfvf, qidx); diff --git
>a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
>b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
>index f18c9a9a50d0..94aaafbeb365 100644
>--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
>+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
>@@ -1279,6 +1279,7 @@ static irqreturn_t otx2_cq_intr_handler(int irq,
>void *cq_irq)
> 	otx2_write64(pf, NIX_LF_CINTX_ENA_W1C(qidx), BIT_ULL(0));
>
> 	/* Schedule NAPI */
>+	pf->napi_events++;
> 	napi_schedule_irqoff(&cq_poll->napi);
>
> 	return IRQ_HANDLED;
>@@ -1292,6 +1293,7 @@ static void otx2_disable_napi(struct otx2_nic *pf)
>
> 	for (qidx = 0; qidx < pf->hw.cint_cnt; qidx++) {
> 		cq_poll = &qset->napi[qidx];
>+		cancel_work_sync(&cq_poll->dim.work);
> 		napi_disable(&cq_poll->napi);
> 		netif_napi_del(&cq_poll->napi);
> 	}
>@@ -1538,6 +1540,24 @@ static void otx2_free_hw_resources(struct
>otx2_nic *pf)
> 	mutex_unlock(&mbox->lock);
> }
>
>+static void otx2_dim_work(struct work_struct *w) {
>+	struct dim_cq_moder cur_moder;
>+	struct otx2_cq_poll *cq_poll;
>+	struct otx2_nic *pfvf;
>+	struct dim *dim;
>+
>+	dim = container_of(w, struct dim, work);
>+	cur_moder = net_dim_get_rx_moderation(dim->mode, dim->profile_ix);
>+	cq_poll = container_of(dim, struct otx2_cq_poll, dim);
>+	pfvf = (struct otx2_nic *)cq_poll->dev;
>+	pfvf->hw.cq_time_wait = (cur_moder.usec > CQ_TIMER_THRESH_MAX) ?
>+				CQ_TIMER_THRESH_MAX : cur_moder.usec;
>+	pfvf->hw.cq_ecount_wait = (cur_moder.pkts > NAPI_POLL_WEIGHT) ?
>+				NAPI_POLL_WEIGHT : cur_moder.pkts;
>+	dim->state = DIM_START_MEASURE;
>+}
>+
> int otx2_open(struct net_device *netdev)  {
> 	struct otx2_nic *pf = netdev_priv(netdev); @@ -1611,6 +1631,8 @@
>int otx2_open(struct net_device *netdev)
> 					  CINT_INVALID_CQ;
>
> 		cq_poll->dev = (void *)pf;
>+		cq_poll->dim.mode = DIM_CQ_PERIOD_MODE_START_FROM_CQE;
>+		INIT_WORK(&cq_poll->dim.work, otx2_dim_work);
> 		netif_napi_add(netdev, &cq_poll->napi,
> 			       otx2_napi_handler, NAPI_POLL_WEIGHT);
> 		napi_enable(&cq_poll->napi);
>diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c
>b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c
>index 459b94b99ddb..927dd12b4f4e 100644
>--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c
>+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c
>@@ -512,6 +512,22 @@ static int otx2_tx_napi_handler(struct otx2_nic
>*pfvf,
> 	return 0;
> }
>
>+static void otx2_adjust_adaptive_coalese(struct otx2_nic *pfvf, struct
>+otx2_cq_poll *cq_poll) {
>+	struct dim_sample dim_sample;
>+	u64 rx_frames, rx_bytes;
>+
>+	rx_frames = OTX2_GET_RX_STATS(RX_BCAST) +
>OTX2_GET_RX_STATS(RX_MCAST) +
>+			OTX2_GET_RX_STATS(RX_UCAST);
>+	rx_bytes = OTX2_GET_RX_STATS(RX_OCTS);
>+	dim_update_sample(pfvf->napi_events,
>+			  rx_frames,
>+			  rx_bytes,
>+			  &dim_sample);
>+
>+	net_dim(&cq_poll->dim, dim_sample);
>+}
>+
> int otx2_napi_handler(struct napi_struct *napi, int budget)  {
> 	struct otx2_cq_queue *rx_cq = NULL;
>@@ -549,6 +565,18 @@ int otx2_napi_handler(struct napi_struct *napi, int
>budget)
> 		if (pfvf->flags & OTX2_FLAG_INTF_DOWN)
> 			return workdone;
>
>+		/* Check for adaptive interrupt coalesce */
>+		if (workdone != 0 &&
>+		    ((pfvf->flags & OTX2_FLAG_ADPTV_INT_COAL_ENABLED) ==
>+		    OTX2_FLAG_ADPTV_INT_COAL_ENABLED)) {
>+			/* Adjust irq coalese using net_dim */
>+			otx2_adjust_adaptive_coalese(pfvf, cq_poll);
>+
>+			/* Update irq coalescing */
>+			for (i = 0; i < pfvf->hw.cint_cnt; i++)
>+				otx2_config_irq_coalescing(pfvf, i);
>+		}
>+
> 		/* Re-enable interrupts */
> 		otx2_write64(pfvf, NIX_LF_CINTX_ENA_W1S(cq_poll->cint_idx),
> 			     BIT_ULL(0));
>diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.h
>b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.h
>index a2ac2b3bdbf5..ed41a68d3ec6 100644
>--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.h
>+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.h
>@@ -107,6 +107,7 @@ struct otx2_cq_poll {
> #define CINT_INVALID_CQ		255
> 	u8			cint_idx;
> 	u8			cq_ids[CQS_PER_CINT];
>+	struct dim		dim;
> 	struct napi_struct	napi;
> };
>
>--
>2.25.1

Re:

[Yonghong Song]

On 5/20/23 2:47 AM, Ze Gao wrote:
> 
> Hi Jiri,
> 
> Would you like to consider to add rcu_is_watching check in
> to solve this from the viewpoint of kprobe_multi_link_prog_run
> itself? And accounting of missed runs can be added as well
> to imporve observability.
> 
> Regards,
> Ze
> 
> 
> -----------------
>  From 29fd3cd713e65461325c2703cf5246a6fae5d4fe Mon Sep 17 00:00:00 2001
> From: Ze Gao <zegao@tencent.com>
> Date: Sat, 20 May 2023 17:32:05 +0800
> Subject: [PATCH] bpf: kprobe_multi runs bpf progs only when rcu_is_watching
> 
>  From the perspective of kprobe_multi_link_prog_run, any traceable
> functions can be attached while bpf progs need specical care and
> ought to be under rcu protection. To solve the likely rcu lockdep
> warns once for good, when (future) functions in idle path were
> attached accidentally, we better paying some cost to check at least
> in kernel-side, and return when rcu is not watching, which helps
> to avoid any unpredictable results.

kprobe_multi/fprobe share the same set of attachments with fentry.
Currently, fentry does not filter with !rcu_is_watching, maybe
because this is an extreme corner case. Not sure whether it is
worthwhile or not.

Maybe if you can give a concrete example (e.g., attachment point)
with current code base to show what the issue you encountered and
it will make it easier to judge whether adding !rcu_is_watching()
is necessary or not.

> 
> Signed-off-by: Ze Gao <zegao@tencent.com>
> ---
>   kernel/trace/bpf_trace.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
> index 9a050e36dc6c..3e6ea7274765 100644
> --- a/kernel/trace/bpf_trace.c
> +++ b/kernel/trace/bpf_trace.c
> @@ -2622,7 +2622,7 @@ kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link,
>   	struct bpf_run_ctx *old_run_ctx;
>   	int err;
>   
> -	if (unlikely(__this_cpu_inc_return(bpf_prog_active) != 1)) {
> +	if (unlikely(__this_cpu_inc_return(bpf_prog_active) != 1 || !rcu_is_watching())) {
>   		err = 0;
>   		goto out;
>   	}

Re:

[Jiri Olsa]

On Sat, May 20, 2023 at 05:47:24PM +0800, Ze Gao wrote:
> 
> Hi Jiri,
> 
> Would you like to consider to add rcu_is_watching check in
> to solve this from the viewpoint of kprobe_multi_link_prog_run

I think this was discussed in here:
  https://lore.kernel.org/bpf/20230321020103.13494-1-laoar.shao@gmail.com/

and was considered a bug, there's fix mentioned later in the thread

there's also this recent patchset:
  https://lore.kernel.org/bpf/20230517034510.15639-3-zegao@tencent.com/

that solves related problems

> itself? And accounting of missed runs can be added as well
> to imporve observability.

right, we count fprobe->nmissed but it's not exposed, we should allow
to get 'missed' stats from both fprobe and kprobe_multi later, which
is missing now, will check

thanks,
jirka

> 
> Regards,
> Ze
> 
> 
> -----------------
> From 29fd3cd713e65461325c2703cf5246a6fae5d4fe Mon Sep 17 00:00:00 2001
> From: Ze Gao <zegao@tencent.com>
> Date: Sat, 20 May 2023 17:32:05 +0800
> Subject: [PATCH] bpf: kprobe_multi runs bpf progs only when rcu_is_watching
> 
> From the perspective of kprobe_multi_link_prog_run, any traceable
> functions can be attached while bpf progs need specical care and
> ought to be under rcu protection. To solve the likely rcu lockdep
> warns once for good, when (future) functions in idle path were
> attached accidentally, we better paying some cost to check at least
> in kernel-side, and return when rcu is not watching, which helps
> to avoid any unpredictable results.
> 
> Signed-off-by: Ze Gao <zegao@tencent.com>
> ---
>  kernel/trace/bpf_trace.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
> index 9a050e36dc6c..3e6ea7274765 100644
> --- a/kernel/trace/bpf_trace.c
> +++ b/kernel/trace/bpf_trace.c
> @@ -2622,7 +2622,7 @@ kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link,
>  	struct bpf_run_ctx *old_run_ctx;
>  	int err;
>  
> -	if (unlikely(__this_cpu_inc_return(bpf_prog_active) != 1)) {
> +	if (unlikely(__this_cpu_inc_return(bpf_prog_active) != 1 || !rcu_is_watching())) {
>  		err = 0;
>  		goto out;
>  	}
> -- 
> 2.40.1
> 

Re:

[Masami Hiramatsu]

On Sun, 21 May 2023 10:08:46 +0200
Jiri Olsa <olsajiri@gmail.com> wrote:

> On Sat, May 20, 2023 at 05:47:24PM +0800, Ze Gao wrote:
> > 
> > Hi Jiri,
> > 
> > Would you like to consider to add rcu_is_watching check in
> > to solve this from the viewpoint of kprobe_multi_link_prog_run
> 
> I think this was discussed in here:
>   https://lore.kernel.org/bpf/20230321020103.13494-1-laoar.shao@gmail.com/
> 
> and was considered a bug, there's fix mentioned later in the thread
> 
> there's also this recent patchset:
>   https://lore.kernel.org/bpf/20230517034510.15639-3-zegao@tencent.com/
> 
> that solves related problems

I think this rcu_is_watching() is a bit different issue. This rcu_is_watching()
check is required if the kprobe_multi_link_prog_run() uses any RCU API.
E.g. rethook_try_get() is also checks rcu_is_watching() because it uses
call_rcu().

Thank you,

> 
> > itself? And accounting of missed runs can be added as well
> > to imporve observability.
> 
> right, we count fprobe->nmissed but it's not exposed, we should allow
> to get 'missed' stats from both fprobe and kprobe_multi later, which
> is missing now, will check
> 
> thanks,
> jirka
> 
> > 
> > Regards,
> > Ze
> > 
> > 
> > -----------------
> > From 29fd3cd713e65461325c2703cf5246a6fae5d4fe Mon Sep 17 00:00:00 2001
> > From: Ze Gao <zegao@tencent.com>
> > Date: Sat, 20 May 2023 17:32:05 +0800
> > Subject: [PATCH] bpf: kprobe_multi runs bpf progs only when rcu_is_watching
> > 
> > From the perspective of kprobe_multi_link_prog_run, any traceable
> > functions can be attached while bpf progs need specical care and
> > ought to be under rcu protection. To solve the likely rcu lockdep
> > warns once for good, when (future) functions in idle path were
> > attached accidentally, we better paying some cost to check at least
> > in kernel-side, and return when rcu is not watching, which helps
> > to avoid any unpredictable results.
> > 
> > Signed-off-by: Ze Gao <zegao@tencent.com>
> > ---
> >  kernel/trace/bpf_trace.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
> > index 9a050e36dc6c..3e6ea7274765 100644
> > --- a/kernel/trace/bpf_trace.c
> > +++ b/kernel/trace/bpf_trace.c
> > @@ -2622,7 +2622,7 @@ kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link,
> >  	struct bpf_run_ctx *old_run_ctx;
> >  	int err;
> >  
> > -	if (unlikely(__this_cpu_inc_return(bpf_prog_active) != 1)) {
> > +	if (unlikely(__this_cpu_inc_return(bpf_prog_active) != 1 || !rcu_is_watching())) {
> >  		err = 0;
> >  		goto out;
> >  	}
> > -- 
> > 2.40.1
> > 


-- 
Masami Hiramatsu (Google) <mhiramat@kernel.org>

Re:

[Ze Gao]

On Sun, May 21, 2023 at 6:09 PM Masami Hiramatsu <mhiramat@kernel.org> wrote:
>
> On Sun, 21 May 2023 10:08:46 +0200
> Jiri Olsa <olsajiri@gmail.com> wrote:
>
> > On Sat, May 20, 2023 at 05:47:24PM +0800, Ze Gao wrote:
> > >
> > > Hi Jiri,
> > >
> > > Would you like to consider to add rcu_is_watching check in
> > > to solve this from the viewpoint of kprobe_multi_link_prog_run
> >
> > I think this was discussed in here:
> >   https://lore.kernel.org/bpf/20230321020103.13494-1-laoar.shao@gmail.com/
> >
> > and was considered a bug, there's fix mentioned later in the thread
> >
> > there's also this recent patchset:
> >   https://lore.kernel.org/bpf/20230517034510.15639-3-zegao@tencent.com/
> >
> > that solves related problems
>
> I think this rcu_is_watching() is a bit different issue. This rcu_is_watching()
> check is required if the kprobe_multi_link_prog_run() uses any RCU API.
> E.g. rethook_try_get() is also checks rcu_is_watching() because it uses
> call_rcu().

Yes, that's my point!

Regards,
Ze

>
> >
> > > itself? And accounting of missed runs can be added as well
> > > to imporve observability.
> >
> > right, we count fprobe->nmissed but it's not exposed, we should allow
> > to get 'missed' stats from both fprobe and kprobe_multi later, which
> > is missing now, will check
> >
> > thanks,
> > jirka
> >
> > >
> > > Regards,
> > > Ze
> > >
> > >
> > > -----------------
> > > From 29fd3cd713e65461325c2703cf5246a6fae5d4fe Mon Sep 17 00:00:00 2001
> > > From: Ze Gao <zegao@tencent.com>
> > > Date: Sat, 20 May 2023 17:32:05 +0800
> > > Subject: [PATCH] bpf: kprobe_multi runs bpf progs only when rcu_is_watching
> > >
> > > From the perspective of kprobe_multi_link_prog_run, any traceable
> > > functions can be attached while bpf progs need specical care and
> > > ought to be under rcu protection. To solve the likely rcu lockdep
> > > warns once for good, when (future) functions in idle path were
> > > attached accidentally, we better paying some cost to check at least
> > > in kernel-side, and return when rcu is not watching, which helps
> > > to avoid any unpredictable results.
> > >
> > > Signed-off-by: Ze Gao <zegao@tencent.com>
> > > ---
> > >  kernel/trace/bpf_trace.c | 2 +-
> > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > >
> > > diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
> > > index 9a050e36dc6c..3e6ea7274765 100644
> > > --- a/kernel/trace/bpf_trace.c
> > > +++ b/kernel/trace/bpf_trace.c
> > > @@ -2622,7 +2622,7 @@ kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link,
> > >     struct bpf_run_ctx *old_run_ctx;
> > >     int err;
> > >
> > > -   if (unlikely(__this_cpu_inc_return(bpf_prog_active) != 1)) {
> > > +   if (unlikely(__this_cpu_inc_return(bpf_prog_active) != 1 || !rcu_is_watching())) {
> > >             err = 0;
> > >             goto out;
> > >     }
> > > --
> > > 2.40.1
> > >
>
>
> --
> Masami Hiramatsu (Google) <mhiramat@kernel.org>

Re:

[Ze Gao]

> kprobe_multi/fprobe share the same set of attachments with fentry.
> Currently, fentry does not filter with !rcu_is_watching, maybe
> because this is an extreme corner case. Not sure whether it is
> worthwhile or not.

Agreed, it's rare, especially after Peter's patches which push narrow
down rcu eqs regions
in the idle path and reduce the chance of any traceable functions
happening in between.

However, from RCU's perspective, we ought to check if rcu_is_watching
theoretically
when there's a chance our code will run in the idle path and also we
need rcu to be alive,
And also we cannot simply make assumptions for any future changes in
the idle path.
You know, just like what was hit in the thread.

> Maybe if you can give a concrete example (e.g., attachment point)
> with current code base to show what the issue you encountered and
> it will make it easier to judge whether adding !rcu_is_watching()
> is necessary or not.

I can reproduce likely warnings on v6.1.18 where arch_cpu_idle is
traceable but not on the latest version
so far. But as I state above, in theory we need it. So here is a
gentle ping :) .

Re:

[Jiri Olsa]

On Sun, May 21, 2023 at 11:10:16PM +0800, Ze Gao wrote:
> > kprobe_multi/fprobe share the same set of attachments with fentry.
> > Currently, fentry does not filter with !rcu_is_watching, maybe
> > because this is an extreme corner case. Not sure whether it is
> > worthwhile or not.
> 
> Agreed, it's rare, especially after Peter's patches which push narrow
> down rcu eqs regions
> in the idle path and reduce the chance of any traceable functions
> happening in between.
> 
> However, from RCU's perspective, we ought to check if rcu_is_watching
> theoretically
> when there's a chance our code will run in the idle path and also we
> need rcu to be alive,
> And also we cannot simply make assumptions for any future changes in
> the idle path.
> You know, just like what was hit in the thread.
> 
> > Maybe if you can give a concrete example (e.g., attachment point)
> > with current code base to show what the issue you encountered and
> > it will make it easier to judge whether adding !rcu_is_watching()
> > is necessary or not.
> 
> I can reproduce likely warnings on v6.1.18 where arch_cpu_idle is
> traceable but not on the latest version
> so far. But as I state above, in theory we need it. So here is a
> gentle ping :) .

hum, this change [1] added rcu_is_watching check to ftrace_test_recursion_trylock,
which we use in fprobe_handler and is coming to fprobe_exit_handler in [2]

I might be missing something, but it seems like we don't need another
rcu_is_watching call on kprobe_multi level

jirka


[1] d099dbfd3306 cpuidle: tracing: Warn about !rcu_is_watching()
[2] https://lore.kernel.org/bpf/20230517034510.15639-4-zegao@tencent.com/

Re:

[Masami Hiramatsu]

On Sun, 21 May 2023 22:26:37 +0200
Jiri Olsa <olsajiri@gmail.com> wrote:

> On Sun, May 21, 2023 at 11:10:16PM +0800, Ze Gao wrote:
> > > kprobe_multi/fprobe share the same set of attachments with fentry.
> > > Currently, fentry does not filter with !rcu_is_watching, maybe
> > > because this is an extreme corner case. Not sure whether it is
> > > worthwhile or not.
> > 
> > Agreed, it's rare, especially after Peter's patches which push narrow
> > down rcu eqs regions
> > in the idle path and reduce the chance of any traceable functions
> > happening in between.
> > 
> > However, from RCU's perspective, we ought to check if rcu_is_watching
> > theoretically
> > when there's a chance our code will run in the idle path and also we
> > need rcu to be alive,
> > And also we cannot simply make assumptions for any future changes in
> > the idle path.
> > You know, just like what was hit in the thread.
> > 
> > > Maybe if you can give a concrete example (e.g., attachment point)
> > > with current code base to show what the issue you encountered and
> > > it will make it easier to judge whether adding !rcu_is_watching()
> > > is necessary or not.
> > 
> > I can reproduce likely warnings on v6.1.18 where arch_cpu_idle is
> > traceable but not on the latest version
> > so far. But as I state above, in theory we need it. So here is a
> > gentle ping :) .
> 
> hum, this change [1] added rcu_is_watching check to ftrace_test_recursion_trylock,
> which we use in fprobe_handler and is coming to fprobe_exit_handler in [2]
> 
> I might be missing something, but it seems like we don't need another
> rcu_is_watching call on kprobe_multi level

Good point! OK, then it seems we don't need it. The rethook continues to
use the rcu_is_watching() because it is also used from kprobes, but the
kprobe_multi doesn't need it.

Thank you,

> 
> jirka
> 
> 
> [1] d099dbfd3306 cpuidle: tracing: Warn about !rcu_is_watching()
> [2] https://lore.kernel.org/bpf/20230517034510.15639-4-zegao@tencent.com/


-- 
Masami Hiramatsu (Google) <mhiramat@kernel.org>

Re:

[Ze Gao]

Oops, I missed that. Thanks for pointing that out, which I thought is
conditional use of rcu_is_watching before.

One last point, I think we should double check on this
     "fentry does not filter with !rcu_is_watching"
as quoted from Yonghong and argue whether it needs
the same check for fentry as well.

Regards,
Ze

Re:

[Yonghong Song]

On 5/21/23 7:07 PM, Ze Gao wrote:
> Oops, I missed that. Thanks for pointing that out, which I thought is
> conditional use of rcu_is_watching before.
> 
> One last point, I think we should double check on this
>       "fentry does not filter with !rcu_is_watching"
> as quoted from Yonghong and argue whether it needs
> the same check for fentry as well.

I would suggest that we address rcu_is_watching issue for fentry
only if we do have a reproducible case to show something goes wrong...

> 
> Regards,
> Ze

Re:

[Masami Hiramatsu]

On Mon, 22 May 2023 10:07:42 +0800
Ze Gao <zegao2021@gmail.com> wrote:

> Oops, I missed that. Thanks for pointing that out, which I thought is
> conditional use of rcu_is_watching before.
> 
> One last point, I think we should double check on this
>      "fentry does not filter with !rcu_is_watching"
> as quoted from Yonghong and argue whether it needs
> the same check for fentry as well.

rcu_is_watching() comment says;

 * if the current CPU is not in its idle loop or is in an interrupt or
 * NMI handler, return true.

Thus it returns *fault* if the current CPU is in the idle loop and not
any interrupt(including NMI) context. This means if any tracable function
is called from idle loop, it can be !rcu_is_watching(). I meant, this is
'context' based check, thus fentry can not filter out that some commonly
used functions is called from that context but it can be detected.

Thank you,

> 
> Regards,
> Ze


-- 
Masami Hiramatsu (Google) <mhiramat@kernel.org>

Re:

[Paul E. McKenney]

On Tue, May 23, 2023 at 01:30:19PM +0800, Masami Hiramatsu wrote:
> On Mon, 22 May 2023 10:07:42 +0800
> Ze Gao <zegao2021@gmail.com> wrote:
> 
> > Oops, I missed that. Thanks for pointing that out, which I thought is
> > conditional use of rcu_is_watching before.
> > 
> > One last point, I think we should double check on this
> >      "fentry does not filter with !rcu_is_watching"
> > as quoted from Yonghong and argue whether it needs
> > the same check for fentry as well.
> 
> rcu_is_watching() comment says;
> 
>  * if the current CPU is not in its idle loop or is in an interrupt or
>  * NMI handler, return true.
> 
> Thus it returns *fault* if the current CPU is in the idle loop and not
> any interrupt(including NMI) context. This means if any tracable function
> is called from idle loop, it can be !rcu_is_watching(). I meant, this is
> 'context' based check, thus fentry can not filter out that some commonly
> used functions is called from that context but it can be detected.

It really does return false (rather than faulting?) if the current CPU
is deep within the idle loop.

In addition, the recent x86/entry rework (thank you Peter and
Thomas!) mean that the "idle loop" is quite restricted, as can be
seen by the invocations of ct_cpuidle_enter() and ct_cpuidle_exit().
For example, in default_idle_call(), these are immediately before and
after the call to arch_cpu_idle().

Would the following help?  Or am I missing your point?

							Thanx, Paul

------------------------------------------------------------------------

diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 1449cb69a0e0..fae9b4e29c93 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -679,10 +679,14 @@ static void rcu_disable_urgency_upon_qs(struct rcu_data *rdp)
 /**
  * rcu_is_watching - see if RCU thinks that the current CPU is not idle
  *
- * Return true if RCU is watching the running CPU, which means that this
- * CPU can safely enter RCU read-side critical sections.  In other words,
- * if the current CPU is not in its idle loop or is in an interrupt or
- * NMI handler, return true.
+ * Return @true if RCU is watching the running CPU and @false otherwise.
+ * An @true return means that this CPU can safely enter RCU read-side
+ * critical sections.
+ *
+ * More specifically, if the current CPU is not deep within its idle
+ * loop, return @true.  Note that rcu_is_watching() will return @true if
+ * invoked from an interrupt or NMI handler, even if that interrupt or
+ * NMI interrupted the CPU while it was deep within its idle loop.
  *
  * Make notrace because it can be called by the internal functions of
  * ftrace, and making this notrace removes unnecessary recursion calls.

Re:

[Masami Hiramatsu]

On Mon, 22 May 2023 23:59:28 -0700
"Paul E. McKenney" <paulmck@kernel.org> wrote:

> On Tue, May 23, 2023 at 01:30:19PM +0800, Masami Hiramatsu wrote:
> > On Mon, 22 May 2023 10:07:42 +0800
> > Ze Gao <zegao2021@gmail.com> wrote:
> > 
> > > Oops, I missed that. Thanks for pointing that out, which I thought is
> > > conditional use of rcu_is_watching before.
> > > 
> > > One last point, I think we should double check on this
> > >      "fentry does not filter with !rcu_is_watching"
> > > as quoted from Yonghong and argue whether it needs
> > > the same check for fentry as well.
> > 
> > rcu_is_watching() comment says;
> > 
> >  * if the current CPU is not in its idle loop or is in an interrupt or
> >  * NMI handler, return true.
> > 
> > Thus it returns *fault* if the current CPU is in the idle loop and not
> > any interrupt(including NMI) context. This means if any tracable function
> > is called from idle loop, it can be !rcu_is_watching(). I meant, this is
> > 'context' based check, thus fentry can not filter out that some commonly
> > used functions is called from that context but it can be detected.
> 
> It really does return false (rather than faulting?) if the current CPU
> is deep within the idle loop.
> 
> In addition, the recent x86/entry rework (thank you Peter and
> Thomas!) mean that the "idle loop" is quite restricted, as can be
> seen by the invocations of ct_cpuidle_enter() and ct_cpuidle_exit().
> For example, in default_idle_call(), these are immediately before and
> after the call to arch_cpu_idle().

Thanks! I also found that the default_idle_call() is enough small and
it seems not happening on fentry because there are no commonly used
functions on that path.

> 
> Would the following help?  Or am I missing your point?

Yes, thank you for the update!

> 
> 							Thanx, Paul
> 
> ------------------------------------------------------------------------
> 
> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> index 1449cb69a0e0..fae9b4e29c93 100644
> --- a/kernel/rcu/tree.c
> +++ b/kernel/rcu/tree.c
> @@ -679,10 +679,14 @@ static void rcu_disable_urgency_upon_qs(struct rcu_data *rdp)
>  /**
>   * rcu_is_watching - see if RCU thinks that the current CPU is not idle
>   *
> - * Return true if RCU is watching the running CPU, which means that this
> - * CPU can safely enter RCU read-side critical sections.  In other words,
> - * if the current CPU is not in its idle loop or is in an interrupt or
> - * NMI handler, return true.
> + * Return @true if RCU is watching the running CPU and @false otherwise.
> + * An @true return means that this CPU can safely enter RCU read-side
> + * critical sections.
> + *
> + * More specifically, if the current CPU is not deep within its idle
> + * loop, return @true.  Note that rcu_is_watching() will return @true if
> + * invoked from an interrupt or NMI handler, even if that interrupt or
> + * NMI interrupted the CPU while it was deep within its idle loop.
>   *
>   * Make notrace because it can be called by the internal functions of
>   * ftrace, and making this notrace removes unnecessary recursion calls.


-- 
Masami Hiramatsu (Google) <mhiramat@kernel.org>

[PATCH 0/3] net: microchip_t1s: additional phy support and collision detect handling

[Ramón N.Rodriguez]

Hi,

This patch series adds support for LAN867x Rev.C and sets the collision
enable bit conditionally on plca enable for LAN865X and LAN867X phys.

The addition of Rev.C support is pretty straight forward (but weird), it 
follows the recommended magic and unexplained steps in Microchip
Application Note 1699.

The second change, conditionally setting collision detection has a
significant impact on link performance. Having both PLCA and collision
detection enabled led to a lot of dropped packets in the setup I've been
developing on.
The datasheets recommends disabling collision detect when PLCA is
enabled.
Took me a couple of weeks to find said footnote, hoping this series can
save the next dev some headache.

Following is a brief description of my test/eval setup:

LAN867x Rev.C
This was tested with Microchips usb eval board, the only testing I did
was that I got the driver attached and got a link.
No performance/stress testing has been performed.

Collision detection
This has been tested on a setup where one ARM system with a LAN8650
mac-phy is daisy-chained to 8 mcus using lan8670 phys. Without the patch we
were limited to really short cables, about 1m per node, but we were
still getting a lot of connection drops.
With the patch we could increase the total cable length to at least 40M.
Electrical properties play in here, both cable types and how the
termination is designed, so these results might not be reproducible in
another setup.
The only testing that has been performed has been a best effort of
estimating dropped packets on the linux node, with low frequency traffic.

Ramón Nordin Rodriguez (3):
  net: microchip_t1s: refactor reset functionality
  net: microchip_t1s: add support for LAN867x Rev.C1
  net: microchip_t1s: conditional collision detection

 drivers/net/phy/microchip_t1s.c | 135 +++++++++++++++++++++++++++++++-
 1 file changed, 132 insertions(+), 3 deletions(-)

-- 
2.40.1

[PATCH 1/3] net: microchip_t1s: refactor reset functionality

[Ramón N.Rodriguez]

From: Ramón Nordin Rodriguez <ramon.nordin.rodriguez@ferroamp.se>

This commit moves the reset functionality for lan867x from the revb1
init function to a separate function. The intention with this minor
refactor is to prepare for adding support for lan867x rev C1.

Signed-off-by: Ramón Nordin Rodriguez <ramon.nordin.rodriguez@ferroamp.se>
---
 drivers/net/phy/microchip_t1s.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/drivers/net/phy/microchip_t1s.c b/drivers/net/phy/microchip_t1s.c
index 534ca7d1b061..ace2bf35a18a 100644
--- a/drivers/net/phy/microchip_t1s.c
+++ b/drivers/net/phy/microchip_t1s.c
@@ -213,7 +213,7 @@ static int lan865x_revb0_config_init(struct phy_device *phydev)
 	return lan865x_setup_cfgparam(phydev);
 }
 
-static int lan867x_revb1_config_init(struct phy_device *phydev)
+static int lan867x_wait_for_reset_complete(struct phy_device *phydev)
 {
 	int err;
 
@@ -234,6 +234,16 @@ static int lan867x_revb1_config_init(struct phy_device *phydev)
 			return -ENODEV;
 		}
 	}
+	return 0;
+}
+
+static int lan867x_revb1_config_init(struct phy_device *phydev)
+{
+	int err;
+
+	err = lan867x_wait_for_reset_complete(phydev);
+	if (err)
+		return err;
 
 	/* Reference to AN1699
 	 * https://ww1.microchip.com/downloads/aemDocuments/documents/AIS/ProductDocuments/SupportingCollateral/AN-LAN8670-1-2-config-60001699.pdf
-- 
2.40.1

[PATCH 2/3] net: microchip_t1s: add support for LAN867x Rev.C1

[Ramón N.Rodriguez]

From: Ramón Nordin Rodriguez <ramon.nordin.rodriguez@ferroamp.se>

This commit adds support for yet another Microchip T1S lan867x phy
revision. The only meaningful difference between Rev.B that already is
supported and Rev.C is the init change where other undocumented regs
needs writes with unexplained values.
The changes introduced here attempts to follow the manufacturer
recommendations in AN1699.

Signed-off-by: Ramón Nordin Rodriguez <ramon.nordin.rodriguez@ferroamp.se>
---
 drivers/net/phy/microchip_t1s.c | 96 +++++++++++++++++++++++++++++++++
 1 file changed, 96 insertions(+)

diff --git a/drivers/net/phy/microchip_t1s.c b/drivers/net/phy/microchip_t1s.c
index ace2bf35a18a..db84d850b165 100644
--- a/drivers/net/phy/microchip_t1s.c
+++ b/drivers/net/phy/microchip_t1s.c
@@ -12,6 +12,7 @@
 #include <linux/phy.h>
 
 #define PHY_ID_LAN867X_REVB1 0x0007C162
+#define PHY_ID_LAN867X_REVC1 0x0007C164
 #define PHY_ID_LAN865X_REVB0 0x0007C1B3
 
 #define LAN867X_REG_STS2 0x0019
@@ -59,6 +60,22 @@ static const u16 lan867x_revb1_fixup_masks[12] = {
 	0x0600, 0x7F00, 0x2000, 0xFFFF,
 };
 
+static const u16 lan867x_revc1_fixup_registers[12] = {
+	0x00D0, 0x00E0, 0x0084, 0x008A,
+	0x00E9, 0x00F5, 0x00F4, 0x00F8,
+	0x00F9, 0x0081, 0x0091, 0x0093,
+};
+
+/* Index 2 & 3 will not be used, these are runtime populated/calculated.
+ * It makes the code a lot easier to read having this arr the same len
+ * as the _fixup_registers arr though
+ */
+static const u16 lan867x_revc1_fixup_values[12] = {
+	0x3F31, 0xC000, 0xFFFF, 0xFFFF,
+	0x9E50, 0x1CF8, 0xC020, 0x9B00,
+	0x4E53, 0x0080, 0x9660, 0x06E9,
+};
+
 /* LAN865x Rev.B0 configuration parameters from AN1760 */
 static const u32 lan865x_revb0_fixup_registers[28] = {
 	0x0091, 0x0081, 0x0043, 0x0044,
@@ -263,6 +280,74 @@ static int lan867x_revb1_config_init(struct phy_device *phydev)
 	return 0;
 }
 
+static int lan867x_revc1_read_fixup_value(struct phy_device *phydev, u16 addr)
+{
+	int regval;
+	/* The AN pretty much just states 'trust us' regarding these magic vals */
+	const u16 magic_or = 0xE0;
+	const u16 magic_reg_mask = 0x1F;
+	const u16 magic_check_mask = 0x10;
+
+	regval = lan865x_revb0_indirect_read(phydev, addr);
+	if (regval < 0)
+		return regval;
+
+	regval &= magic_reg_mask;
+
+	return (regval & magic_check_mask) ? regval | magic_or : regval;
+}
+
+static int lan867x_revc1_config_init(struct phy_device *phydev)
+{
+	int err;
+	int regval;
+	u16 override0;
+	u16 override1;
+	const u16 override_addr0 = 0x4;
+	const u16 override_addr1 = 0x8;
+	const u8 index_to_override0 = 2;
+	const u8 index_to_override1 = 3;
+
+	err = lan867x_wait_for_reset_complete(phydev);
+	if (err)
+		return err;
+
+	/* The application note specifies a super convenient process
+	 * where 2 of the fixup regs needs a write with a value that is
+	 * a modified result of another reg read.
+	 * Enjoy the magic show.
+	 */
+	regval = lan867x_revc1_read_fixup_value(phydev, override_addr0);
+	if (regval < 0)
+		return regval;
+	override0 = ((regval + 9) << 10) | ((regval + 14) << 4) | 0x3;
+
+	regval = lan867x_revc1_read_fixup_value(phydev, override_addr1);
+	if (regval < 0)
+		return regval;
+	override1 = (regval + 40) << 10;
+
+	for (int i = 0; i < ARRAY_SIZE(lan867x_revc1_fixup_registers); i++) {
+		/* The hardcoded  */
+		if (i == index_to_override0)
+			err = phy_write_mmd(phydev, MDIO_MMD_VEND2,
+					    lan867x_revc1_fixup_registers[i],
+					    override0);
+		else if (i == index_to_override1)
+			err = phy_write_mmd(phydev, MDIO_MMD_VEND2,
+					    lan867x_revc1_fixup_registers[i],
+					    override1);
+		else
+			err = phy_write_mmd(phydev, MDIO_MMD_VEND2,
+					    lan867x_revc1_fixup_registers[i],
+					    lan867x_revc1_fixup_values[i]);
+		if (err)
+			return err;
+	}
+
+	return 0;
+}
+
 static int lan86xx_read_status(struct phy_device *phydev)
 {
 	/* The phy has some limitations, namely:
@@ -289,6 +374,16 @@ static struct phy_driver microchip_t1s_driver[] = {
 		.set_plca_cfg	    = genphy_c45_plca_set_cfg,
 		.get_plca_status    = genphy_c45_plca_get_status,
 	},
+	{
+		PHY_ID_MATCH_EXACT(PHY_ID_LAN867X_REVC1),
+		.name               = "LAN867X Rev.C1",
+		.features           = PHY_BASIC_T1S_P2MP_FEATURES,
+		.config_init        = lan867x_revc1_config_init,
+		.read_status        = lan86xx_read_status,
+		.get_plca_cfg	    = genphy_c45_plca_get_cfg,
+		.set_plca_cfg	    = genphy_c45_plca_set_cfg,
+		.get_plca_status    = genphy_c45_plca_get_status,
+	},
 	{
 		PHY_ID_MATCH_EXACT(PHY_ID_LAN865X_REVB0),
 		.name               = "LAN865X Rev.B0 Internal Phy",
@@ -305,6 +400,7 @@ module_phy_driver(microchip_t1s_driver);
 
 static struct mdio_device_id __maybe_unused tbl[] = {
 	{ PHY_ID_MATCH_EXACT(PHY_ID_LAN867X_REVB1) },
+	{ PHY_ID_MATCH_EXACT(PHY_ID_LAN867X_REVC1) },
 	{ PHY_ID_MATCH_EXACT(PHY_ID_LAN865X_REVB0) },
 	{ }
 };
-- 
2.40.1

[PATCH 3/3] net: microchip_t1s: conditional collision detection

[Ramón N.Rodriguez]

From: Ramón Nordin Rodriguez <ramon.nordin.rodriguez@ferroamp.se>

This commit conditionally sets the collision detection bit on lan867x
and lan865x phys on changing PLCA enabled on/off. The intended realworld
scenario is that all nodes on the network run the same settings with
regards to plca, and when plca is enabled the physical layer guarantees
that no collisions should occur.
In a practical setting where it was tested running one node with
collision detection on and other off, the node with collision detection
on dropped a lot of packets, leading to a poor performing link.
Worth noting here is that the phys default/reset to plca disabled and
collision detection enabled. Thus this change would only have an effect
when changing settings via ethtool.

Signed-off-by: Ramón Nordin Rodriguez <ramon.nordin.rodriguez@ferroamp.se>
---
 drivers/net/phy/microchip_t1s.c | 29 ++++++++++++++++++++++++++---
 1 file changed, 26 insertions(+), 3 deletions(-)

diff --git a/drivers/net/phy/microchip_t1s.c b/drivers/net/phy/microchip_t1s.c
index db84d850b165..3b1e82ecdf69 100644
--- a/drivers/net/phy/microchip_t1s.c
+++ b/drivers/net/phy/microchip_t1s.c
@@ -23,8 +23,10 @@
 #define LAN865X_REG_CFGPARAM_DATA 0x00D9
 #define LAN865X_REG_CFGPARAM_CTRL 0x00DA
 #define LAN865X_REG_STS2 0x0019
+#define LAN86XX_REG_COLLISION_DETECT 0x0087
 
 #define LAN865X_CFGPARAM_READ_ENABLE BIT(1)
+#define LAN86XX_COLLISION_DETECT_ENABLE BIT(15)
 
 /* The arrays below are pulled from the following table from AN1699
  * Access MMD Address Value Mask
@@ -363,6 +365,27 @@ static int lan86xx_read_status(struct phy_device *phydev)
 	return 0;
 }
 
+static int lan86xx_plca_set_cfg(struct phy_device *phydev,
+				const struct phy_plca_cfg *plca_cfg)
+{
+	int err;
+
+	err = genphy_c45_plca_set_cfg(phydev, plca_cfg);
+	if (err)
+		return err;
+
+	/* Disable collision detect on the phy when PLCA is enabled.
+	 * Noise can be picked up as a false positive for collisions
+	 * leading to the phy dropping legitimate packets.
+	 * No collisions should be possible when all nodes are setup
+	 * for running PLCA.
+	 */
+	return phy_modify_mmd(phydev, MDIO_MMD_VEND2,
+			LAN86XX_REG_COLLISION_DETECT,
+			LAN86XX_COLLISION_DETECT_ENABLE,
+			plca_cfg->enabled ? 0 : LAN86XX_COLLISION_DETECT_ENABLE);
+}
+
 static struct phy_driver microchip_t1s_driver[] = {
 	{
 		PHY_ID_MATCH_EXACT(PHY_ID_LAN867X_REVB1),
@@ -371,7 +394,7 @@ static struct phy_driver microchip_t1s_driver[] = {
 		.config_init        = lan867x_revb1_config_init,
 		.read_status        = lan86xx_read_status,
 		.get_plca_cfg	    = genphy_c45_plca_get_cfg,
-		.set_plca_cfg	    = genphy_c45_plca_set_cfg,
+		.set_plca_cfg	    = lan86xx_plca_set_cfg,
 		.get_plca_status    = genphy_c45_plca_get_status,
 	},
 	{
@@ -381,7 +404,7 @@ static struct phy_driver microchip_t1s_driver[] = {
 		.config_init        = lan867x_revc1_config_init,
 		.read_status        = lan86xx_read_status,
 		.get_plca_cfg	    = genphy_c45_plca_get_cfg,
-		.set_plca_cfg	    = genphy_c45_plca_set_cfg,
+		.set_plca_cfg	    = lan86xx_plca_set_cfg,
 		.get_plca_status    = genphy_c45_plca_get_status,
 	},
 	{
@@ -391,7 +414,7 @@ static struct phy_driver microchip_t1s_driver[] = {
 		.config_init        = lan865x_revb0_config_init,
 		.read_status        = lan86xx_read_status,
 		.get_plca_cfg	    = genphy_c45_plca_get_cfg,
-		.set_plca_cfg	    = genphy_c45_plca_set_cfg,
+		.set_plca_cfg	    = lan86xx_plca_set_cfg,
 		.get_plca_status    = genphy_c45_plca_get_status,
 	},
 };
-- 
2.40.1

Re: [PATCH 1/3] net: microchip_t1s: refactor reset functionality

[Andrew Lunn]

On Mon, Nov 27, 2023 at 11:40:43AM +0100, Ramón N.Rodriguez wrote:
> From: Ramón Nordin Rodriguez <ramon.nordin.rodriguez@ferroamp.se>
> 
> This commit moves the reset functionality for lan867x from the revb1
> init function to a separate function. The intention with this minor
> refactor is to prepare for adding support for lan867x rev C1.
> 
> Signed-off-by: Ramón Nordin Rodriguez <ramon.nordin.rodriguez@ferroamp.se>

Reviewed-by: Andrew Lunn <andrew@lunn.ch>

    Andrew

Re: [PATCH 2/3] net: microchip_t1s: add support for LAN867x Rev.C1

[Andrew Lunn]

>  #define PHY_ID_LAN867X_REVB1 0x0007C162
> +#define PHY_ID_LAN867X_REVC1 0x0007C164

So there is a gap in the revisions. Maybe a B2 exists?

> +static int lan867x_revc1_read_fixup_value(struct phy_device *phydev, u16 addr)
> +{
> +	int regval;
> +	/* The AN pretty much just states 'trust us' regarding these magic vals */
> +	const u16 magic_or = 0xE0;
> +	const u16 magic_reg_mask = 0x1F;
> +	const u16 magic_check_mask = 0x10;

Reverse christmass tree please. Longest first, shorted last.

> +	regval = lan865x_revb0_indirect_read(phydev, addr);
> +	if (regval < 0)
> +		return regval;
> +
> +	regval &= magic_reg_mask;
> +
> +	return (regval & magic_check_mask) ? regval | magic_or : regval;
> +}
> +
> +static int lan867x_revc1_config_init(struct phy_device *phydev)
> +{
> +	int err;
> +	int regval;
> +	u16 override0;
> +	u16 override1;
> +	const u16 override_addr0 = 0x4;
> +	const u16 override_addr1 = 0x8;
> +	const u8 index_to_override0 = 2;
> +	const u8 index_to_override1 = 3;

Same here.

> +
> +	err = lan867x_wait_for_reset_complete(phydev);
> +	if (err)
> +		return err;
> +
> +	/* The application note specifies a super convenient process
> +	 * where 2 of the fixup regs needs a write with a value that is
> +	 * a modified result of another reg read.
> +	 * Enjoy the magic show.
> +	 */

I really do hope that by revision D1 they get the firmware sorted out
so none of this undocumented magic is needed.

	Andrew

Re: [PATCH 0/3] net: microchip_t1s: additional phy support and collision detect handling

[Andrew Lunn]

> Collision detection
> This has been tested on a setup where one ARM system with a LAN8650
> mac-phy is daisy-chained to 8 mcus using lan8670 phys. Without the patch we
> were limited to really short cables, about 1m per node, but we were
> still getting a lot of connection drops.
> With the patch we could increase the total cable length to at least 40M.

Did you do any testing of collision detection enabled, PLCA disabled?

You say you think this is noise related. But the noise should be the
same with or without PLCA. I'm just thinking maybe collision detection
is just plain broken and should always be disabled?

I've not read much about T1S, but if we assume it is doing good old
fashioned CSMA/CD, with short cables the CS bit works well and the CD
is less important. CD was needed when you have 1000m cable, and you
can fit 64 bytes on the 1000m cable. So always turning of CD might be
appropriate.

	Andrew

Re: [PATCH 2/3] net: microchip_t1s: add support for LAN867x Rev.C1

[Ramón Nordin Rodriguez]

On Mon, Nov 27, 2023 at 02:37:41PM +0100, Andrew Lunn wrote:
> >  #define PHY_ID_LAN867X_REVB1 0x0007C162
> > +#define PHY_ID_LAN867X_REVC1 0x0007C164
> 
> So there is a gap in the revisions. Maybe a B2 exists?

The datasheet lists A0, B1 and C1, seems like Microchip removes the
application notes for old revisions, so no way that I can see to add the
init-fixup for A0.

I'm guessing there is a rev.c0 that was never released to the public.

> > +	const u16 magic_or = 0xE0;
> > +	const u16 magic_reg_mask = 0x1F;
> > +	const u16 magic_check_mask = 0x10;
> 
> Reverse christmass tree please. Longest first, shorted last.

My bad, I was just thinking 'christmas tree' forgot about the reverse.
I'll fix that.

> > +	int err;
> > +	int regval;
> > +	u16 override0;
> > +	u16 override1;
> > +	const u16 override_addr0 = 0x4;
> > +	const u16 override_addr1 = 0x8;
> > +	const u8 index_to_override0 = 2;
> > +	const u8 index_to_override1 = 3;
> 
> Same here.
I'll fix this.

> 
> > +
> > +	err = lan867x_wait_for_reset_complete(phydev);
> > +	if (err)
> > +		return err;
> > +
> > +	/* The application note specifies a super convenient process
> > +	 * where 2 of the fixup regs needs a write with a value that is
> > +	 * a modified result of another reg read.
> > +	 * Enjoy the magic show.
> > +	 */
> 
> I really do hope that by revision D1 they get the firmware sorted out
> so none of this undocumented magic is needed.
> 
> 	Andrew

Really do hope so.. 

Re: [PATCH 0/3] net: microchip_t1s: additional phy support and collision detect handling

[Ramón Nordin Rodriguez]

On Mon, Nov 27, 2023 at 02:58:32PM +0100, Andrew Lunn wrote:
> > Collision detection
> > This has been tested on a setup where one ARM system with a LAN8650
> > mac-phy is daisy-chained to 8 mcus using lan8670 phys. Without the patch we
> > were limited to really short cables, about 1m per node, but we were
> > still getting a lot of connection drops.
> > With the patch we could increase the total cable length to at least 40M.
> 
> Did you do any testing of collision detection enabled, PLCA disabled?
> 

In our dev system we've only tested with PLCA enabled, bit too tricky
changing internals on the microcontrollers.
But I have a lot of usb eval dongles that I can test with.

> You say you think this is noise related. But the noise should be the
> same with or without PLCA. I'm just thinking maybe collision detection
> is just plain broken and should always be disabled?
> 

I don't have access to the equipment to measure noise or reflections,
I've looked at the link with an oscilloscope and it looked fine to me.
The reason I'm mentioning noise is just me parroting the datasheet, for
context I'll quote the footnote here

"No physical collisions will occur when all nodes in a mixing segment are properly
configured for PLCA operation. As a result, for improved performance in high noise
environments where false collisions may be detected leading to dropped packets, it is
recommended that the user write this bit to a ‘0’ to disable collision detection when PLCA
is enabled. When collision detection is disabled, the PLCA reconciliation sublayer will still
assert logical collisions to the MAC as part of normal operation."
LAN8650 datasheet 11.5.51

> I've not read much about T1S, but if we assume it is doing good old
> fashioned CSMA/CD, with short cables the CS bit works well and the CD
> is less important. CD was needed when you have 1000m cable, and you
> can fit 64 bytes on the 1000m cable. So always turning of CD might be
> appropriate.
> 
> 	Andrew

As you assume when PLCA is disabled the phy runs in CSMA/CD mode.

I'll do some tests with both PLCA and CD off/disabled. My thinking is that a
adequate test bench would look like

* 3-4 nodes (depending on how many usb ports and dongles I have)
* run iperf with long cables and CSMA/CD
* run iperf with long cables and CMSA/No CD

I'll report back the results. Anything you'd like to add/focus on with
evaluation?

Ramón

Re: [PATCH 3/3] net: microchip_t1s: conditional collision detection

[Parthiban.Veerasooran]

Hi,

This implementation was introduced in the below patch itself.

https://lore.kernel.org/netdev/20230426205049.xlfqluzwcvlm6ihh@soft-dev3-1/T/#m9a52b6c03b7fa637f70aed306b50b442590e24a3

As it is recommended to do it in a separate patch and also the 
datasheets of LAN867X Rev.B1 and LAN865X Rev.B0 internal PHY have these 
register is reserved, we were working for a feasible solution to 
describe this for customer and mainline. By the time many other things 
messed up and couldn't reach the mainline on time.

We also implemented LAN867X Rev.C1 support already in the driver and 
published in our product site and in the process of preparing mainline 
patches. But unfortunately it took little more time to make it.

https://ww1.microchip.com/downloads/aemDocuments/documents/AIS/ProductDocuments/CodeExamples/EVB-LAN8670-USB_Linux_Driver_1v0.zip

Anyway, thank you for the support. Good luck!

Best regards,
Parthiban V

On 27/11/23 4:10 pm, Ramón N.Rodriguez wrote:
> EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
> 
> From: Ramón Nordin Rodriguez <ramon.nordin.rodriguez@ferroamp.se>
> 
> This commit conditionally sets the collision detection bit on lan867x
> and lan865x phys on changing PLCA enabled on/off. The intended realworld
> scenario is that all nodes on the network run the same settings with
> regards to plca, and when plca is enabled the physical layer guarantees
> that no collisions should occur.
> In a practical setting where it was tested running one node with
> collision detection on and other off, the node with collision detection
> on dropped a lot of packets, leading to a poor performing link.
> Worth noting here is that the phys default/reset to plca disabled and
> collision detection enabled. Thus this change would only have an effect
> when changing settings via ethtool.
> 
> Signed-off-by: Ramón Nordin Rodriguez <ramon.nordin.rodriguez@ferroamp.se>
> ---
>   drivers/net/phy/microchip_t1s.c | 29 ++++++++++++++++++++++++++---
>   1 file changed, 26 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/net/phy/microchip_t1s.c b/drivers/net/phy/microchip_t1s.c
> index db84d850b165..3b1e82ecdf69 100644
> --- a/drivers/net/phy/microchip_t1s.c
> +++ b/drivers/net/phy/microchip_t1s.c
> @@ -23,8 +23,10 @@
>   #define LAN865X_REG_CFGPARAM_DATA 0x00D9
>   #define LAN865X_REG_CFGPARAM_CTRL 0x00DA
>   #define LAN865X_REG_STS2 0x0019
> +#define LAN86XX_REG_COLLISION_DETECT 0x0087
> 
>   #define LAN865X_CFGPARAM_READ_ENABLE BIT(1)
> +#define LAN86XX_COLLISION_DETECT_ENABLE BIT(15)
> 
>   /* The arrays below are pulled from the following table from AN1699
>    * Access MMD Address Value Mask
> @@ -363,6 +365,27 @@ static int lan86xx_read_status(struct phy_device *phydev)
>          return 0;
>   }
> 
> +static int lan86xx_plca_set_cfg(struct phy_device *phydev,
> +                               const struct phy_plca_cfg *plca_cfg)
> +{
> +       int err;
> +
> +       err = genphy_c45_plca_set_cfg(phydev, plca_cfg);
> +       if (err)
> +               return err;
> +
> +       /* Disable collision detect on the phy when PLCA is enabled.
> +        * Noise can be picked up as a false positive for collisions
> +        * leading to the phy dropping legitimate packets.
> +        * No collisions should be possible when all nodes are setup
> +        * for running PLCA.
> +        */
> +       return phy_modify_mmd(phydev, MDIO_MMD_VEND2,
> +                       LAN86XX_REG_COLLISION_DETECT,
> +                       LAN86XX_COLLISION_DETECT_ENABLE,
> +                       plca_cfg->enabled ? 0 : LAN86XX_COLLISION_DETECT_ENABLE);
> +}
> +
>   static struct phy_driver microchip_t1s_driver[] = {
>          {
>                  PHY_ID_MATCH_EXACT(PHY_ID_LAN867X_REVB1),
> @@ -371,7 +394,7 @@ static struct phy_driver microchip_t1s_driver[] = {
>                  .config_init        = lan867x_revb1_config_init,
>                  .read_status        = lan86xx_read_status,
>                  .get_plca_cfg       = genphy_c45_plca_get_cfg,
> -               .set_plca_cfg       = genphy_c45_plca_set_cfg,
> +               .set_plca_cfg       = lan86xx_plca_set_cfg,
>                  .get_plca_status    = genphy_c45_plca_get_status,
>          },
>          {
> @@ -381,7 +404,7 @@ static struct phy_driver microchip_t1s_driver[] = {
>                  .config_init        = lan867x_revc1_config_init,
>                  .read_status        = lan86xx_read_status,
>                  .get_plca_cfg       = genphy_c45_plca_get_cfg,
> -               .set_plca_cfg       = genphy_c45_plca_set_cfg,
> +               .set_plca_cfg       = lan86xx_plca_set_cfg,
>                  .get_plca_status    = genphy_c45_plca_get_status,
>          },
>          {
> @@ -391,7 +414,7 @@ static struct phy_driver microchip_t1s_driver[] = {
>                  .config_init        = lan865x_revb0_config_init,
>                  .read_status        = lan86xx_read_status,
>                  .get_plca_cfg       = genphy_c45_plca_get_cfg,
> -               .set_plca_cfg       = genphy_c45_plca_set_cfg,
> +               .set_plca_cfg       = lan86xx_plca_set_cfg,
>                  .get_plca_status    = genphy_c45_plca_get_status,
>          },
>   };
> --
> 2.40.1
> 
> 

Re: [PATCH 0/3] net: microchip_t1s: additional phy support and collision detect handling

[Andrew Lunn]

> * 3-4 nodes (depending on how many usb ports and dongles I have)
> * run iperf with long cables and CSMA/CD
> * run iperf with long cables and CMSA/No CD
> 
> I'll report back the results. Anything you'd like to add/focus on with
> evaluation?

Humm, thinking about how CSMA/CD works...

Maybe look at what counters the MAC provides. Does it have collisions
and bad FCS? A collision should result in a bad FCS, if you are not
using CD. So if things are working correctly, the count for CD should
move to FCS if you turn CD off. If CD is falsely triggering, FCS as a
% should not really change, but you probably get more frames over the
link?

	Andrew

Re: [PATCH 3/3] net: microchip_t1s: conditional collision detection

[Ramón Nordin Rodriguez]

On Mon, Nov 27, 2023 at 04:00:18PM +0000, Parthiban.Veerasooran@microchip.com wrote:
> Hi,
> 
> This implementation was introduced in the below patch itself.
> 
> https://lore.kernel.org/netdev/20230426205049.xlfqluzwcvlm6ihh@soft-dev3-1/T/#m9a52b6c03b7fa637f70aed306b50b442590e24a3
> 

But the change was dropped in that patchset right? It's not present in
netdev-next.

> As it is recommended to do it in a separate patch and also the 
> datasheets of LAN867X Rev.B1 and LAN865X Rev.B0 internal PHY have these 
> register is reserved, we were working for a feasible solution to 
> describe this for customer and mainline. By the time many other things 
> messed up and couldn't reach the mainline on time.
> 

Far as I can tell 'collision detect' is described in the following
sections of respective datasheet:

* 11.5.51 - LAN8650
* 5.4.48  - LAN8670

The rest of the bits are reserved though. The change I propose only
manipulate the documented (bit 15) collision bit.

Is your point that the lan8670 datasheet is only valid for rev.c and not
rev.b?

Andrew suggested on the cover letter that it be interesting to look at
completly disabling collision detect, any strings you can pull at
Microchip to investigate that?
Also any input on my suggested testing methodology is more than welcome.

> We also implemented LAN867X Rev.C1 support already in the driver and 
> published in our product site and in the process of preparing mainline 
> patches. But unfortunately it took little more time to make it.
> 
> https://ww1.microchip.com/downloads/aemDocuments/documents/AIS/ProductDocuments/CodeExamples/EVB-LAN8670-USB_Linux_Driver_1v0.zip

I'm aware, we've been using a derivative of that work at ferroamp for
development. But it's been driving me nuts, being the 't1s guy' at work,
and maintaining out of tree drivers for weird dev boxes.

It's not my intention to beat you to the punch, I just want a mainlined
driver so that I can spend less of my time on plumbing.

> 
> Anyway, thank you for the support. Good luck!

Likewise!
R

Re: [PATCH 3/3] net: microchip_t1s: conditional collision detection

[Parthiban.Veerasooran]

Hi,

On 27/11/23 10:02 pm, Ramón Nordin Rodriguez wrote:
> EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
> 
> On Mon, Nov 27, 2023 at 04:00:18PM +0000, Parthiban.Veerasooran@microchip.com wrote:
>> Hi,
>>
>> This implementation was introduced in the below patch itself.
>>
>> https://lore.kernel.org/netdev/20230426205049.xlfqluzwcvlm6ihh@soft-dev3-1/T/#m9a52b6c03b7fa637f70aed306b50b442590e24a3
>>
> 
> But the change was dropped in that patchset right? It's not present in
> netdev-next.
Yes, it was dropped. The reason why I gave this info is, you mentioned 
in the cover letter that it took some time for you to find this in the 
datasheet.
> 
>> As it is recommended to do it in a separate patch and also the
>> datasheets of LAN867X Rev.B1 and LAN865X Rev.B0 internal PHY have these
>> register is reserved, we were working for a feasible solution to
>> describe this for customer and mainline. By the time many other things
>> messed up and couldn't reach the mainline on time.
>>
> 
> Far as I can tell 'collision detect' is described in the following
> sections of respective datasheet:
> 
> * 11.5.51 - LAN8650
> * 5.4.48  - LAN8670
> 
> The rest of the bits are reserved though. The change I propose only
> manipulate the documented (bit 15) collision bit.
> 
> Is your point that the lan8670 datasheet is only valid for rev.c and not
> rev.b?
It is valid for rev.b1 as well but the current datasheet for rev.c1 
doesn't show that info.
> 
> Andrew suggested on the cover letter that it be interesting to look at
> completly disabling collision detect, any strings you can pull at
> Microchip to investigate that?
Unfortunately I can't commit anything from my side as we are occupied 
with other activities. But definitely I will try my level best if time 
permits. Alternatively you can contact our Microchip customer support 
team if you are interested to do this testing at Microchip.
> Also any input on my suggested testing methodology is more than welcome.
> 
>> We also implemented LAN867X Rev.C1 support already in the driver and
>> published in our product site and in the process of preparing mainline
>> patches. But unfortunately it took little more time to make it.
>>
>> https://ww1.microchip.com/downloads/aemDocuments/documents/AIS/ProductDocuments/CodeExamples/EVB-LAN8670-USB_Linux_Driver_1v0.zip
> 
> I'm aware, we've been using a derivative of that work at ferroamp for
> development. But it's been driving me nuts, being the 't1s guy' at work,
> and maintaining out of tree drivers for weird dev boxes.
> 
> It's not my intention to beat you to the punch, I just want a mainlined
> driver so that I can spend less of my time on plumbing.
I completely understand. Also it was not my intention too. Just to let 
you know why it is delayed in reaching mainline and a quick reference 
for the existing implementation. Enjoy!

Best regards,
Parthiban V
> 
>>
>> Anyway, thank you for the support. Good luck!
> 
> Likewise!
> R

Re: [PATCH 0/3] net: microchip_t1s: additional phy support and collision detect handling

[Ramón Nordin Rodriguez]

On Mon, Nov 27, 2023 at 05:03:54PM +0100, Andrew Lunn wrote:
> > * 3-4 nodes (depending on how many usb ports and dongles I have)
> > * run iperf with long cables and CSMA/CD
> > * run iperf with long cables and CMSA/No CD
> > 
> > I'll report back the results. Anything you'd like to add/focus on with
> > evaluation?
> 
> Humm, thinking about how CSMA/CD works...
> 
> Maybe look at what counters the MAC provides. Does it have collisions
> and bad FCS? A collision should result in a bad FCS, if you are not
> using CD. So if things are working correctly, the count for CD should
> move to FCS if you turn CD off. If CD is falsely triggering, FCS as a
> % should not really change, but you probably get more frames over the
> link?
> 

That is some really cool input, I have to do some datasheet digging and
hacking. I'll try to set everything up today, tomorrow I can hang in
the lab after hours and test things out!

Partihban suggested that Microchips support might be able to help with
testing, might give them a ping soon as I a solid plan.

Really appreciate the insight! 
R

Re: [PATCH 3/3] net: microchip_t1s: conditional collision detection

[Ramón Nordin Rodriguez]

> > 
> > But the change was dropped in that patchset right? It's not present in
> > netdev-next.
> Yes, it was dropped. The reason why I gave this info is, you mentioned 
> in the cover letter that it took some time for you to find this in the 
> datasheet.

Ha, sometimes I have bad luck while thinking. I guess I never understood
that change and subsequently forgot about it.

> > 
> >> As it is recommended to do it in a separate patch and also the
> >> datasheets of LAN867X Rev.B1 and LAN865X Rev.B0 internal PHY have these
> >> register is reserved, we were working for a feasible solution to
> >> describe this for customer and mainline. By the time many other things
> >> messed up and couldn't reach the mainline on time.
> >>
> > 
> > Far as I can tell 'collision detect' is described in the following
> > sections of respective datasheet:
> > 
> > * 11.5.51 - LAN8650
> > * 5.4.48  - LAN8670
> > 
> > The rest of the bits are reserved though. The change I propose only
> > manipulate the documented (bit 15) collision bit.
> > 
> > Is your point that the lan8670 datasheet is only valid for rev.c and not
> > rev.b?
> It is valid for rev.b1 as well but the current datasheet for rev.c1 
> doesn't show that info.

Thank you for clearing that up! So if I get you correctly this change
would in fact be correct for both lan867x rev.b and rev.c.

> > 
> > Andrew suggested on the cover letter that it be interesting to look at
> > completly disabling collision detect, any strings you can pull at
> > Microchip to investigate that?
> Unfortunately I can't commit anything from my side as we are occupied 
> with other activities. But definitely I will try my level best if time 
> permits. Alternatively you can contact our Microchip customer support 
> team if you are interested to do this testing at Microchip.

I get that, might do as you suggest.

> > Also any input on my suggested testing methodology is more than welcome.
> > 
> >> We also implemented LAN867X Rev.C1 support already in the driver and
> >> published in our product site and in the process of preparing mainline
> >> patches. But unfortunately it took little more time to make it.
> >>
> >> https://ww1.microchip.com/downloads/aemDocuments/documents/AIS/ProductDocuments/CodeExamples/EVB-LAN8670-USB_Linux_Driver_1v0.zip
> > 
> > I'm aware, we've been using a derivative of that work at ferroamp for
> > development. But it's been driving me nuts, being the 't1s guy' at work,
> > and maintaining out of tree drivers for weird dev boxes.
> > 
> > It's not my intention to beat you to the punch, I just want a mainlined
> > driver so that I can spend less of my time on plumbing.
> I completely understand. Also it was not my intention too. Just to let 
> you know why it is delayed in reaching mainline and a quick reference 
> for the existing implementation. Enjoy!

I get that, in an ideal world FOSS would be the top prio for all
industry actors. I'm lucky enough to get some time reserved for it.

R

(no subject)

[Félix Piédallu]

Subject: Re: [PATCH 2/3] net: microchip_t1s: add support for LAN867x Rev.C1

Hi, 

> So there is a gap in the revisions. Maybe a B2 exists?

Actually, probably not. Some search gives this datasheet:

https://ww1.microchip.com/downloads/aemDocuments/documents/AIS/ProductDocuments/DataSheets/LAN8670-1-2-Data-Sheet-60001573.pdf

And page 2 (table 1) shows only revisions A0 (rev0), B1, (rev2), C1 (rev4).
Not sure about why only even revision numbers are released ?

Page 193 (table 10-1) also shows only B1 and C1. So you can be confident that only those exist.

@Ramón, thank you for your work on this driver!

Félix Piédallu

Re:

[Ramón Nordin Rodriguez]

> > So there is a gap in the revisions. Maybe a B2 exists?
> 
> Actually, probably not. Some search gives this datasheet:
> 
> https://ww1.microchip.com/downloads/aemDocuments/documents/AIS/ProductDocuments/DataSheets/LAN8670-1-2-Data-Sheet-60001573.pdf
> 
> And page 2 (table 1) shows only revisions A0 (rev0), B1, (rev2), C1 (rev4).
> Not sure about why only even revision numbers are released ?
> 
> Page 193 (table 10-1) also shows only B1 and C1. So you can be confident that only those exist.
> 

Thanks for clearing that up!

> @Ramón, thank you for your work on this driver!

Much appreciated
R

Re: [PATCH 0/3] net: microchip_t1s: additional phy support and collision detect handling

[Ramón Nordin Rodriguez]

On Mon, Nov 27, 2023 at 05:03:54PM +0100, Andrew Lunn wrote:
> > * 3-4 nodes (depending on how many usb ports and dongles I have)
> > * run iperf with long cables and CSMA/CD
> > * run iperf with long cables and CMSA/No CD
> > 
> > I'll report back the results. Anything you'd like to add/focus on with
> > evaluation?
> 
> Humm, thinking about how CSMA/CD works...
> 
> Maybe look at what counters the MAC provides. Does it have collisions
> and bad FCS? A collision should result in a bad FCS, if you are not
> using CD. So if things are working correctly, the count for CD should
> move to FCS if you turn CD off. If CD is falsely triggering, FCS as a
> % should not really change, but you probably get more frames over the
> link?
> 

# setup

Andrew suggested that I try to get statistics from the MAC, I did some
investigation here but could not figure it out.

Using iperf3
Client: Arm based system running lan865x macphy 
Server: PC running lan867x revB usb dongle


# test results

The results below should be considered fairly represenative but far from
perfect. There was some bounce up and down when rerunning, but these resutls
are an eye-ball average.

No meaningful difference was seen with short (2m) cables or long (12m).

## with collision detection enabled

iperf3 normal
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  5.54 MBytes  4.65 Mbits/sec    0             sender
[  5]   0.00-10.01  sec  5.40 MBytes  4.53 Mbits/sec                  receiver

iperf3 reverse
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   929 KBytes   761 Kbits/sec  293             sender
[  5]   0.00-10.00  sec   830 KBytes   680 Kbits/sec                  receiver


## with collision detection disabled

iperf3 normal
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  6.39 MBytes  5.36 Mbits/sec    0             sender
[  5]   0.00-10.04  sec  6.19 MBytes  5.17 Mbits/sec                  receiver

iperf3 reverse
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.27  sec  1.10 MBytes   897 Kbits/sec  268             sender
[  5]   0.00-10.00  sec  1.01 MBytes   843 Kbits/sec                  receiver

# Conclusions

The arm system running the lan865x macphy uses a not yet mainlined driver, see
https://lore.kernel.org/all/20231023154649.45931-1-Parthiban.Veerasooran@microchip.com/

The lan865x driver crashed out every once in a while on reverse mode, there
is definetly something biased in the driver for tx over rx.
Then again it's not accepted yet.

Disabling collision detection seemes to have an positive effect.
Slightly higher speeds and slightly fewer retransmissions.

I don't have a black and white result to present, but things seems to work
slightly better with CD disabled, so I'm leaning towards just unconditionally 
disabling it for the lan865x and lan867x phys for the v2 patch.

I'll wait with submitting v2 for a day so anyone interested gets a
chance to weigh in on this.

R

Re: [PATCH 0/3] net: microchip_t1s: additional phy support and collision detect handling

[Andrew Lunn]

> ## with collision detection enabled
> 
> iperf3 normal
> - - - - - - - - - - - - - - - - - - - - - - - - -
> [ ID] Interval           Transfer     Bitrate         Retr
> [  5]   0.00-10.00  sec  5.54 MBytes  4.65 Mbits/sec    0             sender
> [  5]   0.00-10.01  sec  5.40 MBytes  4.53 Mbits/sec                  receiver
> 
> iperf3 reverse
> - - - - - - - - - - - - - - - - - - - - - - - - -
> [ ID] Interval           Transfer     Bitrate         Retr
> [  5]   0.00-10.00  sec   929 KBytes   761 Kbits/sec  293             sender
> [  5]   0.00-10.00  sec   830 KBytes   680 Kbits/sec                  receiver
> 
> 
> ## with collision detection disabled
> 
> iperf3 normal
> - - - - - - - - - - - - - - - - - - - - - - - - -
> [ ID] Interval           Transfer     Bitrate         Retr
> [  5]   0.00-10.00  sec  6.39 MBytes  5.36 Mbits/sec    0             sender
> [  5]   0.00-10.04  sec  6.19 MBytes  5.17 Mbits/sec                  receiver
> 
> iperf3 reverse
> - - - - - - - - - - - - - - - - - - - - - - - - -
> [ ID] Interval           Transfer     Bitrate         Retr
> [  5]   0.00-10.27  sec  1.10 MBytes   897 Kbits/sec  268             sender
> [  5]   0.00-10.00  sec  1.01 MBytes   843 Kbits/sec                  receiver
> 
> # Conclusions
> 
> The arm system running the lan865x macphy uses a not yet mainlined driver, see
> https://lore.kernel.org/all/20231023154649.45931-1-Parthiban.Veerasooran@microchip.com/
> 
> The lan865x driver crashed out every once in a while on reverse mode, there
> is definetly something biased in the driver for tx over rx.
>
> Then again it's not accepted yet.
> 
> Disabling collision detection seemes to have an positive effect.
> Slightly higher speeds and slightly fewer retransmissions.

I would want to first understand why there is such a big difference
with the direction. Is it TCP backing off because of the packet loss?
Or is there some other problem.

Maybe try with UDP streams, say with a bandwidth of 5Mbps. Do you
loose 4Mbps in one direction? Or a much smaller number of packets.

Are there any usable statistics? FCS errors?

    Andrew

Re:

[Jakub Kicinski]

On Tue, 15 Oct 2024 15:48:03 -0700 Daniel Yang wrote:
> Subject: 
> Date: Tue, 15 Oct 2024 15:48:03 -0700
> X-Mailer: git-send-email 2.39.2
> 
> Date: Tue, 15 Oct 2024 15:31:12 -0700
> Subject: [PATCH v3 0/2 RESEND] resolve gtp possible deadlock warning

This is garbled as well. 

Before you repost please make sure you take a look at:
https://www.kernel.org/doc/html/next/process/maintainer-netdev.html#tl-dr
-- 
pw-bot: cr

Re:

[Alexei Starovoitov]

On Tue, Apr 22, 2025 at 1:04 AM Feng Yang <yangfeng59949@163.com> wrote:
>
> Subject: Re: [PATCH bpf-next] bpf: Remove bpf_get_smp_processor_id_proto
>
> On Mon, 21 Apr 2025 18:53:07 -0700 Alexei Starovoitov <alexei.starovoitov@gmail.com> wrote:
>
> > On Thu, Apr 17, 2025 at 8:41 PM Feng Yang <yangfeng59949@163.com> wrote:
> > >
> > > From: Feng Yang <yangfeng@kylinos.cn>
> > >
> > > All BPF programs either disable CPU preemption or CPU migration,
> > > so the bpf_get_smp_processor_id_proto can be safely removed,
> > > and the bpf_get_raw_smp_processor_id_proto in bpf_base_func_proto works perfectly.
> > >
> > > Suggested-by: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > > Signed-off-by: Feng Yang <yangfeng@kylinos.cn>
> > > ---
> > >  include/linux/bpf.h      |  1 -
> > >  kernel/bpf/core.c        |  1 -
> > >  kernel/bpf/helpers.c     | 12 ------------
> > >  kernel/trace/bpf_trace.c |  2 --
> > >  net/core/filter.c        |  6 ------
> > >  5 files changed, 22 deletions(-)
> > >
> > > diff --git a/include/linux/bpf.h b/include/linux/bpf.h
> > > index 3f0cc89c0622..36e525141556 100644
> > > --- a/include/linux/bpf.h
> > > +++ b/include/linux/bpf.h
> > > @@ -3316,7 +3316,6 @@ extern const struct bpf_func_proto bpf_map_peek_elem_proto;
> > >  extern const struct bpf_func_proto bpf_map_lookup_percpu_elem_proto;
> > >
> > >  extern const struct bpf_func_proto bpf_get_prandom_u32_proto;
> > > -extern const struct bpf_func_proto bpf_get_smp_processor_id_proto;
> > >  extern const struct bpf_func_proto bpf_get_numa_node_id_proto;
> > >  extern const struct bpf_func_proto bpf_tail_call_proto;
> > >  extern const struct bpf_func_proto bpf_ktime_get_ns_proto;
> > > diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
> > > index ba6b6118cf50..1ad41a16b86e 100644
> > > --- a/kernel/bpf/core.c
> > > +++ b/kernel/bpf/core.c
> > > @@ -2943,7 +2943,6 @@ const struct bpf_func_proto bpf_spin_unlock_proto __weak;
> > >  const struct bpf_func_proto bpf_jiffies64_proto __weak;
> > >
> > >  const struct bpf_func_proto bpf_get_prandom_u32_proto __weak;
> > > -const struct bpf_func_proto bpf_get_smp_processor_id_proto __weak;
> > >  const struct bpf_func_proto bpf_get_numa_node_id_proto __weak;
> > >  const struct bpf_func_proto bpf_ktime_get_ns_proto __weak;
> > >  const struct bpf_func_proto bpf_ktime_get_boot_ns_proto __weak;
> > > diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c
> > > index e3a2662f4e33..2d2bfb2911f8 100644
> > > --- a/kernel/bpf/helpers.c
> > > +++ b/kernel/bpf/helpers.c
> > > @@ -149,18 +149,6 @@ const struct bpf_func_proto bpf_get_prandom_u32_proto = {
> > >         .ret_type       = RET_INTEGER,
> > >  };
> > >
> > > -BPF_CALL_0(bpf_get_smp_processor_id)
> > > -{
> > > -       return smp_processor_id();
> > > -}
> > > -
> > > -const struct bpf_func_proto bpf_get_smp_processor_id_proto = {
> > > -       .func           = bpf_get_smp_processor_id,
> > > -       .gpl_only       = false,
> > > -       .ret_type       = RET_INTEGER,
> > > -       .allow_fastcall = true,
> > > -};
> > > -
> >
> > bpf_get_raw_smp_processor_id_proto doesn't have
> > allow_fastcall = true
> >
> > so this breaks tests.
> >
> > Instead of removing BPF_CALL_0(bpf_get_smp_processor_id)
> > we should probably remove BPF_CALL_0(bpf_get_raw_cpu_id)
> > and adjust SKF_AD_OFF + SKF_AD_CPU case.
> > I don't recall why raw_ version was used back in 2014.
> >
>
> The following two seem to explain the reason:
> https://lore.kernel.org/all/7103e2085afa29c006cd5b94a6e4a2ac83efc30d.1467106475.git.daniel@iogearbox.net/
> https://lore.kernel.org/all/02fa71ebe1c560cad489967aa29c653b48932596.1474586162.git.daniel@iogearbox.net/
>

Ahh. socket filters run in RCU CS. They don't disable preemption or migration.
Then let's keep things as-is.
We still want debugging provided by smp_processor_id().
If we switch everything to raw_ may miss things. Like this example with
socket filters.

Re:

[Jiayuan Chen]

April 24, 2025 at 08:40, "Cong Wang" <xiyou.wangcong@gmail.com> wrote:



> 
> netdev@vger.kernel.org, bpf@vger.kernel.org
> 
> Bcc: 
> 
> Subject: test_sockmap failures on the latest bpf-next
> 
> Reply-To: 
> 
> Hi all,
> 
> The latest bpf-next failed on test_sockmap tests, I got the following
> 
> failures (including 1 kernel warning). It is 100% reproducible here.
> 
> I don't have time to look into them, a quick glance at the changelog
> 
> shows quite some changes from Jiayuan. So please take a look, Jiayuan.
> 
> Meanwhile, please let me know if you need more information from me.
> 
> Thanks!
> 
> --------------->

Thanks, I'm working on it.

> 
> [root@localhost bpf]# ./test_sockmap 
> 
> # 1/ 6 sockmap::txmsg test passthrough:OK
> 
> # 2/ 6 sockmap::txmsg test redirect:OK
> 
> # 3/ 2 sockmap::txmsg test redirect wait send mem:OK
> 
> # 4/ 6 sockmap::txmsg test drop:OK
> 
> [ 182.498017] perf: interrupt took too long (3406 > 3238), lowering kernel.perf_event_max_sample_rate to 58500
> 
> # 5/ 6 sockmap::txmsg test ingress redirect:OK
> 
> # 6/ 7 sockmap::txmsg test skb:OK
> 
> # 7/12 sockmap::txmsg test apply:OK
> 
> # 8/12 sockmap::txmsg test cork:OK
> 
> # 9/ 3 sockmap::txmsg test hanging corks:OK
> 
> #10/11 sockmap::txmsg test push_data:OK
> 
> #11/17 sockmap::txmsg test pull-data:OK
> 
> #12/ 9 sockmap::txmsg test pop-data:OK
> 
> #13/ 6 sockmap::txmsg test push/pop data:OK
> 
> #14/ 1 sockmap::txmsg test ingress parser:OK
> 
> #15/ 1 sockmap::txmsg test ingress parser2:OK
> 
> #16/ 6 sockhash::txmsg test passthrough:OK
> 
> #17/ 6 sockhash::txmsg test redirect:OK
> 
> #18/ 2 sockhash::txmsg test redirect wait send mem:OK
> 
> #19/ 6 sockhash::txmsg test drop:OK
> 
> #20/ 6 sockhash::txmsg test ingress redirect:OK
> 
> #21/ 7 sockhash::txmsg test skb:OK
> 
> #22/12 sockhash::txmsg test apply:OK
> 
> #23/12 sockhash::txmsg test cork:OK
> 
> #24/ 3 sockhash::txmsg test hanging corks:OK
> 
> #25/11 sockhash::txmsg test push_data:OK
> 
> #26/17 sockhash::txmsg test pull-data:OK
> 
> #27/ 9 sockhash::txmsg test pop-data:OK
> 
> #28/ 6 sockhash::txmsg test push/pop data:OK
> 
> #29/ 1 sockhash::txmsg test ingress parser:OK
> 
> #30/ 1 sockhash::txmsg test ingress parser2:OK
> 
> #31/ 6 sockhash:ktls:txmsg test passthrough:OK
> 
> #32/ 6 sockhash:ktls:txmsg test redirect:OK
> 
> #33/ 2 sockhash:ktls:txmsg test redirect wait send mem:OK
> 
> [ 263.509707] ------------[ cut here ]------------
> 
> [ 263.510439] WARNING: CPU: 1 PID: 40 at net/ipv4/af_inet.c:156 inet_sock_destruct+0x173/0x1d5
> 
> [ 263.511450] CPU: 1 UID: 0 PID: 40 Comm: kworker/1:1 Tainted: G W 6.15.0-rc3+ #238 PREEMPT(voluntary) 
> 
> [ 263.512683] Tainted: [W]=WARN
> 
> [ 263.513062] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.15.0-1 04/01/2014
> 
> [ 263.514763] Workqueue: events sk_psock_destroy
> 
> [ 263.515332] RIP: 0010:inet_sock_destruct+0x173/0x1d5
> 
> [ 263.515916] Code: e8 dc dc 3f ff 41 83 bc 24 c0 02 00 00 00 74 02 0f 0b 49 8d bc 24 ac 02 00 00 e8 c2 dc 3f ff 41 83 bc 24 ac 02 00 00 00 74 02 <0f> 0b e8 c7 95 3d 00 49 8d bc 24 b0 05 00 00 e8 c0 dd 3f ff 49 8b
> 
> [ 263.518899] RSP: 0018:ffff8880085cfc18 EFLAGS: 00010202
> 
> [ 263.519596] RAX: 1ffff11003dbfc00 RBX: ffff88801edfe3e8 RCX: ffffffff822f5af4
> 
> [ 263.520502] RDX: 0000000000000007 RSI: dffffc0000000000 RDI: ffff88801edfe16c
> 
> [ 263.522128] RBP: ffff88801edfe184 R08: ffffed1003dbfc31 R09: 0000000000000000
> 
> [ 263.523008] R10: ffffffff822f5ab7 R11: ffff88801edfe187 R12: ffff88801edfdec0
> 
> [ 263.523822] R13: ffff888020376ac0 R14: ffff888020376ac0 R15: ffff888020376a60
> 
> [ 263.524682] FS: 0000000000000000(0000) GS:ffff8880b0e88000(0000) knlGS:0000000000000000
> 
> [ 263.525999] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> 
> [ 263.526765] CR2: 0000556365155830 CR3: 000000001d6aa000 CR4: 0000000000350ef0
> 
> [ 263.527700] Call Trace:
> 
> [ 263.528037] <TASK>
> 
> [ 263.528339] __sk_destruct+0x46/0x222
> 
> [ 263.528856] sk_psock_destroy+0x22f/0x242
> 
> [ 263.529471] process_one_work+0x504/0x8a8
> 
> [ 263.530029] ? process_one_work+0x39d/0x8a8
> 
> [ 263.530587] ? __pfx_process_one_work+0x10/0x10
> 
> [ 263.531195] ? worker_thread+0x44/0x2ae
> 
> [ 263.531721] ? __list_add_valid_or_report+0x83/0xea
> 
> [ 263.532395] ? srso_return_thunk+0x5/0x5f
> 
> [ 263.532929] ? __list_add+0x45/0x52
> 
> [ 263.533482] process_scheduled_works+0x73/0x82
> 
> [ 263.534079] worker_thread+0x1ce/0x2ae
> 
> [ 263.534582] ? _raw_spin_unlock_irqrestore+0x2e/0x44
> 
> [ 263.535243] ? __pfx_worker_thread+0x10/0x10
> 
> [ 263.535822] kthread+0x32a/0x33c
> 
> [ 263.536278] ? kthread+0x13c/0x33c
> 
> [ 263.536724] ? __pfx_kthread+0x10/0x10
> 
> [ 263.537225] ? srso_return_thunk+0x5/0x5f
> 
> [ 263.537869] ? find_held_lock+0x2b/0x75
> 
> [ 263.538388] ? __pfx_kthread+0x10/0x10
> 
> [ 263.538866] ? srso_return_thunk+0x5/0x5f
> 
> [ 263.539523] ? local_clock_noinstr+0x32/0x9c
> 
> [ 263.540128] ? srso_return_thunk+0x5/0x5f
> 
> [ 263.540677] ? srso_return_thunk+0x5/0x5f
> 
> [ 263.541228] ? __lock_release+0xd3/0x1ad
> 
> [ 263.541890] ? srso_return_thunk+0x5/0x5f
> 
> [ 263.542442] ? tracer_hardirqs_on+0x17/0x149
> 
> [ 263.543047] ? _raw_spin_unlock_irq+0x24/0x39
> 
> [ 263.543589] ? __pfx_kthread+0x10/0x10
> 
> [ 263.544069] ? __pfx_kthread+0x10/0x10
> 
> [ 263.544543] ret_from_fork+0x21/0x41
> 
> [ 263.545000] ? __pfx_kthread+0x10/0x10
> 
> [ 263.545557] ret_from_fork_asm+0x1a/0x30
> 
> [ 263.546095] </TASK>
> 
> [ 263.546374] irq event stamp: 1094079
> 
> [ 263.546798] hardirqs last enabled at (1094089): [<ffffffff813be0f6>] __up_console_sem+0x47/0x4e
> 
> [ 263.547762] hardirqs last disabled at (1094098): [<ffffffff813be0d6>] __up_console_sem+0x27/0x4e
> 
> [ 263.548817] softirqs last enabled at (1093692): [<ffffffff812f2906>] handle_softirqs+0x48c/0x4de
> 
> [ 263.550127] softirqs last disabled at (1094117): [<ffffffff812f29b3>] __irq_exit_rcu+0x4b/0xc3
> 
> [ 263.551104] ---[ end trace 0000000000000000 ]---
> 
> #34/ 6 sockhash:ktls:txmsg test drop:OK
> 
> #35/ 6 sockhash:ktls:txmsg test ingress redirect:OK
> 
> #36/ 7 sockhash:ktls:txmsg test skb:OK
> 
> #37/12 sockhash:ktls:txmsg test apply:OK
> 
> [ 278.915147] perf: interrupt took too long (4331 > 4257), lowering kernel.perf_event_max_sample_rate to 46000
> 
> [ 282.974989] test_sockmap (1077) used greatest stack depth: 25072 bytes left
> 
> #38/12 sockhash:ktls:txmsg test cork:OK
> 
> #39/ 3 sockhash:ktls:txmsg test hanging corks:OK
> 
> #40/11 sockhash:ktls:txmsg test push_data:OK
> 
> #41/17 sockhash:ktls:txmsg test pull-data:OK
> 
> recv failed(): Invalid argument
> 
> rx thread exited with err 1.
> 
> recv failed(): Invalid argument
> 
> rx thread exited with err 1.
> 
> recv failed(): Bad message
> 
> rx thread exited with err 1.
> 
> #42/ 9 sockhash:ktls:txmsg test pop-data:FAIL
> 
> recv failed(): Bad message
> 
> rx thread exited with err 1.
> 
> recv failed(): Message too long
> 
> rx thread exited with err 1.
> 
> #43/ 6 sockhash:ktls:txmsg test push/pop data:FAIL
> 
> #44/ 1 sockhash:ktls:txmsg test ingress parser:OK
> 
> #45/ 0 sockhash:ktls:txmsg test ingress parser2:OK
> 
> Pass: 43 Fail: 5
>

Re:

[Jiayuan Chen]

April 24, 2025 at 08:59, "Jiayuan Chen" <jiayuan.chen@linux.dev> wrote:

> 
> April 24, 2025 at 08:40, "Cong Wang" <xiyou.wangcong@gmail.com> wrote:
> 
> > 
> > netdev@vger.kernel.org, bpf@vger.kernel.org
> > 
> >  Bcc: 
> > 
> > 
> >  Subject: test_sockmap failures on the latest bpf-next
> > 
> >  Reply-To: 
> > 
> >  
> > 
> >  Hi all,
> > 
> >  
> > 
> >  The latest bpf-next failed on test_sockmap tests, I got the following
> > 
> >  failures (including 1 kernel warning). It is 100% reproducible here.
> > 
> >  I don't have time to look into them, a quick glance at the changelog
> >  
> >  shows quite some changes from Jiayuan. So please take a look, Jiayuan.
> > 
> >  Meanwhile, please let me know if you need more information from me.
> > 
> >  Thanks!
> > 
> >  
> > 
> >  --------------->
> > 
> 
> Thanks, I'm working on it.
> 

After resetting my commit to 0bb2f7a1ad1f, which is before my changes, the warning still exists.

The warning originates from test_txmsg_redir_wait_sndmem(), which performs
'KTLS + sockmap with redir EGRESS and limited receive buffer'.

The memory charge/uncharge logic is problematic, I need some time to investigate and fix it.

Thanks.

Re:

[David Lechner]

On 7/20/25 1:26 PM, > wrote:
> Changes in v2:
> - Fixed commit message grammar
> - Fixed subject line style as per DT convention
> - Added missing reviewers/maintainers in CC
> 

By placing this before the headers, our email clients think this
message doesn't have a subject. It should go after the ---.

> From 5c00524cbb47e30ee04223fe9502af2eb003ddf1 Mon Sep 17 00:00:00 2001
> From: sanjay suthar <sanjaysuthar661996@gmail.com>
> Date: Sun, 20 Jul 2025 01:11:00 +0530
> Subject: [PATCH v2] dt-bindings: cleanup: fix duplicated 'is is' in YAML docs
> 
> Fix minor grammatical issues by removing duplicated "is" in two devicetree
> binding documents:
> 
> - net/amlogic,meson-dwmac.yaml
> - iio/dac/ti,dac7612.yaml
> 
> Signed-off-by: sanjay suthar <sanjaysuthar661996@gmail.com>
> ---

This is where the changelog belongs.

>  Documentation/devicetree/bindings/iio/dac/ti,dac7612.yaml      | 2 +-
>  Documentation/devicetree/bindings/net/amlogic,meson-dwmac.yaml | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/Documentation/devicetree/bindings/iio/dac/ti,dac7612.yaml b/Documentation/devicetree/bindings/iio/dac/ti,dac7612.yaml
> index 20dd1370660d..624c640be4c8 100644
> --- a/Documentation/devicetree/bindings/iio/dac/ti,dac7612.yaml
> +++ b/Documentation/devicetree/bindings/iio/dac/ti,dac7612.yaml
> @@ -9,7 +9,7 @@ title: Texas Instruments DAC7612 family of DACs
>  description:
>    The DAC7612 is a dual, 12-bit digital-to-analog converter (DAC) with
>    guaranteed 12-bit monotonicity performance over the industrial temperature
> -  range. Is is programmable through an SPI interface.
> +  range. It is programmable through an SPI interface.
>  
>  maintainers:
>    - Ricardo Ribalda Delgado <ricardo@ribalda.com>
> diff --git a/Documentation/devicetree/bindings/net/amlogic,meson-dwmac.yaml b/Documentation/devicetree/bindings/net/amlogic,meson-dwmac.yaml
> index 0cd78d71768c..5c91716d1f21 100644
> --- a/Documentation/devicetree/bindings/net/amlogic,meson-dwmac.yaml
> +++ b/Documentation/devicetree/bindings/net/amlogic,meson-dwmac.yaml
> @@ -149,7 +149,7 @@ properties:
>        - description:
>            The first register range should be the one of the DWMAC controller
>        - description:
> -          The second range is is for the Amlogic specific configuration
> +          The second range is for the Amlogic specific configuration
>            (for example the PRG_ETHERNET register range on Meson8b and newer)
>  
>    interrupts:

I would be tempted to split this into two patches. It's a bit odd to have
a single patch touching two unrelated bindings.

Re:

[Krzysztof Kozlowski]

On 20/07/2025 21:30, David Lechner wrote:
>>    - Ricardo Ribalda Delgado <ricardo@ribalda.com>
>> diff --git a/Documentation/devicetree/bindings/net/amlogic,meson-dwmac.yaml b/Documentation/devicetree/bindings/net/amlogic,meson-dwmac.yaml
>> index 0cd78d71768c..5c91716d1f21 100644
>> --- a/Documentation/devicetree/bindings/net/amlogic,meson-dwmac.yaml
>> +++ b/Documentation/devicetree/bindings/net/amlogic,meson-dwmac.yaml
>> @@ -149,7 +149,7 @@ properties:
>>        - description:
>>            The first register range should be the one of the DWMAC controller
>>        - description:
>> -          The second range is is for the Amlogic specific configuration
>> +          The second range is for the Amlogic specific configuration
>>            (for example the PRG_ETHERNET register range on Meson8b and newer)
>>  
>>    interrupts:
> 
> I would be tempted to split this into two patches. It's a bit odd to have


No, it's a churn to split this into more than one patch.

> a single patch touching two unrelated bindings.




Best regards,
Krzysztof

Re:

[Andy Shevchenko]

On Sun, Jul 20, 2025 at 11:56:27PM +0530, > wrote:
> Changes in v2:
> - Fixed commit message grammar
> - Fixed subject line style as per DT convention
> - Added missing reviewers/maintainers in CC
> 
> From 5c00524cbb47e30ee04223fe9502af2eb003ddf1 Mon Sep 17 00:00:00 2001
> From: sanjay suthar <sanjaysuthar661996@gmail.com>
> Date: Sun, 20 Jul 2025 01:11:00 +0530
> Subject: [PATCH v2] dt-bindings: cleanup: fix duplicated 'is is' in YAML docs
> 
> Fix minor grammatical issues by removing duplicated "is" in two devicetree
> binding documents:
> 
> - net/amlogic,meson-dwmac.yaml
> - iio/dac/ti,dac7612.yaml

This mail is b0rken.

-- 
With Best Regards,
Andy Shevchenko

Re:

[David Lechner]

On 7/21/25 5:15 AM, Sanjay Suthar wrote:
> On Mon, Jul 21, 2025 at 12:22 PM Krzysztof Kozlowski <krzk@kernel.org <mailto:krzk@kernel.org>> wrote:
>>
>> On 20/07/2025 21:30, David Lechner wrote:
>> >>    - Ricardo Ribalda Delgado <ricardo@ribalda.com <mailto:ricardo@ribalda.com>>
>> >> diff --git a/Documentation/devicetree/bindings/net/amlogic,meson-dwmac.yaml b/Documentation/devicetree/bindings/net/amlogic,meson-dwmac.yaml
>> >> index 0cd78d71768c..5c91716d1f21 100644
>> >> --- a/Documentation/devicetree/bindings/net/amlogic,meson-dwmac.yaml
>> >> +++ b/Documentation/devicetree/bindings/net/amlogic,meson-dwmac.yaml
>> >> @@ -149,7 +149,7 @@ properties:
>> >>        - description:
>> >>            The first register range should be the one of the DWMAC controller
>> >>        - description:
>> >> -          The second range is is for the Amlogic specific configuration
>> >> +          The second range is for the Amlogic specific configuration
>> >>            (for example the PRG_ETHERNET register range on Meson8b and newer)
>> >>
>> >>    interrupts:
>> >
>> > I would be tempted to split this into two patches. It's a bit odd to have
>>
>>
>> No, it's a churn to split this into more than one patch.
>>
> 
> Thanks for the reply. Since there are suggestions on patch split as it is touching different subsystems, still not clear if I should split the patch or single patch is fine. I would appreciate if you can guide on the next steps to be taken
> 
> Best Regards,
> Sanjay Suthar

Krzysztof is one of the devicetree maintainers and I am not, so you
should do what Krzysztof says - leave it as one patch. :-)