From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7769CC43381 for ; Thu, 7 Mar 2019 19:03:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 44C5120851 for ; Thu, 7 Mar 2019 19:03:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="KlFRa3H8" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726616AbfCGTC7 (ORCPT ); Thu, 7 Mar 2019 14:02:59 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:44010 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726127AbfCGTC7 (ORCPT ); Thu, 7 Mar 2019 14:02:59 -0500 Received: by mail-wr1-f65.google.com with SMTP id d17so18718508wre.10 for ; Thu, 07 Mar 2019 11:02:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=F/6bDOOHCu9/tFslvw9H1kBE7T1fMg7kZ7N9ZCeikws=; b=KlFRa3H88z8AMZa6G29iO2L0VkKerqo2ETBUuOK4d9WS80YZFuUGfS3T/GY/ZEmsxX LhPHiU5Z9mnWRlRWX8EHVR/PH1lxz9m9j4rrB5KUDc7/XTTJyT0vOwc55THF+vd7hG9h kL31VzmiF3XQ+kKLblLCwTaPhGJ5u8PP955LLqw1Lj1WH34wjcvlpRHnMRMAAV9Mv4gR SOj1C11GBOJuhdQCLj1njT+n+LejIUU3UTRqxjCrVw8jPZvR025GGbIRhuPeFvPS83DR JAzKLunbnjFoQ+YkkeYEv1QF+2dles/CiwSQ2GgWFSzZnbmVvvCks1jYm+Du53N7AUa4 sd9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=F/6bDOOHCu9/tFslvw9H1kBE7T1fMg7kZ7N9ZCeikws=; b=kl7OdB0ehaWYnLLXE06/95bz61G6oRyScUWBORg6DVDzieVTvjOWa9j7zh13U+fJjP w+6PMFm8wQ5x1X6eNR748FSa5znplCw5UEdQvuA66wcFUTTXa2HN0bjDV+VyzOf+OCf0 HrlyusxpObJ4O4QzWbocC72ROMVs6P/+Z5X13RIl6o9EPAPrqNn+Ss4Jn0V6u4jACLiY SVWJiQP2pPmpIDYPGCcpine7ymCZj8cRzWikvdrDHEm2+9vJnuH4/SCZYs/MufzzXqM7 2AgTU1O7TWhRo6ZEaM0rLZCKLYRN4mLv53lPHiLdlAsSt2obubG7TEMnWbnHgsw9ZaFM 8GSw== X-Gm-Message-State: APjAAAWktjv+ij/QnZV0SWDW98q+yhtAPJz27RRy1cF+qRUzMEER42oF +eNznGfceo64rJg1LiuOH6QkiV2Bp5G1XBpwpSuG9DZw X-Google-Smtp-Source: APXvYqzXYlO2K+/L00/PIJU3Wehm9TOdlUM7QXEUuIV2Bt6TrLKLFJHQkZs37l3aK5Ld+WfewJPz9Z3GcmErM6SRnGA= X-Received: by 2002:adf:fd46:: with SMTP id h6mr7809667wrs.170.1551985376884; Thu, 07 Mar 2019 11:02:56 -0800 (PST) MIME-Version: 1.0 References: <20190228185758.6f37i2sfnr2mdrce@localhost> In-Reply-To: From: Paul Thomas Date: Thu, 7 Mar 2019 14:02:44 -0500 Message-ID: Subject: Re: [Linuxptp-devel] strangeness To: Harini Katakam Cc: Richard Cochran , "linuxptp-devel@lists.sourceforge.net" , netdev@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On Thu, Mar 7, 2019 at 12:32 AM Harini Katakam wrote: > > Hi Paul, > On Thu, Mar 7, 2019 at 4:38 AM Paul Thomas wrote: > > > > On Fri, Mar 1, 2019 at 1:24 AM Harini Katakam wrote: > > > > > > +netdev > > > > > > Hi Paul, > > > On Fri, Mar 1, 2019 at 12:29 AM Richard Cochran > > > wrote: > > > > > > > > On Thu, Feb 28, 2019 at 12:33:26PM -0500, Paul Thomas wrote: > > > > > Yes changing it to TSTAMP_ALL_PTP_FRAMES instead of TSTAMP_ALL_FRAMES > > > > > does seem to fix the ssh issue. My worry is that there is still a bug > > > > > somewhere in the network stack that this is just masking. > > > > > > Ok thanks. > > > One place to check in the driver will be: > > > if (gem_ptp_do_txstamp(queue, skb, desc) == 0) { > > > /* skb now belongs to timestamp buffer > > > * and will be removed later > > > */ > > > tx_skb->skb = NULL; > > > } > > > When all TX packets are timestamped, the skb always belongs to the > > > timestamp buffer. > > > > > > > > > > > Or the HW isn't sending the frames in the first place. > > > > > > > > Check that first! > > > > > > To check this, the statistics registers in MAC will be one way. > > > But if there is no TX completion interrupt, then I wouldn't expect > > > these statistics to increase either. The used bit status in BD dump > > > might be of more use. > > > > > > I will also try to reproduce (with TX timestamp ALL) and see if any of > > > the above gives some clue. > > > > > > Regards, > > > Harini > > > > Hi Harini, any luck looking at this? > > I'm sorry, I was not able to debug this further. > > > > > I didn't get very far, even in the "broken" state I see plenty of tx_frames: > > root@xu5:/opt/linuxptp# ethtool -S eth0 > > NIC statistics: > > ... > > tx_frames: 39763 > > ... > > > > When you said "registers in the MAC" is ethtool -S displaying that? > > Yes, ethtool does display these statistics. > I was referring to the registers starting offset 0xFF0B0108 (for GEM0) here: > https://www.xilinx.com/html_docs/registers/ug1087/ug1087-zynq-ultrascale-registers.html > If you see this value increasing, then the MAC is transmitting successfully. > Although, I realize it could be other traffic. To see if specific > packets (for the > failed SSH connection) are not being queued, a BD dump might help. > > Regards, > Harini OK, I think things are becoming more clear. After just doing ioctl(fd, SIOCSHWTSTAMP, &ifreq) from userspace (tx_bd_control = TSTAMP_ALL_FRAMES in macb_ptp.c) then with the nc experiment some udp transmits do not make it to macb_start_xmit() until receive traffic on the nc connection comes in (one-to-one, one new rx packet means one old tx packet goes out). Working setup: Before the tx_bd_control = TSTAMP_ALL_FRAMES. Every time I hit "sN Enter" from nc I see a macb_start_xmit print_hex_dump() and I see the packet on the nc client side: # nc -l -u -p 9999 ... s11 [ 347.517080] macb_start_xmit data: 00000000: 20 b0 f7 04 0a 29 20 b0 f7 04 0a 26 08 00 45 00 ....) ....&..E. s12 [ 348.964369] macb_start_xmit data: 00000000: 20 b0 f7 04 0a 29 20 b0 f7 04 0a 26 08 00 45 00 ....) ....&..E. ... Broken setup: After the tx_bd_control = TSTAMP_ALL_FRAMES. Not the first nc packet, but many of the subsequent ones never make it to macb_start_xmit() # nc -l -u -p 9999 ... s3 s4 s5 ... Eventually after I send data from the client nc I do see the macb_start_xmit() lines. Hopefully this helps point us in the right direction, I would very much like to be able to do timestamping with my zynqmp board. thanks, Paul