From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ot1-f49.google.com (mail-ot1-f49.google.com [209.85.210.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6304432C85 for ; Wed, 1 May 2024 18:16:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.49 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714587404; cv=none; b=h8hIXgdi+F2KI3yII7fgUC64MrP5MJ26BPMiqLiCNHGEf8yIcGgXL1PGt+dIVCTRolq2aGt/EXZc3ez4S6Jaj9lrCFfWMsW9VyYYzySEoQr1DifxdFVhaesqHWweF/HiKSgpW3CGcF97HTSLFAOKvZw3ko6bfXJbHKa5wXRAU4M= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714587404; c=relaxed/simple; bh=um3QI+jS2eRa0Wi7aE0atVKG7SKSi6QmpIMsI9TiMbA=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=J6+vaIRCxkCL2C8ZMEyWxXT1DjYnINkkBFFgfwK9AoHy9NGrrdS5h7b9qJPH9zqqaXOkX3imtlVRhjTr8ZJWyiQ2vpqcxBFEhSB77LugWhu21hKmI5nd5l6ziEUy8BJsCkdIN4klDAvtntibFhvEHwfD7r0otD2WtmJcAvgSQyk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=gYKo6iSn; arc=none smtp.client-ip=209.85.210.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="gYKo6iSn" Received: by mail-ot1-f49.google.com with SMTP id 46e09a7af769-6ef6ef5c5faso1008447a34.3 for ; Wed, 01 May 2024 11:16:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714587402; x=1715192202; darn=vger.kernel.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=h07FLRwiamUXF7mmKRoLxadvWj3yvk6Y3EfSpj1/B9U=; b=gYKo6iSnd6sisJNgMWQLkhgX+8HU3LeJzzyQxGyZGmLavPCLIJNZzRCWBYjrjUD45T nx6pRElvf3Mg0FXOpT+7LbJjTsOXF3MJEntOn71TGk+iJbUSFKBdjldt8wYuhKW7BFxp Wjv8j8MLVuaWIvio6OPwawtXbTZRUvpU+WyAgYpY3oTV6ct4dCStfaoroEr/qhG5wh0e KvC+1hajBYpfetusJmUBsHPigZJCPiuLHBABzT9H6qGIjRlql11n7xBQPst9Bf6qH6zC gxirJjNHG7W+fCiMgEvigP4YIwwzr8uMllYOJig5bo36XJKvTbd9uFiI89/7coAEPzUN bf7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714587402; x=1715192202; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=h07FLRwiamUXF7mmKRoLxadvWj3yvk6Y3EfSpj1/B9U=; b=tgCHMbX+dBKK3OGLQ894wo/YLVCo4oNJO58LaQF3Y4sTM5FQVF2nGIWXt3TeAUyynA Xcvg3zxZVN+hFsxrA08bh541jTwtwdGSV+ycKrXsSe9CXYv1RR0W+tmIT/VlcP1UiGZ+ sZ8iJ35JBD9F61xMU5job5KelWqQ4DA/8zMs3XITtfr5rF3O2pKoObZsETvvGLIqsTCv e4xrx1vyeC50J3eNBoOq3vw/YZY/x0K7B6Ko9bGqfmgUmSBAs72tSdJJTA5k5tLnGm74 +djvPD4TmZXit6Jmjfsr6+CTL9MrSPF9LQCSMm37LFz4V8HPPGqBK+fkBcXuYHinkXvm mdmg== X-Forwarded-Encrypted: i=1; AJvYcCWzISOOyxLbHRLY7+t8i3II3fx0MHhWYpLOex3Z119hl9eX7Suecsfq3dQQ0Oa55NLIkZjvYXXUESrAH/W7lZIkkSVqpIcr X-Gm-Message-State: AOJu0YxVeKWACnqHbPEZm6SkvsEV/hNdoWZI5YxKNaM3MSMUzVAZQzXv rNpBFROXf0pdf0tORPkCPMryZ/+RT+nfsOVRo8xFLqsTkclF/fop X-Google-Smtp-Source: AGHT+IE1+sOI98iQqkK1I1qG0QYLtfPeq+B9BbPlq8q3xsrAoJJNXVGYw2Hj4WWIyl3IEKNHZGwrUw== X-Received: by 2002:a05:6830:25c3:b0:6ee:6675:fb47 with SMTP id d3-20020a05683025c300b006ee6675fb47mr4112010otu.0.1714587402357; Wed, 01 May 2024 11:16:42 -0700 (PDT) Received: from localhost (24-122-67-147.resi.cgocable.ca. [24.122.67.147]) by smtp.gmail.com with ESMTPSA id u5-20020a05620a022500b0078f13d368f3sm12058890qkm.95.2024.05.01.11.16.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 May 2024 11:16:41 -0700 (PDT) Date: Wed, 1 May 2024 14:16:41 -0400 From: Benjamin Poirier To: Shane Miller Cc: Jiri Pirko , netdev@vger.kernel.org Subject: Re: SR-IOV + switchdev + vlan + Mellanox: Cannot ping Message-ID: References: Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: On 2024-04-30 17:29 -0400, Shane Miller wrote: > On Mon, Apr 29, 2024 at 7:29 AM Jiri Pirko wrote: > > Nope. Think of it as another switch inside the NIC that connects VFs and > > uplink port. You have representors that represent the switch port. Each > > representor has counter part VF. You have to configure the forwarding > > between the representor, similar to switch ports. In switch, there is > > also no default forwarding. > > The salient phrase is "forward between the representor". You seem to > be saying to forward ARP packets from the uplink port (ieth3 e.g. > the NIC that was virtualized) to a port representer (ieth3r0)? Are those > the correct endpoints? > > Second, what UNIX tool do I use to forward? As far as I can tell, the > correct methodology is to first create a bridge: > > ip link add name br0 type bridge > ip link set br0 up > I recently learned about this too and here is what I noted down: In switchdev mode, two netdevs are created for each VF: 1) port representor (PR) `ethtool -i` shows "driver: mlx5e_rep" sysfs device/ is the PF `devlink port` shows "flavour pcivf" 2) actual VF driver: mlx5_core sysfs device/ is unique `devlink port` shows "flavour virtual" In order to be able to pass traffic, the PR must be added into a bridge with the PF: ip link add br0 up type bridge ip link set dev eth2 up master br0 # PF ip link set dev eth4 up master br0 # PR