* Re: [bpf-next PATCH v2 05/18] bpf: create tcp_bpf_ulp allowing BPF to monitor socket TX/RX data
From: John Fastabend @ 2018-03-16 16:47 UTC (permalink / raw)
To: Daniel Borkmann, Alexei Starovoitov; +Cc: davem, ast, davejwatson, netdev
In-Reply-To: <7791bf7d-5ca8-7e28-b740-bd6c40be402c@iogearbox.net>
On 03/15/2018 05:37 PM, Daniel Borkmann wrote:
> On 03/16/2018 12:06 AM, Alexei Starovoitov wrote:
>> On Thu, Mar 15, 2018 at 11:55:39PM +0100, Daniel Borkmann wrote:
>>> On 03/15/2018 11:20 PM, Alexei Starovoitov wrote:
>>>> On Thu, Mar 15, 2018 at 11:17:12PM +0100, Daniel Borkmann wrote:
>>>>> On 03/15/2018 10:59 PM, Alexei Starovoitov wrote:
>>>>>> On Mon, Mar 12, 2018 at 12:23:29PM -0700, John Fastabend wrote:
>>>>>>>
>>>>>>> +/* User return codes for SK_MSG prog type. */
>>>>>>> +enum sk_msg_action {
>>>>>>> + SK_MSG_DROP = 0,
>>>>>>> + SK_MSG_PASS,
>>>>>>> +};
>>>>>>
>>>>>> do we really need new enum here?
>>>>>> It's the same as 'enum sk_action' and SK_DROP == SK_MSG_DROP
>>>>>> and there will be only drop/pass in both enums.
>>>>>> Also I don't see where these two new SK_MSG_* are used...
>>>>>>
>>>>>>> +
>>>>>>> +/* user accessible metadata for SK_MSG packet hook, new fields must
>>>>>>> + * be added to the end of this structure
>>>>>>> + */
>>>>>>> +struct sk_msg_md {
>>>>>>> + __u32 data;
>>>>>>> + __u32 data_end;
>>>>>>> +};
>>>>>>
>>>>>> I think it's time for me to ask for forgiveness :)
>>>>>
>>>>> :-)
>>>>>
>>>>>> I used __u32 for data and data_end only because all other fields
>>>>>> in __sk_buff were __u32 at the time and I couldn't easily figure out
>>>>>> how to teach verifier to recognize 8-byte rewrites.
>>>>>> Unfortunately my mistake stuck and was copied over into xdp.
>>>>>> Since this is new struct let's do it right and add
>>>>>> 'void *data, *data_end' here,
>>>>>> since bpf prog will use them as 'void *' pointers.
>>>>>> There are no compat issues here, since bpf is always 64-bit.
>>>>>
>>>>> But at least offset-wise when you do the ctx rewrite this would then
>>>>> be a bit more tricky when you have 64 bit kernel with 32 bit user
>>>>> space since void * members are in each cases at different offset. So
>>>>> unless I'm missing something, this still should either be __u32 or
>>>>> __u64 instead of void *, no?
>>>>
>>>> there is no 32-bit user space. these structs are seen by bpf progs only
>>>> and bpf is 64-bit only too.
>>>> unless I'm missing your point.
>>>
>>> Ok, so lets say you have 32 bit LLVM binary and compile the prog where
>>> you access md->data_end. Given the void * in the struct will that access
>>> end up being BPF_W at ctx offset 4 or BPF_DW at ctx offset 8 from clang
>>> perspective (iow, is the back end treating this special and always use
>>> fixed BPF_DW in such case)? If not and it would be the first case with
>>> offset 4, then we could have the case that underlying 64 bit kernel is
>>> expecting ctx offset 8 for doing the md ctx conversion.
>>
>> i'm still not quite following.
>> Whether llvm itself is 32-bit binary or it's arm32 or sprac32 binary
>> doesn't matter. It will produce the same 64-bit bpf code.
>> It will see 'void *' deref from this struct and will emit DW.
>> May be confusion is from newly added -mattr=+alu32 flag?
>> That option doesn't change that sizeof(void*)==8.
>> It only allows backend to emit 32-bit alu insns.
>
> Ok, so conclusion we had is that while BPF target is unconditionally 64 bit,
> it depends which clang front end you use for compilation wrt structs. E.g.
> on 32 bit native (e.g. arm) clang front end it would compile the ctx void *
> pointers as 4 byte while using clang -target bpf it would compile it as 8
> byte. The native clang front end is needed in case of tracing when accessing
> pt_regs for walking data structures, but not for networking use case, so
> always using -target bpf there is proper way. Meaning there would be no
> confusion on the void * since size will always be 8 regardless of underlying
> arch being 32 or 64 bit or clang/llvm binary being 32 bit on 64 bit kernel.
> Thus, sticking to void * would be fine, but definitely samples/sockmap/Makefile
> must be fixed as well, such that people don't copy it wrongly.
>
> Cheers,
> Danie
I'll send a fix for sockmap/Makefile then as a separate series. And
go ahead and change this series to use 'void *'.
Thanks for the follow-up on this.
^ permalink raw reply
* Re: sctp: use proc_remove_subtree()
From: David Miller @ 2018-03-16 16:44 UTC (permalink / raw)
To: viro; +Cc: netdev, linux-sctp
In-Reply-To: <20180314205119.GZ30522@ZenIV.linux.org.uk>
From: Al Viro <viro@ZenIV.linux.org.uk>
Date: Wed, 14 Mar 2018 20:51:19 +0000
> On Wed, Mar 14, 2018 at 08:46:21PM +0000, Al Viro wrote:
>> use proc_remove_subtree() for subtree removal, both on setup failure
>> halfway through and on teardown. No need to make simple things
>> complex...
>
> ... and sctp_dbg_objcnt_exit() can be removed as well - remove_proc_subtree()
> will get that sucker automatically. Either a trivial addition, or equally
> trivial followup...
Please add it to this patch and resubmit.
Thanks Al.
^ permalink raw reply
* Re: [PATCH 00/30] Netfilter/IPVS updates for net-next
From: David Miller @ 2018-03-16 16:41 UTC (permalink / raw)
To: sguy; +Cc: pablo, netfilter-devel, netdev
In-Reply-To: <d8eec20f-5580-d7c7-860b-14953b55809d@mellanox.com>
From: Guy Shattah <sguy@mellanox.com>
Date: Fri, 16 Mar 2018 18:39:03 +0200
> Would one driver support as demonstration suffice?
It would certinaly improve the reviewability of the changes.
^ permalink raw reply
* Re: [virtio-dev] [pci PATCH v7 2/5] virtio_pci: Add support for unmanaged SR-IOV on virtio_pci devices
From: Alexander Duyck @ 2018-03-16 16:40 UTC (permalink / raw)
To: Michael S. Tsirkin, Daly, Dan
Cc: Bjorn Helgaas, Duyck, Alexander H, linux-pci, virtio-dev, kvm,
Netdev, LKML, linux-nvme, Keith Busch, netanel, Don Dutile,
Maximilian Heyne, Wang, Liang-min, Rustad, Mark D,
David Woodhouse, Christoph Hellwig, dwmw
In-Reply-To: <20180316183042-mutt-send-email-mst@kernel.org>
On Fri, Mar 16, 2018 at 9:34 AM, Michael S. Tsirkin <mst@redhat.com> wrote:
> On Thu, Mar 15, 2018 at 11:42:41AM -0700, Alexander Duyck wrote:
>> From: Alexander Duyck <alexander.h.duyck@intel.com>
>>
>> Hardware-realized virtio_pci devices can implement SR-IOV, so this
>> patch enables its use. The device in question is an upcoming Intel
>> NIC that implements both a virtio_net PF and virtio_net VFs. These
>> are hardware realizations of what has been up to now been a software
>> interface.
>>
>> The device in question has the following 4-part PCI IDs:
>>
>> PF: vendor: 1af4 device: 1041 subvendor: 8086 subdevice: 15fe
>> VF: vendor: 1af4 device: 1041 subvendor: 8086 subdevice: 05fe
>>
>> The patch currently needs no check for device ID, because the callback
>> will never be made for devices that do not assert the capability or
>> when run on a platform incapable of SR-IOV.
>>
>> One reason for this patch is because the hardware requires the
>> vendor ID of a VF to be the same as the vendor ID of the PF that
>> created it. So it seemed logical to simply have a fully-functioning
>> virtio_net PF create the VFs. This patch makes that possible.
>>
>> Reviewed-by: Christoph Hellwig <hch@lst.de>
>> Signed-off-by: Mark Rustad <mark.d.rustad@intel.com>
>> Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
>
> So if and when virtio PFs can manage the VFs, then we can
> add a feature bit for that?
> Seems reasonable.
Yes. If nothing else you may not even need a feature bit depending on
how things go. One of the reasons why Mark called out the
subvendor/subdevice was because that might be able to be used to
identify the specific hardware that is providing the SR-IOV feature so
in the future if it is added to virtio itself then you could exclude
devices like this by just limiting things based on subvendor/subdevice
IDs.
> Also, I am guessing that hardware implementations will want
> to add things like stong memory barriers - I guess we
> will add new feature bits for that too down the road?
That piece I don't have visibility into at this time. Perhaps Dan
might have more visibility into future plans on what this might need.
Thanks.
- Alex
^ permalink raw reply
* Re: [PATCH 00/30] Netfilter/IPVS updates for net-next
From: Guy Shattah @ 2018-03-16 16:39 UTC (permalink / raw)
To: David Miller, pablo; +Cc: netfilter-devel, netdev
In-Reply-To: <20180312.145843.1054152977291695095.davem@davemloft.net>
On 12/03/2018 20:58, David Miller wrote:
> From: Pablo Neira Ayuso <pablo@netfilter.org>
> Date: Mon, 12 Mar 2018 18:58:50 +0100
>
>> The following patchset contains Netfilter/IPVS updates for your net-next
>> tree. This batch comes with more input sanitization for xtables to
>> address bug reports from fuzzers, preparation works to the flowtable
>> infrastructure and assorted updates. In no particular order, they are:
> Sorry, I've seen enough. I'm not pulling this.
>
> What is the story with this flow table stuff? I tried to ask you
> about this before, but the response I was given was extremely vague
> and did not answer my question at all.
>
> This is a lot of code, and a lot of infrastructure, yet I see
> no device using the infrastructure to offload conntack.
Hi David,
Pablo's code is a very welcome addition to the flow tables infrastructure.
We at Mellanox already have customers asking for Offload of Connection
Tracking.
While a complete hardware implementation is yet to arrive. Pablo's
contribution is blessed.
Using this infrastructure we are capable of completely offloading
connection tracking (Without TCP window validation)
and possibly do a complete offload once hardware support for TCP window
Validation shows up.
I'm currently working closely with Pablo to create the first driver
implementation to utilize
hardware offloading. Needless to say - prior to having hardware
implementation a software infrastructure is required.
> Nor can I see how this can possibly be even useful for such an
> application. What conntrack offload needs are things completely
> outside of what the flow table stuff provides. Mainly, they
> require that the SKB is completely abstracted away from all of
> the contrack code paths, and that the conntrack infrastructure
> operates on an abstract packet metadata concept.
Assuming that the software maintains the flow in the system,
it is reasonable to allow software do the connection establishment and
termination
and let the hardware do all the rest between (again - without TCP window
validation, unless a specialized hardware exists)
>
> This, as has been the case in the past, is what is wrong with
> netfilter approach to supporting offloading. We see all of this
> infrastructure before an actual working use case is provided for a
> specific piece of hardware for a specific driver in the tree.
>
> Nobody can evaluate whether the approach is good or not without
> a clear driver change implementing support for it.
I'm speaking on behalf of Mellanox.
Would one driver support as demonstration suffice?
Thanks,
Guy
^ permalink raw reply
* Re: [PATCH net-next 1/2] net/ipv6: Handle onlink flag with multipath routes
From: David Miller @ 2018-03-16 16:38 UTC (permalink / raw)
To: dsahern; +Cc: netdev
In-Reply-To: <3e2977f7-156e-aad9-ef6a-43733fc7ae4d@gmail.com>
From: David Ahern <dsahern@gmail.com>
Date: Fri, 16 Mar 2018 08:45:10 -0700
> On 3/16/18 8:40 AM, David Miller wrote:
>> Hmmm, this actually "accumulates" the flag rather than sets it.
>>
>> Have you thought about what should happen if the cfg has RTNH_F_ONLINK
>> set?
>
> yes, that's why the test script adds cases with the ONLINK flag set for
> both nexthops, then one with it on the first nexthop only, and one with
> the flag only on the second nexthop.
>
> If you look at the full loop 'cfg' is the variable with the user data,
> and r_cfg is the 'local loop version' so r_cfg.fc_flags gets reset for
> each nexthop:
>
> while (rtnh_ok(rtnh, remaining)) {
> memcpy(&r_cfg, cfg, sizeof(*cfg));
> ...
> r_cfg.fc_flags |= (rtnh->rtnh_flags & RTNH_F_ONLINK);
> rt = ip6_route_info_create(&r_cfg, extack);
Right.
>> I think you should either change this logic to a true 'set', or adjust
>> your commit message to address this aspect of the new behavior.
>
> I can update the commit message.
Please do, thanks David.
^ permalink raw reply
* Re: [PATCH net] skbuff: Fix not waking applications when errors are enqueued
From: David Miller @ 2018-03-16 16:37 UTC (permalink / raw)
To: vinicius.gomes; +Cc: netdev, randy.e.witt, eric.dumazet
In-Reply-To: <20180314203209.26532-1-vinicius.gomes@intel.com>
From: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Date: Wed, 14 Mar 2018 13:32:09 -0700
> When errors are enqueued to the error queue via sock_queue_err_skb()
> function, it is possible that the waiting application is not notified.
>
> Calling 'sk->sk_data_ready()' would not notify applications that
> selected only POLLERR events in poll() (for example).
>
> Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2")
> Reported-by: Randy E. Witt <randy.e.witt@intel.com>
> Reviewed-by: Eric Dumazet <edumazet@google.com>
> Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Applied and queued up for -stable, thank you.
Since this is going into 'net', it doesn't need to be in your net-next
submission as 'net' changes will (eventually) propagate into net-next.
Please fix that up, thank you.
^ permalink raw reply
* Re: [PATCH net] netlink: avoid a double skb free in genlmsg_mcast()
From: David Miller @ 2018-03-16 16:36 UTC (permalink / raw)
To: nicolas.dichtel
Cc: ben.hutchings, netdev, stable, johannes.berg, gregkh,
linux-kernel
In-Reply-To: <20180314201023.12407-1-nicolas.dichtel@6wind.com>
From: Nicolas Dichtel <nicolas.dichtel@6wind.com>
Date: Wed, 14 Mar 2018 21:10:23 +0100
> nlmsg_multicast() consumes always the skb, thus the original skb must be
> freed only when this function is called with a clone.
>
> Fixes: cb9f7a9a5c96 ("netlink: ensure to loop over all netns in genlmsg_multicast_allns()")
> Reported-by: Ben Hutchings <ben.hutchings@codethink.co.uk>
> Signed-off-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
Yeah these "clone until final send" loops can be tricky to manage.
Good catch, applied and queued up for -stable, thanks.
^ permalink raw reply
* Re: [virtio-dev] [pci PATCH v7 2/5] virtio_pci: Add support for unmanaged SR-IOV on virtio_pci devices
From: Michael S. Tsirkin @ 2018-03-16 16:34 UTC (permalink / raw)
To: Alexander Duyck
Cc: bhelgaas, alexander.h.duyck, linux-pci, virtio-dev, kvm, netdev,
dan.daly, linux-kernel, linux-nvme, keith.busch, netanel, ddutile,
mheyne, liang-min.wang, mark.d.rustad, dwmw2, hch, dwmw
In-Reply-To: <20180315184132.3102.90947.stgit@localhost.localdomain>
On Thu, Mar 15, 2018 at 11:42:41AM -0700, Alexander Duyck wrote:
> From: Alexander Duyck <alexander.h.duyck@intel.com>
>
> Hardware-realized virtio_pci devices can implement SR-IOV, so this
> patch enables its use. The device in question is an upcoming Intel
> NIC that implements both a virtio_net PF and virtio_net VFs. These
> are hardware realizations of what has been up to now been a software
> interface.
>
> The device in question has the following 4-part PCI IDs:
>
> PF: vendor: 1af4 device: 1041 subvendor: 8086 subdevice: 15fe
> VF: vendor: 1af4 device: 1041 subvendor: 8086 subdevice: 05fe
>
> The patch currently needs no check for device ID, because the callback
> will never be made for devices that do not assert the capability or
> when run on a platform incapable of SR-IOV.
>
> One reason for this patch is because the hardware requires the
> vendor ID of a VF to be the same as the vendor ID of the PF that
> created it. So it seemed logical to simply have a fully-functioning
> virtio_net PF create the VFs. This patch makes that possible.
>
> Reviewed-by: Christoph Hellwig <hch@lst.de>
> Signed-off-by: Mark Rustad <mark.d.rustad@intel.com>
> Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
So if and when virtio PFs can manage the VFs, then we can
add a feature bit for that?
Seems reasonable.
Also, I am guessing that hardware implementations will want
to add things like stong memory barriers - I guess we
will add new feature bits for that too down the road?
> ---
>
> v4: Dropped call to pci_disable_sriov in virtio_pci_remove function
> v5: Replaced call to pci_sriov_configure_unmanaged with
> pci_sriov_configure_simple
> v6: Dropped "#ifdef" checks for IOV wrapping sriov_configure definition
> v7: No code change, added Reviewed-by
>
> drivers/virtio/virtio_pci_common.c | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
> index 48d4d1cf1cb6..67a227fd7aa0 100644
> --- a/drivers/virtio/virtio_pci_common.c
> +++ b/drivers/virtio/virtio_pci_common.c
> @@ -596,6 +596,7 @@ static void virtio_pci_remove(struct pci_dev *pci_dev)
> #ifdef CONFIG_PM_SLEEP
> .driver.pm = &virtio_pci_pm_ops,
> #endif
> + .sriov_configure = pci_sriov_configure_simple,
> };
>
> module_pci_driver(virtio_pci_driver);
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply
* Re: [Intel-wired-lan] [PATCH v3 09/18] fm10k: Eliminate duplicate barriers on weakly-ordered archs
From: Sinan Kaya @ 2018-03-16 16:33 UTC (permalink / raw)
To: Alexander Duyck
Cc: Netdev, Timur Tabi, sulrich, linux-arm-msm, LKML, intel-wired-lan,
linux-arm-kernel
In-Reply-To: <CAKgT0UcKdCpzFgX4qn7EdYnedOsJa9as5g24tfb6qFLXTxZ29A@mail.gmail.com>
On 3/16/2018 12:30 PM, Alexander Duyck wrote:
> On Fri, Mar 16, 2018 at 9:16 AM, Sinan Kaya <okaya@codeaurora.org> wrote:
>> Code includes wmb() followed by writel(). writel() already has a
>> barrier on some architectures like arm64.
>>
>> This ends up CPU observing two barriers back to back before executing
>> the register write.
>>
>> Since code already has an explicit barrier call, changing writel() to
>> writel_relaxed().
>>
>> Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
>
> You can update the writel call in fm10k_tx_map as well.
>
> Of the drivers updated in drivers/net/ethernet/intel/* it looks like
> this is the only one that still requires any additional changes.
will do. thanks for the feedback.
>
> Thanks.
>
> - Alex
>
--
Sinan Kaya
Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm Technologies, Inc.
Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.
^ permalink raw reply
* Re: [PATCH net-next 0/2] Introduce rtnl_lock_killable()
From: David Miller @ 2018-03-16 16:31 UTC (permalink / raw)
To: ktkhai; +Cc: vyasevic, edumazet, nicolas.dichtel, netdev
In-Reply-To: <152105492788.24797.10467675660981296096.stgit@localhost.localdomain>
From: Kirill Tkhai <ktkhai@virtuozzo.com>
Date: Wed, 14 Mar 2018 22:17:11 +0300
> rtnl_lock() is widely used mutex in kernel. Some of kernel code
> does memory allocations under it. In case of memory deficit this
> may invoke OOM killer, but the problem is a killed task can't
> exit if it's waiting for the mutex. This may be a reason of deadlock
> and panic.
>
> This patchset adds a new primitive, which responds on SIGKILL,
> and it allows to use it in the places, where we don't want
> to sleep forever. Also, the first place is made to use it.
This looks reasonable. Series applied, thank you.
^ permalink raw reply
* Re: [Intel-wired-lan] [PATCH v3 09/18] fm10k: Eliminate duplicate barriers on weakly-ordered archs
From: Alexander Duyck @ 2018-03-16 16:30 UTC (permalink / raw)
To: Sinan Kaya
Cc: Netdev, Timur Tabi, sulrich, linux-arm-msm, LKML, intel-wired-lan,
linux-arm-kernel
In-Reply-To: <1521216991-28706-10-git-send-email-okaya@codeaurora.org>
On Fri, Mar 16, 2018 at 9:16 AM, Sinan Kaya <okaya@codeaurora.org> wrote:
> Code includes wmb() followed by writel(). writel() already has a
> barrier on some architectures like arm64.
>
> This ends up CPU observing two barriers back to back before executing
> the register write.
>
> Since code already has an explicit barrier call, changing writel() to
> writel_relaxed().
>
> Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
You can update the writel call in fm10k_tx_map as well.
Of the drivers updated in drivers/net/ethernet/intel/* it looks like
this is the only one that still requires any additional changes.
Thanks.
- Alex
^ permalink raw reply
* Re: [PATCH net-next v3 4/4] net: qualcomm: rmnet: Implement fill_info
From: David Miller @ 2018-03-16 16:30 UTC (permalink / raw)
To: subashab; +Cc: joe, netdev
In-Reply-To: <1521054064-22775-5-git-send-email-subashab@codeaurora.org>
From: Subash Abhinov Kasiviswanathan <subashab@codeaurora.org>
Date: Wed, 14 Mar 2018 13:01:04 -0600
> + if (nla_put_u16(skb, IFLA_VLAN_ID, priv->mux_id))
> + goto nla_put_failure;
> +
> + port = rmnet_get_port_rtnl(real_dev);
> +
> + f.flags = port->data_format;
> + f.mask = ~0;
> +
> + if (nla_put(skb, IFLA_VLAN_FLAGS, sizeof(f), &f))
> + goto nla_put_failure;
Please don't use VLAN netlink attributes to report rmnet device
specific values.
Instead, please add appropriate new netlink attributes which
clearly show that they are the rmnet mux_id and flags. You'll
also need to provide UAPI definitions for these flag bits
to go along with the attribute name.
Thanks.
^ permalink raw reply
* Re: [PATCH 00/30] Netfilter/IPVS updates for net-next
From: David Miller @ 2018-03-16 16:23 UTC (permalink / raw)
To: pablo; +Cc: fw, nbd, netfilter-devel, netdev
In-Reply-To: <20180314183848.vyosb6rpwiqz3jxx@salvia>
From: Pablo Neira Ayuso <pablo@netfilter.org>
Date: Wed, 14 Mar 2018 19:38:48 +0100
> Just for the record, this is a summary of what we have discussed so
> far:
...
> Note that this batch was coming with a patch to reduce cache footprint
> of the flowtable entries, so there is already working-in-progress
> targeted at improving performance of this new software fast path.
> Also, preparation works to introduce iptables support has been in the
> radar while working on this.
>
> We understand, we may have have spent more time in explaining all this
> in the mailing list, we are trying to amend this now. Therefore, we
> can probably convince someone here to write design documentation to be
> placed on the Documentation/flowtable/ directory in the next pull
> request if that makes it easier for the broader audience to understand
> our effort and rise concerns, if any.
Noted.
Please resend your pull request.
Thanks.
^ permalink raw reply
* Re: [PATCH v2 iproute2-next 0/6] cm_id, cq, mr, and pd resource tracking
From: Leon Romanovsky @ 2018-03-16 16:23 UTC (permalink / raw)
To: David Ahern; +Cc: Jason Gunthorpe, stephen, Steve Wise, netdev, linux-rdma
In-Reply-To: <0f008585-9276-35b6-f259-9d1e754fdd22@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 1213 bytes --]
On Fri, Mar 16, 2018 at 09:08:55AM -0700, David Ahern wrote:
> On 3/14/18 8:29 PM, Jason Gunthorpe wrote:
> > On Wed, Mar 14, 2018 at 08:14:53PM -0700, David Ahern wrote:
> >> On 3/13/18 2:13 PM, Jason Gunthorpe wrote:
> >>> Could you pull the uapi headers from linux-next? That tree will have
> >>> both netdev and rdma stuff merged together properly.
> >>
> >> What's the merge history between linux-next, Linus' tree, net-next +
> >> rdma-next?
> >
> > Not sure I understand the question?
> >
> > linux-next is thrown away every cycle, so for instance you can't say
> > "took uapi headers from linux-next commit XYZ" in the iproute git
> > commit..
> >
> > Otherwise, linux-next is built, I think daily/weekly and includes the
> > latest of both rdma and netdev next trees, so it certainly has the
> > right content.
> >
> > A script could figure out the stable netdev and rdma commit IDs from
> > linux-next and record those, if that is interest..
>
> I'd prefer not to sync headers to a tree that disappears; I like the
> traceability to specific points in time.
The headers itself will have real SHAs, the same as netdev and RDMA
trees have, so it will give needed traceability as you are looking.
Thanks
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply
* Re: interdependencies with cxgb4 and iw_cxgb4
From: David Miller @ 2018-03-16 16:21 UTC (permalink / raw)
To: swise
Cc: rajur, dledford, linux-rdma, jgg, netdev, bharat, ganeshgr,
rahul.lakkireddy
In-Reply-To: <022f01d3bba9$82e94870$88bbd950$@opengridcomputing.com>
From: "Steve Wise" <swise@opengridcomputing.com>
Date: Wed, 14 Mar 2018 10:31:24 -0500
> This issue has also been dealt-with for Mellanox drivers, I believe. I take
> it the solution for them was a k.o. signed repo, that they maintain, where
> both linux-rdma and netdev take PRs from for commits that are needed in both
> repos. Then these are reconciled when both repos are merged into Linus'
> repo. (I hope my understanding of this is correct)
>
> For Chelsio, this is perhaps a possibility, but I'm wondering if there is a
> simpler solution? A few other option we've been discussing include:
>
> 1) submit the cxgb4-only changes to netdev in release cycle X, and then only
> submit the iw_cxgb4 (or other upper drivers) changes that use them in
> release cycle X+1. The pro of this is simplicity. The con is timeliness -
> it takes 2 release cycles to get the feature upstream.
>
> 2) run the entire series through one maintainer's repo (with all
> maintainers' ACK on the content and plan, of course), and ensuring no
> conflicting commits are submitted for the rest of that release cycle. I'm
> not really sure that this is feasible given anyone could create commits for
> upstream drivers. So how could Chelsio really control this?
>
> Do you have any suggestions on how we should proceed?
I think the Mellanox setup is working well currently.
If the changes get pulled into both the rdma and networking tree, then it
all gets resolved cleanly no matter which of rdma or networking goes into
Linus's tree first during the merge window.
It doesn't have the delay issues of suggestion #1, and I think avoiding
conflicts in situation #2 is next to impossible.
In fact, such conflict problems are how we arrived at the approach
Mellanox is using in the first place.
^ permalink raw reply
* Re: [PATCH v2 04/15] ice: Get switch config, scheduler config and device capabilities
From: Shannon Nelson @ 2018-03-16 16:21 UTC (permalink / raw)
To: Anirudh Venkataramanan, intel-wired-lan; +Cc: netdev
In-Reply-To: <20180315234802.31336-5-anirudh.venkataramanan@intel.com>
On 3/15/2018 4:47 PM, Anirudh Venkataramanan wrote:
> This patch adds to the initialization flow by getting switch
> configuration, scheduler configuration and device capabilities.
>
> Switch configuration:
> On boot, an L2 switch element is created in the firmware per physical
> function. Each physical function is also mapped to a port, to which its
> switch element is connected. In other words, this switch can be visualized
> as an embedded vSwitch that can connect a physical functions's virtual
> station interfaces (VSIs) to the egress/ingress port. Egress/ingress
> filters will be eventually created and applied on this switch element.
> As part of the initialization flow, the driver gets configuration data
> from this switch element and stores it.
>
> Scheduler configuration:
> The Tx scheduler is a subsystem responsible for setting and enforcing QoS.
> As part of the initialization flow, the driver queries and stores the
> default scheduler configuration for the given physical function.
>
> Device capabilities:
> As part of initialization, the driver has to determine what the device is
> capable of (ex. max queues, VSIs, etc). This information is obtained from
> the firmware and stored by the driver.
>
> CC: Shannon Nelson <shannon.nelson@oracle.com>
> Signed-off-by: Anirudh Venkataramanan <anirudh.venkataramanan@intel.com>
> ---
> v2: Addressed Shannon Nelson's review comment by changing retry count value
> to 2.
The fix looks fine.
sln
Acked-by: Shannon Nelson <shannon.nelson@oracle.com>
^ permalink raw reply
* Re: [PATCH v2 07/15] ice: Add support for VSI allocation and deallocation
From: Shannon Nelson @ 2018-03-16 16:21 UTC (permalink / raw)
To: Anirudh Venkataramanan, intel-wired-lan; +Cc: netdev
In-Reply-To: <20180315234802.31336-8-anirudh.venkataramanan@intel.com>
On 3/15/2018 4:47 PM, Anirudh Venkataramanan wrote:
> This patch introduces data structures and functions to alloc/free
> VSIs. The driver represents a VSI using the ice_vsi structure.
>
> Some noteworthy points about VSI allocation:
>
> 1) A VSI is allocated in the firmware using the "add VSI" admin queue
> command (implemented as ice_aq_add_vsi). The firmware returns an
> identifier for the allocated VSI. The VSI context is used to program
> certain aspects (loopback, queue map, etc.) of the VSI's configuration.
>
> 2) A VSI is deleted using the "free VSI" admin queue command (implemented
> as ice_aq_free_vsi).
>
> 3) The driver represents a VSI using struct ice_vsi. This is allocated
> and initialized as part of the ice_vsi_alloc flow, and deallocated
> as part of the ice_vsi_delete flow.
>
> 4) Once the VSI is created, a netdev is allocated and associated with it.
> The VSI's ring and vector related data structures are also allocated
> and initialized.
>
> 5) A VSI's queues can either be contiguous or scattered. To do this, the
> driver maintains a bitmap (vsi->avail_txqs) which is kept in sync with
> the firmware's VSI queue allocation imap. If the VSI can't get a
> contiguous queue allocation, it will fallback to scatter. This is
> implemented in ice_vsi_get_qs which is called as part of the VSI setup
> flow. In the release flow, the VSI's queues are released and the bitmap
> is updated to reflect this by ice_vsi_put_qs.
>
> CC: Shannon Nelson <shannon.nelson@oracle.com>
> Signed-off-by: Anirudh Venkataramanan <anirudh.venkataramanan@intel.com>
> ---
> v2: Addressed Shannon Nelson's comments by
> 1) using a new define ICE_NO_VSI instead of the magic number 0xffff.
> 2) adding missing curly braces and break statements.
>
> Also, ice_set_def_vsi_ctx was changed to ice_set_dflt_vsi_ctx for clarity.
Thanks,
sln
Acked-by: Shannon Nelson <shannon.nelson@oracle.com>
^ permalink raw reply
* Re: [PATCH v2 06/15] ice: Initialize PF and setup miscellaneous interrupt
From: Shannon Nelson @ 2018-03-16 16:21 UTC (permalink / raw)
To: Anirudh Venkataramanan, intel-wired-lan; +Cc: netdev
In-Reply-To: <20180315234802.31336-7-anirudh.venkataramanan@intel.com>
On 3/15/2018 4:47 PM, Anirudh Venkataramanan wrote:
> This patch continues the initialization flow as follows:
>
> 1) Allocate and initialize necessary fields (like vsi, num_alloc_vsi,
> irq_tracker, etc) in the ice_pf instance.
>
> 2) Setup the miscellaneous interrupt handler. This also known as the
> "other interrupt causes" (OIC) handler and is used to handle non
> hotpath interrupts (like control queue events, link events,
> exceptions, etc.
>
> 3) Implement a background task to process admin queue receive (ARQ)
> events received by the driver.
>
> CC: Shannon Nelson <shannon.nelson@oracle.com>
> Signed-off-by: Anirudh Venkataramanan <anirudh.venkataramanan@intel.com>
> ---
> v2: Removed reference to "lump" as suggested by Shannon Nelson.
Yep,
sln
Acked-by: Shannon Nelson <shannon.nelson@oracle.com>
^ permalink raw reply
* Re: [PATCH v2 03/15] ice: Start hardware initialization
From: Shannon Nelson @ 2018-03-16 16:21 UTC (permalink / raw)
To: Anirudh Venkataramanan, intel-wired-lan; +Cc: netdev
In-Reply-To: <20180315234802.31336-4-anirudh.venkataramanan@intel.com>
On 3/15/2018 4:47 PM, Anirudh Venkataramanan wrote:
> This patch implements multiple pieces of the initialization flow
> as follows:
>
> 1) A reset is issued to ensure a clean device state, followed
> by initialization of admin queue interface.
>
> 2) Once the admin queue interface is up, clear the PF config
> and transition the device to non-PXE mode.
>
> 3) Get the NVM configuration stored in the device's non-volatile
> memory (NVM) using ice_init_nvm.
>
> CC: Shannon Nelson <shannon.nelson@oracle.com>
> Signed-off-by: Anirudh Venkataramanan <anirudh.venkataramanan@intel.com>
> ---
> v2: Addressed Shannon Nelson's review comments by
> 1) removing an unnecessary register write in ice_aq_clear_pxe_mode.
> 2) adding a comment explaining the need to convert word sized values
> to byte sized values.
These two changes look fine.
sln
Acked-by: Shannon Nelson <shannon.nelson@oracle.com>
^ permalink raw reply
* Re: [PATCH v2 iproute2-next 0/6] cm_id, cq, mr, and pd resource tracking
From: David Ahern @ 2018-03-16 16:18 UTC (permalink / raw)
To: Doug Ledford, Leon Romanovsky, stephen; +Cc: Steve Wise, netdev, linux-rdma
In-Reply-To: <1520974685.18703.11.camel@redhat.com>
On 3/13/18 1:58 PM, Doug Ledford wrote:
> On Tue, 2018-03-13 at 13:45 -0700, David Ahern wrote:
>> On 3/13/18 1:32 AM, Leon Romanovsky wrote:
>>> On Mon, Mar 12, 2018 at 10:53:03AM -0700, David Ahern wrote:
>>>> On 3/12/18 8:16 AM, Steve Wise wrote:
>>>>> Hey all,
>>>>>
>>>>> The kernel side of this series has been merged for rdma-next [1]. Let me
>>>>> know if this iproute2 series can be merged, of if it needs more changes.
>>>>>
>>>>
>>>> The problem is that iproute2 headers are synced to kernel headers from
>>>> DaveM's tree (net-next mainly). I take it this series will not appear in
>>>> Dave's tree until after a merge through Linus' tree. Correct?
>>>
>>> David,
>>>
>>> Technically, you are right, and we would like to ask you for an extra tweak
>>> to the flow for the RDMAtool, because current scheme causes delays at least
>>> cycle.
>>>
>>> Every RDMAtool's patchset which requires changes to headers is always
>>> includes header patch, can you please accept those series and once you
>>> are bringing new net-next headers from Linus, simply overwrite all our
>>> headers?
>>
>> I did not follow the discussion back when this decision was made, so how
>> did rdma tool end up in iproute2?
>
> It is modeled after the ip command, and for better or worse, the
> iproute2 package has become the standard drop box for low level kernel
> network configuring tools. The RDMA subsystem may not be IP networking,
> but it is still networking, so it seemed an appropriate fit.
why doesn't the rdma tree go through Dave then?
^ permalink raw reply
* [PATCH v3 18/18] infiniband: cxgb4: Eliminate duplicate barriers on weakly-ordered archs
From: Sinan Kaya @ 2018-03-16 16:16 UTC (permalink / raw)
To: netdev, timur, sulrich
Cc: linux-arm-msm, linux-arm-kernel, Sinan Kaya, Steve Wise,
Doug Ledford, Jason Gunthorpe, linux-rdma, linux-kernel
In-Reply-To: <1521216991-28706-1-git-send-email-okaya@codeaurora.org>
Code includes wmb() followed by writel(). writel() already has a barrier on
some architectures like arm64.
This ends up CPU observing two barriers back to back before executing the
register write.
Since code already has an explicit barrier call, changing writel() to
writel_relaxed().
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
---
drivers/infiniband/hw/cxgb4/t4.h | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/drivers/infiniband/hw/cxgb4/t4.h b/drivers/infiniband/hw/cxgb4/t4.h
index 8369c7c..7a48c9e 100644
--- a/drivers/infiniband/hw/cxgb4/t4.h
+++ b/drivers/infiniband/hw/cxgb4/t4.h
@@ -477,15 +477,15 @@ static inline void t4_ring_sq_db(struct t4_wq *wq, u16 inc, union t4_wr *wqe)
(u64 *)wqe);
} else {
pr_debug("DB wq->sq.pidx = %d\n", wq->sq.pidx);
- writel(PIDX_T5_V(inc) | QID_V(wq->sq.bar2_qid),
- wq->sq.bar2_va + SGE_UDB_KDOORBELL);
+ writel_relaxed(PIDX_T5_V(inc) | QID_V(wq->sq.bar2_qid),
+ wq->sq.bar2_va + SGE_UDB_KDOORBELL);
}
/* Flush user doorbell area writes. */
wmb();
return;
}
- writel(QID_V(wq->sq.qid) | PIDX_V(inc), wq->db);
+ writel_relaxed(QID_V(wq->sq.qid) | PIDX_V(inc), wq->db);
}
static inline void t4_ring_rq_db(struct t4_wq *wq, u16 inc,
@@ -502,15 +502,15 @@ static inline void t4_ring_rq_db(struct t4_wq *wq, u16 inc,
(void *)wqe);
} else {
pr_debug("DB wq->rq.pidx = %d\n", wq->rq.pidx);
- writel(PIDX_T5_V(inc) | QID_V(wq->rq.bar2_qid),
- wq->rq.bar2_va + SGE_UDB_KDOORBELL);
+ writel_relaxed(PIDX_T5_V(inc) | QID_V(wq->rq.bar2_qid),
+ wq->rq.bar2_va + SGE_UDB_KDOORBELL);
}
/* Flush user doorbell area writes. */
wmb();
return;
}
- writel(QID_V(wq->rq.qid) | PIDX_V(inc), wq->db);
+ writel_relaxed(QID_V(wq->rq.qid) | PIDX_V(inc), wq->db);
}
static inline int t4_wq_in_error(struct t4_wq *wq)
--
2.7.4
^ permalink raw reply related
* [PATCH v3 17/18] RDMA/i40iw: Eliminate duplicate barriers on weakly-ordered archs
From: Sinan Kaya @ 2018-03-16 16:16 UTC (permalink / raw)
To: netdev, timur, sulrich
Cc: linux-arm-msm, linux-arm-kernel, Sinan Kaya, Faisal Latif,
Shiraz Saleem, Doug Ledford, Jason Gunthorpe, linux-rdma,
linux-kernel
In-Reply-To: <1521216991-28706-1-git-send-email-okaya@codeaurora.org>
Code includes wmb() followed by writel(). writel() already has a barrier on
some architectures like arm64.
This ends up CPU observing two barriers back to back before executing the
register write.
Create a new wrapper function with relaxed write operator. Use the new
wrapper when a write is following a wmb().
Since code already has an explicit barrier call, changing writel() to
writel_relaxed().
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
---
drivers/infiniband/hw/i40iw/i40iw_ctrl.c | 6 ++++--
drivers/infiniband/hw/i40iw/i40iw_osdep.h | 1 +
drivers/infiniband/hw/i40iw/i40iw_uk.c | 2 +-
drivers/infiniband/hw/i40iw/i40iw_utils.c | 11 +++++++++++
4 files changed, 17 insertions(+), 3 deletions(-)
diff --git a/drivers/infiniband/hw/i40iw/i40iw_ctrl.c b/drivers/infiniband/hw/i40iw/i40iw_ctrl.c
index c74fd33..47f473e 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_ctrl.c
+++ b/drivers/infiniband/hw/i40iw/i40iw_ctrl.c
@@ -706,9 +706,11 @@ static void i40iw_sc_ccq_arm(struct i40iw_sc_cq *ccq)
wmb(); /* make sure shadow area is updated before arming */
if (ccq->dev->is_pf)
- i40iw_wr32(ccq->dev->hw, I40E_PFPE_CQARM, ccq->cq_uk.cq_id);
+ i40iw_wr32_relaxed(ccq->dev->hw, I40E_PFPE_CQARM,
+ ccq->cq_uk.cq_id);
else
- i40iw_wr32(ccq->dev->hw, I40E_VFPE_CQARM1, ccq->cq_uk.cq_id);
+ i40iw_wr32_relaxed(ccq->dev->hw, I40E_VFPE_CQARM1,
+ ccq->cq_uk.cq_id);
}
/**
diff --git a/drivers/infiniband/hw/i40iw/i40iw_osdep.h b/drivers/infiniband/hw/i40iw/i40iw_osdep.h
index f27be3e..e06f4b9 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_osdep.h
+++ b/drivers/infiniband/hw/i40iw/i40iw_osdep.h
@@ -213,5 +213,6 @@ void i40iw_hw_stats_start_timer(struct i40iw_sc_vsi *vsi);
void i40iw_hw_stats_stop_timer(struct i40iw_sc_vsi *vsi);
#define i40iw_mmiowb() mmiowb()
void i40iw_wr32(struct i40iw_hw *hw, u32 reg, u32 value);
+void i40iw_wr32_relaxed(struct i40iw_hw *hw, u32 reg, u32 value);
u32 i40iw_rd32(struct i40iw_hw *hw, u32 reg);
#endif /* _I40IW_OSDEP_H_ */
diff --git a/drivers/infiniband/hw/i40iw/i40iw_uk.c b/drivers/infiniband/hw/i40iw/i40iw_uk.c
index 8afa5a6..7f0ebed 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_uk.c
+++ b/drivers/infiniband/hw/i40iw/i40iw_uk.c
@@ -723,7 +723,7 @@ static void i40iw_cq_request_notification(struct i40iw_cq_uk *cq,
wmb(); /* make sure WQE is populated before valid bit is set */
- writel(cq->cq_id, cq->cqe_alloc_reg);
+ writel_relaxed(cq->cq_id, cq->cqe_alloc_reg);
}
/**
diff --git a/drivers/infiniband/hw/i40iw/i40iw_utils.c b/drivers/infiniband/hw/i40iw/i40iw_utils.c
index ddc1056..99aa6f8 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_utils.c
+++ b/drivers/infiniband/hw/i40iw/i40iw_utils.c
@@ -125,6 +125,17 @@ inline void i40iw_wr32(struct i40iw_hw *hw, u32 reg, u32 value)
}
/**
+ * i40iw_wr32_relaxed - write 32 bits to hw register without ordering
+ * @hw: hardware information including registers
+ * @reg: register offset
+ * @value: vvalue to write to register
+ */
+inline void i40iw_wr32_relaxed(struct i40iw_hw *hw, u32 reg, u32 value)
+{
+ writel_relaxed(value, hw->hw_addr + reg);
+}
+
+/**
* i40iw_rd32 - read a 32 bit hw register
* @hw: hardware information including registers
* @reg: register offset
--
2.7.4
^ permalink raw reply related
* [PATCH v3 16/18] IB/mlx4: Eliminate duplicate barriers on weakly-ordered archs
From: Sinan Kaya @ 2018-03-16 16:16 UTC (permalink / raw)
To: netdev, timur, sulrich
Cc: linux-arm-msm, linux-arm-kernel, Sinan Kaya, Yishai Hadas,
Doug Ledford, Jason Gunthorpe, linux-rdma, linux-kernel
In-Reply-To: <1521216991-28706-1-git-send-email-okaya@codeaurora.org>
Code includes wmb() followed by writel(). writel() already has a barrier on
some architectures like arm64.
This ends up CPU observing two barriers back to back before executing the
register write.
Since code already has an explicit barrier call, changing writel() to
writel_relaxed().
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
---
drivers/infiniband/hw/mlx4/qp.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c
index f045491..74b27b0 100644
--- a/drivers/infiniband/hw/mlx4/qp.c
+++ b/drivers/infiniband/hw/mlx4/qp.c
@@ -3880,8 +3880,8 @@ int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
*/
wmb();
- writel(qp->doorbell_qpn,
- to_mdev(ibqp->device)->uar_map + MLX4_SEND_DOORBELL);
+ writel_relaxed(qp->doorbell_qpn,
+ to_mdev(ibqp->device)->uar_map + MLX4_SEND_DOORBELL);
/*
* Make sure doorbells don't leak out of SQ spinlock
--
2.7.4
^ permalink raw reply related
* [PATCH v3 15/18] RDMA/bnxt_re: Eliminate duplicate barriers on weakly-ordered archs
From: Sinan Kaya @ 2018-03-16 16:16 UTC (permalink / raw)
To: netdev, timur, sulrich
Cc: linux-arm-msm, linux-arm-kernel, Sinan Kaya, Selvin Xavier,
Devesh Sharma, Somnath Kotur, Sriharsha Basavapatna, Doug Ledford,
Jason Gunthorpe, linux-rdma, linux-kernel
In-Reply-To: <1521216991-28706-1-git-send-email-okaya@codeaurora.org>
Code includes wmb() followed by writel(). writel() already has a barrier on
some architectures like arm64.
This ends up CPU observing two barriers back to back before executing the
register write.
Since code already has an explicit barrier call, changing writel() to
writel_relaxed().
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
---
drivers/infiniband/hw/bnxt_re/qplib_rcfw.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c
index 8329ec6..4a6b981 100644
--- a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c
+++ b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c
@@ -181,10 +181,10 @@ static int __send_message(struct bnxt_qplib_rcfw *rcfw, struct cmdq_base *req,
/* ring CMDQ DB */
wmb();
- writel(cmdq_prod, rcfw->cmdq_bar_reg_iomem +
- rcfw->cmdq_bar_reg_prod_off);
- writel(RCFW_CMDQ_TRIG_VAL, rcfw->cmdq_bar_reg_iomem +
- rcfw->cmdq_bar_reg_trig_off);
+ writel_relaxed(cmdq_prod, rcfw->cmdq_bar_reg_iomem +
+ rcfw->cmdq_bar_reg_prod_off);
+ writel_relaxed(RCFW_CMDQ_TRIG_VAL, rcfw->cmdq_bar_reg_iomem +
+ rcfw->cmdq_bar_reg_trig_off);
done:
spin_unlock_irqrestore(&cmdq->lock, flags);
/* Return the CREQ response pointer */
--
2.7.4
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox