diff for duplicates of <20160809055848.GE19025@dastard> diff --git a/a/1.txt b/N1/1.txt index 8529927..ef26c7e 100644 --- a/a/1.txt +++ b/N1/1.txt @@ -178,4 +178,8 @@ Cheers, Dave. -- Dave Chinner -david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org +david@fromorbit.com +_______________________________________________ +Linux-nvdimm mailing list +Linux-nvdimm@lists.01.org +https://lists.01.org/mailman/listinfo/linux-nvdimm diff --git a/a/content_digest b/N1/content_digest index d828a7b..0c2200f 100644 --- a/a/content_digest +++ b/N1/content_digest @@ -8,17 +8,16 @@ "ref\0CS1PR84MB0119314ACA9B4823C0FE33318E180@CS1PR84MB0119.NAMPRD84.PROD.OUTLOOK.COM\0" "ref\020160808231225.GD19025@dastard\0" "ref\01470704418.32015.51.camel@hpe.com\0" - "ref\01470704418.32015.51.camel-ZPxbGqLxI0U@public.gmane.org\0" - "From\0Dave Chinner <david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org>\0" + "From\0Dave Chinner <david@fromorbit.com>\0" "Subject\0Re: Subtle races between DAX mmap fault and write path\0" "Date\0Tue, 9 Aug 2016 15:58:48 +1000\0" "To\0Kani" - " Toshimitsu <toshi.kani-ZPxbGqLxI0U@public.gmane.org>\0" - "Cc\0jack-AlSwsSmVLrQ@public.gmane.org <jack-AlSwsSmVLrQ@public.gmane.org>" - linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org <linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org> - xfs-VZNHf3L845pBDgjK7y7TUQ@public.gmane.org <xfs-VZNHf3L845pBDgjK7y7TUQ@public.gmane.org> - linux-fsdevel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org <linux-fsdevel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org> - " linux-ext4-u79uwXL29TY76Z2rM5mHXA@public.gmane.org <linux-ext4-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>\0" + " Toshimitsu <toshi.kani@hpe.com>\0" + "Cc\0jack@suse.cz <jack@suse.cz>" + linux-nvdimm@lists.01.org <linux-nvdimm@lists.01.org> + xfs@oss.sgi.com <xfs@oss.sgi.com> + linux-fsdevel@vger.kernel.org <linux-fsdevel@vger.kernel.org> + " linux-ext4@vger.kernel.org <linux-ext4@vger.kernel.org>\0" "\00:1\0" "b\0" "On Tue, Aug 09, 2016 at 01:00:30AM +0000, Kani, Toshimitsu wrote:\n" @@ -201,6 +200,10 @@ "Dave.\n" "-- \n" "Dave Chinner\n" - david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org + "david@fromorbit.com\n" + "_______________________________________________\n" + "Linux-nvdimm mailing list\n" + "Linux-nvdimm@lists.01.org\n" + https://lists.01.org/mailman/listinfo/linux-nvdimm -759d64cc6cefd0b908edfda79cdc890831f0e47c291cf4acec38daab3204e06d +4093a17b43d04c3b278d47bb4a7bdb3809b0bdd996e64278a88cf09be64b1ce9
diff --git a/a/1.txt b/N2/1.txt index 8529927..5e66486 100644 --- a/a/1.txt +++ b/N2/1.txt @@ -178,4 +178,9 @@ Cheers, Dave. -- Dave Chinner -david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org +david@fromorbit.com + +_______________________________________________ +xfs mailing list +xfs@oss.sgi.com +http://oss.sgi.com/mailman/listinfo/xfs diff --git a/a/content_digest b/N2/content_digest index d828a7b..68bfff8 100644 --- a/a/content_digest +++ b/N2/content_digest @@ -8,17 +8,18 @@ "ref\0CS1PR84MB0119314ACA9B4823C0FE33318E180@CS1PR84MB0119.NAMPRD84.PROD.OUTLOOK.COM\0" "ref\020160808231225.GD19025@dastard\0" "ref\01470704418.32015.51.camel@hpe.com\0" - "ref\01470704418.32015.51.camel-ZPxbGqLxI0U@public.gmane.org\0" - "From\0Dave Chinner <david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org>\0" + "From\0Dave Chinner <david@fromorbit.com>\0" "Subject\0Re: Subtle races between DAX mmap fault and write path\0" "Date\0Tue, 9 Aug 2016 15:58:48 +1000\0" "To\0Kani" - " Toshimitsu <toshi.kani-ZPxbGqLxI0U@public.gmane.org>\0" - "Cc\0jack-AlSwsSmVLrQ@public.gmane.org <jack-AlSwsSmVLrQ@public.gmane.org>" - linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org <linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org> - xfs-VZNHf3L845pBDgjK7y7TUQ@public.gmane.org <xfs-VZNHf3L845pBDgjK7y7TUQ@public.gmane.org> - linux-fsdevel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org <linux-fsdevel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org> - " linux-ext4-u79uwXL29TY76Z2rM5mHXA@public.gmane.org <linux-ext4-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>\0" + " Toshimitsu <toshi.kani@hpe.com>\0" + "Cc\0jack@suse.cz <jack@suse.cz>" + linux-nvdimm@lists.01.org <linux-nvdimm@lists.01.org> + xfs@oss.sgi.com <xfs@oss.sgi.com> + Boylston + Brian <brian.boylston@hpe.com> + linux-fsdevel@vger.kernel.org <linux-fsdevel@vger.kernel.org> + " linux-ext4@vger.kernel.org <linux-ext4@vger.kernel.org>\0" "\00:1\0" "b\0" "On Tue, Aug 09, 2016 at 01:00:30AM +0000, Kani, Toshimitsu wrote:\n" @@ -201,6 +202,11 @@ "Dave.\n" "-- \n" "Dave Chinner\n" - david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org + "david@fromorbit.com\n" + "\n" + "_______________________________________________\n" + "xfs mailing list\n" + "xfs@oss.sgi.com\n" + http://oss.sgi.com/mailman/listinfo/xfs -759d64cc6cefd0b908edfda79cdc890831f0e47c291cf4acec38daab3204e06d +ae77c2e86f849bdf191a9b15733b9c4b8a3c699b1fd14ec12d964095d7030f23
diff --git a/a/1.txt b/N3/1.txt index 8529927..e339134 100644 --- a/a/1.txt +++ b/N3/1.txt @@ -16,9 +16,9 @@ On Tue, Aug 09, 2016 at 01:00:30AM +0000, Kani, Toshimitsu wrote: > > > > > > > > > > I do not think the test results are relevant on this point > > > > > because both buffered and dax write() paths use uncached copy -> > > > > to avoid clflush. The buffered path uses cached copy to the +> > > > > to avoid clflush. �The buffered path uses cached copy to the > > > > > page cache and then use uncached copy to PMEM via writeback. -> > > > > Therefore, the buffered IO path also benefits from using +> > > > > �Therefore, the buffered IO path also benefits from using > > > > > uncached copy to avoid clflush. > > > > > > > > Except that I tested without the writeback path for buffered IO, @@ -37,7 +37,7 @@ On Tue, Aug 09, 2016 at 01:00:30AM +0000, Kani, Toshimitsu wrote: > > > > than cached. How about you two get together, do some benchmarking > > > > and get your story straight, eh? > > > > -> > > > > and should be used for writing to the page cache. For writing +> > > > > and should be used for writing to the page cache. �For writing > > > > > to PMEM, however, additional clflush can be expensive, and > > > > > allocating cachelines for PMEM leads to evict application's > > > > > cachelines. @@ -55,22 +55,22 @@ On Tue, Aug 09, 2016 at 01:00:30AM +0000, Kani, Toshimitsu wrote: > > > below. > > > > > > Time (in seconds) to copy a 16KiB buffer 1M times to a 4MiB NVDIMM -> > > buffer (1M total memcpy()s). For the cached+clflush case, the +> > > buffer (1M total memcpy()s).��For the cached+clflush case, the > > > flushes are done every 4MiB (which seems slightly faster than > > > flushing every 16KiB): > > > -> > > NUMA local NUMA remote -> > > Cached+clflush 13.5 37.1 -> > > movnt 1.0 1.3 +> > > ������������������NUMA local����NUMA remote +> > > Cached+clflush������13.5�����������37.1 +> > > movnt����������������1.0������������1.3� > > > > So let's put that in memory bandwidth terms. You wrote 16GB to the -> > NVDIMM. That means: +> > NVDIMM.��That means: > > -> > NUMA local NUMA remote -> > Cached+clflush 1.2GB/s 0.43GB/s -> > movnt 16.0GB/s 12.3GB/s +> > ������������������NUMA local����NUMA remote +> > Cached+clflush������1.2GB/s���������0.43GB/s +> > movnt��������������16.0GB/s���������12.3GB/s > > -> > That smells wrong. The DAX code (using movnt) is not 1-2 orders of +> > That smells wrong.��The DAX code (using movnt) is not 1-2 orders of > > magnitude faster than a page cache copy, so I don't believe your > > benchmark reflects what I'm proposing. > > @@ -78,7 +78,7 @@ On Tue, Aug 09, 2016 at 01:00:30AM +0000, Kani, Toshimitsu wrote: > > after every 16k write when we use the page cache, nor will we do > > that if we use cached copies, dirty tracking and clflush on fsync(). > -> As I mentioned before, we do not use clflush on the write path. So, +> As I mentioned before, we do not use clflush on the write path. �So, > your tests did not issue clflush at all. Uh, yes, I made that clear by saying "using volatile, cached @@ -103,12 +103,12 @@ basically the same cost as the volatile page cache copy up front. > > memcpy(dstp, src, src_sz); // pwrite(); > > dstp += src_sz; > > } -> > pmem_persist(dst, dstsz); // fsync(); +> > ��������pmem_persist(dst, dstsz); // fsync(); > > > > i.e. The cache flushes occur only at the user defined > > synchronisation point not on every syscall. > -> Brian's test is (16 KiB pwrite + fsync) repeated 1M times. It compared +> Brian's test is (16 KiB pwrite + fsync) repeated 1M times. �It compared > two approaches in the case of 16 KiB persistent write. Yes, Brian tested synchronous writes. But: @@ -139,7 +139,7 @@ CPUs. > > existing storage and the mechanisms are already there; we just need > > the dirty tracking to optimise it. > -> Perhaps, you are referring flushing on disk write cache? I do not +> Perhaps, you are referring flushing on disk write cache? �I do not > think clflush as a x86 instruction is used for exisiting storage. I'm talking about *whatever volatile caches are in layers below the @@ -178,4 +178,4 @@ Cheers, Dave. -- Dave Chinner -david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org +david@fromorbit.com diff --git a/a/content_digest b/N3/content_digest index d828a7b..5d07a7a 100644 --- a/a/content_digest +++ b/N3/content_digest @@ -8,17 +8,18 @@ "ref\0CS1PR84MB0119314ACA9B4823C0FE33318E180@CS1PR84MB0119.NAMPRD84.PROD.OUTLOOK.COM\0" "ref\020160808231225.GD19025@dastard\0" "ref\01470704418.32015.51.camel@hpe.com\0" - "ref\01470704418.32015.51.camel-ZPxbGqLxI0U@public.gmane.org\0" - "From\0Dave Chinner <david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org>\0" + "From\0Dave Chinner <david@fromorbit.com>\0" "Subject\0Re: Subtle races between DAX mmap fault and write path\0" "Date\0Tue, 9 Aug 2016 15:58:48 +1000\0" "To\0Kani" - " Toshimitsu <toshi.kani-ZPxbGqLxI0U@public.gmane.org>\0" - "Cc\0jack-AlSwsSmVLrQ@public.gmane.org <jack-AlSwsSmVLrQ@public.gmane.org>" - linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org <linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org> - xfs-VZNHf3L845pBDgjK7y7TUQ@public.gmane.org <xfs-VZNHf3L845pBDgjK7y7TUQ@public.gmane.org> - linux-fsdevel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org <linux-fsdevel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org> - " linux-ext4-u79uwXL29TY76Z2rM5mHXA@public.gmane.org <linux-ext4-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>\0" + " Toshimitsu <toshi.kani@hpe.com>\0" + "Cc\0Boylston" + Brian <brian.boylston@hpe.com> + linux-ext4@vger.kernel.org <linux-ext4@vger.kernel.org> + jack@suse.cz <jack@suse.cz> + linux-nvdimm@lists.01.org <linux-nvdimm@lists.01.org> + xfs@oss.sgi.com <xfs@oss.sgi.com> + " linux-fsdevel@vger.kernel.org <linux-fsdevel@vger.kernel.org>\0" "\00:1\0" "b\0" "On Tue, Aug 09, 2016 at 01:00:30AM +0000, Kani, Toshimitsu wrote:\n" @@ -39,9 +40,9 @@ "> > > > > \n" "> > > > > I do not think the test results are relevant on this point\n" "> > > > > because both buffered and dax write() paths use uncached copy\n" - "> > > > > to avoid clflush. \302\240The buffered path uses cached copy to the\n" + "> > > > > to avoid clflush. \303\257\302\277\302\275The buffered path uses cached copy to the\n" "> > > > > page cache and then use uncached copy to PMEM via writeback.\n" - "> > > > > \302\240Therefore, the buffered IO path also benefits from using\n" + "> > > > > \303\257\302\277\302\275Therefore, the buffered IO path also benefits from using\n" "> > > > > uncached copy to avoid clflush.\n" "> > > > \n" "> > > > Except that I tested without the writeback path for buffered IO,\n" @@ -60,7 +61,7 @@ "> > > > than cached. How about you two get together, do some benchmarking\n" "> > > > and get your story straight, eh?\n" "> > > > \n" - "> > > > > and should be used for writing to the page cache. \302\240For writing\n" + "> > > > > and should be used for writing to the page cache. \303\257\302\277\302\275For writing\n" "> > > > > to PMEM, however, additional clflush can be expensive, and\n" "> > > > > allocating cachelines for PMEM leads to evict application's\n" "> > > > > cachelines.\n" @@ -78,22 +79,22 @@ "> > > below.\n" "> > > \n" "> > > Time (in seconds) to copy a 16KiB buffer 1M times to a 4MiB NVDIMM\n" - "> > > buffer (1M total memcpy()s).\302\240\302\240For the cached+clflush case, the\n" + "> > > buffer (1M total memcpy()s).\303\257\302\277\302\275\303\257\302\277\302\275For the cached+clflush case, the\n" "> > > flushes are done every 4MiB (which seems slightly faster than\n" "> > > flushing every 16KiB):\n" "> > > \n" - "> > > \302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240NUMA local\302\240\302\240\302\240\302\240NUMA remote\n" - "> > > Cached+clflush\302\240\302\240\302\240\302\240\302\240\302\24013.5\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\24037.1\n" - "> > > movnt\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\2401.0\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\2401.3\302\240\n" + "> > > \303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275NUMA local\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275NUMA remote\n" + "> > > Cached+clflush\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\27513.5\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\27537.1\n" + "> > > movnt\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\2751.0\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\2751.3\303\257\302\277\302\275\n" "> > \n" "> > So let's put that in memory bandwidth terms. You wrote 16GB to the\n" - "> > NVDIMM.\302\240\302\240That means:\n" + "> > NVDIMM.\303\257\302\277\302\275\303\257\302\277\302\275That means:\n" "> > \n" - "> > \302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240NUMA local\302\240\302\240\302\240\302\240NUMA remote\n" - "> > Cached+clflush\302\240\302\240\302\240\302\240\302\240\302\2401.2GB/s\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\2400.43GB/s\n" - "> > movnt\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\24016.0GB/s\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\24012.3GB/s\n" + "> > \303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275NUMA local\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275NUMA remote\n" + "> > Cached+clflush\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\2751.2GB/s\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\2750.43GB/s\n" + "> > movnt\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\27516.0GB/s\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\27512.3GB/s\n" "> > \n" - "> > That smells wrong.\302\240\302\240The DAX code (using movnt) is not 1-2 orders of\n" + "> > That smells wrong.\303\257\302\277\302\275\303\257\302\277\302\275The DAX code (using movnt) is not 1-2 orders of\n" "> > magnitude faster than a page cache copy, so I don't believe your\n" "> > benchmark reflects what I'm proposing.\n" "> >\n" @@ -101,7 +102,7 @@ "> > after every 16k write when we use the page cache, nor will we do\n" "> > that if we use cached copies, dirty tracking and clflush on fsync().\n" "> \n" - "> As I mentioned before, we do not use clflush on the write path. \302\240So,\n" + "> As I mentioned before, we do not use clflush on the write path. \303\257\302\277\302\275So,\n" "> your tests did not issue clflush at all.\n" "\n" "Uh, yes, I made that clear by saying \"using volatile, cached\n" @@ -126,12 +127,12 @@ "> > \t\tmemcpy(dstp, src, src_sz);\t// pwrite();\n" "> > \t\tdstp += src_sz;\n" "> > \t}\n" - "> > \302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240pmem_persist(dst, dstsz);\t// fsync();\n" + "> > \303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275\303\257\302\277\302\275pmem_persist(dst, dstsz);\t// fsync();\n" "> > \n" "> > i.e. The cache flushes occur only at the user defined\n" "> > synchronisation point not on every syscall.\n" "> \n" - "> Brian's test is (16 KiB pwrite + fsync) repeated 1M times. \302\240It compared\n" + "> Brian's test is (16 KiB pwrite + fsync) repeated 1M times. \303\257\302\277\302\275It compared\n" "> two approaches in the case of 16 KiB persistent write.\n" "\n" "Yes, Brian tested synchronous writes. But:\n" @@ -162,7 +163,7 @@ "> > existing storage and the mechanisms are already there; we just need\n" "> > the dirty tracking to optimise it.\n" "> \n" - "> Perhaps, you are referring flushing on disk write cache? \302\240I do not\n" + "> Perhaps, you are referring flushing on disk write cache? \303\257\302\277\302\275I do not\n" "> think clflush as a x86 instruction is used for exisiting storage.\n" "\n" "I'm talking about *whatever volatile caches are in layers below the\n" @@ -201,6 +202,6 @@ "Dave.\n" "-- \n" "Dave Chinner\n" - david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org + david@fromorbit.com -759d64cc6cefd0b908edfda79cdc890831f0e47c291cf4acec38daab3204e06d +aca802bdb76bc611ccd1ba38ff13010dc90dfe7f9e9b8923edf6f9a2d09f410c
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.