* [layerindex-web][patch v3 0/1] recipes.html: Require keyword for recipe search @ 2017-11-07 22:28 Amanda Brindle 2017-11-07 22:31 ` [layerindex-web][patch v3 1/1] " Amanda Brindle 0 siblings, 1 reply; 7+ messages in thread From: Amanda Brindle @ 2017-11-07 22:28 UTC (permalink / raw) To: yocto; +Cc: paul.eggleton, Amanda Brindle For v3, updated recipes.html to use bootstrap's error state and added the browse option for Classes. The following changes since commit 78c2561181f07b1c39f1dc3516a9110a39d874f2: templates/layerindex/classes.html: Add bbclass search (2017-11-07 16:54:46 +1300) are available in the git repository at: git://git.yoctoproject.org/layerindex-web abrindle/recipe_search http://git.yoctoproject.org/cgit.cgi/layerindex-web/log/?h=abrindle/recipe_search Amanda Brindle (1): recipes.html: Require keyword for recipe search layerindex/views.py | 15 ++++++++++++--- templates/layerindex/classes.html | 3 ++- templates/layerindex/distros.html | 3 ++- templates/layerindex/machines.html | 3 ++- templates/layerindex/recipes.html | 27 +++++++++++++++++++++------ 5 files changed, 39 insertions(+), 12 deletions(-) -- 2.7.4 ^ permalink raw reply [flat|nested] 7+ messages in thread
* [layerindex-web][patch v3 1/1] recipes.html: Require keyword for recipe search 2017-11-07 22:28 [layerindex-web][patch v3 0/1] recipes.html: Require keyword for recipe search Amanda Brindle @ 2017-11-07 22:31 ` Amanda Brindle 2017-11-07 22:47 ` Mark Hatle 0 siblings, 1 reply; 7+ messages in thread From: Amanda Brindle @ 2017-11-07 22:31 UTC (permalink / raw) To: yocto; +Cc: paul.eggleton, Amanda Brindle Use JavaScript to check if the search box for recipe search is empty before querying the database. This will prevent the "502 Bad Gateway" error that occurs when the query takes too long due to the large list of recipes. Since there are so many recipes spread across the layers in the OE index, there's no point in allowing a user to search without a keyword in order to browse the list; it simply isn't digestible as a whole. Add a browse button for the Machines, Classes, and Distros pages. Fixes [YOCTO #11930] Signed-off-by: Amanda Brindle <amanda.r.brindle@intel.com> --- layerindex/views.py | 15 ++++++++++++--- templates/layerindex/classes.html | 3 ++- templates/layerindex/distros.html | 3 ++- templates/layerindex/machines.html | 3 ++- templates/layerindex/recipes.html | 27 +++++++++++++++++++++------ 5 files changed, 39 insertions(+), 12 deletions(-) diff --git a/layerindex/views.py b/layerindex/views.py index 03d47f2..414c770 100644 --- a/layerindex/views.py +++ b/layerindex/views.py @@ -656,7 +656,10 @@ class MachineSearchView(ListView): def get_queryset(self): _check_url_branch(self.kwargs) - query_string = self.request.GET.get('q', '') + if self.request.GET.get('search', ''): + query_string = self.request.GET.get('q', '') + else: + query_string = "" init_qs = Machine.objects.filter(layerbranch__branch__name=self.kwargs['branch']) if query_string.strip(): entry_query = simplesearch.get_query(query_string, ['name', 'description']) @@ -705,7 +708,10 @@ class DistroSearchView(ListView): def get_queryset(self): _check_url_branch(self.kwargs) - query_string = self.request.GET.get('q', '') + if self.request.GET.get('search', ''): + query_string = self.request.GET.get('q', '') + else: + query_string = "" init_qs = Distro.objects.filter(layerbranch__branch__name=self.kwargs['branch']) if query_string.strip(): entry_query = simplesearch.get_query(query_string, ['name', 'description']) @@ -730,7 +736,10 @@ class ClassSearchView(ListView): def get_queryset(self): _check_url_branch(self.kwargs) - query_string = self.request.GET.get('q', '') + if self.request.GET.get('search', ''): + query_string = self.request.GET.get('q', '') + else: + query_string = "" init_qs = BBClass.objects.filter(layerbranch__branch__name=self.kwargs['branch']) if query_string.strip(): entry_query = simplesearch.get_query(query_string, ['name']) diff --git a/templates/layerindex/classes.html b/templates/layerindex/classes.html index 34ac5aa..574cdb8 100644 --- a/templates/layerindex/classes.html +++ b/templates/layerindex/classes.html @@ -35,7 +35,8 @@ <div class="input-append"> <form id="filter-form" action="{% url 'class_search' url_branch %}" method="get"> <input type="text" class="input-xxlarge" id="appendedInputButtons" placeholder="Search classes" name="q" value="{{ search_keyword }}" /> - <button class="btn" type="submit">search</button> + <button class="btn" type="submit" name="search" value="1">search</button> + <button class="btn" type="submit" name="browse" value="1">browse</button> </form> </div> </div> diff --git a/templates/layerindex/distros.html b/templates/layerindex/distros.html index 5b6995a..3266bf6 100644 --- a/templates/layerindex/distros.html +++ b/templates/layerindex/distros.html @@ -35,7 +35,8 @@ <div class="input-append"> <form id="filter-form" action="{% url 'distro_search' url_branch %}" method="get"> <input type="text" class="input-xxlarge" id="appendedInputButtons" placeholder="Search distros" name="q" value="{{ search_keyword }}" /> - <button class="btn" type="submit">search</button> + <button class="btn" type="submit" name="search" value="1">search</button> + <button class="btn" type="submit" name="browse" value="1">browse</button> </form> </div> </div> diff --git a/templates/layerindex/machines.html b/templates/layerindex/machines.html index c0c6f33..e963376 100644 --- a/templates/layerindex/machines.html +++ b/templates/layerindex/machines.html @@ -34,7 +34,8 @@ <div class="input-append"> <form id="filter-form" action="{% url 'machine_search' url_branch %}" method="get"> <input type="text" class="input-xxlarge" id="appendedInputButtons" placeholder="Search machines" name="q" value="{{ search_keyword }}" /> - <button class="btn" type="submit">search</button> + <button class="btn" type="submit" name="search" value="1">search</button> + <button class="btn" type="submit" name="browse" value="1">browse</button> </form> </div> </div> diff --git a/templates/layerindex/recipes.html b/templates/layerindex/recipes.html index 1322750..5ff92ab 100644 --- a/templates/layerindex/recipes.html +++ b/templates/layerindex/recipes.html @@ -32,14 +32,20 @@ <div class="row-fluid"> - <div class="input-append"> - <form id="filter-form" action="{% url 'recipe_search' url_branch %}" method="get"> - <input type="text" class="input-xxlarge" id="appendedInputButtons" placeholder="Search recipes" name="q" value="{{ search_keyword }}" /> - <button class="btn" type="submit">search</button> - </form> - </div> + <form id="filter-form" action="{% url 'recipe_search' url_branch %}" method="get" onsubmit="return validate()"> + <div class="control-group" id="searchfield"> + <div class="controls"> + <div class="input-append"> + <input type="text" class="input-xxlarge" id="appendedInputButtons" placeholder="Search recipes" name="q" value="{{ search_keyword }}" /> + <button class="btn" type="submit">search</button> + </div> + <span class="help-inline" id="errortext"></span> + </div> + </div> + </form> </div> + <div id="error"> </div> {% if recipe_list %} <table class="table table-striped table-bordered recipestable"> <thead> @@ -88,5 +94,14 @@ $('.icon-hdd').tooltip({title:"Inherits image"}); $('.label-inverse').tooltip(); }); + + function validate(){ + if (!$("#appendedInputButtons").val()){ + $("#errortext").html("<p>Please specify search text</p>"); + $("#searchfield").addClass("error"); + return false; + } + } + </script> {% endblock %} -- 2.7.4 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [layerindex-web][patch v3 1/1] recipes.html: Require keyword for recipe search 2017-11-07 22:31 ` [layerindex-web][patch v3 1/1] " Amanda Brindle @ 2017-11-07 22:47 ` Mark Hatle 2017-11-08 2:43 ` Paul Eggleton 0 siblings, 1 reply; 7+ messages in thread From: Mark Hatle @ 2017-11-07 22:47 UTC (permalink / raw) To: Amanda Brindle, yocto; +Cc: paul.eggleton On 11/7/17 4:31 PM, Amanda Brindle wrote: > Use JavaScript to check if the search box for recipe search is > empty before querying the database. This will prevent the "502 > Bad Gateway" error that occurs when the query takes too long due > to the large list of recipes. Since there are so many recipes > spread across the layers in the OE index, there's no point in > allowing a user to search without a keyword in order to browse > the list; it simply isn't digestible as a whole. > > Add a browse button for the Machines, Classes, and Distros pages. There are reasons to view all of the recipes, machines, classes, distros, etc. (Not necessarily good reasons, but I know people do it.) If the query is too long, it would be better to figure out a way to get a partial response and formulate the first page based on partial responses... having a multipage response that the user can look at. --Mark > Fixes [YOCTO #11930] > > Signed-off-by: Amanda Brindle <amanda.r.brindle@intel.com> > --- > layerindex/views.py | 15 ++++++++++++--- > templates/layerindex/classes.html | 3 ++- > templates/layerindex/distros.html | 3 ++- > templates/layerindex/machines.html | 3 ++- > templates/layerindex/recipes.html | 27 +++++++++++++++++++++------ > 5 files changed, 39 insertions(+), 12 deletions(-) > > diff --git a/layerindex/views.py b/layerindex/views.py > index 03d47f2..414c770 100644 > --- a/layerindex/views.py > +++ b/layerindex/views.py > @@ -656,7 +656,10 @@ class MachineSearchView(ListView): > > def get_queryset(self): > _check_url_branch(self.kwargs) > - query_string = self.request.GET.get('q', '') > + if self.request.GET.get('search', ''): > + query_string = self.request.GET.get('q', '') > + else: > + query_string = "" > init_qs = Machine.objects.filter(layerbranch__branch__name=self.kwargs['branch']) > if query_string.strip(): > entry_query = simplesearch.get_query(query_string, ['name', 'description']) > @@ -705,7 +708,10 @@ class DistroSearchView(ListView): > > def get_queryset(self): > _check_url_branch(self.kwargs) > - query_string = self.request.GET.get('q', '') > + if self.request.GET.get('search', ''): > + query_string = self.request.GET.get('q', '') > + else: > + query_string = "" > init_qs = Distro.objects.filter(layerbranch__branch__name=self.kwargs['branch']) > if query_string.strip(): > entry_query = simplesearch.get_query(query_string, ['name', 'description']) > @@ -730,7 +736,10 @@ class ClassSearchView(ListView): > > def get_queryset(self): > _check_url_branch(self.kwargs) > - query_string = self.request.GET.get('q', '') > + if self.request.GET.get('search', ''): > + query_string = self.request.GET.get('q', '') > + else: > + query_string = "" > init_qs = BBClass.objects.filter(layerbranch__branch__name=self.kwargs['branch']) > if query_string.strip(): > entry_query = simplesearch.get_query(query_string, ['name']) > diff --git a/templates/layerindex/classes.html b/templates/layerindex/classes.html > index 34ac5aa..574cdb8 100644 > --- a/templates/layerindex/classes.html > +++ b/templates/layerindex/classes.html > @@ -35,7 +35,8 @@ > <div class="input-append"> > <form id="filter-form" action="{% url 'class_search' url_branch %}" method="get"> > <input type="text" class="input-xxlarge" id="appendedInputButtons" placeholder="Search classes" name="q" value="{{ search_keyword }}" /> > - <button class="btn" type="submit">search</button> > + <button class="btn" type="submit" name="search" value="1">search</button> > + <button class="btn" type="submit" name="browse" value="1">browse</button> > </form> > </div> > </div> > diff --git a/templates/layerindex/distros.html b/templates/layerindex/distros.html > index 5b6995a..3266bf6 100644 > --- a/templates/layerindex/distros.html > +++ b/templates/layerindex/distros.html > @@ -35,7 +35,8 @@ > <div class="input-append"> > <form id="filter-form" action="{% url 'distro_search' url_branch %}" method="get"> > <input type="text" class="input-xxlarge" id="appendedInputButtons" placeholder="Search distros" name="q" value="{{ search_keyword }}" /> > - <button class="btn" type="submit">search</button> > + <button class="btn" type="submit" name="search" value="1">search</button> > + <button class="btn" type="submit" name="browse" value="1">browse</button> > </form> > </div> > </div> > diff --git a/templates/layerindex/machines.html b/templates/layerindex/machines.html > index c0c6f33..e963376 100644 > --- a/templates/layerindex/machines.html > +++ b/templates/layerindex/machines.html > @@ -34,7 +34,8 @@ > <div class="input-append"> > <form id="filter-form" action="{% url 'machine_search' url_branch %}" method="get"> > <input type="text" class="input-xxlarge" id="appendedInputButtons" placeholder="Search machines" name="q" value="{{ search_keyword }}" /> > - <button class="btn" type="submit">search</button> > + <button class="btn" type="submit" name="search" value="1">search</button> > + <button class="btn" type="submit" name="browse" value="1">browse</button> > </form> > </div> > </div> > diff --git a/templates/layerindex/recipes.html b/templates/layerindex/recipes.html > index 1322750..5ff92ab 100644 > --- a/templates/layerindex/recipes.html > +++ b/templates/layerindex/recipes.html > @@ -32,14 +32,20 @@ > > > <div class="row-fluid"> > - <div class="input-append"> > - <form id="filter-form" action="{% url 'recipe_search' url_branch %}" method="get"> > - <input type="text" class="input-xxlarge" id="appendedInputButtons" placeholder="Search recipes" name="q" value="{{ search_keyword }}" /> > - <button class="btn" type="submit">search</button> > - </form> > - </div> > + <form id="filter-form" action="{% url 'recipe_search' url_branch %}" method="get" onsubmit="return validate()"> > + <div class="control-group" id="searchfield"> > + <div class="controls"> > + <div class="input-append"> > + <input type="text" class="input-xxlarge" id="appendedInputButtons" placeholder="Search recipes" name="q" value="{{ search_keyword }}" /> > + <button class="btn" type="submit">search</button> > + </div> > + <span class="help-inline" id="errortext"></span> > + </div> > + </div> > + </form> > </div> > > + <div id="error"> </div> > {% if recipe_list %} > <table class="table table-striped table-bordered recipestable"> > <thead> > @@ -88,5 +94,14 @@ > $('.icon-hdd').tooltip({title:"Inherits image"}); > $('.label-inverse').tooltip(); > }); > + > + function validate(){ > + if (!$("#appendedInputButtons").val()){ > + $("#errortext").html("<p>Please specify search text</p>"); > + $("#searchfield").addClass("error"); > + return false; > + } > + } > + > </script> > {% endblock %} > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [layerindex-web][patch v3 1/1] recipes.html: Require keyword for recipe search 2017-11-07 22:47 ` Mark Hatle @ 2017-11-08 2:43 ` Paul Eggleton 2017-11-08 15:42 ` Mark Hatle 0 siblings, 1 reply; 7+ messages in thread From: Paul Eggleton @ 2017-11-08 2:43 UTC (permalink / raw) To: Mark Hatle; +Cc: yocto, Amanda Brindle On Wednesday, 8 November 2017 11:47:49 AM NZDT Mark Hatle wrote: > On 11/7/17 4:31 PM, Amanda Brindle wrote: > > Use JavaScript to check if the search box for recipe search is > > empty before querying the database. This will prevent the "502 > > Bad Gateway" error that occurs when the query takes too long due > > to the large list of recipes. Since there are so many recipes > > spread across the layers in the OE index, there's no point in > > allowing a user to search without a keyword in order to browse > > the list; it simply isn't digestible as a whole. > > > > Add a browse button for the Machines, Classes, and Distros pages. > > There are reasons to view all of the recipes, machines, classes, distros, > etc. (Not necessarily good reasons, but I know people do it.) > > If the query is too long, it would be better to figure out a way to get a > partial response and formulate the first page based on partial responses... > having a multipage response that the user can look at. I'm willing to be proven wrong, but unfortunately it looks to me like this will require major re-engineering to sort out for a fairly weak use case. If you consider this important, would you be able to look into making it work? Cheers, Paul -- Paul Eggleton Intel Open Source Technology Centre ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [layerindex-web][patch v3 1/1] recipes.html: Require keyword for recipe search 2017-11-08 2:43 ` Paul Eggleton @ 2017-11-08 15:42 ` Mark Hatle 2017-11-08 19:08 ` Paul Eggleton 0 siblings, 1 reply; 7+ messages in thread From: Mark Hatle @ 2017-11-08 15:42 UTC (permalink / raw) To: Paul Eggleton; +Cc: yocto, Amanda Brindle On 11/7/17 8:43 PM, Paul Eggleton wrote: > On Wednesday, 8 November 2017 11:47:49 AM NZDT Mark Hatle wrote: >> On 11/7/17 4:31 PM, Amanda Brindle wrote: >>> Use JavaScript to check if the search box for recipe search is >>> empty before querying the database. This will prevent the "502 >>> Bad Gateway" error that occurs when the query takes too long due >>> to the large list of recipes. Since there are so many recipes >>> spread across the layers in the OE index, there's no point in >>> allowing a user to search without a keyword in order to browse >>> the list; it simply isn't digestible as a whole. >>> >>> Add a browse button for the Machines, Classes, and Distros pages. >> >> There are reasons to view all of the recipes, machines, classes, distros, >> etc. (Not necessarily good reasons, but I know people do it.) >> >> If the query is too long, it would be better to figure out a way to get a >> partial response and formulate the first page based on partial responses... >> having a multipage response that the user can look at. > > I'm willing to be proven wrong, but unfortunately it looks to me like this > will require major re-engineering to sort out for a fairly weak use case. If > you consider this important, would you be able to look into making it work? I suspect much of the display engine probably needs a rework as the contents of the layer index have grown. I can't promise anything, but I'll see if I can find anyone who can work on it. For recipes, the use-case is very weak. I have talked with people though who have just wanted a list of what is available (one big list). Primarily so they can compare what is available to some magic spec sheet they are looking at. For the Machines, Distros pages.. people do want a full list here, as they simply won't know what is available or what something is called. The scope of the machines and distros of course is nowhere near the scope for the full recipe list. --Mark > Cheers, > Paul > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [layerindex-web][patch v3 1/1] recipes.html: Require keyword for recipe search 2017-11-08 15:42 ` Mark Hatle @ 2017-11-08 19:08 ` Paul Eggleton 2017-11-08 19:30 ` Mark Hatle 0 siblings, 1 reply; 7+ messages in thread From: Paul Eggleton @ 2017-11-08 19:08 UTC (permalink / raw) To: Mark Hatle; +Cc: yocto, Amanda Brindle On Thursday, 9 November 2017 4:42:50 AM NZDT Mark Hatle wrote: > On 11/7/17 8:43 PM, Paul Eggleton wrote: > > On Wednesday, 8 November 2017 11:47:49 AM NZDT Mark Hatle wrote: > >> On 11/7/17 4:31 PM, Amanda Brindle wrote: > >>> Use JavaScript to check if the search box for recipe search is > >>> empty before querying the database. This will prevent the "502 > >>> Bad Gateway" error that occurs when the query takes too long due > >>> to the large list of recipes. Since there are so many recipes > >>> spread across the layers in the OE index, there's no point in > >>> allowing a user to search without a keyword in order to browse > >>> the list; it simply isn't digestible as a whole. > >>> > >>> Add a browse button for the Machines, Classes, and Distros pages. > >> > >> There are reasons to view all of the recipes, machines, classes, distros, > >> etc. (Not necessarily good reasons, but I know people do it.) > >> > >> If the query is too long, it would be better to figure out a way to get a > >> partial response and formulate the first page based on partial > >> responses... having a multipage response that the user can look at. > > > > I'm willing to be proven wrong, but unfortunately it looks to me like this > > will require major re-engineering to sort out for a fairly weak use case. > > If you consider this important, would you be able to look into making it > > work? > > I suspect much of the display engine probably needs a rework as the contents > of the layer index have grown. I can't promise anything, but I'll see if I > can find anyone who can work on it. Could be yes, thanks. > For recipes, the use-case is very weak. I have talked with people though > who have just wanted a list of what is available (one big list). Primarily > so they can compare what is available to some magic spec sheet they are > looking at. If that's the use case though we should provide a proper export capability, because comparing big lists manually 50 items at a time isn't how you'd best support that. Theoretically the API could be used there but an explicit export function would be more accessible. > For the Machines, Distros pages.. people do want a full list here, as they > simply won't know what is available or what something is called. The scope > of the machines and distros of course is nowhere near the scope for the full > recipe list. Right, and this patch doesn't remove the ability to browse those - in fact it makes it easier to do so by adding an explicit browse button, so I think we're OK there. Cheers, Paul -- Paul Eggleton Intel Open Source Technology Centre ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [layerindex-web][patch v3 1/1] recipes.html: Require keyword for recipe search 2017-11-08 19:08 ` Paul Eggleton @ 2017-11-08 19:30 ` Mark Hatle 0 siblings, 0 replies; 7+ messages in thread From: Mark Hatle @ 2017-11-08 19:30 UTC (permalink / raw) To: Paul Eggleton; +Cc: yocto, Amanda Brindle On 11/8/17 1:08 PM, Paul Eggleton wrote: > On Thursday, 9 November 2017 4:42:50 AM NZDT Mark Hatle wrote: >> On 11/7/17 8:43 PM, Paul Eggleton wrote: >>> On Wednesday, 8 November 2017 11:47:49 AM NZDT Mark Hatle wrote: >>>> On 11/7/17 4:31 PM, Amanda Brindle wrote: >>>>> Use JavaScript to check if the search box for recipe search is >>>>> empty before querying the database. This will prevent the "502 >>>>> Bad Gateway" error that occurs when the query takes too long due >>>>> to the large list of recipes. Since there are so many recipes >>>>> spread across the layers in the OE index, there's no point in >>>>> allowing a user to search without a keyword in order to browse >>>>> the list; it simply isn't digestible as a whole. >>>>> >>>>> Add a browse button for the Machines, Classes, and Distros pages. >>>> >>>> There are reasons to view all of the recipes, machines, classes, distros, >>>> etc. (Not necessarily good reasons, but I know people do it.) >>>> >>>> If the query is too long, it would be better to figure out a way to get a >>>> partial response and formulate the first page based on partial >>>> responses... having a multipage response that the user can look at. >>> >>> I'm willing to be proven wrong, but unfortunately it looks to me like this >>> will require major re-engineering to sort out for a fairly weak use case. >>> If you consider this important, would you be able to look into making it >>> work? >> >> I suspect much of the display engine probably needs a rework as the contents >> of the layer index have grown. I can't promise anything, but I'll see if I >> can find anyone who can work on it. > > Could be yes, thanks. > >> For recipes, the use-case is very weak. I have talked with people though >> who have just wanted a list of what is available (one big list). Primarily >> so they can compare what is available to some magic spec sheet they are >> looking at. > > If that's the use case though we should provide a proper export capability, > because comparing big lists manually 50 items at a time isn't how you'd best > support that. Theoretically the API could be used there but an explicit export > function would be more accessible. For my own uses, I export this using the restapi. So it can certainly be exported easily enough and processed externally. (Of course, few people know about the restapi or how to use it.) >> For the Machines, Distros pages.. people do want a full list here, as they >> simply won't know what is available or what something is called. The scope >> of the machines and distros of course is nowhere near the scope for the full >> recipe list. > > Right, and this patch doesn't remove the ability to browse those - in fact it > makes it easier to do so by adding an explicit browse button, so I think we're > OK there. I misunderstood that. I though the machine/distro would be restricted like recipes. > Cheers, > Paul > ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2017-11-08 19:30 UTC | newest] Thread overview: 7+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2017-11-07 22:28 [layerindex-web][patch v3 0/1] recipes.html: Require keyword for recipe search Amanda Brindle 2017-11-07 22:31 ` [layerindex-web][patch v3 1/1] " Amanda Brindle 2017-11-07 22:47 ` Mark Hatle 2017-11-08 2:43 ` Paul Eggleton 2017-11-08 15:42 ` Mark Hatle 2017-11-08 19:08 ` Paul Eggleton 2017-11-08 19:30 ` Mark Hatle
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.