[PATCH] Limited git-gui to displaying 5000 new files

[PATCH] Limited git-gui to displaying 5000 new files

[Dan Zwell]

When there is a large number of new or modified files,
"display_all_files" takes a long time, and git-gui appears to
hang. Limit the display to 5000 files, by default. This number
is configurable as gui.maxfilesdisplayed.

Show a warning if the list of files is truncated.

Signed-off-by: Dan Zwell <dzwell@zwell.net>
---
 git-gui.sh |   17 ++++++++++++++++-
 1 files changed, 16 insertions(+), 1 deletions(-)

diff --git a/git-gui.sh b/git-gui.sh
index 14b92ba..5a20923 100755
--- a/git-gui.sh
+++ b/git-gui.sh
@@ -745,6 +745,8 @@ set default_config(gui.newbranchtemplate) {}
 set default_config(gui.spellingdictionary) {}
 set default_config(gui.fontui) [font configure font_ui]
 set default_config(gui.fontdiff) [font configure font_diff]
+# TODO: this option should be added to the git-config documentation
+set default_config(gui.maxfilesdisplayed) 5000
 set font_descs {
 	{fontui   font_ui   {mc "Main Font"}}
 	{fontdiff font_diff {mc "Diff/Console Font"}}
@@ -1702,6 +1704,8 @@ proc display_all_files {} {
 	global ui_index ui_workdir
 	global file_states file_lists
 	global last_clicked
+	global files_warning
+	global default_config
 
 	$ui_index conf -state normal
 	$ui_workdir conf -state normal
@@ -1713,7 +1717,18 @@ proc display_all_files {} {
 	set file_lists($ui_index) [list]
 	set file_lists($ui_workdir) [list]
 
-	foreach path [lsort [array names file_states]] {
+	set to_display [lsort [array names file_states]]
+	set display_limit $default_config(gui.maxfilesdisplayed)
+	if {[llength $to_display] > $display_limit} {
+		if {![info exists files_warning] || !$files_warning} {
+			set warning "Displaying only $display_limit of "
+			append warning "[llength $to_display] files."
+			info_popup [mc $warning]
+			set files_warning 1
+		}
+		set to_display [lrange $to_display 0 [expr {$display_limit-1}]]
+	}
+	foreach path $to_display {
 		set s $file_states($path)
 		set m [lindex $s 0]
 		set icon_name [lindex $s 1]
-- 
1.6.3.3

Re: [PATCH] Limited git-gui to displaying 5000 new files

[Shawn O. Pearce]

Dan Zwell <dzwell@gmail.com> wrote:
> When there is a large number of new or modified files,
> "display_all_files" takes a long time, and git-gui appears to
> hang. Limit the display to 5000 files, by default. This number
> is configurable as gui.maxfilesdisplayed.
>
> Show a warning if the list of files is truncated.

> @@ -1713,7 +1717,18 @@ proc display_all_files {} {
> 	set file_lists($ui_index) [list]
> 	set file_lists($ui_workdir) [list]
>
> -	foreach path [lsort [array names file_states]] {
> +	set to_display [lsort [array names file_states]]
> +	set display_limit $default_config(gui.maxfilesdisplayed)

This should use [get_config gui.maxfilesdisplayed] so that the
user can actually set this property in a configuration file and
have git-gui honor it.  Reading from $default_config means you are
only looking at the hardcoded value you set in git-gui.sh.

> +	if {[llength $to_display] > $display_limit} {
> +		if {![info exists files_warning] || !$files_warning} {

Wouldn't it be easier to just set files_warning to 0 at the start
of the script, so that you don't need to do this info exists test?

> +			set warning "Displaying only $display_limit of "
> +			append warning "[llength $to_display] files."
> +			info_popup [mc $warning]

This needs to be in the translated strings.

-- 
Shawn.

Re: [PATCH] Limited git-gui to displaying 5000 new files

[Alex Riesen]

On Mon, Aug 10, 2009 at 17:38, Shawn O. Pearce<spearce@spearce.org> wrote:
>> +                     set warning "Displaying only $display_limit of "
>> +                     append warning "[llength $to_display] files."
>> +                     info_popup [mc $warning]
>
> This needs to be in the translated strings.
>

Will do, as soon as this hits git://repo.or.cz/git-gui.git.
Or should I have looked at the internationalization repo
git://repo.or.cz/git-gui/git-gui-i18n.git?

Re: [PATCH] Limited git-gui to displaying 5000 new files

[Shawn O. Pearce]

Alex Riesen <raa.lkml@gmail.com> wrote:
> On Mon, Aug 10, 2009 at 17:38, Shawn O. Pearce<spearce@spearce.org> wrote:
> >> + ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? set warning "Displaying only $display_limit of "
> >> + ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? append warning "[llength $to_display] files."
> >> + ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? info_popup [mc $warning]
> >
> > This needs to be in the translated strings.
> 
> Will do, as soon as this hits git://repo.or.cz/git-gui.git.
> Or should I have looked at the internationalization repo
> git://repo.or.cz/git-gui/git-gui-i18n.git?

No, I meant that the patch needs to use [mc] or whatever it is
to enable the string to be localized.  After its marked with the
necessary code change, I can apply the patch, regenerate the .pot,
and let translators update their .po when they have time.

-- 
Shawn.

Re: [PATCH] Limited git-gui to displaying 5000 new files

[Dan Zwell]

Shawn O. Pearce wrote:
> Alex Riesen <raa.lkml@gmail.com> wrote:
>> On Mon, Aug 10, 2009 at 17:38, Shawn O. Pearce<spearce@spearce.org> wrote:
>>>> + ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? set warning "Displaying only $display_limit of "
>>>> + ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? append warning "[llength $to_display] files."
>>>> + ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? info_popup [mc $warning]
>>> This needs to be in the translated strings.
>> Will do, as soon as this hits git://repo.or.cz/git-gui.git.
>> Or should I have looked at the internationalization repo
>> git://repo.or.cz/git-gui/git-gui-i18n.git?
> 
> No, I meant that the patch needs to use [mc] or whatever it is
> to enable the string to be localized.  After its marked with the
> necessary code change, I can apply the patch, regenerate the .pot,
> and let translators update their .po when they have time.
> 

I will change it to use [mc] (and make the other changes that Shawn 
mentioned) and resubmit the patch.

-Dan

Re: [PATCH] Limit git-gui to display a maximum number of files

[Dan Zwell]

When there is a large number of new or modified files,
"display_all_files" takes a long time, and git-gui appears to
hang. This change limits the number of files that are displayed.
This limit can be set as gui.maxfilesdisplayed, and is
5000 by default.

A warning is shown when the list of files is truncated.

Signed-off-by: Dan Zwell <dzwell@zwell.net>
---
By the way, is the right way to deal with strings to be
translated? See the end of the patch.

 git-gui.sh     |   18 +++++++++++++++++-
 po/git-gui.pot |    5 +++++
 2 files changed, 22 insertions(+), 1 deletions(-)

diff --git a/git-gui.sh b/git-gui.sh
index 3c0ce26..a4dde9e 100755
--- a/git-gui.sh
+++ b/git-gui.sh
@@ -745,6 +745,8 @@ set default_config(gui.newbranchtemplate) {}
 set default_config(gui.spellingdictionary) {}
 set default_config(gui.fontui) [font configure font_ui]
 set default_config(gui.fontdiff) [font configure font_diff]
+# TODO: this option should be added to the git-config documentation
+set default_config(gui.maxfilesdisplayed) 5000
 set font_descs {
 	{fontui   font_ui   {mc "Main Font"}}
 	{fontdiff font_diff {mc "Diff/Console Font"}}
@@ -1698,10 +1700,12 @@ proc display_all_files_helper {w path icon_name m} {
 	$w insert end "[escape_path $path]\n"
 }
 
+set files_warning 0
 proc display_all_files {} {
 	global ui_index ui_workdir
 	global file_states file_lists
 	global last_clicked
+	global files_warning
 
 	$ui_index conf -state normal
 	$ui_workdir conf -state normal
@@ -1713,7 +1717,19 @@ proc display_all_files {} {
 	set file_lists($ui_index) [list]
 	set file_lists($ui_workdir) [list]
 
-	foreach path [lsort [array names file_states]] {
+	set to_display [lsort [array names file_states]]
+	set display_limit [get_config gui.maxfilesdisplayed]
+	if {[llength $to_display] > $display_limit} {
+		if {!$files_warning} {
+			# do not repeatedly warn:
+			set files_warning 1
+			set warning "Displaying only $display_limit of "
+			append warning "[llength $to_display] files."
+			info_popup [mc $warning]
+		}
+		set to_display [lrange $to_display 0 [expr {$display_limit-1}]]
+	}
+	foreach path $to_display {
 		set s $file_states($path)
 		set m [lindex $s 0]
 		set icon_name [lindex $s 1]
diff --git a/po/git-gui.pot b/po/git-gui.pot
index 53b7d36..fb60472 100644
--- a/po/git-gui.pot
+++ b/po/git-gui.pot
@@ -90,6 +90,11 @@ msgstr ""
 msgid "Ready."
 msgstr ""
 
+#: git-gui.sh:1725
+#, tcl-format
+msgid "Displaying only %s of %s files."
+msgstr ""
+
 #: git-gui.sh:1819
 msgid "Unmodified"
 msgstr ""
-- 
1.6.4


Shawn O. Pearce wrote:

> Dan Zwell <dzwell@gmail.com> wrote:
>   
>> When there is a large number of new or modified files,
>> "display_all_files" takes a long time, and git-gui appears to
>> hang. Limit the display to 5000 files, by default. This number
>> is configurable as gui.maxfilesdisplayed.
>>
>> Show a warning if the list of files is truncated.
>>     
>
>   
>> @@ -1713,7 +1717,18 @@ proc display_all_files {} {
>> 	set file_lists($ui_index) [list]
>> 	set file_lists($ui_workdir) [list]
>>
>> -	foreach path [lsort [array names file_states]] {
>> +	set to_display [lsort [array names file_states]]
>> +	set display_limit $default_config(gui.maxfilesdisplayed)
>>     
>
> This should use [get_config gui.maxfilesdisplayed] so that the
> user can actually set this property in a configuration file and
> have git-gui honor it.  Reading from $default_config means you are
> only looking at the hardcoded value you set in git-gui.sh.
>
>   
>> +	if {[llength $to_display] > $display_limit} {
>> +		if {![info exists files_warning] || !$files_warning} {
>>     
>
> Wouldn't it be easier to just set files_warning to 0 at the start
> of the script, so that you don't need to do this info exists test?
>
>   
>> +			set warning "Displaying only $display_limit of "
>> +			append warning "[llength $to_display] files."
>> +			info_popup [mc $warning]
>>     
>
> This needs to be in the translated strings.
>
>   

Re: [PATCH] Limit git-gui to display a maximum number of files

[Shawn O. Pearce]

Dan Zwell <dzwell@zwell.net> wrote:
> When there is a large number of new or modified files,
> "display_all_files" takes a long time, and git-gui appears to
> hang. This change limits the number of files that are displayed.
> This limit can be set as gui.maxfilesdisplayed, and is
> 5000 by default.
>
> A warning is shown when the list of files is truncated.
>
> Signed-off-by: Dan Zwell <dzwell@zwell.net>
> ---
> By the way, is the right way to deal with strings to be
> translated? See the end of the patch.

No.

> +			set warning "Displaying only $display_limit of "
> +			append warning "[llength $to_display] files."
> +			info_popup [mc $warning]

This should be:

info_popup [mc "Displaying only %s of %s files." $display_limit [llength $to_display]]

> +msgid "Displaying only %s of %s files."
> +msgstr ""

So that then the placeholders are available here...

-- 
Shawn.

Re: [PATCH] Limit git-gui to display a maximum number of files

[Dan Zwell]

When there is a large number of new or modified files,
"display_all_files" takes a long time, and git-gui appears to
hang. This change limits the number of files that are displayed.
This limit can be set as gui.maxfilesdisplayed, and is
5000 by default.

A warning is shown when the list of files is truncated.

Signed-off-by: Dan Zwell <dzwell@zwell.net>
---
 git-gui.sh     |   17 ++++++++++++++++-
 po/git-gui.pot |    5 +++++
 2 files changed, 21 insertions(+), 1 deletions(-)

diff --git a/git-gui.sh b/git-gui.sh
index 3c0ce26..eae1f81 100755
--- a/git-gui.sh
+++ b/git-gui.sh
@@ -745,6 +745,8 @@ set default_config(gui.newbranchtemplate) {}
 set default_config(gui.spellingdictionary) {}
 set default_config(gui.fontui) [font configure font_ui]
 set default_config(gui.fontdiff) [font configure font_diff]
+# TODO: this option should be added to the git-config documentation
+set default_config(gui.maxfilesdisplayed) 5000
 set font_descs {
 	{fontui   font_ui   {mc "Main Font"}}
 	{fontdiff font_diff {mc "Diff/Console Font"}}
@@ -1698,10 +1700,12 @@ proc display_all_files_helper {w path icon_name m} {
 	$w insert end "[escape_path $path]\n"
 }
 
+set files_warning 0
 proc display_all_files {} {
 	global ui_index ui_workdir
 	global file_states file_lists
 	global last_clicked
+	global files_warning
 
 	$ui_index conf -state normal
 	$ui_workdir conf -state normal
@@ -1713,7 +1717,18 @@ proc display_all_files {} {
 	set file_lists($ui_index) [list]
 	set file_lists($ui_workdir) [list]
 
-	foreach path [lsort [array names file_states]] {
+	set to_display [lsort [array names file_states]]
+	set display_limit [get_config gui.maxfilesdisplayed]
+	if {[llength $to_display] > $display_limit} {
+		if {!$files_warning} {
+			# do not repeatedly warn:
+			set files_warning 1
+			info_popup [mc "Displaying only %s of %s files." \
+				$display_limit [llength $to_display]]
+		}
+		set to_display [lrange $to_display 0 [expr {$display_limit-1}]]
+	}
+	foreach path $to_display {
 		set s $file_states($path)
 		set m [lindex $s 0]
 		set icon_name [lindex $s 1]
diff --git a/po/git-gui.pot b/po/git-gui.pot
index 53b7d36..074582d 100644
--- a/po/git-gui.pot
+++ b/po/git-gui.pot
@@ -90,6 +90,11 @@ msgstr ""
 msgid "Ready."
 msgstr ""
 
+#: git-gui.sh:1726
+#, tcl-format
+msgid "Displaying only %s of %s files."
+msgstr ""
+
 #: git-gui.sh:1819
 msgid "Unmodified"
 msgstr ""
-- 
1.6.4

Re: [PATCH] Limit git-gui to display a maximum number of files

[Shawn O. Pearce]

Dan Zwell <dzwell@zwell.net> wrote:
> When there is a large number of new or modified files,
> "display_all_files" takes a long time, and git-gui appears to
> hang. This change limits the number of files that are displayed.
> This limit can be set as gui.maxfilesdisplayed, and is
> 5000 by default.
>
> A warning is shown when the list of files is truncated.

Thanks, applied.

-- 
Shawn.

[PATCH] git-gui: Update russian translation

[Alex Riesen]

Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
---

Shawn O. Pearce, Wed, Aug 12, 2009 16:43:45 +0200:
> Dan Zwell <dzwell@zwell.net> wrote:
> > When there is a large number of new or modified files,
> > "display_all_files" takes a long time, and git-gui appears to
> > hang. This change limits the number of files that are displayed.
> > This limit can be set as gui.maxfilesdisplayed, and is
> > 5000 by default.
> >
> > A warning is shown when the list of files is truncated.
> 
> Thanks, applied.
> 

And the updated translation.

But now, when I really think about the change, it looks useless.
What has the _number_ of files has to do with the files you actually
have to handle? As the sorting of the file list cannot be changed (and
it wouldn't be a big help anyway), you have no chance to get to your
file if it happens to be past the limit!

Wouldn't a pathname/glob filter in the command-line (or file/path
selection dialog) to limit the scope be more appropriate and useful?
And have the file list reading to happen in background, as gitk does?

 po/ru.po |   28 ++++++++++++++++------------
 1 files changed, 16 insertions(+), 12 deletions(-)

diff --git a/po/ru.po b/po/ru.po
index 0ffc4a4..364c074 100644
--- a/po/ru.po
+++ b/po/ru.po
@@ -90,12 +90,18 @@ msgstr "Вызов программы поддержки репозитория
 
 #: git-gui.sh:1384
 msgid "Commit declined by prepare-commit-msg hook."
-msgstr "Сохранение прервано программой поддержки репозитория prepare-commit-msg"
+msgstr ""
+"Сохранение прервано программой поддержки репозитория prepare-commit-msg"
 
 #: git-gui.sh:1542 lib/browser.tcl:246
 msgid "Ready."
 msgstr "Готово."
 
+#: git-gui.sh:1726
+#, tcl-format
+msgid "Displaying only %s of %s files."
+msgstr "Показано %s из %s файлов."
+
 #: git-gui.sh:1819
 msgid "Unmodified"
 msgstr "Не изменено"
@@ -1297,8 +1303,8 @@ msgid ""
 msgstr ""
 "Невозможно исправить состояние во время операции слияния.\n"
 "\n"
-"Текущее слияние не завершено. Невозможно исправить предыдущее "
-"сохраненное состояние, не прерывая эту операцию.\n"
+"Текущее слияние не завершено. Невозможно исправить предыдущее сохраненное "
+"состояние, не прерывая эту операцию.\n"
 
 #: lib/commit.tcl:48
 msgid "Error loading commit data for amend:"
@@ -1723,8 +1729,7 @@ msgid ""
 msgstr ""
 "Невозможно выполнить слияние во время исправления.\n"
 "\n"
-"Завершите исправление данного состояния перед выполнением операции "
-"слияния.\n"
+"Завершите исправление данного состояния перед выполнением операции слияния.\n"
 
 #: lib/merge.tcl:27
 msgid ""
@@ -1888,8 +1893,8 @@ msgstr ""
 #, tcl-format
 msgid "File %s seems to have unresolved conflicts, still stage?"
 msgstr ""
-"Файл %s кажется содержит необработаные конфликты. "
-"Продолжить подготовку к сохранению?"
+"Файл %s кажется содержит необработаные конфликты. Продолжить подготовку к "
+"сохранению?"
 
 #: lib/mergetool.tcl:60
 #, tcl-format
@@ -2213,8 +2218,8 @@ msgid ""
 "One or more of the merge tests failed because you have not fetched the "
 "necessary commits.  Try fetching from %s first."
 msgstr ""
-"Некоторые тесты на слияние не прошли, потому что Вы не "
-"получили необходимые состояния. Попытайтесь получить их из %s."
+"Некоторые тесты на слияние не прошли, потому что Вы не получили необходимые "
+"состояния. Попытайтесь получить их из %s."
 
 #: lib/remote_branch_delete.tcl:207
 msgid "Please select one or more branches to delete."
@@ -2381,8 +2386,8 @@ msgstr "Выполнение: %s"
 
 #: lib/tools.tcl:149
 #, tcl-format
-msgid "Tool completed succesfully: %s"
-msgstr "Программа %s успешно завершилась."
+msgid "Tool completed successfully: %s"
+msgstr "Программа %s завершилась успешно."
 
 #: lib/tools.tcl:151
 #, tcl-format
@@ -2538,4 +2543,3 @@ msgstr "Использовать thin pack (для медленных сетев
 #: lib/transport.tcl:179
 msgid "Include tags"
 msgstr "Передать метки"
-
-- 
1.6.4.140.gc6dfd

Re: [PATCH] git-gui: Update russian translation

[Shawn O. Pearce]

Alex Riesen <raa.lkml@gmail.com> wrote:
> Shawn O. Pearce, Wed, Aug 12, 2009 16:43:45 +0200:
> > Dan Zwell <dzwell@zwell.net> wrote:
> > > When there is a large number of new or modified files,
> > > "display_all_files" takes a long time, and git-gui appears to
> > > hang. This change limits the number of files that are displayed.
> > > This limit can be set as gui.maxfilesdisplayed, and is
> > > 5000 by default.
> > >
> > > A warning is shown when the list of files is truncated.
> > 
> > Thanks, applied.
> 
> But now, when I really think about the change, it looks useless.
> What has the _number_ of files has to do with the files you actually
> have to handle? As the sorting of the file list cannot be changed (and
> it wouldn't be a big help anyway), you have no chance to get to your
> file if it happens to be past the limit!
> 
> Wouldn't a pathname/glob filter in the command-line (or file/path
> selection dialog) to limit the scope be more appropriate and useful?
> And have the file list reading to happen in background, as gitk does?

Good point.  I suspect the problem wasn't so much with Tcl doing the
list processing as it was with Tk actually creating the underlying
icons and stuff for each file name.  But with the list clipped,
you are right, you are basically SOL.  You can't do much beyond
dropping back to the CLI and using the CLI tools.

IMHO, if we aren't going to handle 20k file names, we should at least
punt and tell the user we aren't going to handle 20k file names,
rather than just play Outlook wannabe and lockup the entire UI until
the user gets bored and kill -9's us.  So this patch is better than
nothing, it at least lets the user know we have given up on them.

-- 
Shawn.

Re: [PATCH] git-gui: Update russian translation

[Dan Zwell]

> Alex Riesen <raa.lkml@gmail.com> wrote:
>> But now, when I really think about the change, it looks useless.
>> What has the _number_ of files has to do with the files you actually
>> have to handle? As the sorting of the file list cannot be changed (and
>> it wouldn't be a big help anyway), you have no chance to get to your
>> file if it happens to be past the limit!

> Good point.  I suspect the problem wasn't so much with Tcl doing the
> list processing as it was with Tk actually creating the underlying
> icons and stuff for each file name.  But with the list clipped,
> you are right, you are basically SOL.  You can't do much beyond
> dropping back to the CLI and using the CLI tools.
> 
> IMHO, if we aren't going to handle 20k file names, we should at least
> punt and tell the user we aren't going to handle 20k file names,
> rather than just play Outlook wannabe and lockup the entire UI until
> the user gets bored and kill -9's us.  So this patch is better than
> nothing, it at least lets the user know we have given up on them.
> 

A warning would be good, but this gives users more information. In my 
case, I only wanted to see a few files, but the rest were from a 
directory that should have been in .gitignore. If I had seen the 
filenames, I would have known which directory was the culprit. This way, 
you don't need to drop to the command line to fix the problem.

-Dan