git-ls-files(1) avoiding symlinks

git-ls-files(1) avoiding symlinks

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 1822 bytes --]

Hi!

Yesterday, we were discussing alternative methods for grepping source
code.  Here's a list of similar alternatives:

	$ find . -type f | xargs grep ARRAY_END;
	$ git ls-files | xargs grep ARRAY_END;
	$ git grep ARRAY_END;

The version using find(1) has some problems: it is slow, especially
after having built a kernel, when the directory is full of temporary
objects.

The version using git-grep(1) without IPC is the fastest, and doesn't
have any problems, but it is limited to just grep(1).  If we want to use
sed(1), or any other programs, we're out of luck.

A common program I want to use is grepc(1), which searches source code.
See for example this code for finding the creat(2) system call:

	$ git ls-files | grep '\.[ch]$' | xargs grepc creat;
	fs/open.c:SYSCALL_DEFINE2(creat, const char __user *, pathname, umode_t, mode)
	{
		int flags = O_CREAT | O_WRONLY | O_TRUNC;

		if (force_o_largefile())
			flags |= O_LARGEFILE;
		return do_sys_open(AT_FDCWD, pathname, flags, mode);
	}
	include/linux/syscalls.h:asmlinkage long sys_creat(const char __user *pathname, umode_t mode);
	tools/include/nolibc/fcntl.h:static __attribute__((unused))
	int creat(const char *path, mode_t mode)
	{
		return open(path, O_CREAT | O_WRONLY | O_TRUNC, mode);
	}

A problem with git-ls-files(1) is that it doesn't have a way to filter
files by type.  That is, it will list symlinks, which will affect the
search.  With find(1), I can control that with '-type f', which makes
sure I only search in regular files.  I'd like to have some feature in
git-ls-files(1) that allows me to print files of the specified type.
Since git(1) knows the type of the files, this should be doable without
much overhead, right?


Have a lovely day!
Alex

-- 
<https://www.alejandro-colomar.es>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: git-ls-files(1) avoiding symlinks

[D. Ben Knoble]

On Fri, Nov 6, 2026 at 8:35 AM Alejandro Colomar <alx+git@kernel.org> wrote:
>
> Hi!
>
> Yesterday, we were discussing alternative methods for grepping source
> code.  Here's a list of similar alternatives:
>
>         $ find . -type f | xargs grep ARRAY_END;
>         $ git ls-files | xargs grep ARRAY_END;
>         $ git grep ARRAY_END;
>
> The version using find(1) has some problems: it is slow, especially
> after having built a kernel, when the directory is full of temporary
> objects.
>
> The version using git-grep(1) without IPC is the fastest, and doesn't
> have any problems, but it is limited to just grep(1).  If we want to use
> sed(1), or any other programs, we're out of luck.
>
> A common program I want to use is grepc(1), which searches source code.
> See for example this code for finding the creat(2) system call:
>
>         $ git ls-files | grep '\.[ch]$' | xargs grepc creat;
>         fs/open.c:SYSCALL_DEFINE2(creat, const char __user *, pathname, umode_t, mode)
>         {
>                 int flags = O_CREAT | O_WRONLY | O_TRUNC;
>
>                 if (force_o_largefile())
>                         flags |= O_LARGEFILE;
>                 return do_sys_open(AT_FDCWD, pathname, flags, mode);
>         }
>         include/linux/syscalls.h:asmlinkage long sys_creat(const char __user *pathname, umode_t mode);
>         tools/include/nolibc/fcntl.h:static __attribute__((unused))
>         int creat(const char *path, mode_t mode)
>         {
>                 return open(path, O_CREAT | O_WRONLY | O_TRUNC, mode);
>         }
>
> A problem with git-ls-files(1) is that it doesn't have a way to filter
> files by type.  That is, it will list symlinks, which will affect the
> search.  With find(1), I can control that with '-type f', which makes
> sure I only search in regular files.  I'd like to have some feature in
> git-ls-files(1) that allows me to print files of the specified type.
> Since git(1) knows the type of the files, this should be doable without
> much overhead, right?

Perhaps something with "git ls-files --format='%(objectmode) %(path)'
-z"? Of course, your examples already omit -z, so if you can assume
that it might be simpler.

-- 
D. Ben Knoble

Re: git-ls-files(1) avoiding symlinks

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 1442 bytes --]

Hi Ben,

> Date: 2026-11-06 09:15:22-0500
> From: "D. Ben Knoble" <ben.knoble@gmail.com>
>
[...]
> > A problem with git-ls-files(1) is that it doesn't have a way to filter
> > files by type.  That is, it will list symlinks, which will affect the
> > search.  With find(1), I can control that with '-type f', which makes
> > sure I only search in regular files.  I'd like to have some feature in
> > git-ls-files(1) that allows me to print files of the specified type.
> > Since git(1) knows the type of the files, this should be doable without
> > much overhead, right?
> 
> Perhaps something with "git ls-files --format='%(objectmode) %(path)'
> -z"? Of course, your examples already omit -z, so if you can assume
> that it might be simpler.

Hmmm, that could work.  Thanks!

	$ git ls-files --format='%(objectmode) %(path)' \
	| grep -v ^120000 \
	| sed 's/^.* //' \
	| grep '\.[ch]$' \
	| xargs grepc creat;

With -z it becomes a bit tedious:

	$ git ls-files -z --format='%(objectmode) %(path)' \
	| grep -zv ^120000 \
	| sed -z 's/^.* //' \
	| grep -z '\.[ch]$' \
	| xargs -0 grepc creat;

But, indeed, I don't care about files with spaces often, because I don't
create them in my stuff.

I expect git-ls-files(1) could make this much simpler, though:

	$ git ls-files --type f \
	| grep '\.[ch]$' \
	| xargs grepc creat;


Have a lovely day!
Alex

-- 
<https://www.alejandro-colomar.es>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: git-ls-files(1) avoiding symlinks

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 1260 bytes --]

Hi Ben,

> Date: 2026-11-06 12:34:20-0500
> From: Ben Knoble <ben.knoble@gmail.com>
>
[...]
> > Hmmm, that could work.  Thanks!
> > 
> >    $ git ls-files --format='%(objectmode) %(path)' \
> >    | grep -v ^120000 \
> >    | sed 's/^.* //' \
> >    | grep '\.[ch]$' \
> >    | xargs grepc creat;
> > 
> > With -z it becomes a bit tedious:
> > 
> >    $ git ls-files -z --format='%(objectmode) %(path)' \
> >    | grep -zv ^120000 \
> >    | sed -z 's/^.* //' \
> >    | grep -z '\.[ch]$' \
> >    | xargs -0 grepc creat;
> 
> The filtering for C files can also be done by Git, I believe:
> 
>     git ls-files … '*.[ch]'

I prefer regexes over globs; they're more powerful, and they're hot in
brain memory.  :)

> > But, indeed, I don't care about files with spaces often, because I don't
> > create them in my stuff.
> > 
> > I expect git-ls-files(1) could make this much simpler, though:
> > 
> >    $ git ls-files --type f \
> >    | grep '\.[ch]$' \
> >    | xargs grepc creat;
> 
> I believe the expression is “patches [scratching your own itch are] welcome” ;) 

Nice!  If the idea is welcome, I can certainly work on a patch.  :)


Have a lovely night!
Alex

-- 
<https://www.alejandro-colomar.es>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: git-ls-files(1) avoiding symlinks

[Junio C Hamano]

Alejandro Colomar <alx+git@kernel.org> writes:

>> > With -z it becomes a bit tedious:
>> > 
>> >    $ git ls-files -z --format='%(objectmode) %(path)' \
>> >    | grep -zv ^120000 \
>> >    | sed -z 's/^.* //' \
>> >    | grep -z '\.[ch]$' \
>> >    | xargs -0 grepc creat;

Shouldn't the grep|sed|grep be doable with a single sed invocation?

>> > I expect git-ls-files(1) could make this much simpler, though:
>> > 
>> >    $ git ls-files --type f \
>> >    | grep '\.[ch]$' \
>> >    | xargs grepc creat;

Not really.  Why is "--type" have to be so special?  What would the
next person ask for?  --size?

The "ls-files" plumbing command is designed as a component to build
your own pipeline with.  Don't throw in random unrelated features to
it.  In the longer run, it is not a good substitute for improving
the skill to construct a good pipeline.

Learn the tools available to you that you already have heard of, so
that you can make better uses of them (in this case "sed").  If you
squash the grep|sed|grep pipeline into a single sed

    git ls-files -z -s "*.[ch]" |
    sed -z -n -e '/^120000/d' -e '/^.* 0	/s///p' |
    xargs -r -0 ls -ld

the result does not look too bad.

One unrelated tip for better readability is to end, not begin, your
lines with the "|" pipe characters.  That way you do not have to
sprinkle so many unsightly backslashes ;-).

HTH.

Re: git-ls-files(1) avoiding symlinks

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 3284 bytes --]

Hi Junio,

> Date: 2026-11-06 12:16:48-0800
> From: Junio C Hamano <gitster@pobox.com>
>
> Alejandro Colomar <alx+git@kernel.org> writes:
> 
> >> > With -z it becomes a bit tedious:
> >> > 
> >> >    $ git ls-files -z --format='%(objectmode) %(path)' \
> >> >    | grep -zv ^120000 \
> >> >    | sed -z 's/^.* //' \
> >> >    | grep -z '\.[ch]$' \
> >> >    | xargs -0 grepc creat;
> 
> Shouldn't the grep|sed|grep be doable with a single sed invocation?
> 
> >> > I expect git-ls-files(1) could make this much simpler, though:
> >> > 
> >> >    $ git ls-files --type f \
> >> >    | grep '\.[ch]$' \
> >> >    | xargs grepc creat;
> 
> Not really.  Why is "--type" have to be so special?  What would the
> next person ask for?  --size?

Indeed.  While looking at the implementation of git-ls-files(1), I
realized it wasn't easy to decide how to implement this, as it has other
types (objecttype, which is blob/commit/...).  I thought perhaps
--objectmode, but then I'm not inderested in searching specific modes,
but rather filtering out some.  Perhaps in this case it's more
appropriate to have my own local command 'git-find' that does what
I want.

I've ended up doing this, and now don't think a flag in git-ls-files(1)
would make much sense.

	$ cat $(which git-find)
	#!/bin/bash
	# Copyright 2026, Alejandro Colomar <alx@kernel.org>
	# SPDX-License-Identifier: GPL-3.0-or-later

	set -Eeufo pipefail;

	git ls-files --format='%(objectmode) %(path)' \
	| grep -v '^120000' \
	| sed 's/^.* //';

And will eventually add a -z flag for this command when I find I need
it.

> The "ls-files" plumbing command is designed as a component to build
> your own pipeline with.  Don't throw in random unrelated features to
> it.  In the longer run, it is not a good substitute for improving
> the skill to construct a good pipeline.

Indeed.

> Learn the tools available to you that you already have heard of, so
> that you can make better uses of them (in this case "sed").  If you
> squash the grep|sed|grep pipeline into a single sed
> 
>     git ls-files -z -s "*.[ch]" |
>     sed -z -n -e '/^120000/d' -e '/^.* 0	/s///p' |
>     xargs -r -0 ls -ld
> 
> the result does not look too bad.

I find it simpler to have a simple grep -v rejecting the lines I don't
want, and then sed(1) getting rid of the stuff.  The command above seems
interesting for performance, but compared to the tool using the file
names, that's probably negligible.

About git-ls-files(1)'s -s, I think it's easier to parse with the
%(objectmode) in the format, as I don't need all the other information
printed by -s.

> 
> One unrelated tip for better readability is to end, not begin, your
> lines with the "|" pipe characters.  That way you do not have to
> sprinkle so many unsightly backslashes ;-).

I like to have them at the front of the line because it shows more
clearly whether I'm piping a command or starting a new pipeline.  I got
used to the backslashes.  After all, the first few characters of a line
are all aligned, and the most significant ones.  Stuff to the right I
tend to ignore it as noise.  :)

> 
> HTH.

Yup!  Thanks!


Have a lovely night!
Alex

-- 
<https://www.alejandro-colomar.es>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]