[Qemu-devel] [PATCH] build-sys: generate .gitignore

[Qemu-devel] [PATCH] build-sys: generate .gitignore

[Marc-André Lureau]

It's too easy to forget about updating the .gitignore, so this is an
attempt at generating it instead.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 tests/Makefile.include | 23 ++++++++++++--
 tests/.gitignore       | 82 --------------------------------------------------
 2 files changed, 20 insertions(+), 85 deletions(-)
 delete mode 100644 tests/.gitignore

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 93f2ba1..339126d 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -785,6 +785,19 @@ $(patsubst %, check-%, $(check-qapi-schema-y)): check-%.json: $(SRC_PATH)/%.json
 	@diff -q $(SRC_PATH)/$*.exit $*.test.exit
 
 # Consolidated targets
+tests-cleanfiles = *.o
+tests-cleanfiles = .gitignore
+tests-cleanfiles += qht-bench$(EXESUF)
+tests-cleanfiles += qapi-schema/*.test.*
+tests-cleanfiles += test-qapi-event.[ch]
+tests-cleanfiles += test-qapi-types.[ch]
+tests-cleanfiles += test-qapi-visit.[ch]
+tests-cleanfiles += test-qmp-introspect.[ch]
+tests-cleanfiles += test-qmp-commands.h
+tests-cleanfiles += test-qmp-marshal.c
+tests-cleanfiles += $(subst tests/,,$(check-unit-y))
+tests-cleanfiles += $(subst tests/,,$(check-qtest-y))
+tests-cleanfiles += qemu-iotests/socket_scm_helper$(EXESUF)
 
 .PHONY: check-qapi-schema check-qtest check-unit check check-clean
 check-qapi-schema: $(patsubst %,check-%, $(check-qapi-schema-y))
@@ -794,14 +807,18 @@ check-block: $(patsubst %,check-%, $(check-block-y))
 check: check-qapi-schema check-unit check-qtest
 check-clean:
 	$(MAKE) -C tests/tcg clean
-	rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
-	rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y))
-
+	(cd tests && echo rm -rf $(tests-cleanfiles))
 clean: check-clean
 
 # Build the help program automatically
 
 all: $(QEMU_IOTESTS_HELPERS-y)
 
+ifneq ($(filter-out $(UNCHECKED_GOALS),$(MAKECMDGOALS)),$(if $(MAKECMDGOALS),,fail))
+$(SRC_PATH)/tests/.gitignore:
+	echo "$(tests-cleanfiles)" | xargs -n1 | sort > $@
+Makefile: $(SRC_PATH)/tests/.gitignore
+endif
+
 -include $(wildcard tests/*.d)
 -include $(wildcard tests/libqos/*.d)
diff --git a/tests/.gitignore b/tests/.gitignore
deleted file mode 100644
index b4a9cfc..0000000
--- a/tests/.gitignore
+++ /dev/null
@@ -1,82 +0,0 @@
-check-qdict
-check-qfloat
-check-qint
-check-qjson
-check-qlist
-check-qnull
-check-qstring
-check-qom-interface
-check-qom-proplist
-qht-bench
-rcutorture
-test-aio
-test-base64
-test-bitops
-test-blockjob-txn
-test-clone-visitor
-test-coroutine
-test-crypto-afsplit
-test-crypto-block
-test-crypto-cipher
-test-crypto-hash
-test-crypto-ivgen
-test-crypto-pbkdf
-test-crypto-secret
-test-crypto-tlscredsx509
-test-crypto-tlscredsx509-work/
-test-crypto-tlscredsx509-certs/
-test-crypto-tlssession
-test-crypto-tlssession-work/
-test-crypto-tlssession-client/
-test-crypto-tlssession-server/
-test-crypto-xts
-test-cutils
-test-hbitmap
-test-int128
-test-iov
-test-io-channel-buffer
-test-io-channel-command
-test-io-channel-command.fifo
-test-io-channel-file
-test-io-channel-file.txt
-test-io-channel-socket
-test-io-channel-tls
-test-io-task
-test-logging
-test-mul64
-test-opts-visitor
-test-qapi-event.[ch]
-test-qapi-types.[ch]
-test-qapi-visit.[ch]
-test-qdev-global-props
-test-qemu-opts
-test-qdist
-test-qga
-test-qht
-test-qht-par
-test-qmp-commands
-test-qmp-commands.h
-test-qmp-event
-test-qmp-input-strict
-test-qmp-input-visitor
-test-qmp-introspect.[ch]
-test-qmp-marshal.c
-test-qmp-output-visitor
-test-rcu-list
-test-replication
-test-rfifolock
-test-string-input-visitor
-test-string-output-visitor
-test-thread-pool
-test-throttle
-test-timed-average
-test-visitor-serialization
-test-vmstate
-test-write-threshold
-test-x86-cpuid
-test-xbzrle
-test-netfilter
-test-filter-mirror
-test-filter-redirector
-*-test
-qapi-schema/*.test.*
-- 
2.10.0

Re: [Qemu-devel] [PATCH] build-sys: generate .gitignore

[Marc-André Lureau]

Hi

----- Original Message -----
> It's too easy to forget about updating the .gitignore, so this is an
> attempt at generating it instead.
> 
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>

I forgot to mention this is only for tests/.gitignore actually here.

> ---
>  tests/Makefile.include | 23 ++++++++++++--
>  tests/.gitignore       | 82
>  --------------------------------------------------
>  2 files changed, 20 insertions(+), 85 deletions(-)
>  delete mode 100644 tests/.gitignore
> 
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 93f2ba1..339126d 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -785,6 +785,19 @@ $(patsubst %, check-%, $(check-qapi-schema-y)):
> check-%.json: $(SRC_PATH)/%.json
>  	@diff -q $(SRC_PATH)/$*.exit $*.test.exit
>  
>  # Consolidated targets
> +tests-cleanfiles = *.o
> +tests-cleanfiles = .gitignore
> +tests-cleanfiles += qht-bench$(EXESUF)
> +tests-cleanfiles += qapi-schema/*.test.*
> +tests-cleanfiles += test-qapi-event.[ch]
> +tests-cleanfiles += test-qapi-types.[ch]
> +tests-cleanfiles += test-qapi-visit.[ch]
> +tests-cleanfiles += test-qmp-introspect.[ch]
> +tests-cleanfiles += test-qmp-commands.h
> +tests-cleanfiles += test-qmp-marshal.c
> +tests-cleanfiles += $(subst tests/,,$(check-unit-y))
> +tests-cleanfiles += $(subst tests/,,$(check-qtest-y))
> +tests-cleanfiles += qemu-iotests/socket_scm_helper$(EXESUF)
>  
>  .PHONY: check-qapi-schema check-qtest check-unit check check-clean
>  check-qapi-schema: $(patsubst %,check-%, $(check-qapi-schema-y))
> @@ -794,14 +807,18 @@ check-block: $(patsubst %,check-%, $(check-block-y))
>  check: check-qapi-schema check-unit check-qtest
>  check-clean:
>  	$(MAKE) -C tests/tcg clean
> -	rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
> -	rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST),
> $(check-qtest-$(target)-y)) $(check-qtest-generic-y))
> -
> +	(cd tests && echo rm -rf $(tests-cleanfiles))
>  clean: check-clean
>  
>  # Build the help program automatically
>  
>  all: $(QEMU_IOTESTS_HELPERS-y)
>  
> +ifneq ($(filter-out $(UNCHECKED_GOALS),$(MAKECMDGOALS)),$(if
> $(MAKECMDGOALS),,fail))
> +$(SRC_PATH)/tests/.gitignore:
> +	echo "$(tests-cleanfiles)" | xargs -n1 | sort > $@
> +Makefile: $(SRC_PATH)/tests/.gitignore
> +endif
> +
>  -include $(wildcard tests/*.d)
>  -include $(wildcard tests/libqos/*.d)
> diff --git a/tests/.gitignore b/tests/.gitignore
> deleted file mode 100644
> index b4a9cfc..0000000
> --- a/tests/.gitignore
> +++ /dev/null
> @@ -1,82 +0,0 @@
> -check-qdict
> -check-qfloat
> -check-qint
> -check-qjson
> -check-qlist
> -check-qnull
> -check-qstring
> -check-qom-interface
> -check-qom-proplist
> -qht-bench
> -rcutorture
> -test-aio
> -test-base64
> -test-bitops
> -test-blockjob-txn
> -test-clone-visitor
> -test-coroutine
> -test-crypto-afsplit
> -test-crypto-block
> -test-crypto-cipher
> -test-crypto-hash
> -test-crypto-ivgen
> -test-crypto-pbkdf
> -test-crypto-secret
> -test-crypto-tlscredsx509
> -test-crypto-tlscredsx509-work/
> -test-crypto-tlscredsx509-certs/
> -test-crypto-tlssession
> -test-crypto-tlssession-work/
> -test-crypto-tlssession-client/
> -test-crypto-tlssession-server/
> -test-crypto-xts
> -test-cutils
> -test-hbitmap
> -test-int128
> -test-iov
> -test-io-channel-buffer
> -test-io-channel-command
> -test-io-channel-command.fifo
> -test-io-channel-file
> -test-io-channel-file.txt
> -test-io-channel-socket
> -test-io-channel-tls
> -test-io-task
> -test-logging
> -test-mul64
> -test-opts-visitor
> -test-qapi-event.[ch]
> -test-qapi-types.[ch]
> -test-qapi-visit.[ch]
> -test-qdev-global-props
> -test-qemu-opts
> -test-qdist
> -test-qga
> -test-qht
> -test-qht-par
> -test-qmp-commands
> -test-qmp-commands.h
> -test-qmp-event
> -test-qmp-input-strict
> -test-qmp-input-visitor
> -test-qmp-introspect.[ch]
> -test-qmp-marshal.c
> -test-qmp-output-visitor
> -test-rcu-list
> -test-replication
> -test-rfifolock
> -test-string-input-visitor
> -test-string-output-visitor
> -test-thread-pool
> -test-throttle
> -test-timed-average
> -test-visitor-serialization
> -test-vmstate
> -test-write-threshold
> -test-x86-cpuid
> -test-xbzrle
> -test-netfilter
> -test-filter-mirror
> -test-filter-redirector
> -*-test
> -qapi-schema/*.test.*
> --
> 2.10.0
> 
> 

Re: [Qemu-devel] [PATCH] build-sys: generate .gitignore

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 3022 bytes --]

On 09/22/2016 03:19 PM, Marc-André Lureau wrote:
> It's too easy to forget about updating the .gitignore, so this is an
> attempt at generating it instead.
> 
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>  tests/Makefile.include | 23 ++++++++++++--
>  tests/.gitignore       | 82 --------------------------------------------------
>  2 files changed, 20 insertions(+), 85 deletions(-)
>  delete mode 100644 tests/.gitignore

Interesting that applying this to an incremental tree will show lots of
unignored files until the next 'make check' run.  But once that is done,
I can confirm that it did result in the same 'git status' as pre-patch.

> 
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 93f2ba1..339126d 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -785,6 +785,19 @@ $(patsubst %, check-%, $(check-qapi-schema-y)): check-%.json: $(SRC_PATH)/%.json
>  	@diff -q $(SRC_PATH)/$*.exit $*.test.exit
>  
>  # Consolidated targets
> +tests-cleanfiles = *.o
> +tests-cleanfiles = .gitignore
> +tests-cleanfiles += qht-bench$(EXESUF)
> +tests-cleanfiles += qapi-schema/*.test.*
> +tests-cleanfiles += test-qapi-event.[ch]
> +tests-cleanfiles += test-qapi-types.[ch]
> +tests-cleanfiles += test-qapi-visit.[ch]
> +tests-cleanfiles += test-qmp-introspect.[ch]
> +tests-cleanfiles += test-qmp-commands.h
> +tests-cleanfiles += test-qmp-marshal.c
> +tests-cleanfiles += $(subst tests/,,$(check-unit-y))
> +tests-cleanfiles += $(subst tests/,,$(check-qtest-y))
> +tests-cleanfiles += qemu-iotests/socket_scm_helper$(EXESUF)
>  
>  .PHONY: check-qapi-schema check-qtest check-unit check check-clean
>  check-qapi-schema: $(patsubst %,check-%, $(check-qapi-schema-y))
> @@ -794,14 +807,18 @@ check-block: $(patsubst %,check-%, $(check-block-y))
>  check: check-qapi-schema check-unit check-qtest
>  check-clean:
>  	$(MAKE) -C tests/tcg clean
> -	rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
> -	rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y))
> -
> +	(cd tests && echo rm -rf $(tests-cleanfiles))
>  clean: check-clean
>  
>  # Build the help program automatically
>  
>  all: $(QEMU_IOTESTS_HELPERS-y)
>  
> +ifneq ($(filter-out $(UNCHECKED_GOALS),$(MAKECMDGOALS)),$(if $(MAKECMDGOALS),,fail))
> +$(SRC_PATH)/tests/.gitignore:
> +	echo "$(tests-cleanfiles)" | xargs -n1 | sort > $@

This line is TOO noisy; you'll want to wrap it into a $(call
quiet-command...).

You could also do it with fewer processes and less typing, as:

printf %s\\n $(tests-cleanfiles) | sort > $@

> +Makefile: $(SRC_PATH)/tests/.gitignore
> +endif
> +
>  -include $(wildcard tests/*.d)
>  -include $(wildcard tests/libqos/*.d)

But the idea seems interesting; looking forward to v2 and comments from
anyone else.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH] build-sys: generate .gitignore

[Marc-André Lureau]

Hi

On Fri, Sep 23, 2016 at 12:38 AM Eric Blake <eblake@redhat.com> wrote:

> On 09/22/2016 03:19 PM, Marc-André Lureau wrote:
> > It's too easy to forget about updating the .gitignore, so this is an
> > attempt at generating it instead.
> >
> > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> > ---
> >  tests/Makefile.include | 23 ++++++++++++--
> >  tests/.gitignore       | 82
> --------------------------------------------------
> >  2 files changed, 20 insertions(+), 85 deletions(-)
> >  delete mode 100644 tests/.gitignore
>
> Interesting that applying this to an incremental tree will show lots of
> unignored files until the next 'make check' run.  But once that is done,
> I can confirm that it did result in the same 'git status' as pre-patch.
>
>
The next 'make' should be enough.


> >
> > diff --git a/tests/Makefile.include b/tests/Makefile.include
> > index 93f2ba1..339126d 100644
> > --- a/tests/Makefile.include
> > +++ b/tests/Makefile.include
> > @@ -785,6 +785,19 @@ $(patsubst %, check-%, $(check-qapi-schema-y)):
> check-%.json: $(SRC_PATH)/%.json
> >       @diff -q $(SRC_PATH)/$*.exit $*.test.exit
> >
> >  # Consolidated targets
> > +tests-cleanfiles = *.o
> > +tests-cleanfiles = .gitignore
> > +tests-cleanfiles += qht-bench$(EXESUF)
> > +tests-cleanfiles += qapi-schema/*.test.*
> > +tests-cleanfiles += test-qapi-event.[ch]
> > +tests-cleanfiles += test-qapi-types.[ch]
> > +tests-cleanfiles += test-qapi-visit.[ch]
> > +tests-cleanfiles += test-qmp-introspect.[ch]
> > +tests-cleanfiles += test-qmp-commands.h
> > +tests-cleanfiles += test-qmp-marshal.c
> > +tests-cleanfiles += $(subst tests/,,$(check-unit-y))
> > +tests-cleanfiles += $(subst tests/,,$(check-qtest-y))
> > +tests-cleanfiles += qemu-iotests/socket_scm_helper$(EXESUF)
> >
> >  .PHONY: check-qapi-schema check-qtest check-unit check check-clean
> >  check-qapi-schema: $(patsubst %,check-%, $(check-qapi-schema-y))
> > @@ -794,14 +807,18 @@ check-block: $(patsubst %,check-%,
> $(check-block-y))
> >  check: check-qapi-schema check-unit check-qtest
> >  check-clean:
> >       $(MAKE) -C tests/tcg clean
> > -     rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
> > -     rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST),
> $(check-qtest-$(target)-y)) $(check-qtest-generic-y))
> > -
> > +     (cd tests && echo rm -rf $(tests-cleanfiles))
> >  clean: check-clean
> >
> >  # Build the help program automatically
> >
> >  all: $(QEMU_IOTESTS_HELPERS-y)
> >
> > +ifneq ($(filter-out $(UNCHECKED_GOALS),$(MAKECMDGOALS)),$(if
> $(MAKECMDGOALS),,fail))
> > +$(SRC_PATH)/tests/.gitignore:
> > +     echo "$(tests-cleanfiles)" | xargs -n1 | sort > $@
>
> This line is TOO noisy; you'll want to wrap it into a $(call
> quiet-command...).
>

Yep


>
> You could also do it with fewer processes and less typing, as:
>
> printf %s\\n $(tests-cleanfiles) | sort > $@
>

Is there a way for printf to split the variable?

echo "*.o foo" | xargs -n1
*.o
foo

vs

pritnf %s\\n "*.o foo"
*.o foo

If I remove the quote in the Makefile, then it will expand to all the
non/exisiting .o files.


>
> > +Makefile: $(SRC_PATH)/tests/.gitignore
> > +endif
> > +
> >  -include $(wildcard tests/*.d)
> >  -include $(wildcard tests/libqos/*.d)
>
> But the idea seems interesting; looking forward to v2 and comments from
> anyone else.
>
> --
> Eric Blake   eblake redhat com    +1-919-301-3266
> Libvirt virtualization library http://libvirt.org
>
> --
Marc-André Lureau

Re: [Qemu-devel] [PATCH] build-sys: generate .gitignore

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1812 bytes --]

On 09/22/2016 03:47 PM, Marc-André Lureau wrote:

>> 
>>  # Consolidated targets
>> +tests-cleanfiles = *.o
>> +tests-cleanfiles = .gitignore
>> +tests-cleanfiles += qht-bench$(EXESUF)

Missed this on the first round: you meant to use += on the .gitignore
line.  Without it, you aren't excluding ANY .o files.

>>> +ifneq ($(filter-out $(UNCHECKED_GOALS),$(MAKECMDGOALS)),$(if
>> $(MAKECMDGOALS),,fail))
>>> +$(SRC_PATH)/tests/.gitignore:
>>> +     echo "$(tests-cleanfiles)" | xargs -n1 | sort > $@
>>
>> This line is TOO noisy; you'll want to wrap it into a $(call
>> quiet-command...).
>>
> 
> Yep
> 
> 
>>
>> You could also do it with fewer processes and less typing, as:
>>
>> printf %s\\n $(tests-cleanfiles) | sort > $@
>>
> 
> Is there a way for printf to split the variable?
> 
> echo "*.o foo" | xargs -n1
> *.o
> foo
> 
> vs
> 
> pritnf %s\\n "*.o foo"
> *.o foo
> 
> If I remove the quote in the Makefile, then it will expand to all the
> non/exisiting .o files.

I thought you WANTED shell globbing; but now I see that you don't (you
want a literal glob output into the .gitignore).  I guess I got thrown
by the fact that I didn't see *.o in the generated .gitignore (never
mind that I didn't see ANY .o...), from the bug pointed out above.

Okay, so you DO want to convert make's spaces into newlines, while
avoiding globbing.  My initial thought was that 'xargs -n1' feels a bit
heavy-handed; tr ' ' '\n' would do the same thing but then you aren't
shaving off any processes or any typing.  You can temporarily disable
file globbing by playing with 'set -f', but that also feels awkward.  So
maybe your approach is best after all.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]