[Qemu-devel] [PATCH v6 0/2] qemu-img: Document --force-share / -U

[Qemu-devel] [PATCH v6 0/2] qemu-img: Document --force-share / -U

[Fam Zheng]

v6: Tweak wording ("concurrent writers"). [Max]

Fam Zheng (2):
  qemu-img.texi: Clean up parameter list
  qemu-img: Document --force-share / -U

 qemu-img.texi | 73 +++++++++++++++++++++++++++++++++++++----------------------
 1 file changed, 46 insertions(+), 27 deletions(-)

-- 
2.14.3

[Qemu-devel] [PATCH v6 1/2] qemu-img.texi: Clean up parameter list

[Fam Zheng]

Split options out of the "@table @var" section and create a "@table
@option", then use whitespaces and blank lines consistently.

Suggested-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
---
 qemu-img.texi | 66 +++++++++++++++++++++++++++++++++++------------------------
 1 file changed, 39 insertions(+), 27 deletions(-)

diff --git a/qemu-img.texi b/qemu-img.texi
index fdcf120f36..60a0e080c6 100644
--- a/qemu-img.texi
+++ b/qemu-img.texi
@@ -33,38 +33,14 @@ The following commands are supported:
 
 Command parameters:
 @table @var
+
 @item filename
- is a disk image filename
-
-@item --object @var{objectdef}
-
-is a QEMU user creatable object definition. See the @code{qemu(1)} manual
-page for a description of the object properties. The most common object
-type is a @code{secret}, which is used to supply passwords and/or encryption
-keys.
-
-@item --image-opts
-
-Indicates that the source @var{filename} parameter is to be interpreted as a
-full option string, not a plain filename. This parameter is mutually
-exclusive with the @var{-f} parameter.
-
-@item --target-image-opts
-
-Indicates that the @var{output_filename} parameter(s) are to be interpreted as
-a full option string, not a plain filename. This parameter is mutually
-exclusive with the @var{-O} parameters. It is currently required to also use
-the @var{-n} parameter to skip image creation. This restriction may be relaxed
-in a future release.
+is a disk image filename
 
 @item fmt
 is the disk image format. It is guessed automatically in most cases. See below
 for a description of the supported disk formats.
 
-@item --backing-chain
-will enumerate information about backing files in a disk image chain. Refer
-below for further description.
-
 @item size
 is the disk image size in bytes. Optional suffixes @code{k} or @code{K}
 (kilobyte, 1024) @code{M} (megabyte, 1024k) and @code{G} (gigabyte, 1024M)
@@ -74,42 +50,78 @@ and T (terabyte, 1024G) are supported.  @code{b} is ignored.
 is the destination disk image filename
 
 @item output_fmt
- is the destination format
+is the destination format
+
 @item options
 is a comma separated list of format specific options in a
 name=value format. Use @code{-o ?} for an overview of the options supported
 by the used format or see the format descriptions below for details.
+
 @item snapshot_param
 is param used for internal snapshot, format is
 'snapshot.id=[ID],snapshot.name=[NAME]' or '[ID_OR_NAME]'
+
 @item snapshot_id_or_name
 is deprecated, use snapshot_param instead
 
+@end table
+
+@table @option
+
+@item --object @var{objectdef}
+is a QEMU user creatable object definition. See the @code{qemu(1)} manual
+page for a description of the object properties. The most common object
+type is a @code{secret}, which is used to supply passwords and/or encryption
+keys.
+
+@item --image-opts
+Indicates that the source @var{filename} parameter is to be interpreted as a
+full option string, not a plain filename. This parameter is mutually
+exclusive with the @var{-f} parameter.
+
+@item --target-image-opts
+Indicates that the @var{output_filename} parameter(s) are to be interpreted as
+a full option string, not a plain filename. This parameter is mutually
+exclusive with the @var{-O} parameters. It is currently required to also use
+the @var{-n} parameter to skip image creation. This restriction may be relaxed
+in a future release.
+
+@item --backing-chain
+will enumerate information about backing files in a disk image chain. Refer
+below for further description.
+
 @item -c
 indicates that target image must be compressed (qcow format only)
+
 @item -h
 with or without a command shows help and lists the supported formats
+
 @item -p
 display progress bar (compare, convert and rebase commands only).
 If the @var{-p} option is not used for a command that supports it, the
 progress is reported when the process receives a @code{SIGUSR1} or
 @code{SIGINFO} signal.
+
 @item -q
 Quiet mode - do not print any output (except errors). There's no progress bar
 in case both @var{-q} and @var{-p} options are used.
+
 @item -S @var{size}
 indicates the consecutive number of bytes that must contain only zeros
 for qemu-img to create a sparse image during conversion. This value is rounded
 down to the nearest 512 bytes. You may use the common size suffixes like
 @code{k} for kilobytes.
+
 @item -t @var{cache}
 specifies the cache mode that should be used with the (destination) file. See
 the documentation of the emulator's @code{-drive cache=...} option for allowed
 values.
+
 @item -T @var{src_cache}
 specifies the cache mode that should be used with the source file(s). See
 the documentation of the emulator's @code{-drive cache=...} option for allowed
 values.
+
 @end table
 
 Parameters to snapshot subcommand:
-- 
2.14.3

[Qemu-devel] [PATCH v6 2/2] qemu-img: Document --force-share / -U

[Fam Zheng]

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 qemu-img.texi | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/qemu-img.texi b/qemu-img.texi
index 60a0e080c6..ec7e2f5d1e 100644
--- a/qemu-img.texi
+++ b/qemu-img.texi
@@ -86,6 +86,13 @@ exclusive with the @var{-O} parameters. It is currently required to also use
 the @var{-n} parameter to skip image creation. This restriction may be relaxed
 in a future release.
 
+@item --force-share (-U)
+If specified, @code{qemu-img} will open the image in shared mode, allowing
+concurrent writers. For example, this can be used to get the image information
+(with 'info' subcommand) when the image is used by a running guest.  Note that
+this could produce inconsistent results because of concurrent metadata changes,
+etc. This option is only allowed when opening images in read-only mode.
+
 @item --backing-chain
 will enumerate information about backing files in a disk image chain. Refer
 below for further description.
-- 
2.14.3

Re: [Qemu-devel] [PATCH v6 2/2] qemu-img: Document --force-share / -U

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1195 bytes --]

On Tue, Jan 30, 2018 at 02:34:33PM +0800, Fam Zheng wrote:
> Signed-off-by: Fam Zheng <famz@redhat.com>
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
>  qemu-img.texi | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/qemu-img.texi b/qemu-img.texi
> index 60a0e080c6..ec7e2f5d1e 100644
> --- a/qemu-img.texi
> +++ b/qemu-img.texi
> @@ -86,6 +86,13 @@ exclusive with the @var{-O} parameters. It is currently required to also use
>  the @var{-n} parameter to skip image creation. This restriction may be relaxed
>  in a future release.
>  
> +@item --force-share (-U)
> +If specified, @code{qemu-img} will open the image in shared mode, allowing
> +concurrent writers.

This wording confuses me.  It makes me think of multiple processes
writing to the image at the same time...

> This option is only allowed when opening images in read-only mode.

...but later it turns out "concurrent writers" means there can be at
most 1 writing process and multiple reading processes.

How about merging the two statements?

"If specified, allows @code{qemu-img} to open the image in read-only
mode even if another process already has it open for writing."

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [Qemu-devel] [PATCH v6 1/2] qemu-img.texi: Clean up parameter list

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 597 bytes --]

On 01/30/2018 12:34 AM, Fam Zheng wrote:
> Split options out of the "@table @var" section and create a "@table
> @option", then use whitespaces and blank lines consistently.
> 
> Suggested-by: Kevin Wolf <kwolf@redhat.com>
> Signed-off-by: Fam Zheng <famz@redhat.com>
> ---
>  qemu-img.texi | 66 +++++++++++++++++++++++++++++++++++------------------------
>  1 file changed, 39 insertions(+), 27 deletions(-)

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: [Qemu-devel] [PATCH v6 2/2] qemu-img: Document --force-share / -U

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 392 bytes --]

On 01/30/2018 12:34 AM, Fam Zheng wrote:
> Signed-off-by: Fam Zheng <famz@redhat.com>
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
>  qemu-img.texi | 7 +++++++
>  1 file changed, 7 insertions(+)
> 

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: [Qemu-devel] [PATCH v6 2/2] qemu-img: Document --force-share / -U

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1470 bytes --]

On 01/30/2018 12:34 AM, Fam Zheng wrote:
> Signed-off-by: Fam Zheng <famz@redhat.com>
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
>  qemu-img.texi | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/qemu-img.texi b/qemu-img.texi
> index 60a0e080c6..ec7e2f5d1e 100644
> --- a/qemu-img.texi
> +++ b/qemu-img.texi
> @@ -86,6 +86,13 @@ exclusive with the @var{-O} parameters. It is currently required to also use
>  the @var{-n} parameter to skip image creation. This restriction may be relaxed
>  in a future release.
>  
> +@item --force-share (-U)
> +If specified, @code{qemu-img} will open the image in shared mode, allowing
> +concurrent writers. For example, this can be used to get the image information

Actually, we only permit one writer at a time.  Would it be better to
say "allowing a concurrent writer"?

> +(with 'info' subcommand) when the image is used by a running guest.  Note that
> +this could produce inconsistent results because of concurrent metadata changes,
> +etc. This option is only allowed when opening images in read-only mode.

After all, we are stating that this process (which must be read-only,
because we can't have two writers at once) is permitting some other
process to be the concurrent writer (but not multiple processes to be
concurrent writers)


-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: [Qemu-devel] [PATCH v6 2/2] qemu-img: Document --force-share / -U

[Kashyap Chamarthy]

On Tue, Jan 30, 2018 at 08:23:50AM -0600, Eric Blake wrote:
> On 01/30/2018 12:34 AM, Fam Zheng wrote:

[...]

> > +If specified, @code{qemu-img} will open the image in shared mode, allowing
> > +concurrent writers. For example, this can be used to get the image information
> 
> Actually, we only permit one writer at a time.  Would it be better to
> say "allowing a concurrent writer"?

Definitely worth rewording.  Otherwise the two sentences:

    "If specified, @code{qemu-img} will open the image in shared mode,
     allowing concurrent writers."

    "This option is only allowed when opening images in read-only mode."

are at odds with each other --- it says "concurrent writers" are
allowed, BUT "allowed only when opening images in 'read-only' mode".

> > +(with 'info' subcommand) when the image is used by a running guest.  Note that
> > +this could produce inconsistent results because of concurrent metadata changes,
> > +etc. This option is only allowed when opening images in read-only mode.
> 
> After all, we are stating that this process (which must be read-only,
> because we can't have two writers at once) is permitting some other
> process to be the concurrent writer (but not multiple processes to be
> concurrent writers)

Precisely.  So it's worth being clearer.

With the rewording suggested by Eric:

    Reviewed-by: Kashyap Chamarthy <kchamart@redhat.com>

-- 
/kashyap

Re: [Qemu-devel] [PATCH v6 2/2] qemu-img: Document --force-share / -U

[Kevin Wolf]

[-- Attachment #1: Type: text/plain, Size: 1276 bytes --]

Am 30.01.2018 um 15:23 hat Eric Blake geschrieben:
> On 01/30/2018 12:34 AM, Fam Zheng wrote:
> > Signed-off-by: Fam Zheng <famz@redhat.com>
> > Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> > ---
> >  qemu-img.texi | 7 +++++++
> >  1 file changed, 7 insertions(+)
> > 
> > diff --git a/qemu-img.texi b/qemu-img.texi
> > index 60a0e080c6..ec7e2f5d1e 100644
> > --- a/qemu-img.texi
> > +++ b/qemu-img.texi
> > @@ -86,6 +86,13 @@ exclusive with the @var{-O} parameters. It is currently required to also use
> >  the @var{-n} parameter to skip image creation. This restriction may be relaxed
> >  in a future release.
> >  
> > +@item --force-share (-U)
> > +If specified, @code{qemu-img} will open the image in shared mode, allowing
> > +concurrent writers. For example, this can be used to get the image information
> 
> Actually, we only permit one writer at a time.  Would it be better to
> say "allowing a concurrent writer"?

As far as qemu-img is concerned, multiple writers to the image are
allowed. There may be further restrictions imposed by a writer so that a
second writer isn't allowed, but with raw images and share-rw=on nothing
prevents two qemu instances writing to the same image while qemu-img is
accessing it read-only.

Kevin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [Qemu-devel] [PATCH v6 2/2] qemu-img: Document --force-share / -U

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1885 bytes --]

On Tue, Jan 30, 2018 at 05:38:20PM +0100, Kevin Wolf wrote:
> Am 30.01.2018 um 15:23 hat Eric Blake geschrieben:
> > On 01/30/2018 12:34 AM, Fam Zheng wrote:
> > > Signed-off-by: Fam Zheng <famz@redhat.com>
> > > Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> > > ---
> > >  qemu-img.texi | 7 +++++++
> > >  1 file changed, 7 insertions(+)
> > > 
> > > diff --git a/qemu-img.texi b/qemu-img.texi
> > > index 60a0e080c6..ec7e2f5d1e 100644
> > > --- a/qemu-img.texi
> > > +++ b/qemu-img.texi
> > > @@ -86,6 +86,13 @@ exclusive with the @var{-O} parameters. It is currently required to also use
> > >  the @var{-n} parameter to skip image creation. This restriction may be relaxed
> > >  in a future release.
> > >  
> > > +@item --force-share (-U)
> > > +If specified, @code{qemu-img} will open the image in shared mode, allowing
> > > +concurrent writers. For example, this can be used to get the image information
> > 
> > Actually, we only permit one writer at a time.  Would it be better to
> > say "allowing a concurrent writer"?
> 
> As far as qemu-img is concerned, multiple writers to the image are
> allowed. There may be further restrictions imposed by a writer so that a
> second writer isn't allowed, but with raw images and share-rw=on nothing
> prevents two qemu instances writing to the same image while qemu-img is
> accessing it read-only.

I agree with what you wrote and it's technically correct, but this is a
confusing place to mention concurrent writers.  They are a common source
of user error and that's why locking was introduced in the first place.

There should be a separate paragraph in docs/qemu-block-drivers.texi
explaining that share-rw=on can be used safely with format=raw if the
guests are configured to safely access a shared disk.  It should also
mention that share-rw=on is unsafe for image formats.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [Qemu-devel] [PATCH v6 2/2] qemu-img: Document --force-share / -U

[Kevin Wolf]

[-- Attachment #1: Type: text/plain, Size: 2487 bytes --]

Am 31.01.2018 um 15:12 hat Stefan Hajnoczi geschrieben:
> On Tue, Jan 30, 2018 at 05:38:20PM +0100, Kevin Wolf wrote:
> > Am 30.01.2018 um 15:23 hat Eric Blake geschrieben:
> > > On 01/30/2018 12:34 AM, Fam Zheng wrote:
> > > > Signed-off-by: Fam Zheng <famz@redhat.com>
> > > > Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> > > > ---
> > > >  qemu-img.texi | 7 +++++++
> > > >  1 file changed, 7 insertions(+)
> > > > 
> > > > diff --git a/qemu-img.texi b/qemu-img.texi
> > > > index 60a0e080c6..ec7e2f5d1e 100644
> > > > --- a/qemu-img.texi
> > > > +++ b/qemu-img.texi
> > > > @@ -86,6 +86,13 @@ exclusive with the @var{-O} parameters. It is currently required to also use
> > > >  the @var{-n} parameter to skip image creation. This restriction may be relaxed
> > > >  in a future release.
> > > >  
> > > > +@item --force-share (-U)
> > > > +If specified, @code{qemu-img} will open the image in shared mode, allowing
> > > > +concurrent writers. For example, this can be used to get the image information
> > > 
> > > Actually, we only permit one writer at a time.  Would it be better to
> > > say "allowing a concurrent writer"?
> > 
> > As far as qemu-img is concerned, multiple writers to the image are
> > allowed. There may be further restrictions imposed by a writer so that a
> > second writer isn't allowed, but with raw images and share-rw=on nothing
> > prevents two qemu instances writing to the same image while qemu-img is
> > accessing it read-only.
> 
> I agree with what you wrote and it's technically correct, but this is a
> confusing place to mention concurrent writers.  They are a common source
> of user error and that's why locking was introduced in the first place.

So you understand "concurrent writers" as at least two writers? Because
I wouldn't understood it as a writer that is concurrent to the
(read-only) qemu-img.

Maybe rephrase it as "...allowing other processes to write to the
image" then?

> There should be a separate paragraph in docs/qemu-block-drivers.texi
> explaining that share-rw=on can be used safely with format=raw if the
> guests are configured to safely access a shared disk.  It should also
> mention that share-rw=on is unsafe for image formats.

share-rw=on is a -device option and only about the guest, not about the
backend. It is never unsafe if the guest can cope with external writers.
It just doesn't prevent qemu from locking the image file for image
formats.

Kevin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [Qemu-devel] [PATCH v6 2/2] qemu-img: Document --force-share / -U

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 2114 bytes --]

On Wed, Jan 31, 2018 at 03:22:49PM +0100, Kevin Wolf wrote:
> Am 31.01.2018 um 15:12 hat Stefan Hajnoczi geschrieben:
> > There should be a separate paragraph in docs/qemu-block-drivers.texi
> > explaining that share-rw=on can be used safely with format=raw if the
> > guests are configured to safely access a shared disk.  It should also
> > mention that share-rw=on is unsafe for image formats.
> 
> share-rw=on is a -device option and only about the guest, not about the
> backend. It is never unsafe if the guest can cope with external writers.
> It just doesn't prevent qemu from locking the image file for image
> formats.

Thanks for the explanation.  Documentation on share-rw=on|off would be
nice.

Maybe something like:

  By default the guest has exclusive write access to its disk image.
  This is enforced by QEMU via file locking.  If the guest can safely
  share the disk image with other writers the -device ...,share-rw=on
  parameter can be used.  This is only safe if the guest is running
  software, such as a cluster file system, that coordinates disk
  accesses to avoid corruption.

  Note that share-rw=on only declares the guest's ability to share the
  disk.  Some QEMU features, such as image file formats, require
  exclusive write access to the disk image and this is unaffected by the
  share-rw=on option.

For anyone else reading this, here is the code that protects qcow2 and
other image formats:

  void bdrv_format_default_perms(BlockDriverState *bs, BdrvChild *c,
                                 const BdrvChildRole *role,
                                 BlockReopenQueue *reopen_queue,
                                 uint64_t perm, uint64_t shared,
                                 uint64_t *nperm, uint64_t *nshared)
  {
      ...

      /* bs->file always needs to be consistent because of the metadata. We
       * can never allow other users to resize or write to it. */
      if (!(flags & BDRV_O_NO_IO)) {
          perm |= BLK_PERM_CONSISTENT_READ;
      }
      shared &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE);

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [Qemu-devel] [PATCH v6 2/2] qemu-img: Document --force-share / -U

[Fam Zheng]

On Thu, 02/01 11:44, Stefan Hajnoczi wrote:
> On Wed, Jan 31, 2018 at 03:22:49PM +0100, Kevin Wolf wrote:
> > Am 31.01.2018 um 15:12 hat Stefan Hajnoczi geschrieben:
> > > There should be a separate paragraph in docs/qemu-block-drivers.texi
> > > explaining that share-rw=on can be used safely with format=raw if the
> > > guests are configured to safely access a shared disk.  It should also
> > > mention that share-rw=on is unsafe for image formats.
> > 
> > share-rw=on is a -device option and only about the guest, not about the
> > backend. It is never unsafe if the guest can cope with external writers.
> > It just doesn't prevent qemu from locking the image file for image
> > formats.
> 
> Thanks for the explanation.  Documentation on share-rw=on|off would be
> nice.
> 
> Maybe something like:
> 
>   By default the guest has exclusive write access to its disk image.
>   This is enforced by QEMU via file locking.  If the guest can safely
>   share the disk image with other writers the -device ...,share-rw=on
>   parameter can be used.  This is only safe if the guest is running
>   software, such as a cluster file system, that coordinates disk
>   accesses to avoid corruption.
> 
>   Note that share-rw=on only declares the guest's ability to share the
>   disk.  Some QEMU features, such as image file formats, require
>   exclusive write access to the disk image and this is unaffected by the
>   share-rw=on option.

I will add these paragraphs to the "image locking" seciton in
docs/qemu-block-drivers.texi.

Fam

Re: [Qemu-devel] [PATCH v6 2/2] qemu-img: Document --force-share / -U

[Fam Zheng]

On Thu, 02/01 11:44, Stefan Hajnoczi wrote:
> On Wed, Jan 31, 2018 at 03:22:49PM +0100, Kevin Wolf wrote:
> > Am 31.01.2018 um 15:12 hat Stefan Hajnoczi geschrieben:
> > > There should be a separate paragraph in docs/qemu-block-drivers.texi
> > > explaining that share-rw=on can be used safely with format=raw if the
> > > guests are configured to safely access a shared disk.  It should also
> > > mention that share-rw=on is unsafe for image formats.
> > 
> > share-rw=on is a -device option and only about the guest, not about the
> > backend. It is never unsafe if the guest can cope with external writers.
> > It just doesn't prevent qemu from locking the image file for image
> > formats.
> 
> Thanks for the explanation.  Documentation on share-rw=on|off would be
> nice.
> 
> Maybe something like:
> 
>   By default the guest has exclusive write access to its disk image.
>   This is enforced by QEMU via file locking.  If the guest can safely
>   share the disk image with other writers the -device ...,share-rw=on
>   parameter can be used.  This is only safe if the guest is running
>   software, such as a cluster file system, that coordinates disk
>   accesses to avoid corruption.
> 
>   Note that share-rw=on only declares the guest's ability to share the
>   disk.  Some QEMU features, such as image file formats, require
>   exclusive write access to the disk image and this is unaffected by the
>   share-rw=on option.

I will add these paragraphs to the "image locking" seciton in
docs/qemu-block-drivers.texi.

Fam