All of lore.kernel.org
 help / color / mirror / Atom feed
From: Junio C Hamano <gitster@pobox.com>
To: Steffen Prohaska <prohaska@zib.de>
Cc: "Linus Torvalds" <torvalds@linux-foundation.org>,
	git@vger.kernel.org, "Johannes Sixt" <j6t@kdbg.org>,
	"John Keeping" <john@keeping.me.uk>,
	"Jonathan Nieder" <jrnieder@gmail.com>,
	"Kyle J. McKay" <mackyle@gmail.com>,
	"Torsten Bögershausen" <tboegi@web.de>,
	"Eric Sunshine" <sunshine@sunshineco.com>
Subject: Re: [PATCH v5 1/2] xread, xwrite: Limit size of IO, fixing IO of 2GB and more on Mac OS X
Date: Tue, 20 Aug 2013 12:37:49 -0700	[thread overview]
Message-ID: <xmqqli3w5f1u.fsf@gitster.dls.corp.google.com> (raw)
In-Reply-To: <1376981035-23284-2-git-send-email-prohaska@zib.de> (Steffen Prohaska's message of "Tue, 20 Aug 2013 08:43:54 +0200")

Steffen Prohaska <prohaska@zib.de> writes:

> diff --git a/wrapper.c b/wrapper.c
> index 6a015de..97e3cf7 100644
> --- a/wrapper.c
> +++ b/wrapper.c
> @@ -131,6 +131,14 @@ void *xcalloc(size_t nmemb, size_t size)
>  }
>  
>  /*
> + * Limit size of IO chunks, because huge chunks only cause pain.  OS X 64-bit
> + * buggy, returning EINVAL if len >= INT_MAX; and even in the absense of bugs,

s/buggy/is &/ perhaps?

> + * large chunks can result in bad latencies when you decide to kill the
> + * process.
> + */
> +#define MAX_IO_SIZE (8*1024*1024)
> +
> +/*
>   * xread() is the same a read(), but it automatically restarts read()
>   * operations with a recoverable error (EAGAIN and EINTR). xread()
>   * DOES NOT GUARANTEE that "len" bytes is read even if the data is available.
> @@ -138,6 +146,8 @@ void *xcalloc(size_t nmemb, size_t size)
>  ssize_t xread(int fd, void *buf, size_t len)
>  {
>  	ssize_t nr;
> +	if (len > MAX_IO_SIZE)
> +	    len = MAX_IO_SIZE;
>  	while (1) {
>  		nr = read(fd, buf, len);
>  		if ((nr < 0) && (errno == EAGAIN || errno == EINTR))
> @@ -154,6 +164,8 @@ ssize_t xread(int fd, void *buf, size_t len)
>  ssize_t xwrite(int fd, const void *buf, size_t len)
>  {
>  	ssize_t nr;
> +	if (len > MAX_IO_SIZE)
> +	    len = MAX_IO_SIZE;
>  	while (1) {
>  		nr = write(fd, buf, len);
>  		if ((nr < 0) && (errno == EAGAIN || errno == EINTR))

  reply	other threads:[~2013-08-20 19:37 UTC|newest]

Thread overview: 37+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-08-17 12:40 [PATCH] xread(): Fix read error when filtering >= 2GB on Mac OS X Steffen Prohaska
2013-08-17 15:27 ` John Keeping
2013-08-17 15:56 ` Torsten Bögershausen
2013-08-17 17:16 ` Johannes Sixt
2013-08-17 18:57 ` Jonathan Nieder
2013-08-17 20:25 ` Kyle J. McKay
2013-08-17 21:23   ` Jonathan Nieder
2013-08-19  6:38 ` [PATCH v2] compat: Fix read() of 2GB and more " Steffen Prohaska
2013-08-19  7:54   ` John Keeping
2013-08-19  8:20     ` Steffen Prohaska
2013-08-19  8:20   ` Johannes Sixt
2013-08-19  8:25     ` Stefan Beller
2013-08-19  8:40       ` Johannes Sixt
2013-08-19  8:28     ` Steffen Prohaska
2013-08-19  8:21   ` [PATCH v3] " Steffen Prohaska
2013-08-19 13:59     ` Eric Sunshine
2013-08-19 16:33       ` Junio C Hamano
2013-08-19 15:41     ` [PATCH v4] " Steffen Prohaska
2013-08-19 16:04       ` Linus Torvalds
2013-08-19 16:37         ` Steffen Prohaska
2013-08-19 17:24           ` Junio C Hamano
2013-08-19 17:16         ` Junio C Hamano
2013-08-19 17:28           ` Linus Torvalds
2013-08-19 21:56           ` Kyle J. McKay
2013-08-19 22:51             ` Linus Torvalds
2013-08-27  4:59               ` Junio C Hamano
2013-08-20  6:43       ` [PATCH v5 0/2] Fix IO of >=2GB on Mac OS X by limiting IO chunks Steffen Prohaska
2013-08-20  6:43         ` [PATCH v5 1/2] xread, xwrite: Limit size of IO, fixing IO of 2GB and more on Mac OS X Steffen Prohaska
2013-08-20 19:37           ` Junio C Hamano [this message]
2013-08-21 19:50           ` Torsten Bögershausen
2013-08-20  6:43         ` [PATCH v5 2/2] Revert "compate/clipped-write.c: large write(2) fails on Mac OS X/XNU" Steffen Prohaska
2013-08-21 13:46         ` [PATCH v6 0/2] Fix IO >= 2GB on Mac, fixed typo Steffen Prohaska
2013-08-21 13:46           ` [PATCH v5 1/2] xread, xwrite: Limit size of IO, fixing IO of 2GB and more on Mac OS X Steffen Prohaska
2013-08-21 13:46           ` [PATCH v5 2/2] Revert "compate/clipped-write.c: large write(2) fails on Mac OS X/XNU" Steffen Prohaska
2013-08-21 15:58           ` [PATCH v6 0/2] Fix IO >= 2GB on Mac, fixed typo Junio C Hamano
2013-08-19  8:27   ` [PATCH v2] compat: Fix read() of 2GB and more on Mac OS X Johannes Sixt
2013-08-19 14:41   ` Torsten Bögershausen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=xmqqli3w5f1u.fsf@gitster.dls.corp.google.com \
    --to=gitster@pobox.com \
    --cc=git@vger.kernel.org \
    --cc=j6t@kdbg.org \
    --cc=john@keeping.me.uk \
    --cc=jrnieder@gmail.com \
    --cc=mackyle@gmail.com \
    --cc=prohaska@zib.de \
    --cc=sunshine@sunshineco.com \
    --cc=tboegi@web.de \
    --cc=torvalds@linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.