[patch] problem w/ arecord "split interleaved files" feature

[patch] problem w/ arecord "split interleaved files" feature

[Dave Maley]

[-- Attachment #1: Type: text/plain, Size: 1424 bytes --]

Hello,

My first post to the alsa-devel list, and I'm not subscribed so please
cc: me directly on any responses ...  This is in reference to a patch
which was incorporated into alsa-utils-1.0.11, originally posted here:

http://thread.gmane.org/gmane.linux.alsa.devel/29025/focus=29098

The max_filesize for wave files as currently implemented it's actually
splitting files a good bit before reaching 2GB.

A little background as to why this matters (to me at least).  I use
arecord to transfer DAT's to my computer, and many of these DAT's are 3
hours in length.  So for this I'd typically do something to the effect
of:


[dave@flanders ~]$ arecord -t wav -f dat -d 10800 foo.wav


Since max_filesize is set to 2000000000 arecord splits the resulting
file a bit over 170 minutes.  Not a huge problem, except I pipe the
above command to sox for down-sampling to 44.1KHz:


[dave@flanders ~]$ arecord -q -t wav -f dat -d 10800 | sox -t wav -r
48000 -c 2 - -t wav -r 44100 -c 2 foo.wav resample


It seems this new split feature doesn't work properly when writing to a
pipe (or at least when piping to sox).  A quick look at an strace shows
constant write errors, and foo.wav grows in size until the disk is full.

A quick fix for this is to set max_filesize to 2147483648.  Not sure if
this is a "proper" fix however I have tested it and it seems to work
just fine for me.  Please consider the attached patch.

Thanks,
Dave

[-- Attachment #2: arecord_split.patch --]
[-- Type: text/x-patch, Size: 456 bytes --]

--- alsa-utils-1.0.11/aplay/aplay.c.orig	2006-04-25 01:37:13.000000000 -0400
+++ alsa-utils-1.0.11/aplay/aplay.c	2006-04-25 01:38:24.000000000 -0400
@@ -128,7 +128,7 @@
 } fmt_rec_table[] = {
 	{	NULL,		end_raw,	N_("raw data"),		LLONG_MAX },
 	{	begin_voc,	end_voc,	N_("VOC"),		16000000 },
-	{	begin_wave,	end_wave,	N_("WAVE"),		2000000000 },
+	{	begin_wave,	end_wave,	N_("WAVE"),		2147483648 },
 	{	begin_au,	end_au,		N_("Sparc Audio"),	LLONG_MAX }
 };
 

Re: [patch] problem w/ arecord "split interleaved files" feature

[Takashi Iwai]

At Tue, 25 Apr 2006 01:43:24 -0400,
Dave Maley wrote:
> 
> Hello,
> 
> My first post to the alsa-devel list, and I'm not subscribed so please
> cc: me directly on any responses ...  This is in reference to a patch
> which was incorporated into alsa-utils-1.0.11, originally posted here:
> 
> http://thread.gmane.org/gmane.linux.alsa.devel/29025/focus=29098
> 
> The max_filesize for wave files as currently implemented it's actually
> splitting files a good bit before reaching 2GB.
> 
> A little background as to why this matters (to me at least).  I use
> arecord to transfer DAT's to my computer, and many of these DAT's are 3
> hours in length.  So for this I'd typically do something to the effect
> of:
> 
> 
> [dave@flanders ~]$ arecord -t wav -f dat -d 10800 foo.wav
> 
> 
> Since max_filesize is set to 2000000000 arecord splits the resulting
> file a bit over 170 minutes.  Not a huge problem, except I pipe the
> above command to sox for down-sampling to 44.1KHz:
> 
> 
> [dave@flanders ~]$ arecord -q -t wav -f dat -d 10800 | sox -t wav -r
> 48000 -c 2 - -t wav -r 44100 -c 2 foo.wav resample
> 
> 
> It seems this new split feature doesn't work properly when writing to a
> pipe (or at least when piping to sox).  A quick look at an strace shows
> constant write errors, and foo.wav grows in size until the disk is full.

I guess it's rather a bug of sox.  arecord quits when the data size
reaches to max_filesize in the case of pipe output (since obviously
you cannot split the output).  sox should quits, too, when receiving
EPIPE error.

> A quick fix for this is to set max_filesize to 2147483648.  Not sure if
> this is a "proper" fix however I have tested it and it seems to work
> just fine for me.  Please consider the attached patch.

Well, I'm not sure whether exactly 2GB is allowed.

Does anyone know the exact specification of WAV (PCM) format?


Takashi


-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

Re: [patch] problem w/ arecord "split interleaved files" feature

[Dave Maley]

Thanks for the quick response ...

> > [dave@flanders ~]$ arecord -q -t wav -f dat -d 10800 | sox -t wav -r
> > 48000 -c 2 - -t wav -r 44100 -c 2 foo.wav resample
> > 
> > 
> > It seems this new split feature doesn't work properly when writing to a
> > pipe (or at least when piping to sox).  A quick look at an strace shows
> > constant write errors, and foo.wav grows in size until the disk is full.
> 
> I guess it's rather a bug of sox.  arecord quits when the data size
> reaches to max_filesize in the case of pipe output (since obviously
> you cannot split the output).  sox should quits, too, when receiving
> EPIPE error.

Hmm, this doesn't seem to match the behavior I'm seeing.  It's very
possible I'm misunderstanding something here, but this is what I see:

Using the above arecord | sox command when the foo.wav file reaches
2000000000 in size and the split occurs it'll actually be split into
foo-01.wav and foo-02.wav, and I believe that this rename of foo.wav to
foo-01.wav is where the problem may be occurring.  I can capture strace
output at the time of the split, if it's at all helpful.

> > A quick fix for this is to set max_filesize to 2147483648.  Not sure if
> > this is a "proper" fix however I have tested it and it seems to work
> > just fine for me.  Please consider the attached patch.
> 
> Well, I'm not sure whether exactly 2GB is allowed.

If not then I'd be happy with just increasing this max_filesize to
2073600044 as that's what my 180 minute 48KHz files end up being ... 8)

Thanks again for your feedback on this.

~Dave



-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

Re: [patch] problem w/ arecord "split interleaved files" feature

[Takashi Iwai]

At Tue, 25 Apr 2006 17:07:35 -0400,
Dave Maley wrote:
> 
> Thanks for the quick response ...
> 
> > > [dave@flanders ~]$ arecord -q -t wav -f dat -d 10800 | sox -t wav -r
> > > 48000 -c 2 - -t wav -r 44100 -c 2 foo.wav resample
> > > 
> > > 
> > > It seems this new split feature doesn't work properly when writing to a
> > > pipe (or at least when piping to sox).  A quick look at an strace shows
> > > constant write errors, and foo.wav grows in size until the disk is full.
> > 
> > I guess it's rather a bug of sox.  arecord quits when the data size
> > reaches to max_filesize in the case of pipe output (since obviously
> > you cannot split the output).  sox should quits, too, when receiving
> > EPIPE error.
> 
> Hmm, this doesn't seem to match the behavior I'm seeing.  It's very
> possible I'm misunderstanding something here, but this is what I see:
> 
> Using the above arecord | sox command when the foo.wav file reaches
> 2000000000 in size and the split occurs it'll actually be split into
> foo-01.wav and foo-02.wav, and I believe that this rename of foo.wav to
> foo-01.wav is where the problem may be occurring.  I can capture strace
> output at the time of the split, if it's at all helpful.

I found a stupid bug there.  arecord doesn't quit properly at
max_filesize due to a wrong comparison.  The fix patch is below.


> > > A quick fix for this is to set max_filesize to 2147483648.  Not sure if
> > > this is a "proper" fix however I have tested it and it seems to work
> > > just fine for me.  Please consider the attached patch.
> > 
> > Well, I'm not sure whether exactly 2GB is allowed.
> 
> If not then I'd be happy with just increasing this max_filesize to
> 2073600044 as that's what my 180 minute 48KHz files end up being ... 8)

Maybe 2GB is OK.  I'll fix it too.


Takashi

diff -r 2dacb18877f3 aplay/aplay.c
--- a/aplay/aplay.c	Wed Apr 19 12:29:49 2006 +0200
+++ b/aplay/aplay.c	Wed Apr 26 16:07:21 2006 +0200
@@ -2165,13 +2165,15 @@ static void capture(char *name_)
 			   2) all requested samples/bytes have been captured (cur>0)
 			   3) filesize threshold was reached (fdcount<wrthreshold)
 			*/
-		} while (file_type == FORMAT_RAW && !timelimit && cur>0 && fdcount<fmt_rec_table[file_type].max_filesize);
+		} while (file_type == FORMAT_RAW && !timelimit && cur>0 &&
+			 fdcount<fmt_rec_table[file_type].max_filesize);
 
 		/* finish sample container */
 		fmt_rec_table[file_type].end(fd);
 
 		/* repeat the loop when format is raw without timelimit or filesize threshold was reached */
-	} while((file_type == FORMAT_RAW && !timelimit) || fdcount>=fmt_rec_table[file_type].max_filesize);
+	} while((file_type == FORMAT_RAW && !timelimit) ||
+		fdcount < fmt_rec_table[file_type].max_filesize);
 }
 
 void playbackv_go(int* fds, unsigned int channels, size_t loaded, off64_t count, int rtype, char **names)


-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

Re: [patch] problem w/ arecord "split interleaved files" feature

[Dave Maley]

> I found a stupid bug there.  arecord doesn't quit properly at
> max_filesize due to a wrong comparison.  The fix patch is below.

Excellent - Thanks much.  I'll test this tonight and report back the
results.

> Maybe 2GB is OK.  I'll fix it too.

No idea what the spec says but I've recorded test wave files which were
exactly 2GB and they didn't seem to have any problems (played fine in
totem and opened properly in a wave editor).

Thanks again for you prompt responses and patch!

~Dave



-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

Re: [patch] problem w/ arecord "split interleaved files" feature

[Dave Maley]

Hi Takashi,

> I found a stupid bug there.  arecord doesn't quit properly at
> max_filesize due to a wrong comparison.  The fix patch is below.

I finally had a chance to test this patch and it indeed resolves the
problem I was running into.

> Maybe 2GB is OK.  I'll fix it too.

Excellent.  Thanks much for looking into this.

~Dave



-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642