help with joining files

help with joining files

[James Miller]

Flemming's script did the job of getting wierd characters out of file 
names for me, so that part of dealing with these recordings is nicely 
resolved. Just run the comand in the dir where the files are, and in 
about a second you have renamed 170 files. Now, if cat'ing them together 
were even 1/10th as efficient. I'm discovering that this process really 
needs to be automated, too. Maybe someone could offer advice on how to do 
this?

Currently, I surf to the dir where the 170 files are, issue "cat" 
followied by part of the beginning of a file's name and hit <tab>, which 
gives me about 2/3's of the rest of the name. I then have to start looking 
at names of files more closely in another window, look at their time 
signatures (the most pertinent piece of info, really), choose the oldest 
one, enter some letter from it where cat left off, and hit <tab> again to 
get the complete name of the first file. Then, I copy that file's name and 
paste it after >~/MyMusic/classical/ and strip off the more particluarized 
parts so I have a more generic name for my full-piece file, then hit 
enter. At that point, I have the beginning of the cat'd together piece, 
and I follow a similar procedure--but using >>~/MyMusic/classical--to cat 
the rest of the components of the piece together into one.

To give a more complete picture using fake names, I do something like 
this:

cat Snooty-Composer1-highflownsymphony-inGmajor-Op.89674-movement1.mp3 
>~/MyMusic/classical/Snooty-Composer1-highflownsymphony-inGmajor-Op.89674.mp3

cat Snooty-Composer1-highflownsymphony-inGmajor-Op.89674-movement2.mp3 
>>~/MyMusic/classical/Snooty-Composer1-highflownsymphony-inGmajor-Op.89674.mp3

cat Snooty-Composer1-highflownsymphony-inGmajor-Op.89674-movement3.mp3 
>>~/MyMusic/classical/Snooty-Composer1-highflownsymphony-inGmajor-Op.89674.mp3

I then have the entire piece in a single file called 
Snooty-Composer1-highflownsymphony-inGmajor-Op.89674.mp3. Incidentally, 
there is alot more variation in the actual names than in these examples. 
For example, not very many have the movements numbered (the numeral right 
before .mp3). Rather, the movements are called by their more generic 
titles like "Adagio," "Andante," and so forth. So, not much consistency 
in that part of the name.

It seems like there should be a less labor-intensive way of doing this, 
and, it seems to me, the salient factor in such a scheme is the time of 
file creation rather than file name. Each file is written to the HD in the 
order it finishes playing from the stream I am recording, so that's really 
probably the easiest and surest way to join them. All movments of a given 
piece are all going to have been written to disk within, say, 1/2 hr of 
the beginning of the piece.

Problem is, I'm sort of at a loss for ideas about how this could be done. 
Is there a way to feed cat file creation time rather than file name? It's 
not really so important that the name of the file I am cat'ing be 
preserved, so much as that the file being cat'ed to have an informative 
name. So, I'm casting about for ways of automating this process of joining 
movements together into the full pieces of which the movements are a part. 
Suggestions?

Thanks, James
-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

Re: help with joining files

[J.]

On Fri, 15 Apr 2005, James Miller wrote:

> Flemming's script did the job of getting wierd characters out of file 
> names for me, so that part of dealing with these recordings is nicely 
> resolved. Just run the comand in the dir where the files are, and in 
> about a second you have renamed 170 files. Now, if cat'ing them together 
> were even 1/10th as efficient. I'm discovering that this process really 
> needs to be automated, too. Maybe someone could offer advice on how to do 
> this?
> 
> Currently, I surf to the dir where the 170 files are, issue "cat" 
> followied by part of the beginning of a file's name and hit <tab>, which 
> gives me about 2/3's of the rest of the name. I then have to start looking 
> at names of files more closely in another window, look at their time 
> signatures (the most pertinent piece of info, really), choose the oldest 
> one, enter some letter from it where cat left off, and hit <tab> again to 
> get the complete name of the first file. Then, I copy that file's name and 
> paste it after >~/MyMusic/classical/ and strip off the more particluarized 
> parts so I have a more generic name for my full-piece file, then hit 
> enter. At that point, I have the beginning of the cat'd together piece, 
> and I follow a similar procedure--but using >>~/MyMusic/classical--to cat 
> the rest of the components of the piece together into one.
> 
> To give a more complete picture using fake names, I do something like 
> this:
> 
> cat Snooty-Composer1-highflownsymphony-inGmajor-Op.89674-movement1.mp3 
> >~/MyMusic/classical/Snooty-Composer1-highflownsymphony-inGmajor-Op.89674.mp3
> 
> cat Snooty-Composer1-highflownsymphony-inGmajor-Op.89674-movement2.mp3 
> >>~/MyMusic/classical/Snooty-Composer1-highflownsymphony-inGmajor-Op.89674.mp3
> 
> cat Snooty-Composer1-highflownsymphony-inGmajor-Op.89674-movement3.mp3 
> >>~/MyMusic/classical/Snooty-Composer1-highflownsymphony-inGmajor-Op.89674.mp3

If I read your filenames correctly, the only thing that seperates them are
the movement numbers ? I am assuming that `[0-9]' would work than for
all your files this way ???

~: ls Snooty-Composer1-highflownsymphony-inGmajor-Op.89674-movement[0-9].mp3
Snooty-Composer1-highflownsymphony-inGmajor-Op.89674-movement1.mp3 
Snooty-Composer1-highflownsymphony-inGmajor-Op.89674-movement2.mp3 
Snooty-Composer1-highflownsymphony-inGmajor-Op.89674-movement3.mp3 

`ls' can take a number of options, underwich `sort' entrys by time .
~: ls -lt -c Snooty-Composer1-highflownsymphony-inGmajor-Op.89674-movement[0-9].mp3
Now extract the targetname you want with `sed` [see bottom of email] and
fall into a loop dd'ing with notrunc or contactenating `like you did
above' all files to your target file.
done

> I then have the entire piece in a single file called 
> Snooty-Composer1-highflownsymphony-inGmajor-Op.89674.mp3. Incidentally, 
> there is alot more variation in the actual names than in these examples. 
> For example, not very many have the movements numbered (the numeral right 
> before .mp3). Rather, the movements are called by their more generic 
> titles like "Adagio," "Andante," and so forth. So, not much consistency 
> in that part of the name.
> 
> It seems like there should be a less labor-intensive way of doing this, 
> and, it seems to me, the salient factor in such a scheme is the time of 
> file creation rather than file name. 

Unix and GNU/Linux type systems do not store the file creation time as
static information. There are `three sorts of time` that are stored with
each file:

C time - changed time info of the corresponding inode,
M time - last modified time,
A time - last access time

These can be changed by other programs. For example `touch' . So be
careful how you implement this and how you group corrosponding files by
time .

Personally I would have renamed each file correctly according to the
task I wanted to preform right from the start, thus to include
the last modified UTC time in the filename. That way you can list each
file belonging to a certain group `sort -n' them by time and concat them
as appropiate. 

> Each file is written to the HD in the 
> order it finishes playing from the stream I am recording, so that's really 
> probably the easiest and surest way to join them. All movments of a given 
> piece are all going to have been written to disk within, say, 1/2 hr of 
> the beginning of the piece.
> 
> Problem is, I'm sort of at a loss for ideas about how this could be done. 
> Is there a way to feed cat file creation time rather than file name? 

No .

B.t.w. you could also use `dd', with the `notrunc' parameter . That is
more optimized for this task.

> It's 
> not really so important that the name of the file I am cat'ing be 
> preserved, so much as that the file being cat'ed to have an informative 
> name. So, I'm casting about for ways of automating this process of joining 
> movements together into the full pieces of which the movements are a part. 
> Suggestions?

This will....
~: echo 'Snooty-Composer1-highflownsymphony-inGmajor-Op.89674-movement1.mp3' | \
 sed 's/^\(.*\)-movement\([0-9]*\).mp3$/\1_FULL.mp3/g'
result in: 
 Snooty-Composer1-highflownsymphony-inGmajor-Op.89674_FULL.mp3

Note the nonlinebreak after the pipe is done because of the linewidth in
e-mail.. 

> Thanks, James

The GLUE at the GNU/Linux , Unix type command-line provides was already
there long before. Pls. Have a look at the `UNIX philosophy' since I 
think it will give some more insight in the background of these tasks
you are trying to accomplish.

http://www.google.com/linux?hl=en&lr=&q=unix+philosophy&btnG=Search

Success, 
J.

--
Don't worry Ma'am. We're university students, - we know what we're doing.

-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

Re: help with joining files

[James Miller]

Thanks for your response, J. I'm faced with a fairly complex task here. I 
suppose a more reasonable man would say, "heck with this. I'm gonna go buy 
myself some CD's." But if I were a more reasonable man, I wouldn't be 
using Linux, now would I? . . . :)

Some of what you're saying is a bit beyond me. I'd kinda like to get to 
the point where I could understand it better, and understand the 
philosophy better (at the link you provide). But I'm sort of an old hacker 
(I use that term in its old-fashioned, non-technical sense), so at the 
very least I'll have to get there very slowly and gradually.

Your assumption about movement numbers is wrong, unfortunately. I tried to 
make that clear in my note to the list, but apparently did not succeed. So 
the bash examples you give are not going to get me too far--at least not 
that I can see. But I was able to use some things in your post to help 
automate things a bit more: I'm now using the "ls -l partial_file_name 
<tab>" to get the salient information in a more discrete place from which 
I can copy and paste. I do have to do some "ls partial_file_name <tab>ing" 
to even get to the point where I can use ls -l (-l gives me the time 
information I need in order to cat things together in the right order), 
but I've already reduced the number of steps to get full_piece.mp3.

Here's a more real world example of what I'm doing:

me@mymachine:~/radio/editing/classcl$ ls -l 
Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4-_*
-rw-r--r--  1 me me 9617922 2005-04-15 13:50 
Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4-_Adagio_-_Allegro.mp3
-rw-r--r--  1 me me 5612894 2005-04-15 13:56 
Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4-_Andante_con_Variazioni.mp3
-rw-r--r--  1 me me 8120985 2005-04-15 14:08 
Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4-_Finale-_Allegro_vivace.mp3
-rw-r--r--  1 me me 2760026 2005-04-15 13:59 
Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4-_Minuetto.mp3
me@mymachine:~/radio/editing/classcl$ cat 
Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4-_Adagio_-_Allegro.mp3 
>~/radio/editing/Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4.mp3
me@mymachine:~/radio/editing/classcl$ cat 
Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4-_Andante_con_Variazioni.mp3 
>>~/radio/editing/Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4.mp3
me@mymachine:~/radio/editing/classcl$ cat 
Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4-_Minuetto.mp3 
>>~/radio/editing/Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4.mp3
me@mymachine:~/radio/editing/classcl$ cat 
Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4-_Finale-_Allegro_vivace.mp3 
>>~/radio/editing/Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4.mp3
me@mymachine:~/radio/editing/classcl$ mv 
Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4-_* 
~/radio/editing/parts-done/
me@mymachine:~/radio/editing/classcl$

I don't really need file permissions information, so using the -l switch 
is overkill. But so far I have not determined how I can get the time 
information I need without the other, file permission info.

On Sat, 16 Apr 2005, J. wrote:

> Unix and GNU/Linux type systems do not store the file creation time as
> static information. There are `three sorts of time` that are stored with
> each file:
>
> C time - changed time info of the corresponding inode,
> M time - last modified time,
> A time - last access time
>
> These can be changed by other programs. For example `touch' . So be
> careful how you implement this and how you group corrosponding files by
> time .

I'm not doing anything with these files after they're saved to my HD other 
than listing them (like, with ls) or looking at contents of directories 
with mc. Could any of these change the time information ls -l gives me?

> Personally I would have renamed each file correctly according to the
> task I wanted to preform right from the start, thus to include
> the last modified UTC time in the filename. That way you can list each
> file belonging to a certain group `sort -n' them by time and concat them
> as appropiate.

Good idea. I'm just technically-challenged enough that I have no idea how 
to do this (or even where to start) where it concerns recording streaming 
audio. I suppose I'd need to make streamripper run as part of a larger 
script that would likewise invoke Flemming's remove_invalid, etc. Mind 
boggling!

> B.t.w. you could also use `dd', with the `notrunc' parameter . That is
> more optimized for this task.

I'll look into this.

> The GLUE at the GNU/Linux , Unix type command-line provides was already
> there long before. Pls. Have a look at the `UNIX philosophy' since I
> think it will give some more insight in the background of these tasks
> you are trying to accomplish.
>
> http://www.google.com/linux?hl=en&lr=&q=unix+philosophy&btnG=Search

Thanks for the link.

James
-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

Re: help with joining files

[J.]

On Sat, 16 Apr 2005, James Miller wrote:

> Thanks for your response, J. I'm faced with a fairly complex task here. I 
> suppose a more reasonable man would say, "heck with this. I'm gonna go buy 
> myself some CD's." But if I were a more reasonable man, I wouldn't be 
> using Linux, now would I? . . . :)

;-) That's the spirit.. !

> Some of what you're saying is a bit beyond me. I'd kinda like to get to 
> the point where I could understand it better, and understand the 
> philosophy better (at the link you provide). But I'm sort of an old hacker 
> (I use that term in its old-fashioned, non-technical sense), so at the 
> very least I'll have to get there very slowly and gradually.
> 
> Your assumption about movement numbers is wrong, unfortunately. I tried to 
> make that clear in my note to the list, but apparently did not succeed. So 
> the bash examples you give are not going to get me too far--at least not 
> that I can see. But I was able to use some things in your post to help 
> automate things a bit more: I'm now using the "ls -l partial_file_name 
> <tab>" to get the salient information in a more discrete place from which 
> I can copy and paste. I do have to do some "ls partial_file_name <tab>ing" 
> to even get to the point where I can use ls -l (-l gives me the time 
> information I need in order to cat things together in the right order), 
> but I've already reduced the number of steps to get full_piece.mp3.
> 
> Here's a more real world example of what I'm doing:
> 
> me@mymachine:~/radio/editing/classcl$ ls -l 
> Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4-_*
> -rw-r--r--  1 me me 9617922 2005-04-15 13:50 
> Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4-_Adagio_-_Allegro.mp3
> -rw-r--r--  1 me me 5612894 2005-04-15 13:56 
> Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4-_Andante_con_Variazioni.mp3
> -rw-r--r--  1 me me 8120985 2005-04-15 14:08 
> Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4-_Finale-_Allegro_vivace.mp3
> -rw-r--r--  1 me me 2760026 2005-04-15 13:59 
> Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4-_Minuetto.mp3
> me@mymachine:~/radio/editing/classcl$ cat 
> Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4-_Adagio_-_Allegro.mp3 
> >~/radio/editing/Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4.mp3
> me@mymachine:~/radio/editing/classcl$ cat 
> Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4-_Andante_con_Variazioni.mp3 
> >>~/radio/editing/Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4.mp3
> me@mymachine:~/radio/editing/classcl$ cat 
> Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4-_Minuetto.mp3 
> >>~/radio/editing/Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4.mp3
> me@mymachine:~/radio/editing/classcl$ cat 
> Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4-_Finale-_Allegro_vivace.mp3 
> >>~/radio/editing/Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4.mp3
> me@mymachine:~/radio/editing/classcl$ mv 
> Anton_Reicha-_Albert_Schweitzer_Quintett__-_WInd_Quintet_No.22_in_E_minor_Op.100_No.4-_* 
> ~/radio/editing/parts-done/
> me@mymachine:~/radio/editing/classcl$
> 
> I don't really need file permissions information, so using the -l switch 
> is overkill. But so far I have not determined how I can get the time 
> information I need without the other, file permission info.

The essential problem is; That you are trying to cluster or group strings 
that share similaritys to some extend.

The best way is not to; have to sort at all. 8 times out of 10 people
can save sorting by storing the info in the right way the moment it has
arrived.

Ehmm.. but I guess, for tab completion these files you could use the 
wildcard `*' in combi with `cut' . I will give some examples later down
the e-mail .

Back to the `ls' and file information as time, group etc..
As with many situations on the command-line, there are several ways that
lead to Rome, e.g. that could achieve this , it boils down to what you
personal prefer usually.

Personally I would go for a program that takes a printf statement. That
enables more flexibility generating appropiate filenames with a
bit of luck that makes sorting & classifying entries easier. 

printf used with `find' takes an argument which refers to a file property.
For example:
~: find . -type f -name '*.mp3' -printf "%A@_%f\n"

Now `find' finds all regulare files with names ending on .mp3 
and then calls printf with the arguments %A@ which stands for 
last access UTC time then a `_' underscore
''octal \137'' as seperator and finally %f which stands for the filename
itself ''without prepended directory path'', and \n which stands for a
newline . There are many arguments `find' can take . `man find' .

This previous example results in:
1113653372_Snooty-Composer1-highflownsymphony-inGmajor-Op.89674-movement1.mp3
1113653375_Snooty-Composer1-highflownsymphony-inGmajor-Op.89674-movement2.mp3
1113653379_Snooty-Composer1-highflownsymphony-inGmajor-Op.89674-movement3.mp3 

But let's leave find for a moment .

I recreated all the filenames you have listed above in my /tmp/ dir with:
~: while read LINE ; do touch ${LINE} ; done < filenames.txt

Looking at the filenames I note that each group shares the same first few
characters ? Is that true ? If that is the case you could cut those first
characters off, uniq them with `uniq` and create a directory for each so
that you can move the files into them . That way you sorted/grouped each
file into its own directory and you will only have to access that
directory to access each independent group of files.
Example:

~: ls -1 *.mp3 | while read LINE ; do echo ${LINE} | cut -c -15 ; done

Will result in the output:
Anton_Reicha-_A
Anton_Reicha-_A
Anton_Reicha-_A
Anton_Reicha-_A
Anton_Reicha-_A
Snooty-Composer
Snooty-Composer
Snooty-Composer

If we pipe this on to `uniq`; W'll get:

Anton_Reicha-_A
Snooty-Composer

Read this input, to create an new directory for each entry. The complete
command will become:
 
~: ls -1 *.mp3 | while read LINE ; do echo ${LINE} | cut -c -15 ; done |
uniq | while read DR ; do mkdir ${DR} ; mv ${DR}*.mp3 ${DR} ; done

Now the directory /tmp/ in which I previously created you filenames 
will list:
~: ls /tmp/
Anton_Reicha-_A/ 
Snooty-Composer/ 

With each directory holding the all the shared files:
ls /tmp/Snooty-Composer/
Snooty-Composer1-highflownsymphony-inGmajor-Op.89674-movement1.mp3
Snooty-Composer1-highflownsymphony-inGmajor-Op.89674-movement2.mp3
Snooty-Composer1-highflownsymphony-inGmajor-Op.89674-movement3.mp3

`cd' into the directory and:
~: cat *.mp3 >> /my/target/file.mp3

That last step could be automated without to much effort with the 
knowledge given above.

As an addition:
If you really want to do fancy you can use the levenshtein algorithm
to calculate the distance inbetween strings and sort them accordingly.
This is a great way to for example match character sequences that share
similiarities without positional parameters being out of bound. It works
by calculating the number of steps that it takes to transform one string
into another .

~:  levenshtein
Snooty-Composer1-highflownsymphony-inGmajor-Op.89674-movement1.mp3
Snooty-Composer1-highflownsymphony-inGmajor-Op.89674-movement2.mp3

will result in a score of 1

But again.. that's more off-topic.

> On Sat, 16 Apr 2005, J. wrote:
> 
> > Unix and GNU/Linux type systems do not store the file creation time as
> > static information. There are `three sorts of time` that are stored with
> > each file:
> >
> > C time - changed time info of the corresponding inode,
> > M time - last modified time,
> > A time - last access time
> >
> > These can be changed by other programs. For example `touch' . So be
> > careful how you implement this and how you group corrosponding files by
> > time .
> 
> I'm not doing anything with these files after they're saved to my HD other 
> than listing them (like, with ls) or looking at contents of directories 
> with mc. Could any of these change the time information ls -l gives me?

Good question. Only listing files, the answer is no. As long if you
don't access the content you shouldn't be to worried. Just don't `touch'
them or access them for example with an editor.

> > Personally I would have renamed each file correctly according to the
> > task I wanted to preform right from the start, thus to include
> > the last modified UTC time in the filename. That way you can list each
> > file belonging to a certain group `sort -n' them by time and concat them
> > as appropiate.
> 
> Good idea. I'm just technically-challenged enough that I have no idea how 
> to do this (or even where to start) where it concerns recording streaming 
> audio. I suppose I'd need to make streamripper run as part of a larger 
> script that would likewise invoke Flemming's remove_invalid, etc. Mind 
> boggling!

Indeed good suggestion, 8 times out of ten, sorting etc.. is done
because people don't store the information correctly as it arrives.

> > B.t.w. you could also use `dd', with the `notrunc' parameter . That is
> > more optimized for this task.
> 
> I'll look into this.

`dd' is a great tool, you can copy everything like whole partitions and 
easily created images from disks. It's very standard and included in 
every base system install.

> > The GLUE at the GNU/Linux , Unix type command-line provides was already
> > there long before. Pls. Have a look at the `UNIX philosophy' since I
> > think it will give some more insight in the background of these tasks
> > you are trying to accomplish.
> >
> > http://www.google.com/linux?hl=en&lr=&q=unix+philosophy&btnG=Search
> 
> Thanks for the link.
> 
> James

I hope I left you with some clou's and workable idea's & info... 
Good Luck, The other J.

-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

Re: help with joining files

[James Miller]

On Sat, 16 Apr 2005, James Miller wrote:

> I don't really need file permissions information, so using the -l switch is 
> overkill. But so far I have not determined how I can get the time information 
> I need without the other, file permission info.

I've found the -t switch to give me the time info I need without the 
permissions info. ls -t lists files in the order created, oldest last, so 
far as I can tell.

James
-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

Re: help with joining files

[Ray Olszewski]

Just a couple of minor comments, James.

First, though, a preliminary question: have you looked at the command-line 
options for streamripper? I just read its man page, and that says the app 
has these options available:

        -a [file]
               Rip to single file.

        The default mode of operation is to separate the each track into a sep-
        arate  file.   But  sometimes this is not what you want.  Sometimes you
        want the stream recorded to a single (big) file without splitting  into
        tracks.   The -a option does this.  If you use -a without including the
        [file], a timestamped filename will automatically be used.

        -A     Don't create individual tracks.

        The default mode of operation is to create one  file  for  each  track.
        But  sometimes you don't want these files.  For example, you might pre-
        fer a single file (using the -a option), or you want to use  streamrip-
        per  as  a  relay  (using the -r option), without creating these files.
        Using the -A option, the individual files for each track are  not  cre-
        ated.

Won't using these option avoid the problem you have in the first place?

streamrippeer also has options for moving the track-break position, See the 
man page for the details on this ... it will probably require a bit of 
experimenting in any case.

At 12:27 PM 4/16/2005 -0500, James Miller wrote:
[...]
>  I'm now using the "ls -l partial_file_name <tab>" to get the salient 
> information in a more discrete place from which I can copy and paste. I 
> do have to do some "ls partial_file_name <tab>ing" to even get to the 
> point where I can use ls -l (-l gives me the time information I need in 
> order to cat things together in the right order), but I've already 
> reduced the number of steps to get full_piece.mp3.

Depending on the *exact* problem you are having, the alternative approach 
of using
         ls -l | grep partial_file_name

may give you better results.

>>Personally I would have renamed each file correctly according to the
>>task I wanted to preform right from the start, thus to include
>>the last modified UTC time in the filename. That way you can list each
>>file belonging to a certain group `sort -n' them by time and concat them
>>as appropiate.
>
>Good idea. I'm just technically-challenged enough that I have no idea how 
>to do this (or even where to start) where it concerns recording streaming 
>audio. I suppose I'd need to make streamripper run as part of a larger 
>script that would likewise invoke Flemming's remove_invalid, etc. Mind 
>boggling!

While J's suggestion *sounds* reasonable, in fact it pretty much ignores 
the fact that you are dependent ... and least for starters ... on (a) what 
name data the radio station provides and (b) what streamripper is set up to 
do with it. From the examples you've provided, I'd guess that the radio 
station is giving you names in, or close to, the format the the FreeDB CDDB 
servers use.

Those names *can* be preprocessed into filenames more gracefully than 
streamripper seems to ... for example, abcde (the standard CD-ripping 
wrapper script on Linux, which in turn calls a bunch of task-specific apps 
for getting tracklists, ripping to wav, encoding to mp3 or orr, and so 
forth) includes some preprocssing capability. ot enough to number 
movements, though (unless the number is part of the name); for example, a 
multi-movement Gershwin piece I ripped shows up as

         Concerto_In_F_for_Piano_and_Orchestra_-_Adagio.ogg
         Concerto_In_F_for_Piano_and_Orchestra_-_Allegro.ogg
         Concerto_In_F_for_Piano_and_Orchestra_-_Allegro_agitato.ogg

BTW, abcde uses a script to process CDDB tracknames into more 
Unix-friendlty filenames. Here's the man-page excerpt describing the function:

        mungefilename
               mungefilename() is an abcde shell function that can be  overrid-
               den  via  abcde.conf.  It  takes CDDB data as $1 and outputs the
               resulting filename on stdout.  It  defaults  to  eating  control
               characters,  apostrophes  and question marks, translating spaces
               and forward slashes to underscores, and translating colons to an
               underscore and a hyphen.
               If  you modify this function, it is probably a good idea to keep
               the forward slash munging (UNIX cannot store a file with  a  '/'
               char in it) as well as the control character munging (NULs can't
               be in a filename either, and newlines and such in filenames  are
               typically not desirable).

And here is the actual function in its entirety:

         mungefilename ()
         {
                         echo "$@" | sed s,:,\ -,g | tr \ /\* __+ | tr -d 
\'\"\?\[:cntrl:\]
         }

After writing the above, I checked a bit into what parsing options 
streamripper has. It has an option, -w, that can be used to provide parsing 
rules. Look at the file parse-rules.txt (location distro depsndent; on 
Debian, look in /usr/share/doc/streamripper) for the details. The syntax of 
this file is a bit different from sed or tr or whatever, but you might be 
able to adapt the abcde rule to its syntax and get tidy filenames out of 
the box that way.





-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

Re: help with joining files

[James Miller]

Hello Ray:

On Sat, 16 Apr 2005, Ray Olszewski wrote:

> options for streamripper? I just read its man page, and that says the app has 
> these options available:
>
>       -a [file]
>              Rip to single file.
<snip>
>
> Won't using these option avoid the problem you have in the first place?

I'm not sure. I did see that. Since I'm recording for many hours at a 
time, wouldn't I come up with a 300MB file or something? I chose the 
default method because I would like for pieces to play individually, and 
I'd also like to know what I'm listening to (I'm not much of a classical 
officianado, but through this station I've already discovered a Baroque 
composer I like that I didn't even know about--Boccherini). Both of the 
"single file" options seem to me to mean that the file is the length of 
the recording session, and that the program does not present information 
or make any sort of divisions in that file (say, by composer). But 
documentation is not always as descriptive as it could/should be, so 
likely only by experimentation could I determine for sure exactly what the 
-a and -A options do. Have you tried them, or is it perhaps more clear to 
you from the documentation than it is to me what they do?

Pondering over your other suggestions . . .

James

PS Btw, what does "track" translate to in classical music terms? And does 
the -r option, for relay, make the program play rather than record?
-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

Re: help with joining files

[Ray Olszewski]

At 03:07 PM 4/16/2005 -0500, James Miller wrote:
>Hello Ray:
>
>On Sat, 16 Apr 2005, Ray Olszewski wrote:
>
>>options for streamripper? I just read its man page, and that says the app 
>>has these options available:
>>
>>       -a [file]
>>              Rip to single file.
><snip>
>>
>>Won't using these option avoid the problem you have in the first place?
>
>I'm not sure. I did see that. Since I'm recording for many hours at a 
>time, wouldn't I come up with a 300MB file or something? I chose the 
>default method because I would like for pieces to play individually, and 
>I'd also like to know what I'm listening to (I'm not much of a classical 
>officianado, but through this station I've already discovered a Baroque 
>composer I like that I didn't even know about--Boccherini). Both of the 
>"single file" options seem to me to mean that the file is the length of 
>the recording session, and that the program does not present information 
>or make any sort of divisions in that file (say, by composer). But 
>documentation is not always as descriptive as it could/should be, so 
>likely only by experimentation could I determine for sure exactly what the 
>-a and -A options do. Have you tried them, or is it perhaps more clear to 
>you from the documentation than it is to me what they do?

I've never actually used streamripper at all ... finding suitable streams 
to capture, or even listen too, has always seemed to me like more trouble 
than the benefit is worth (though this is clearly a place where personal 
preferences will vary).  So I have no hands-on experience using these, or 
any other, options.

But I think your interpretations of the -a and -A options are correct. 
Those options would, probably, work better in a setting something like a 
timed recording of "The Classical Hour" at 8 PM every weeknight ... that 
sort of thing.

BTW, what station are you recording? Perhaps I should give it a try, though 
my tastes in classical music run more to a mix of early stuff like Bach and 
some of the Romantics.

>Pondering over your other suggestions . . .
>
>James
>
>PS Btw, what does "track" translate to in classical music terms?

It depends. In the case of streams, it is some sort of mix of the breaks 
... as deduced by silence gaps ... and changes in the accompanying 
metadata. Track separation is an art, not a science, and the heuristics 
that programs use are imperfect. (At least that's true of gramofile, which 
I do have experience with.)

On CDs, tracks are whatever the record company says they are ... in 
practice, each movement of a Symphony is usually a separate track, and each 
shorter piece (a Fugue, say, or a Polonaise ... anything up to a Concerto) 
is a single track.

>And does the -r option, for relay, make the program play rather than record?

Your reading is as good as mine here. My *guess* is that this option makes 
streamripper act just as a proxy, so (for example) a bunch of hosts -- the 
number depends on the -R setting -- on a LAN can share listening in real 
time to a single Internet stream.


-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

Re: help with joining files

[James Miller]

On Sat, 16 Apr 2005, Ray Olszewski wrote:

> BTW, what station are you recording? Perhaps I should give it a try, though 
> my tastes in classical music run more to a mix of early stuff like Bach and 
> some of the Romantics.

http://61.74.65.198:8000 Good sound quality, no commercials or DJ chat. So 
far mostly obscure Baroque period stuff.

James
-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

Re: help with joining files

[James Miller]

I've discovered that ls -tr is what I need to get the files listed in the 
right time sequence (time of creation). ls -t gives, for example:

Boccherini_-_Quintet_for_Strings_in_E_G.275_Mvt.4_Rondo_-_Andante.mp3
Boccherini_-_Quintet_for_Strings_in_E_G.275_Mvt.3_Minuetto.mp3
Boccherini_-_Quintet_for_Strings_in_E_G.275_Mvt.2_Allegro_con_spirito.mp3
Boccherini_-_Quintet_for_Strings_in_E_G.275_Mvt.1_Amoroso.mp3

while ls -tr gives:

Boccherini_-_Quintet_for_Strings_in_E_G.275_Mvt.1_Amoroso.mp3
Boccherini_-_Quintet_for_Strings_in_E_G.275_Mvt.2_Allegro_con_spirito.mp3
Boccherini_-_Quintet_for_Strings_in_E_G.275_Mvt.3_Minuetto.mp3
Boccherini_-_Quintet_for_Strings_in_E_G.275_Mvt.4_Rondo_-_Andante.mp3

(assuming the directory where the command is issued contains only these 
files). Presently, since those are just excerpts and there are many more 
files in that dir, I am navigating to that dir and doing ls -tr 
>ls-tr_date.txt. I then go into that text file with an editor adding "cat" 
at the beginning of the piece, taking out carriage returns between 
movments, and adding >~/MyMusic/name-of-piece.mp3 at the end of the 
movement. Using the above output, this would look like;

cat Boccherini_-_Quintet_for_Strings_in_E_G.275_Mvt.1_Amoroso.mp3
Boccherini_-_Quintet_for_Strings_in_E_G.275_Mvt.2_Allegro_con_spirito.mp3
Boccherini_-_Quintet_for_Strings_in_E_G.275_Mvt.3_Minuetto.mp3
Boccherini_-_Quintet_for_Strings_in_E_G.275_Mvt.4_Rondo_-_Andante.mp3 
>~/MyMusic/Boccherini_-_Quintet_for_Strings_in_E_G.275.mp3

Then I cut and paste this into an xterm wherein I have navigated to the 
dir containing the files, then hit return. This is as streamlined as I've 
been able to get with this so far. Still time-consuming and a bit 
cumbersome, but I've progressed a little.

James

PS Again, the above excerpts, where movement number is actually given, are 
exceptions to the rule. I just selected output that has numbers since I 
thought it would more readily demonstrative of what I'm doing.
-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs