Updates on "mv", "cp" commands

Updates on "mv", "cp" commands

[BRAGA, Bruno]

Hi,

I don't know if that's the place to post this question, but I am sure
someone here will be able to give me hints on how to proceed.

I would like to propose and patch improvements on the "mv" (move) and
"cp" (copy) commands, by adding options for progress information.

The idea is that any GUI actions already do it graphically, and other
CLI applications, such as "rsync", have this as an extended option
"--progress". This is handy for large files processing, for instance.

I looked for the code in the kernel source but I did not find any. I
could find some sources on the GNU OS, but I am not sure if that's the
right place (probably not).

Can anyone give me a little help? Maybe this is even an old idea that
was discarded because of performance reasons, I don't know..

Thanks in advance,

Bruno

Re: Updates on "mv", "cp" commands

[Michal Nazarewicz]

[-- Attachment #1: Type: text/plain, Size: 1399 bytes --]

"BRAGA, Bruno" <bruno.braga@gmail.com> writes:
> I don't know if that's the place to post this question, but I am sure
> someone here will be able to give me hints on how to proceed.
>
> I would like to propose and patch improvements on the "mv" (move) and
> "cp" (copy) commands, by adding options for progress information.

That's something I was wondering about some time ago myself.

> The idea is that any GUI actions already do it graphically, and other
> CLI applications, such as "rsync", have this as an extended option
> "--progress". This is handy for large files processing, for instance.
>
> I looked for the code in the kernel source but I did not find any. I
> could find some sources on the GNU OS, but I am not sure if that's the
> right place (probably not).

You won't find any sources of cp or mv in Linux source code because cp
and mv commands are user space and as such they are not part of Linux
per se.

If you are using some kind of GNU/Linux distribution the place to look
is GNU coreutils package.  Consult gnu.org webpage for it's source code
and information how to submit patches.

-- 
Best regards,                                         _     _
 .o. | Liege of Serenly Enlightened Majesty of      o' \,=./ `o
 ..o | Computer Science,  Michal "mina86" Nazarewicz   (o o)
 ooo +--<mina86*tlen.pl>--<jid:mina86*jabber.org>--ooO--(_)--Ooo--

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: Updates on "mv", "cp" commands

[BRAGA, Bruno]

Thanks for the reply Michal.

I will look into that (I found the source in GNU's source code, but I
am still kinda lost on to whom should I ask stuff, lol). i will let
you know if/when I get things done with the team there.

Cheers,

******************************
Braga, Bruno
http://www.brunobraga.net
bruno.braga@gmail.com
*******************************

"The future belongs to people who see possibilities before they become
obvious!." -- Ted Levitt



2009/11/1 Michal Nazarewicz <mina86@tlen.pl>:
> "BRAGA, Bruno" <bruno.braga@gmail.com> writes:
>> I don't know if that's the place to post this question, but I am sure
>> someone here will be able to give me hints on how to proceed.
>>
>> I would like to propose and patch improvements on the "mv" (move) and
>> "cp" (copy) commands, by adding options for progress information.
>
> That's something I was wondering about some time ago myself.
>
>> The idea is that any GUI actions already do it graphically, and other
>> CLI applications, such as "rsync", have this as an extended option
>> "--progress". This is handy for large files processing, for instance.
>>
>> I looked for the code in the kernel source but I did not find any. I
>> could find some sources on the GNU OS, but I am not sure if that's the
>> right place (probably not).
>
> You won't find any sources of cp or mv in Linux source code because cp
> and mv commands are user space and as such they are not part of Linux
> per se.
>
> If you are using some kind of GNU/Linux distribution the place to look
> is GNU coreutils package.  Consult gnu.org webpage for it's source code
> and information how to submit patches.
>
> --
> Best regards,                                         _     _
>  .o. | Liege of Serenly Enlightened Majesty of      o' \,=./ `o
>  ..o | Computer Science,  Michal "mina86" Nazarewicz   (o o)
>  ooo +--<mina86*tlen.pl>--<jid:mina86*jabber.org>--ooO--(_)--Ooo--
>
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Updates on "mv", "cp" commands

[Renato Callado Borges]

[-- Attachment #1: Type: text/plain, Size: 3362 bytes --]

Hi Bruno!

Wouldn't you prefer to build something on top of cp and mv, like a wrapper script? I did a sketch for such a wrapper for cp, tell me what you think.

#!/bin/bash

if [ ! -n "$2" ]; then
    echo "Usage: $0 input_file output_file"
fi

LS_ALIAS=`alias | grep "ls='" | cut -d "'" -f 2`
if [ -n "$LS_ALIAS" ]; then
    unalias ls
fi

SIZE1=`ls -l "$1" | cut -d " " -f 5`
cp "$1" "$2" &
sleep 1
echo -n "Copying: $1 -> $2 ["
tput sc
SIZE2=`ls -l "$2" | cut -d " " -f 5`
while [ $SIZE2 -lt $SIZE1 ]; do
    PERCENTAGE=`echo "a=$SIZE2/$SIZE1;100*a" | bc -l | cut -d "." -f 1`
    tput rc
    echo -n "$PERCENTAGE%]"
    sleep 1
    SIZE2=`ls -l $2 | cut -d " " -f 5`
done
echo " Done."

if [ -n "$LS_ALIAS" ]; then
    alias ls="$LS_ALIAS"
fi

Be careful: this script might fail if cp has to overwrite files, and there are race conditions (that's the reason for the sleep outside the while loop).

Cheers,
Renato.

On Sun, Nov 01, 2009 at 08:47:21PM +0900, BRAGA, Bruno wrote:
> Thanks for the reply Michal.
> 
> I will look into that (I found the source in GNU's source code, but I
> am still kinda lost on to whom should I ask stuff, lol). i will let
> you know if/when I get things done with the team there.
> 
> Cheers,
> 
> ******************************
> Braga, Bruno
> http://www.brunobraga.net
> bruno.braga@gmail.com
> *******************************
> 
> "The future belongs to people who see possibilities before they become
> obvious!." -- Ted Levitt
> 
> 
> 
> 2009/11/1 Michal Nazarewicz <mina86@tlen.pl>:
> > "BRAGA, Bruno" <bruno.braga@gmail.com> writes:
> >> I don't know if that's the place to post this question, but I am sure
> >> someone here will be able to give me hints on how to proceed.
> >>
> >> I would like to propose and patch improvements on the "mv" (move) and
> >> "cp" (copy) commands, by adding options for progress information.
> >
> > That's something I was wondering about some time ago myself.
> >
> >> The idea is that any GUI actions already do it graphically, and other
> >> CLI applications, such as "rsync", have this as an extended option
> >> "--progress". This is handy for large files processing, for instance.
> >>
> >> I looked for the code in the kernel source but I did not find any. I
> >> could find some sources on the GNU OS, but I am not sure if that's the
> >> right place (probably not).
> >
> > You won't find any sources of cp or mv in Linux source code because cp
> > and mv commands are user space and as such they are not part of Linux
> > per se.
> >
> > If you are using some kind of GNU/Linux distribution the place to look
> > is GNU coreutils package.  Consult gnu.org webpage for it's source code
> > and information how to submit patches.
> >
> > --
> > Best regards,                                         _     _
> >  .o. | Liege of Serenly Enlightened Majesty of      o' \,=./ `o
> >  ..o | Computer Science,  Michal "mina86" Nazarewicz   (o o)
> >  ooo +--<mina86*tlen.pl>--<jid:mina86*jabber.org>--ooO--(_)--Ooo--
> >
> --
> To unsubscribe from this list: send the line "unsubscribe linux-console" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Renato 字. []'s, RCB.

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Updates on "mv", "cp" commands

[BRAGA, Bruno]

Hi Renato,

thanks for the feedback.

In deed, I found some solutions such as wrapper scripts (the simplest
I saw using pv), and the aliases using rsync. For now, I will just
stick with the rsync aliases for simplicity.

pv solution:
pv big_file.tgz > /some_dir/sub_dir/big_file.tgz

aliases solution: (place on ~/.bashrc)
alias rscp=’rsync -aP –no-whole-file –inplace’
alias rsmv=’rscp –remove-source-files’

strace solution:

#!/bin/sh
cp_p()
{
   strace -q -ewrite cp -- "${1}" "${2}" 2>&1 \

      | awk '{
        count += $NF
            if (count % 10 == 0) {
               percent = count / total_size * 100

               printf "%3d%% [", percent
               for (i=0;i<=percent;i++)
                  printf "="
               printf ">"

               for (i=percent;i<100;i++)
                  printf " "
               printf "]\r"
            }
         }
         END { print "" }' total_size=$(stat -c '%s' "${1}") count=0
}

sources:
http://www.debianadmin.com/pv-pipe-viewer-shell-pipeline-element-to-meter-data-passing-through.html
http://chris-lamb.co.uk/2008/01/24/can-you-get-cp-to-give-a-progress-bar-like-wget/

I placed this question under the GNU Coreutils mailing list, and got
lots of feedback too.

The discussion seems to be long, and getting kinda philosophic as
well. Let's see where this goes.
Here is another thread (old one) talking more about this:
https://bugzilla.redhat.com/show_bug.cgi?id=237553

thank you all for the help.

Bruno

On Mon, Nov 2, 2009 at 2:26 PM, Renato Callado Borges
<renato.oferenda@gmail.com> wrote:
>
> Hi Bruno!
>
> Wouldn't you prefer to build something on top of cp and mv, like a wrapper script? I did a sketch for such a wrapper for cp, tell me what you think.
>
> #!/bin/bash
>
> if [ ! -n "$2" ]; then
>    echo "Usage: $0 input_file output_file"
> fi
>
> LS_ALIAS=`alias | grep "ls='" | cut -d "'" -f 2`
> if [ -n "$LS_ALIAS" ]; then
>    unalias ls
> fi
>
> SIZE1=`ls -l "$1" | cut -d " " -f 5`
> cp "$1" "$2" &
> sleep 1
> echo -n "Copying: $1 -> $2 ["
> tput sc
> SIZE2=`ls -l "$2" | cut -d " " -f 5`
> while [ $SIZE2 -lt $SIZE1 ]; do
>    PERCENTAGE=`echo "a=$SIZE2/$SIZE1;100*a" | bc -l | cut -d "." -f 1`
>    tput rc
>    echo -n "$PERCENTAGE%]"
>    sleep 1
>    SIZE2=`ls -l $2 | cut -d " " -f 5`
> done
> echo " Done."
>
> if [ -n "$LS_ALIAS" ]; then
>    alias ls="$LS_ALIAS"
> fi
>
> Be careful: this script might fail if cp has to overwrite files, and there are race conditions (that's the reason for the sleep outside the while loop).
>
> Cheers,
> Renato.
>
> On Sun, Nov 01, 2009 at 08:47:21PM +0900, BRAGA, Bruno wrote:
> > Thanks for the reply Michal.
> >
> > I will look into that (I found the source in GNU's source code, but I
> > am still kinda lost on to whom should I ask stuff, lol). i will let
> > you know if/when I get things done with the team there.
> >
> > Cheers,
> >
> > Braga, Bruno
> >
> >
> >
> > 2009/11/1 Michal Nazarewicz <mina86@tlen.pl>:
> > > "BRAGA, Bruno" <bruno.braga@gmail.com> writes:
> > >> I don't know if that's the place to post this question, but I am sure
> > >> someone here will be able to give me hints on how to proceed.
> > >>
> > >> I would like to propose and patch improvements on the "mv" (move) and
> > >> "cp" (copy) commands, by adding options for progress information.
> > >
> > > That's something I was wondering about some time ago myself.
> > >
> > >> The idea is that any GUI actions already do it graphically, and other
> > >> CLI applications, such as "rsync", have this as an extended option
> > >> "--progress". This is handy for large files processing, for instance.
> > >>
> > >> I looked for the code in the kernel source but I did not find any. I
> > >> could find some sources on the GNU OS, but I am not sure if that's the
> > >> right place (probably not).
> > >
> > > You won't find any sources of cp or mv in Linux source code because cp
> > > and mv commands are user space and as such they are not part of Linux
> > > per se.
> > >
> > > If you are using some kind of GNU/Linux distribution the place to look
> > > is GNU coreutils package.  Consult gnu.org webpage for it's source code
> > > and information how to submit patches.
> > >
> > > --
> > > Best regards,                                         _     _
> > >  .o. | Liege of Serenly Enlightened Majesty of      o' \,=./ `o
> > >  ..o | Computer Science,  Michal "mina86" Nazarewicz   (o o)
> > >  ooo +--<mina86*tlen.pl>--<jid:mina86*jabber.org>--ooO--(_)--Ooo--
> > >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-console" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
> --
> Renato 字. []'s, RCB.
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Updates on "mv", "cp" commands

[renato.oferenda]

[-- Attachment #1: Type: text/plain, Size: 593 bytes --]

Hello Bruno!

On Sun, Nov 01, 2009 at 09:30:21AM +0900, BRAGA, Bruno wrote:
> I looked for the code in the kernel source but I did not find any. I
> could find some sources on the GNU OS, but I am not sure if that's the
> right place (probably not).

A good place to start anything in linux is at the man page for the software. ("man" == shorthand for "manual").

If you "man cp" and then "man mv" you'll see on the last line for each man page that they both belong to the GNU coreutils package. (http://www.gnu.org/software/coreutils/).

Good luck!

-- 
Renato 字. []'s, RCB.

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: Updates on "mv", "cp" commands

[BRAGA, Bruno]

Wow thanks for the tip... I was pretty lost on who to contact.

Cheers,

******************************
Braga, Bruno



On Sun, Nov 1, 2009 at 9:20 PM,  <renato.oferenda@gmail.com> wrote:
> Hello Bruno!
>
> On Sun, Nov 01, 2009 at 09:30:21AM +0900, BRAGA, Bruno wrote:
>> I looked for the code in the kernel source but I did not find any. I
>> could find some sources on the GNU OS, but I am not sure if that's the
>> right place (probably not).
>
> A good place to start anything in linux is at the man page for the software. ("man" == shorthand for "manual").
>
> If you "man cp" and then "man mv" you'll see on the last line for each man page that they both belong to the GNU coreutils package. (http://www.gnu.org/software/coreutils/).
>
> Good luck!
>
> --
> Renato å­—. []'s, RCB.
>
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Updates on "mv", "cp" commands

[renato.oferenda]

[-- Attachment #1: Type: text/plain, Size: 234 bytes --]

On Sun, Nov 01, 2009 at 09:51:03PM +0900, BRAGA, Bruno wrote:
> Wow thanks for the tip... I was pretty lost on who to contact.

No problem! Let us know if you develop pretty console cp's and mv's :)

-- 
Renato 字. []'s, RCB.

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]