[lm-sensors] snapshot generation

[lm-sensors] snapshot generation

[Rudolf Marek]

Hello Axel,

I'm attaching a script that creates snapshots from SVN. It seems there is no
crontab available on the server, so I'm asking you kindly to run it somewhere.

The script is fairly simple, it just needs to know the target dir, plus the dir
where will be the script/temp repo placed. Check first few lines of script please

It may be executed with lm-sensors or i2c as parameter. We would like to have
daily snapshots for lm-sensors and weekly for i2c if possible.

I tested the script briefly and it seems to work.

Thanks,
Rudolf
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: do_snap
Url: http://lists.lm-sensors.org/pipermail/lm-sensors/attachments/20060624/ecb2a3d5/attachment.pl 

[lm-sensors] snapshot generation

[Jean Delvare]

Hi Rudolf,

> I'm attaching a script that creates snapshots from SVN. It seems there is no
> crontab available on the server, so I'm asking you kindly to run it somewhere.
> 
> The script is fairly simple, it just needs to know the target dir, plus the dir
> where will be the script/temp repo placed. Check first few lines of script please
> 
> It may be executed with lm-sensors or i2c as parameter. We would like to have
> daily snapshots for lm-sensors and weekly for i2c if possible.
> 
> I tested the script briefly and it seems to work.

Thanks for working on this. We had this feature on the old lm-sensors
site and it was very convenient for some users, so it would be nice to
have it on the new site too.

I think the following part of your script can be improved:

> # create another local copies
> cp -r "$WHICH" "$WHICH"-tmp
> 	
> # Recursively remove all .svn directories from folder k2
> find "$WHICH"-tmp -name .svn -print0 | xargs -0 rm -rf
> 
> # Create archive
> tar czf "$WWW/$WHICH-r$NEW-$date.tar.gz" "$WHICH"/

First of all I guess you really meant "$WHICH"-tmp on the last line, or
you will be archiving the copy which still has the .svn directories.
But even then, this isn't really efficient, neither as disk space nor
as processing time is concerned. Instead, you could tell tar not to
pick the .svn directories:

tar czf "$WWW/$WHICH-r$NEW-$date.tar.gz" --exclude .svn "$WHICH"/

So you don't have to duplicate the tree.

Thanks,
-- 
Jean Delvare

[lm-sensors] snapshot generation

[Rudolf Marek]

Hi Jean,
> Thanks for working on this. We had this feature on the old lm-sensors
> site and it was very convenient for some users, so it would be nice to
> have it on the new site too.

It did not took too long -in the magnitude of minutes.

> I think the following part of your script can be improved:
> 
>> # create another local copies
>> cp -r "$WHICH" "$WHICH"-tmp
>> 	
>> # Recursively remove all .svn directories from folder k2
>> find "$WHICH"-tmp -name .svn -print0 | xargs -0 rm -rf
>>
>> # Create archive
>> tar czf "$WWW/$WHICH-r$NEW-$date.tar.gz" "$WHICH"/
> 
> First of all I guess you really meant "$WHICH"-tmp on the last line, or
> you will be archiving the copy which still has the .svn directories.
> But even then, this isn't really efficient, neither as disk space nor
> as processing time is concerned. Instead, you could tell tar not to
> pick the .svn directories:
> 
> tar czf "$WWW/$WHICH-r$NEW-$date.tar.gz" --exclude .svn "$WHICH"/

Yes you are correct. This part was based on original script and I was lazy to
change it. And made the mistake ;) I will try to fix it tomorrow/today. I think
there was already a midnight here...

Regards
Rudolf

[lm-sensors] snapshot generation

[Axel Thimm]

Hi,

On Sat, Jun 24, 2006 at 02:30:06PM +0200, Rudolf Marek wrote:
> Hello Axel,
> 
> I'm attaching a script that creates snapshots from SVN. It seems there is no
> crontab available on the server, so I'm asking you kindly to run it somewhere.
> 
> The script is fairly simple, it just needs to know the target dir, plus the dir
> where will be the script/temp repo placed. Check first few lines of script please
> 
> It may be executed with lm-sensors or i2c as parameter. We would like to have
> daily snapshots for lm-sensors and weekly for i2c if possible.

OK. Would it perhaps make sense to have tarballs on demand? I'm doing
this for ivtv and you get the tarballs for possible branches (and even
tags aka releases) for free, too.

> I tested the script briefly and it seems to work.
> 
> Thanks,
> Rudolf

> #!/bin/bash
> #
> #  Create a snapshot copy of lm-sensors Nightly Build 
> #  http://zeo.unic.net.my/notes/nightly-build-script/
> #
> #  Modified by Rudolf Marek
> #
> #  Based on:
> #  (CC) 2006 Safirul Alredha.
> 
> # Show me the current date
> date=`date +%Y-%m-%d`
> 
> # our SVN server
> WHICH="$1"
> SVN="http://lm-sensors.org/svn/$WHICH/trunk"
> WWW="/var/www/assembler.cz/$WHICH"
> REPODIR="/home/ruik/buildsnap"
> 
> if [ $# != "1" ]; then
>     echo "Use with: i2c or lm-sensors as param"
>     exit 255;
> fi
> 
> 
> # Change to working space directory. This where all the magic happens.
> cd "$REPODIR"
> if [ ! -d "$WHICH" ] ; then
>     echo "Creating files for first usage"
>     mkdir -p $WWW
>     svn co "$SVN" "$WHICH"
>     echo "0" > "$WWW/LATEST-IS-r0"
> fi 
> 
> # update local copies
> svn update "$WHICH"	
> 
> OLD=`cat "$WWW/LATEST-IS-r"*`
> NEW=`svn info "$WHICH" | grep -r Revision | cut -f 2 -d " "`
> 
> #no need to update
> if [ "$NEW" -le "$OLD" ] ; then
>     exit 0;
> fi
> 
> # create another local copies
> cp -r "$WHICH" "$WHICH"-tmp
> 	
> # Recursively remove all .svn directories from folder k2
> find "$WHICH"-tmp -name .svn -print0 | xargs -0 rm -rf
> 
> # Create archive
> tar czf "$WWW/$WHICH-r$NEW-$date.tar.gz" "$WHICH"/
> 
> rm "$WWW/LATEST-IS-r"*
> echo "$NEW" > "$WWW/LATEST-IS-r$NEW"
> 
> # Dumping svn commit log
> svn log -v "$WHICH" > "$WWW/CHANGELOG"
> 
> # Delete folder
> rm -rf "$WHICH-tmp"

> _______________________________________________
> lm-sensors mailing list
> lm-sensors at lm-sensors.org
> http://lists.lm-sensors.org/mailman/listinfo/lm-sensors

-- 
Axel.Thimm at ATrpms.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.lm-sensors.org/pipermail/lm-sensors/attachments/20060625/f8171a17/attachment.bin 

[lm-sensors] snapshot generation

[Jean Delvare]

Hi Axel,

> > I'm attaching a script that creates snapshots from SVN. It seems there is no
> > crontab available on the server, so I'm asking you kindly to run it somewhere.
> > 
> > The script is fairly simple, it just needs to know the target dir, plus the dir
> > where will be the script/temp repo placed. Check first few lines of script please
> > 
> > It may be executed with lm-sensors or i2c as parameter. We would like to have
> > daily snapshots for lm-sensors and weekly for i2c if possible.
> 
> OK. Would it perhaps make sense to have tarballs on demand? I'm doing
> this for ivtv and you get the tarballs for possible branches (and even
> tags aka releases) for free, too.

Either way is fine with me. We don't need both. We don't use branches
either. So, whatever is easier for you.

For releases, it would make more sense to have a "static" download
area, like we had on the old site:
http://secure.netroedge.com/~lm78/archive/
So that we can also sign the archives. Can we have such a download area
on the new site?

It would also be convenient if we could have a "Download" link it
trac's top bar, as this is clearly one of the most popular page. Is it
possible? I'm surprised there isn't one by default.

Thanks,
-- 
Jean Delvare

[lm-sensors] snapshot generation

[Axel Thimm]

On Mon, Jun 26, 2006 at 09:56:08PM +0200, Jean Delvare wrote:
> For releases, it would make more sense to have a "static" download
> area, like we had on the old site:
> http://secure.netroedge.com/~lm78/archive/
> So that we can also sign the archives. Can we have such a download area
> on the new site?

I created

http://dl.lm-sensors.org/{lm-sensors,i2c}/{release,snapshot}

and synced over the contents of
http://secure.netroedge.com/~lm78/archive/

Maybe you'd like to restructure a bit, it's all under
/srv/lm-sensors/dl and owned by yourself. I would maybe move some
older releases in an old folder or similar.

I'll return on the subject of snapshots in another mail.

> It would also be convenient if we could have a "Download" link it
> trac's top bar, as this is clearly one of the most popular page. Is it
> possible? I'm surprised there isn't one by default.

That seems to be a bit difficult and is currently advised against
since it may conflict with upgrades

http://projects.edgewall.com/trac/wiki/TracInterfaceCustomization#MainTemplates

-- 
Axel.Thimm at ATrpms.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.lm-sensors.org/pipermail/lm-sensors/attachments/20060627/930f5fa2/attachment.bin 

[lm-sensors] snapshot generation

[Michael Renzmann]

Hi all.

> I'll return on the subject of snapshots in another mail.

If it helps I can provide a bash script that we use at madwifi.org for
generating snapshots once a day (see [1]). Snapshot tarballs are only
generated if there were changes since the last generated tarball. The
script is kept as general as possible and makes use of "configuration
files", so it's easy to reuse it for generating snapshots of more than one
repository and/or several directory trees in one repository.

Bye, Mike

[1] http://snapshots.madwifi.org

[lm-sensors] snapshot generation

[Michael Renzmann]

Hi (again).

Sorry for the noise (by adding yet another mail to this thread), I didn't
notice that part of Axel's mail before:

> On Mon, Jun 26, 2006 at 09:56:08PM +0200, Jean Delvare wrote:
>> It would also be convenient if we could have a "Download" link it
>> trac's top bar, as this is clearly one of the most popular page. Is it
>> possible? I'm surprised there isn't one by default.
>
> That seems to be a bit difficult and is currently advised against
> since it may conflict with upgrades

I wrote a plugin (which should work on Trac 0.9 as well as 0.10-dev, but
tested it on 0.9 only so far) which allows to add custom buttons to the
various navigation bars. You can see it in action on http://madwifi.org -
the "download" button in the mainnav bar is added by the plugin.

I will publish the plugin on trac-hacks.org as soon as I find a bit time
to document it, but can provide it to you along with a few lines of
explanation if you're interested.

Bye, Mike

[lm-sensors] snapshot generation

[Axel Thimm]

Hi,

On Sat, Jun 24, 2006 at 02:30:06PM +0200, Rudolf Marek wrote:
> I'm attaching a script that creates snapshots from SVN. It seems there is no
> crontab available on the server, so I'm asking you kindly to run it somewhere.
> 
> The script is fairly simple, it just needs to know the target dir, plus the dir
> where will be the script/temp repo placed. Check first few lines of script please
> 
> It may be executed with lm-sensors or i2c as parameter. We would like to have
> daily snapshots for lm-sensors and weekly for i2c if possible.
> 
> I tested the script briefly and it seems to work.

I've setup snapshots at

http://dl.lm-sensors.org/{lm-sensors,i2c}/snapshots/

They are checked daily, but only generated if there is an actual
change. I'm keeping the last 32 snapshots of both sub-projects.

I've manually initiated snapshots right now, and the scripts are
running at 06:00-06:10 CET

We can tune any parameter you like (cron frequency & time, URLs,
number of snapshots to keep etc).
-- 
Axel.Thimm at ATrpms.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.lm-sensors.org/pipermail/lm-sensors/attachments/20060627/89521a93/attachment.bin 

[lm-sensors] snapshot generation

[Axel Thimm]

Hi Mike,

On Tue, Jun 27, 2006 at 12:42:05PM +0200, Michael Renzmann wrote:
> Sorry for the noise (by adding yet another mail to this thread), I didn't
> notice that part of Axel's mail before:
> 
> > On Mon, Jun 26, 2006 at 09:56:08PM +0200, Jean Delvare wrote:
> >> It would also be convenient if we could have a "Download" link it
> >> trac's top bar, as this is clearly one of the most popular page. Is it
> >> possible? I'm surprised there isn't one by default.
> >
> > That seems to be a bit difficult and is currently advised against
> > since it may conflict with upgrades
> 
> I wrote a plugin (which should work on Trac 0.9 as well as 0.10-dev, but
> tested it on 0.9 only so far) which allows to add custom buttons to the
> various navigation bars. You can see it in action on http://madwifi.org -
> the "download" button in the mainnav bar is added by the plugin.
> 
> I will publish the plugin on trac-hacks.org as soon as I find a bit time
> to document it, but can provide it to you along with a few lines of
> explanation if you're interested.

Yes, that would be great! Thanks!
-- 
Axel.Thimm at ATrpms.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.lm-sensors.org/pipermail/lm-sensors/attachments/20060627/3f4195ff/attachment.bin 

[lm-sensors] snapshot generation

[Axel Thimm]

On Tue, Jun 27, 2006 at 12:32:41PM +0200, Michael Renzmann wrote:
> > I'll return on the subject of snapshots in another mail.
> 
> If it helps I can provide a bash script that we use at madwifi.org for
> generating snapshots once a day (see [1]). Snapshot tarballs are only
> generated if there were changes since the last generated tarball. The
> script is kept as general as possible and makes use of "configuration
> files", so it's easy to reuse it for generating snapshots of more than one
> repository and/or several directory trees in one repository.

Thanks, again, I think we're done in this part.

But looking at [1] I think we should sync the snapshot naming to what
madwifi is using (rev before date), it makes more sense. OK, done. :)

> [1] http://snapshots.madwifi.org
-- 
Axel.Thimm at ATrpms.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.lm-sensors.org/pipermail/lm-sensors/attachments/20060627/094edb36/attachment.bin 

[lm-sensors] snapshot generation

[Axel Thimm]

On Tue, Jun 27, 2006 at 12:53:44PM +0200, Axel Thimm wrote:
> On Tue, Jun 27, 2006 at 12:42:05PM +0200, Michael Renzmann wrote:
> > > On Mon, Jun 26, 2006 at 09:56:08PM +0200, Jean Delvare wrote:
> > >> It would also be convenient if we could have a "Download" link it
> > >> trac's top bar, as this is clearly one of the most popular page. Is it
> > >> possible? I'm surprised there isn't one by default.

> > I wrote a plugin (which should work on Trac 0.9 as well as 0.10-dev, but
> > tested it on 0.9 only so far) which allows to add custom buttons to the
> > various navigation bars. You can see it in action on http://madwifi.org -
> > the "download" button in the mainnav bar is added by the plugin.
> > 
> > I will publish the plugin on trac-hacks.org as soon as I find a bit time
> > to document it, but can provide it to you along with a few lines of
> > explanation if you're interested.
> 
> Yes, that would be great! Thanks!

Mike sent me the plugin with instructions and it's installed
now. There is a menu button "Downloads" that links to
dl.lm-sensors.org. It works as advertized :=)

Thanks again, Mike!
-- 
Axel.Thimm at ATrpms.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.lm-sensors.org/pipermail/lm-sensors/attachments/20060627/92b4415b/attachment.bin 

[lm-sensors] snapshot generation

[Rudolf Marek]

> I've setup snapshots at
> 
> http://dl.lm-sensors.org/{lm-sensors,i2c}/snapshots/
> 
> They are checked daily, but only generated if there is an actual
> change. I'm keeping the last 32 snapshots of both sub-projects.
> 
> I've manually initiated snapshots right now, and the scripts are
> running at 06:00-06:10 CET
> 
> We can tune any parameter you like (cron frequency & time, URLs,
> number of snapshots to keep etc).

Thank you all. I will put the wiki pointer to that area too.

Is it CET/CEST or just CET?

Thanks,
Regards
Rudolf

[lm-sensors] snapshot generation

[Jean Delvare]

Hi Axel,

> Mike sent me the plugin with instructions and it's installed
> now. There is a menu button "Downloads" that links to
> dl.lm-sensors.org. It works as advertized :=)

What I had in mind was rather a link to wiki/Download, then we remove
that link from our own "Starting points" menu. There are a few hints we
want people to read before they download. Can this be done? (And, for my
own culture, where is it configured?)

Thanks Mike :)

-- 
Jean Delvare

[lm-sensors] snapshot generation

[Axel Thimm]

On Fri, Jun 30, 2006 at 01:37:25PM +0200, Jean Delvare wrote:
> Hi Axel,
> 
> > Mike sent me the plugin with instructions and it's installed
> > now. There is a menu button "Downloads" that links to
> > dl.lm-sensors.org. It works as advertized :=)
> 
> What I had in mind was rather a link to wiki/Download, then we remove
> that link from our own "Starting points" menu. There are a few hints we
> want people to read before they download.

Ah, OK.

> Can this be done? (And, for my own culture, where is it configured?)

Yes, it can (just did). If you can read german I'll forward you Mike's
instructions. Even if not, it's very intuitive and it's all within
trac.ini:

[components]
navadd.* = enabled

[navadd]
add_items = download
download.title = Download
download.url = /wiki/Download
download.target = mainnav

So it can easily be further extended if/when you wish so.

> Thanks Mike :)
-- 
Axel.Thimm at ATrpms.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.lm-sensors.org/pipermail/lm-sensors/attachments/20060630/555eb548/attachment.bin 

[lm-sensors] snapshot generation

[Axel Thimm]

On Fri, Jun 30, 2006 at 10:06:27AM +0200, Rudolf Marek wrote:
> > I've setup snapshots at
> > 
> > http://dl.lm-sensors.org/{lm-sensors,i2c}/snapshots/
> > 
> > They are checked daily, but only generated if there is an actual
> > change. I'm keeping the last 32 snapshots of both sub-projects.
> > 
> > I've manually initiated snapshots right now, and the scripts are
> > running at 06:00-06:10 CET
> > 
> > We can tune any parameter you like (cron frequency & time, URLs,
> > number of snapshots to keep etc).
> 
> Thank you all. I will put the wiki pointer to that area too.
> 
> Is it CET/CEST or just CET?

Currently it's CEST, in the winter it will be CET. de and cz have the
same local time, so it's 06:00-06:10 your local time. :)

If you prefer another scheduling, just let me know.
-- 
Axel.Thimm at ATrpms.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.lm-sensors.org/pipermail/lm-sensors/attachments/20060630/3a3e8a73/attachment.bin 

[lm-sensors] snapshot generation

[Jean Delvare]

Hi Axel,

> I've setup snapshots at
> 
> http://dl.lm-sensors.org/{lm-sensors,i2c}/snapshots/
> 
> They are checked daily, but only generated if there is an actual
> change. I'm keeping the last 32 snapshots of both sub-projects.
> 
> I've manually initiated snapshots right now, and the scripts are
> running at 06:00-06:10 CET
> 
> We can tune any parameter you like (cron frequency & time, URLs,
> number of snapshots to keep etc).

It looks like the timestamps are not preserved by the snapshot
operation. Can it be improved?

Thanks,
-- 
Jean Delvare

[lm-sensors] snapshot generation

[Axel Thimm]

On Sun, Aug 13, 2006 at 10:18:49AM +0200, Jean Delvare wrote:
> > I've setup snapshots at
> > 
> > http://dl.lm-sensors.org/{lm-sensors,i2c}/snapshots/
> > 
> > They are checked daily, but only generated if there is an actual
> > change. I'm keeping the last 32 snapshots of both sub-projects.
> > 
> > I've manually initiated snapshots right now, and the scripts are
> > running at 06:00-06:10 CET
> > 
> > We can tune any parameter you like (cron frequency & time, URLs,
> > number of snapshots to keep etc).
> 
> It looks like the timestamps are not preserved by the snapshot
> operation. Can it be improved?

Hm, that was how it was written, and that's how it ends if invoked
manually. It looks like cron squashed my $HOME, where the subversion
config variables are set, so I hardwired it into the scripts. Can you
check on the next revision?
-- 
Axel.Thimm at ATrpms.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.lm-sensors.org/pipermail/lm-sensors/attachments/20060813/4d6e9b68/attachment.bin 

[lm-sensors] snapshot generation

[Jean Delvare]

Hi Axel,

> On Sun, Aug 13, 2006 at 10:18:49AM +0200, Jean Delvare wrote:
> > It looks like the timestamps are not preserved by the snapshot
> > operation. Can it be improved?
> 
> Hm, that was how it was written, and that's how it ends if invoked
> manually. It looks like cron squashed my $HOME, where the subversion
> config variables are set, so I hardwired it into the scripts. Can you
> check on the next revision?

Timestamps are now properly preserved for regular files, but still not
for directories. Regular files are what I care mostly about anyway.

-- 
Jean Delvare

[lm-sensors] snapshot generation

[Axel Thimm]

Hi,

On Mon, Aug 14, 2006 at 07:49:30AM +0200, Jean Delvare wrote:
> > On Sun, Aug 13, 2006 at 10:18:49AM +0200, Jean Delvare wrote:
> > > It looks like the timestamps are not preserved by the snapshot
> > > operation. Can it be improved?
> > 
> > Hm, that was how it was written, and that's how it ends if invoked
> > manually. It looks like cron squashed my $HOME, where the subversion
> > config variables are set, so I hardwired it into the scripts. Can you
> > check on the next revision?
> 
> Timestamps are now properly preserved for regular files, but still not
> for directories. Regular files are what I care mostly about anyway.

I'm not even sure svn stores folder timestamps at all.

Just FYI the flag to use to have svn co use the real timestamps of
files is use-commit-times under [miscellany] in ~/.subversion/config.

[You should think about switching it on on your own checkouts or not,
 as it may break your make building. See info automake cvs ("26.1.2
 Background: CVS and timestamps"). Though that's not an issue for
 creating tarballs, only for continuous updates/commits work cycles.]

But I didn't find any switch to manipulate folder timestamps on
checkouts/updates.
-- 
Axel.Thimm at ATrpms.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.lm-sensors.org/pipermail/lm-sensors/attachments/20060814/2dd39899/attachment.bin