[PATCH] Adds bittorrent support to 45url-lib for live images

[PATCH] Adds bittorrent support to 45url-lib for live images

[Antony Messerli]

This patch adds bittorrent support to 45url-lib for those that might want to
retrive the same live image for multiple systems at once without
saturating the network.
This patch requires ctorrent to be installed into initramfs.

Torrent kernel command line format:
root=live:torrent://image_location/image.torrent

Start a tracker:
bttrack --bind <tracker_ip> --port 6969 --dfile dstate --reannounce_interval 60

Create the torrent:
ctorrent -t live_image -u http://<tracker_ip>:6969/announce -s
live_image.torrent

Seed the initial torrent:
ctorrent live_image.torrent

Boot the live image.
---
 modules.d/45url-lib/url-lib.sh | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/modules.d/45url-lib/url-lib.sh b/modules.d/45url-lib/url-lib.sh
index c9d143a..d9cebd4 100755
--- a/modules.d/45url-lib/url-lib.sh
+++ b/modules.d/45url-lib/url-lib.sh
@@ -82,6 +82,36 @@ set_http_header() {
     echo "header = \"$1: $2\"" >> $CURL_HOME/.curlrc
 }

+### TORRENT ##########################################################
+
+ctorrent_args="-E 0 -e 0"
+
+ctorrent_fetch_url() {
+    local url="$1" outloc="$2"
+    url=${url#*//}
+    torrent_outloc="$outloc.torrent"
+    echo "$url" > /proc/self/fd/0
+    if [ -n "$outloc" ]; then
+        curl $curl_args --output - -- "$url" > "$torrent_outloc" || return $?
+    else
+        local outdir="$(mkuniqdir /tmp torrent_fetch_url)"
+        ( cd "$outdir"; curl $curl_args --remote-name "$url" || return $? )
+        torrent_outloc="$outdir/$(ls -A $outdir)"
+        outloc=${torrent_outloc%.*}
+    fi
+    if ! [ -f "$torrent_outloc" ]; then
+        warn "Downloading '$url' failed!"
+        return 253
+    fi
+    ctorrent $ctorrent_args -s $outloc $torrent_outloc >&2
+    if ! [ -f "$outloc" ]; then
+        warn "Torrent download of '$url' failed!"
+        return 253
+    fi
+    if [ -z "$2" ]; then echo "$outloc" ; fi
+}
+add_url_handler ctorrent_fetch_url torrent
+
 ### NFS ##############################################################

 [ -e /lib/nfs-lib.sh ] && . /lib/nfs-lib.sh
--
2.0.4

Re: [PATCH] Adds bittorrent support to 45url-lib for live images

[Vasiliy Tolstov]

2014-08-15 0:31 GMT+04:00 Antony Messerli <antony-M9eEzXzj57sUjmZRR9AIpQ@public.gmane.org>:
> This patch adds bittorrent support to 45url-lib for those that might want to
> retrive the same live image for multiple systems at once without
> saturating the network.
> This patch requires ctorrent to be installed into initramfs.
>
> Torrent kernel command line format:
> root=live:torrent://image_location/image.torrent
>
> Start a tracker:
> bttrack --bind <tracker_ip> --port 6969 --dfile dstate --reannounce_interval 60
>
> Create the torrent:
> ctorrent -t live_image -u http://<tracker_ip>:6969/announce -s
> live_image.torrent
>
> Seed the initial torrent:
> ctorrent live_image.torrent


Wow! Very nice. I want to do something some time ago, but this is
looks very good!
Question for all - i have a patch for overlayfs support, also github
have patch for aufs support.
As i see main difference of top layer. How the best rewrite my (may be
aufs) patch to have bottom layer agnostic support?

-- 
Vasiliy Tolstov,
e-mail: v.tolstov-+9FY0jupvH6HXe+LvDLADg@public.gmane.org
jabber: vase-+9FY0jupvH6HXe+LvDLADg@public.gmane.org

Re: [PATCH] Adds bittorrent support to 45url-lib for live images

[Harald Hoyer]

On 14.08.2014 22:31, Antony Messerli wrote:
> This patch adds bittorrent support to 45url-lib for those that might want to
> retrive the same live image for multiple systems at once without
> saturating the network.
> This patch requires ctorrent to be installed into initramfs.
> 
> Torrent kernel command line format:
> root=live:torrent://image_location/image.torrent
> 
> Start a tracker:
> bttrack --bind <tracker_ip> --port 6969 --dfile dstate --reannounce_interval 60
> 
> Create the torrent:
> ctorrent -t live_image -u http://<tracker_ip>:6969/announce -s
> live_image.torrent
> 
> Seed the initial torrent:
> ctorrent live_image.torrent
> 
> Boot the live image.
> ---

Can you add

inst_multiple -o ctorrent

to module-setup.sh ?

Re: [PATCH] Adds bittorrent support to 45url-lib for live images

[Antony Messerli]

I've gone ahead and created a pull request with a few more changes here:

https://github.com/haraldh/dracut/pull/26

On Fri, Aug 15, 2014 at 5:34 AM, Harald Hoyer <harald-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> On 14.08.2014 22:31, Antony Messerli wrote:
>> This patch adds bittorrent support to 45url-lib for those that might want to
>> retrive the same live image for multiple systems at once without
>> saturating the network.
>> This patch requires ctorrent to be installed into initramfs.
>>
>> Torrent kernel command line format:
>> root=live:torrent://image_location/image.torrent
>>
>> Start a tracker:
>> bttrack --bind <tracker_ip> --port 6969 --dfile dstate --reannounce_interval 60
>>
>> Create the torrent:
>> ctorrent -t live_image -u http://<tracker_ip>:6969/announce -s
>> live_image.torrent
>>
>> Seed the initial torrent:
>> ctorrent live_image.torrent
>>
>> Boot the live image.
>> ---
>
> Can you add
>
> inst_multiple -o ctorrent
>
> to module-setup.sh ?
>

Re: [PATCH] Adds bittorrent support to 45url-lib for live images

[Harald Hoyer]

Thanks! Pushed along with a check for the ctorrent binary before adding it to
the url handler.
commit e84af01462b8b7c4b0efa2fd27bd28a01a8ef19a


On 15.08.2014 17:28, Antony Messerli wrote:
> I've gone ahead and created a pull request with a few more changes here:
> 
> https://github.com/haraldh/dracut/pull/26
> 
> On Fri, Aug 15, 2014 at 5:34 AM, Harald Hoyer <harald-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
>> On 14.08.2014 22:31, Antony Messerli wrote:
>>> This patch adds bittorrent support to 45url-lib for those that might want to
>>> retrive the same live image for multiple systems at once without
>>> saturating the network.
>>> This patch requires ctorrent to be installed into initramfs.
>>>
>>> Torrent kernel command line format:
>>> root=live:torrent://image_location/image.torrent
>>>
>>> Start a tracker:
>>> bttrack --bind <tracker_ip> --port 6969 --dfile dstate --reannounce_interval 60
>>>
>>> Create the torrent:
>>> ctorrent -t live_image -u http://<tracker_ip>:6969/announce -s
>>> live_image.torrent
>>>
>>> Seed the initial torrent:
>>> ctorrent live_image.torrent
>>>
>>> Boot the live image.
>>> ---
>>
>> Can you add
>>
>> inst_multiple -o ctorrent
>>
>> to module-setup.sh ?
>>