YAPP server pgm

YAPP server pgm

[Bob Morgan]

I have been watching some of the discussion go past here
about YAPP.  A few years ago I hacked up the source for
call (the linux ax25 packet user pgm) and extracted the yapp
portion of the code out as two separate programs.  I got the
download half of it working to my satisfaction, and I
never have yet had time to finish the upload half of it.
I call them yd.c and yu.c, respectively, for the usual
BBS commands with which users usually associate them.

I am not sure if these programs I wrote are what the
original requestor wanted or not.  Here is what they do:
They sit on an unattended AX.25 capable file server, and users connect
to them and download files.  If I had the upload half of
the program finished, they could upload files to the server also.
The normal way I have implemented them is as an external command
of the NODE program, where I have configured NODE to redirect
transparent stdin/stdout to these programs.  (I also have
made some minor patches to NODE to handle some transparency
issues, and from memory I don't remember if the patches
affected this interface or some others.)  These programs
are called by some hooks in node.conf, or possibly
in ax25d.conf.  They are NOT command line driven in any way,
and are NOT useable from the keyboard locally.  (For this, I
use the standard CALL application.)

Along a little different thread, here are my side-by-side
comparisons of yapp vs. ftp as a file transfer process:

I usually start to see problems with YAPP on ax25 radio
channels with a filesize greater than roughly 20K to 30K.
Yapp protocol just buffers up as much of the file as
there is comm buffer space to handle, after the initial
handshaking headers are exchanged and the data transfer
state starts up, and continues to refill buffers from
there on to the end of the file.  It doesn't have any
application-layer flow control or pacing.  Sure, AX.25 has some
hop-by-hop short term flow control, but that doesn't
seem to be sufficient.  I have also been known to
use it in some mixed networks where some of the packets
are ax.25 over ethernet (BPQ, at ethernet speeds of course),
followed by rerouting these packets out over real radio
ports at traditional lower packet speeds, so sometimes
there is quite a mismatch which of course has to be
buffered up, and this can cause more problems.  Anyhow the
usual symptom with YAPP is that the connection breaks off
during a longer file transfer.  Shorter files do fine.

FTP, on the other hand, seems to be bulletproof.
Ftp does some application layer end-to-end flow control and
pacing, and IP helps out also in the flow control area.
Watch the progress displays on FTP to see it refill
the buffers as the file transfer progresses.  I usually
see it complete one 16K block before buffering up the next one.
Here, I am using IP over AX.25 UI frames with the
standard kernel ax.25 networking.  I don't recall
ever having a file transfer fail in the process of
sending the data, as long as the radio links held up.
Filesize doesn't seem to be a limit.  I once sent
a huge (by ax.25 packet standards at least) 3 or 4 MB
file by ftp at 1200b over a plain old packet link, and
while it took almost 12 hours on an otherwise idle
channel, it worked the first time and didn't stumble,
and correctly transferred the file intact. (It helps
to have a fan on the transmitter PA, particularly the
one sending the file out.)  As someone else observed,
the IP exponential backoff, when it occurs, is quite
painful to watch, and I would like to find out how
to tune it on AX.25 links.  Usually if it misses
a few packets it backs off much too far much too quickly,
and takes too long to take off again.

Comments, anyone?

I can post the source of yd.c (~19Kb) to the list, if that is
appropriate, or maybe send it somewhere if there is
some interest.  I could also post the comments at the front
of that file which explain things a bit more, to the list,
if necessary, without posting the whole file.  Sorry I haven't
had a chance to finish off the upload half of it yet.  In our
local situation, the download portion had a lot more
priority, so it got done first, and I haven't had a chance
to revisit it yet.  (This amounts to a first public
announcement that it exists and is available, gpl of course.)

I also have some semi-useful short patches to NODE that we
use locally that might be of some limited interest, I would
have to take a while to locate and document some of them.

73 de Bob Morgan, WB5AOH
wb5aoh at arrl dot net
or
wb5aoh at wb5aoh.dyndns.org

Re: YAPP server pgm

[Tomi Manninen]

On Thu, 2005-07-14 at 23:10, Bob Morgan wrote:

> I also have some semi-useful short patches to NODE that we
> use locally that might be of some limited interest, I would
> have to take a while to locate and document some of them.

You could also send them to the author of node, namely me.
You know, I might even get interested in this stuff again... :)

-- 
Tomi Manninen / OH2BNS / KP20JF74

Re: YAPP server pgm - Packet File Transfer

[Bill Vodall WA7NWP]

> I am not sure if these programs I wrote are what the
> original requestor wanted or not. 

that was me...

> Here is what they do:
> They sit on an unattended AX.25 capable file server, and users connect
> to them and download files.  If I had the upload half of
> the program finished, they could upload files to the server also.

It sounds like you are very close with the core functionality of
what I'm looking for.

My objective here is to transfer a binary file between two systems with 
packet radio.

The file will be highly compressed standard Email and NNTP posts - the
old UUCP way of doing things.  I believe the existing tools allow 
setting the size of the outgoing file bundle so we'll be able to limit 
it to something like 50K which would probably be fine for most of the 
exchanges.

I'd like the transfer to be very fast, aggressive and efficient.  To get 
the job done as quick as possible and get out of the way.   No doubt 
that FTP/TCP would be better for larger and more robust transfers but in 
this case the overhead of TCP is unacceptable.

I'm attempting to keep this scheme as standard as possible by minimizing 
the value added tools.   A single specialized "ax25" file copy program 
that works through the AX25 tools would be a good solution.

While this is basically Linux, I'm considering the issues and 
possibilities of running this on the other OS's.  (UUPC anyone?)

There's sure been some interesting info presented here in the past 
couple days...

73,
Bill - WA7NWP

Re: YAPP server pgm - Packet File Transfer

[Dave Platt]

> It sounds like you are very close with the core functionality of
> what I'm looking for.
> 
> My objective here is to transfer a binary file between two systems with 
> packet radio.
> 
> The file will be highly compressed standard Email and NNTP posts - the
> old UUCP way of doing things.  I believe the existing tools allow 
> setting the size of the outgoing file bundle so we'll be able to limit 
> it to something like 50K which would probably be fine for most of the 
> exchanges.
> 
> I'd like the transfer to be very fast, aggressive and efficient.  To get 
> the job done as quick as possible and get out of the way.   No doubt 
> that FTP/TCP would be better for larger and more robust transfers but in 
> this case the overhead of TCP is unacceptable.
> 
> I'm attempting to keep this scheme as standard as possible by minimizing 
> the value added tools.   A single specialized "ax25" file copy program 
> that works through the AX25 tools would be a good solution.
> 
> While this is basically Linux, I'm considering the issues and 
> possibilities of running this on the other OS's.  (UUPC anyone?)

That latter is an interesting suggestion.

The uucp protocol was originally designed for efficient host-to-host
file transfer over modem links whose behavior isn't all that difference
from low-speed AX.25 in a lot of way - slow and somewhat noise-prone.
It might actually make sense to implement uucp, wrapped directly in
AX.25 Level 2 UI frames... just steal a Level 2 protocol ID to do it
with.

uucp supports multiple data-transfer protocols, optimized for different
sorts of links.  The 'g' protocol is the standard one - relatively small
packets (64 bytes standard), with a sliding-window acknowledgement 
system using small ACK packets (6 bytes).  It's optimized for full-duplex 
lines, as it can't keep the pipeline full if it doesn't hear ACKs fairly 
frequently.

'g' protocol with 128- or 256-byte packets, and a standard three-packet
window might work out tolerably well for 1200-baud AX.25 use.

Another possibility would be to make use of the 'i' protocol, implemented
in Ian Taylor's uucp package (the standard for Linux systems these days)
and also in some others.  'i' has quite a few sophisticated capabilities,
including the ability to interleave multiple files being sent at once,
the ability to send files in both directions simultaneously, and a
NAK (selective reject) feature to request the retransmission of only
a single packet in the transmission window.

Taylor uucp also has some tweaks which cut down on the per-file or
per-message overhead - it can include the specification of the command
to be run on the received file as part of the "transfer file" command,
rather than having to send two separate file transfers (one with the
data and one with the "Do this to it!" command, as the legacy uucp
packages do).

It's been years since I played around much with this (back when I was
the de facto code-wrangler for the Macintosh port of UUPC), but I still
maintain a uucp link between my system here at work, and my home system.
It's normally done over TCP/IP but I could drop back to using a direct-
dial connection if I had to.  Fun stuff!

Re: YAPP server pgm - Packet File Transfer

[Bill Vodall WA7NWP]

> 
> It's been years since I played around much with this (back when I was
> the de facto code-wrangler for the Macintosh port of UUPC), but I still
> maintain a uucp link between my system here at work, and my home system.
> It's normally done over TCP/IP but I could drop back to using a direct-
> dial connection if I had to.  Fun stuff!

Lots of good UUCP reference info there.  Thanks.  I know it's a rich 
tool but I don't have the gritty details.   I do remember one posting by 
a fellow some years ago where he mentions doing UUCP over packet. 
Thanks to Googles massive cache, I found it:

-----
I've been running this sort of thing for the last 2 or 3 years, and I've 
recently added UUCP to the mix, as I'm now running FreeBSD on my laptop, 
and so use UUCP to transport email from the server to the laptop via 
802.11b, cell phone, or packet radio (no TCP/IP layer needed to slow 
things down - I connect to the server via packet, login, then start the 
UUCP session).
Ed. C---
-----

Search on "wetnet packet uucp" and you'll see it as the 3rd or so posting..

I'm guessing there's some UUCP scripting tricks to establish the 
connection and then put the TNC into transparent mode.   Hmm.  Might 
still work through the AX25 stack which I wouldn't give up for all the 
other tools it enables.

Bill

Re: YAPP server pgm - Packet File Transfer

[w9ya]

Hey Bill and the gang;

In the bad ol' days. i.e. in the first years after AX.25 made it's
appearance, I had a file server on the "duplex (ham 2m es 440) packet
network" in LA. We used a variation on the binary to ascii-text conversion
to prefilter for transfers. Worked pretty well too.

Vy 73;

Bob
w9ya


>
>> I am not sure if these programs I wrote are what the
>> original requestor wanted or not.
>
> that was me...
>
>> Here is what they do:
>> They sit on an unattended AX.25 capable file server, and users connect
>> to them and download files.  If I had the upload half of
>> the program finished, they could upload files to the server also.
>
> It sounds like you are very close with the core functionality of
> what I'm looking for.
>
> My objective here is to transfer a binary file between two systems with
> packet radio.
>
> The file will be highly compressed standard Email and NNTP posts - the
> old UUCP way of doing things.  I believe the existing tools allow
> setting the size of the outgoing file bundle so we'll be able to limit
> it to something like 50K which would probably be fine for most of the
> exchanges.
>
> I'd like the transfer to be very fast, aggressive and efficient.  To get
>  the job done as quick as possible and get out of the way.   No doubt
> that FTP/TCP would be better for larger and more robust transfers but in
>  this case the overhead of TCP is unacceptable.
>
> I'm attempting to keep this scheme as standard as possible by minimizing
>  the value added tools.   A single specialized "ax25" file copy program
> that works through the AX25 tools would be a good solution.
>
> While this is basically Linux, I'm considering the issues and
> possibilities of running this on the other OS's.  (UUPC anyone?)
>
> There's sure been some interesting info presented here in the past
> couple days...
>
> 73,
> Bill - WA7NWP
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-hams" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html