Frozen Fish 1: Amiga/FrozenFish-Apr94.iso/bbs/alib/d2xx/d236/xprzmodem.lha/XprZmodem/xprzmodem.doc

Text File | 1989-08-09 | 26KB | 454 lines

                    Documentation for XPRZModem.library
                Version 1.0, 29 July 1989, by Rick Huebner



1.  Introduction (or "What is this thing, anyway?")
---------------------------------------------------

     XPRZModem.library is an Amiga shared library (with full Manx C source
code) which provides ZModem file transfer capability to any XPR-compatible
communications program.  The XPR external protocol standard describes an
interface method which allows various file transfer protocols to be
implemented as Amiga shared libraries which may then be used
interchangeably in any compatible communications program.  This takes a
heavy load off of the comm program author, who no longer has to support
scads of different file transfer protocols (many of which are quite tricky
to code) in their program in order to make it widely useful and popular.
The comm program is also smaller and more efficient as a result, since all
those obscure protocols (you know, the ones *you* don't need) are no longer
taking up space.  The XPR standard also helps users, who can mix and match
their favorite file transfer protocols (and implementations thereof) with
their favorite comm programs.  And when new protocols are invented, the
user simply plugs in a new library, and voila!, it's ready to use.
Hopefully, making protocols easy to support will allow more and better comm
programs to be written, as authors can concentrate on their programs
instead of constantly re-inventing the wheel.

     Of course, for all of this wonderful stuff to happen, there needs to
be a good selection of these XPR protocol libraries available to the
public.  It's the classic chicken-and-egg problem; comm program authors
won't be motivated to support the XPR standard unless there are a goodly
number of protocols available for it.  And other programmers won't be
motivated to write XPR protocol libraries until there are a goodly number
of comm programs which can use them.  In an effort to do my bit [ B^) ] for
the Amiga community, which has given me so many neat toys to play with over
the past few years, I decided to try and help get the ball rolling.

     It just so happens that I run a BBS (see ad at end of file B-)), and I
help work on the BBS software I use (Opus, a popular MS-DOS program [don't
boo and hiss, that's something PClones are well suited for; why waste a
neat machine like an Amiga on sitting around being a BBS all day?]).  And
in that capacity, I wrote the ZModem implementation for Opus back when
ZModem first came out, around 1986.  I went through a period of serious
frustration in the early days of Amiga comm programs, since I'd gone to the
trouble of writing ZModem for my BBS program, and none of my users had a
program which could handle it.  I wasn't yet knowledgeable enough about
programming the Amiga to fix the problem myself, and comm program authors
weren't interested in my ZModem code since it was so MS-DOS-dependant (I
*tried* to give it away), so I had to wait quite a long time before my own
BBS users were able to take advantage of my own ZModem implementation.
Anyway, this means that when I saw the XPR standard published, I was in a
good position to help out.  I had this perfectly good, heavily tested
ZModem code sitting around, and it wasn't very hard to convert that code
into an XPR-type ZModem library.

     Hopefully, the early availability of a ZModem library will help
stimulate interest in the XPR standard, resulting in better Amiga telecomms
for all of us.  And by making my source code PD, I hope to help others
interested in writing XPR protocol libraries by giving them some serious
example code.  Also, having ZModem library code readily available to John
Q.  Hacker should help ensure a steady stream of new lemon-fresh enhanced
ZModem libraries (with enzymes) for all of us to use in the future.

     Of course, no discussion of the XPR standard would be complete without
giving proper credit to the author, Willy Langeveld of the Stanford Linear
Accelerator Center.  Many thanks are due him for this effort.  If you have
any further questions about the XPR standard, be sure and download the spec;
it should be available on BIX (since he's a sysop there), or on most other
major systems.  And I'll try to keep the current version available on my
BBS, as well.

     All files in this archive which are not copyrighted are hereby
released to the public domain (which they were anyway, by way of not being
copyrighted, but I want to make sure YOU realize that).  Do as you like
with them.  Please make lots of copies and distribute them all over the
place, and make lots of derivative works, and everything!  Heck, you can
even publicly perform and/or display this code if you can figure out how...



2.  Installation (OK, enough chatter; let's get to work)
--------------------------------------------------------

     Couldn't be easier.  Just copy the xprzmodem.library file into your
LIBS: directory.  All XPR-compatible comm programs should provide a way for
you to select which XPR protocol you wish to use, either by giving you a
file requester showing LIBS:xpr*.library, or by automatically detecting
these libraries and adding them into their menus.

WARNING:  VLT 4.058 appears to have a bug in its xpr_sflush() routine which
causes random Guru 3 crashes (at least on my stock A2000).  Luckily, the
sflush() routine isn't that crucial, so I've included a version of the
library with the sflush() calls commented out for VLT users.  The only
problem caused by not using sflush() is that error recovery and resync when
sending files will be a bit erratic, possibly causing extra timeouts.  The
data should still get sent just fine; the transfer just won't be as
efficient as usual.



3.  Options
-----------

     The XPR standard lays out a way for the comm program user to specify
options for the XPR protocol.  The comm program is supposed to provide some
method of passing an option string to the XPR library before transferring
files.  The precise format and usage of this option string is left up to
the XPR protocol author; the comm program just sends it verbatim.  If an
environment variable is found with the same name as the XPR protocol (i.e.
there's a file in the ENV:  directory called "xprzmodem"), the comm program
is supposed to use this string (contents of file) to initialize the
protocol options.  Also, a menu option or some such should normally be
provided which will allow the user to be prompted for the option string
interactively.

     In any case, no matter how your particular comm program feels like
handling it, these are the options supported by this implementation of
ZModem:

          T{Y|N|?}    Text translation mode:
             TY = Text Yes; if receiving, translate CR/LF pairs or solo
                  CR chars to normal Amiga LF chars.  Ignore data past ^Z.
                  If sending, suggests to receiver that they should receive
                  this file in text mode.
             TN = Text No; receive file verbatim, without changes.  If
                  sending, suggest to receiver that they receive this
                  file verbatim, without translations.
             T? = Text status unknown; if receiving, use sender's
                  suggestion as to whether to do EOL translations or not.
                  If sending, tell receiver to use default mode, 'cause we
                  don't know either.

          O{Y|N|R|S}  Overwrite mode:
             OY = Overwrite Yes; if about to receive file with same name as
                  one which already exists, delete the old file and receive
                  the new file in its place.
             ON = Overwrite No; if about to receive file with same name as
                  one which already exists, append ".dup" onto the name of
                  the new file to keep them separate.
             OR = Overwrite Resume; if about to receive file with same name
                  as one which already exists, resume receiving file data
                  from the current end of the existing file.
             OS = Overwrite Skip; if (etc.), tell sender never mind, skip
                  this file, we don't want it.  Batch transfers will move
                  on to the next file in the set, if any.

          Bnnn   Buffer size:
             XPRZModem.library adds a layer of file I/O buffering in
             addition to whatever the comm program may or may not provide.
             This option sets the size of XPRZModem's file I/O buffer in
             kilobytes.  The minimum value is 1 KB, for those using RAM
             drives or fast hard drives, or those whose comm programs
             already provide sufficient buffering.  The maximum value is
             as much contiguous RAM as you have available in your Amiga.
             If you specify more than is actually available, XPRZModem will
             keep decrementing the buffer size requested by 1 KB until the
             memory allocation works.  That way, if your RAM is too
             fragmented to use the amount you request, XPRZModem simply
             uses the largest block available.  Buffering is especially
             helpful for floppy drive users; it keeps your drive from
             continuously gronking and slowing things down all through the
             transfer.

          Fnnn   Frame size:
             Although normally avoided, ZModem has the ability to require
             an ACK to be sent from the receiver to the sender every X-many
             data bytes.  Normally you don't want to use this feature,
             because not waiting for ACKs is part of how ZModem works so
             fast.  However, this feature can be very useful in conjunction
             with file I/O buffering on slow devices (namely those floppy
             drives).  If you set up a large I/O buffer to avoid gronking
             your floppy so often, you'll find that when the buffer finally
             *does* get around to being flushed that it can take a looonng
             time; so long, in fact, that the delay can cause timeouts and
             errors.  But if you set your ZModem to require the sender to
             wait for an ACK every buffer's-worth of data, the sender will
             politely wait for you to flush your buffer to the slow floppy
             and send it an ACK saying it's OK to continue now.  This value
             should be set to 0 to disable ACKs (normal mode), or set it to
             the actual number of data bytes allowed between ACKs.  For
             example, if you set B64 because of your floppy, you should
             also set F65536.

     Note that the setting for the option must immediately follow the
option character with no intervening characters ("TY", not "T Y" or "T=Y").
When setting multiple options at once, separate the options from each other
with commas and/or spaces; for example, "TN,OR,F0".  You don't have to
specify all options every time; the options you specify will be merged into
the current option settings, replacing their old values.  Upper/lower case
is not significant.  The default option settings if you don't change
anything are "T?,ON,B16,F0".



4.  Serial port settings
------------------------

     ZModem (at least this implementation of it) requires that your serial
port be set to 8 data bits, no parity, 1 stop bit.  This allows ZModem to
send full 8-bit binary data bytes without having them munged on the way
through the modem.  If your comm program supports the xpr_setserial()
function, XPRZModem will use it to set your serial port to 8N1 before doing
a transfer, and will set your port back the way it was again after it's
done.  If your comm program doesn't support xpr_setserial(), you'll need to
make sure it's in 8N1 mode yourself.

     ZModem works well in all serial port handshaking modes; none,
XON/XOFF, or 7-wire/RTS/CTS.  Since any or all of those handshaking modes
may be appropriate at different times, with different modems or remote
systems, XPRZModem lets you set the handshaking mode and doesn't mess with
it.



5.  Receiving files
-------------------

     Once you get the ZModem options and your serial port configuration set
up properly, you're ready to actually use this thing (gasp!).  Receiving
files via ZModem is quite simple.  First, get the file sender going by
using whatever command it wants.  ZModem is a batch file transfer protocol,
meaning that it's capable of transferring several files in a single
exchange, so the remote system may allow you to specify multiple files to
be sent to you at one time.  It may also allow you to use wildcard
characters in the filename(s); this is all system dependant.

     Once you've gotten the sender going, use whatever option your comm
program provides to begin receiving files with ZModem; this will usually be
a menu option or button gadget.  The comm program should now prompt you for
the name of the file to be received.  Here's the only non-obvious part of
receiving files; because of the interchangeability of XPR libraries, the
comm program has no way of knowing whether your chosen protocol is a batch
protocol or a single file protocol.  Batch protocols send the names of the
files being transferred along with the data, so the receiving program
doesn't choose the name; you just tell the comm program what directory to
receive the files into, and they are automatically named correctly.
However, single-file protocols like good ol' XModem don't send the filename
with the data, and the receiving comm program has to name the file.  So...
the upshot of all this is that your comm program will probably ask you for
the name of the file to be received, whereas all you need to tell a ZModem
receiver is the name of the directory to put the files into.  The way
XPRZModem handles this is for you to just specify a dummy filename in the
directory which you want to put the files in.  XPRZModem will use the
directory portion of the filename you specify, and will append the actual
names of the received files onto this directory path in order to create the
received files.  So if you want your received files to go into
"HD:Uploads", tell your comm program the name of the file being received is
"HD:Uploads/x" or something similar.  Once you enter the dummy filename,
ZModem should automatically transfer all of the files you specified into
that directory.

     Make sure that you have set the ZModem options properly before
starting the transfer; especially, make sure you only use TY if you know
that all of the files being transferred in this batch are printable ASCII
text files.  If you use TY on normal binary files like .ARCs or .ZOOs,
they'll be mangled beyond use.



6.  Sending files
-----------------

     Sending files using ZModem is fairly straightforward.  First, activate
the file receiver with whatever command the remote system requires.  You
may or may not need to specify a filename or directory to the remote
system; this depends on their implementation of ZModem.  Once the remote
system is ready to receive files, activate your comm program's ZModem send
function.  Your comm program will prompt you for which file(s) to send.
Although ZModem is a batch protocol, your comm program may or may not allow
you to specify multiple file names to be sent; also, wildcards may or may
not be supported.  These decisions are up to the comm program author;
ZModem will handle multiple files and wildcards if the comm program allows
them.  Once you've specified the file name(s), the file(s) will be sent to
the remote system.  Note that the directory paths of the files you send
will not be included in the filenames sent to the other system; only the
filename portion will be sent.  Also, note that the T option serves only as
a suggestion to the receiving system when sending files; the receiver makes
the final decision as to whether to take your advice or to force the files
to be received in text or binary mode.

     If errors occur while sending the file(s), you'll probably notice a
small enhancement I made to the normal ZModem error recovery procedures.
Normally, file transfer protocols have to compromise between efficient data
transmission on good, clean phone lines and quick error recovery on bad,
noisy phone lines.  If you pick a large packet size, you get high
throughput on clean lines due to low packet overhead, but you have slow
recovery times and large amounts of retransmitted data on noisy lines.  If
you've used YModem on noisy lines you've seen this problem.  But, if you
use small packets to reduce retransmitted data on noisy lines, you increase
the amount of time the protocol spends sending packet overhead, and your
throughput suffers.  The solution is to vary the block size according to
the experienced error rate during the transfer.  That way you aren't stuck
with a rigid packet length which will sometimes be the wrong size no matter
what.  I came up with this idea back when I first wrote the ZModem code for
Opus, and cleared it for future compatibility with ZModem's designer, Chuck
Forsberg, back then.  Since then the basic concept has been extensively
tested in the Opus BBS system, and has proven quite effective; it has also
been incorporated into various other ZModem implementations over time.  The
actual algorithm for deciding what size packets to use when is pretty much
up to the protocol author; XPRZModem uses a modified version of the Opus
algorithm which prevents locking the packet size at a small value when a
large one-time burst of errors occurs.



7.  Notes for comm program authors
----------------------------------

     That's pretty much everything you need to know in order to use
XPRZModem properly.  Here are some notes for the "other" XPR standard
users, namely the comm program authors:

     Certain XPR callback functions *must* be implemented by the comm
program author in order for XPRZModem to be used.  If any of these
functions are not supported by your comm program, XPRZModem will display an
error message and abort when invoked.  These required functions are:

          xpr_fopen(), xpr_fclose(), xpr_fread(), xpr_fwrite(),
          xpr_fseek(), xpr_sread(), xpr_swrite(), and xpr_update()

     The xpr_update() function provides many data fields for your comm
program to potentially display to the user.  These are the XPR_UPDATE
struct elements which XPRZModem will keep updated during transfers:

          xpru_protocol, xpru_filename, xpru_filesize, xpru_msg,
          xpru_errormsg, xpru_blocks, xpru_blocksize, xpru_bytes,
          xpru_errors, xpru_timeouts, xpru_blockcheck, xpru_expecttime,
          xpru_elapsedtime, and xpru_datarate

     As you can see, XPRZModem tries to provide as many status fields as
possible.  Although all of them are useful, the ones which are most
important to ZModem users are filename, filesize, msg and/or errormsg, and
bytes.  Please try to provide at least these fields in your status display,
plus as many of the rest as you can manage.

     Although only the XPR callback functions listed above are crucial for
XPRZModem, all of them are used if they are provided.  Although XPRZModem
will function without any of the other routines, its performance or
capabilities may be degraded somewhat.  Specifically, this is what you give
up if you choose not to supply any of these other XPR callback functions:

          xpr_sflush(): Used when performing error recovery and resync
               when sending files.  If not provided, extra timeout errors
               and delayed error recovery will be likely.  The files will
               still be transferred properly, but errors will degrade
               overall throughput more than usual.

          xpr_chkabort(): Called between sending or receiving packets.
               If not provided, there's no way for your comm program user
               to abort a transfer in progress except by trying to somehow
               force it to decide to give up and abort on its own, such as
               by turning off the modem and hoping the protocol will abort
               after enough timeouts.

           xpr_chkmisc(): Also called between sending or receiving packets.
               Actually, this is the one routine which doesn't cause any
               problems if missing.  If you don't need a timeslice while
               the transfer is going, don't provide it.

           xpr_gets(): Called to prompt the user interactively for options
               when your comm program invokes XProtocolSetup() with a null
               xpr_filename field.  If not provided, you'll have to prompt
               the user for the options string yourself, and pass this
               string in xpr_filename when invoking XProtocolSetup().

           xpr_setserial(): Called to obtain the current serial port
               settings, and to change the serial port to 8N1 if it's not
               already set that way.  If not provided, XPRZModem will
               assume all transfers are being done at 2400 bps, which
               won't hurt anything, and your users will have to make sure
               that the serial port is set to 8N1 themselves.

           xpr_ffirst() and xpr_fnext(): If either of these routines are
               missing, XPRZModem will lose the ability to send multiple
               files in a batch.  The xpr_filename pointer passed to
               XProtocolSend() will be assumed to point to the actual full
               filename of the single file to be sent in this batch.  If
               both of these routines are provided, XPRZModem will rely
               upon them completely to obtain the names of the files to
               send, and the xpr_filename pointer will not be used for any
               purpose by XPRZModem except to be passed to ffirst/fnext.
               This gives your comm program a way to send not just a single
               filename template's worth of files in a batch, but a list of
               different filenames.  If, for example, you set xpr_filename
               to point to the first node of a linked list of filenames
               and/or templates to be sent, rather than just having it
               point to a string, you can have your ffirst and fnext
               routines traverse this linked list in order to determine the
               next file to be sent.  Or you could have xpr_filename point
               to a buffer containing a list of filenames separated by
               commas, and your ffirst/fnext routines could return each
               filename in this list in turn.  The key here is that if you
               provide these two routines, you're in complete control over
               the series of names fed to XProtocolSend.  If you omit these
               routines, XPRZModem is stuck with single-file mode.  Once
               again, if these two routines are provided, XPRZModem will
               *always* call them; it makes no attempt to use the
               xpr_filename pointer for anything itself.  This is not
               explicitly spelled out in the XPR standard, but it seems the
               only reasonable way to handle batch protocols to me.
               Hopefully other XPR library authors will follow this
               precedent as well, so that comm program authors will be able
               to count on multiple-filename batch sessions being handled
               properly.

           xpr_finfo(): Used to determine the filesize of files being sent,
               in order to tell the receiving system how big they are.
               Also used to determine the size of a file which already
               exists when in Overwrite Resume mode; XPRZModem must be able
               to get the size of the current portion of the file in order
               to be able to tell the sender where to resume sending from.
               If this routine isn't provided, Overwrite Resume mode is
               not allowed.



8.  The future
--------------

     I don't want or expect this to be the last or only XPR ZModem library
available.  There are a lot of less-commonly-used ZModem features which
have popped up over the past few years, and many people might like to see
some of them made available.  Such as 32-bit CRC (although I consider
CRC-16 perfectly adequate for the max 1K packets sent by ZModem), full
control-character escaping, or maybe 8th bit escaping to allow use of 7-bit
serial channels.  I didn't want to add a bunch of rarely-used bells and
whistles to this version of the library, because I want it to be able to
serve as comprehensible example code.  I just want to provide a good solid
ZModem which reliably handles the majority of people's needs.  Hopefully,
this will serve as a foundation for future enhanced versions, while
providing a safe fallback for people to come back to if that fancy new
enhanced version (with neo-maxi zoomed weebies) turns out to need some more
debugging.

     Bug reports, questions, or comments may be sent to me at:

          BIX: rahuebner
          Usenet: [e-mail]
          Compu$erve: 73105,1022

     Or, if you think it's important, and you want an answer in less than a
week or two, call my BBS:

          Amiga++
          1-402-291-3636
          1200-9600 bps, HST or V.32
          FidoNet node 1:285/614

     I check the messages on my BBS at least once a day, so that's where to
get ahold of me quick.  I'll also try and stock the latest XPR standard
spec and XPR libraries there, in addition to the best collection of Amiga
PD software in the MidWest ( <- blatant ad ).