The Fred Fish Collection 1.5/ffcollection-1-5-1992-11.iso/ff_disks/300-399/ff319.lzh/CNewsSrc/cnews.orig.lzh/notebook/ihave

Text File | 1989-06-27 | 7KB | 350 lines

.TL
Setting Up Netnews Feeds Using the Ihave/sendme Protocol
.AU
Geoff Collyer
.AI
Department of Statistics
University of Toronto
.SH
Introduction
.PP
The ihave/sendme protocol is a means of conserving network bandwidth
at the expense of some real-time delays in getting netnews.
It pre-dates the NetNews Transfer Protocol (NNTP)
for the TCP/IP protocol stack by several years and attempts much
the same optimisation,
but ihave/sendme is batched,
unlike NNTP.
.PP
Ihave/sendme
is sketched in extremely vague terms in
ARPA Internet RFC 1036 (nee 850),
but the description therein is so lofty as
to be useless as a protocol specification.
.SH
Into the Breach
.PP
The ihave/sendme strategy is for a site which has just received a new
netnews article to send an
.I ihave
netnews control message
containing the Message-ID of the new article to each
of its ihave/sendme
neighbours;
the neighbour consults its netnews
.I history
file and if it has not seen the article,
sends back a
.I sendme
control message containing the original Message-ID;
upon receipt of the
.I sendme ,
the first site will transmit the article via normal channels
to the requesting neighbour.
.PP
Due to the high volume of netnews,
sending individual articles is always a performance disaster
(see B News,
NNTP and unbatched ihave/sendme),
so the golden rule of netnews is
``Thou shalt batch thine articles.''.
This makes the above sketch a little slower and more complicated:
now individual articles are not sent,
but batches of
.I ihave ,
.I sendme ,
and
netnews messages are transmitted,
incurring further delay since netnews batchers are usually
run only once per hour,
say.
.PP
To permit
.I sys
file entries to distinguish
.I ihave s
from
.I sendme s
from
ordinary netnews traffic,
two distributions have been usurped for use by ihave/sendme links:
.I ihave
and
.I sendme .
When
.I relaynews
is processing a control message of the form
.DS
Control: keyword system
.DE
where
.I keyword
may be
.I ihave
or
.I sendme ,
.I relaynews
generates a reply
by finding the
.I sys
file line that will match a newsgroup of
.B to. system
and a distribution of
.B ihave
(or
.B sendme
as appropriate),
and sending its reply
(a
.I sendme
control in the case of
.I ihave,
or ordinary articles in the case of
.I sendme )
to the matched channel.
.PP
We will now walk through an example ihave/sendme set up between
two C news sites
(\c
.I utzoo
and
.I utstat ),
with reference to the following
.I sys
file fragments and flow diagram.
.br
.ne 2i
.SH
utzoo's sys file
.LP
.DS L
.ft B
# for testing ihave/sendme control messages: an ihave/sendme link with utstat.
# batcher turns batch file -> giant ihave control message & posts to to.utstat
#1)
utstat:rec.music.synth/all,!sendme,!ihave:I:utstat.ihave/togo
# sends ihave & sendme messages via normal batching (or uux) (#2, #4)
utstat-ctl:to.utstat/all,!sendme,!ihave:f:utstat/togo
# batcher turns batch file -> giant sendme control message & posts to to.utstat
(#3)
utstat-send-ids:to.utstat/ihave:I:utstat.sendme/togo
# final sending of the articles themselves (#5) (optional; can share with #2 &
4)
utstat-real:to.utstat/sendme:f:utstat/togo
.ft R
.DE
.br
.ne 2i
.SH
utstat's sys file
.LP
.DS L
.ft B
# for testing ihave/sendme control messages: an ihave/sendme link with utzoo.
# batcher turns batch file -> giant ihave control message & posts to to.utzoo (
1)
utzoo:rec.music.synth/all,!sendme,!ihave:I:utzoo.ihave/togo
# sends ihave & sendme messages via normal batching (or uux) (#2, #4)
utzoo-ctl:to.utzoo/all,!sendme,!ihave:f:utzoo/togo
# batcher turns batch file -> giant sendme control message & posts to to.utzoo
#3)
utzoo-send-ids:to.utzoo/ihave:I:utzoo.sendme/togo
# final sending of the articles themselves (#5) (optional; can share with #2 &
4)
utzoo-real:to.utzoo/sendme:f:utzoo/togo
.ft R
.DE
.so ihave.pic
.SH
An Example
.PP
A new article arrives on
.I utzoo
in newsgroup
.I rec.music.synth
and is matched by
.I sys
file line
.B "#1 on utzoo" ,
which writes the Message-ID of the article
onto a batch file
(\c
.I $NEWSARTS/out.going/utstat.ihave/togo ).
There are some subtleties here:
it would be essential to add
.B ,!to
to #1's subscription list
if that list contained
.B all
or
.B to
to prevent leaking messages intended for the other
.I sys
entries out #1;
and
it is always worthwhile to prevent leaking local
newsgroups out one's full feeds,
so if #1's subscription list contained
.B all
one would want to add something like
.B ,!general
or
.B ,!utstat .
Some time later,
the batcher runs on #1's batch file.
.PP
The batcher on
.I utzoo
prepares
a batch of Message-IDs as an
.I ihave
control message to
.I utstat
and submits it to
.I "inews -h" ,
which will match
.I sys
file line
.B "#2 on utzoo" ,
which will typically write the file name of the
control message
(in the
.I control
pseudo-newsgroup)
on the usual
.I utstat
batch file
(\c
.I $NEWSARTS/out.going/utstat/togo ).
(For greater speed,
one can have the #2
.I sys
file line simply execute
.I uux (1)
or
.I mail (1)
but this can result is a lot of short batches being sent,
which will increase
.I rnews
processing overhead on
.I utstat .)
.PP
Some time later,
the batcher will run on
.I utzoo
on the usual
.I utstat
batch file
and will produce batches,
including the
.I ihave
control message,
and transmit them to
.I utstat
by means specified in
.I $NEWSCTL/batchparms .
.PP
When the
.I ihave
control message arrives on
.I utstat ,
it will match
.I sys
file line
.B "#3 on utstat"
and have its Message-ID
written on the
.I utzoo
.I sendme
batch file
(\c
.I $NEWSARTS/out.going/utzoo.sendme/togo ).
.PP
Some time later,
the batcher will run on
.I utstat
on that batch file,
which will produce a
.I sendme
control message
and submit it to
.I "inews -h" ,
which will match
.I sys
file line
.B "#4 on utstat" ,
which will write the name of the
.I sendme
control message in the
.I control
pseudo-newsgroup on
the usual
.I utzoo
batch file
(\c
.I $NEWSARTS/out.going/utzoo/togo)
.PP
Some time later,
the batcher will run on
.I utstat
on the usual
.I utzoo
batch file,
and will produce batches including
the
.I sendme
and send them to
.I utzoo
by means specified in
.I $NEWSCTL/batchparms .
.PP
When the
.I sendme
control message arrives on
.I utzoo ,
it is matched by
.I sys
file line
.B "#5 on utzoo" ,
which writes the file names of the articles
named by Message-ID in
the
.I sendme
control message
on the usual
.I utstat
batch file.
.PP
Some time later,
the batcher on
.I utzoo
will run and produce batches,
including the articles requested by the
.I sendme ,
and will send them to
.I utstat
by means specified in
.I $NEWSCTL/batchparms .
.SH
Comparison with B News
.PP
There is a lot of activity involved in ihave/sendme,
and there are at least five distinct channels
and three transactions needed to send a batch of articles.
B news ``simplifies'' the
.I sys
file by requiring certain
.I sys
file options (sic)
to be set to fixed values,
and
by overloading madly.
The resulting
.I sys
file
.I is
shorter,
but it is also much more difficult to decypher,
and if you should want to specify non-default options,
you are out of luck.