Usenet 1994 January

home *** CD-ROM | disk | FTP | other *** search

/ Usenet 1994 January / usenetsourcesnewsgroupsinfomagicjanuary1994.iso / sources / misc / volume3 / bt-rio < prev next >

Wrap

Internet Message Format | 1989-02-03 | 42.6 KB

Path: xanth!mcnc!decvax!decwrl!purdue!umd5!ames!necntc!ncoast!allbery From: mjr@welchsun2.UUCP (Marcus J. Ranum) Newsgroups: comp.sources.misc Subject: v03i050: record I/O library for use with btree package Message-ID: <7953@ncoast.UUCP> Date: 12 Jun 88 21:53:45 GMT Sender: allbery@ncoast.UUCP Reply-To: mjr@welchsun2.UUCP (Marcus J. Ranum) Lines: 1736 Approved: allbery@ncoast.UUCP comp.sources.misc: Volume 3, Issue 50 Submitted-By: "Marcus J. Ranum" <mjr@welchsun2.UUCP> Archive-Name: bt-rio [Repackaged: I applied a patch before posting. ++bsa] #--------------------------------CUT HERE------------------------------------- #! /bin/sh # # This is a shell archive. Save this into a file, edit it # and delete all lines above this comment. Then give this # file to sh by executing the command "sh file". The files # will be extracted into the current directory owned by # you with default permissions. # # The files contained herein are: # # -rw-r--r-- 1 allbery System 1512 Jun 12 17:44 Makefile # -rw-r--r-- 1 allbery System 3119 Jun 12 17:44 README # -rw-r--r-- 1 allbery System 5585 Jun 12 17:44 example.c # -rw-r--r-- 1 allbery System 7001 Jun 12 17:44 recio.3 # -rw-r--r-- 1 allbery System 17221 Jun 12 17:45 recio.c # -rw-r--r-- 1 allbery System 2905 Jun 12 17:44 recio.h # -rw-r--r-- 1 allbery System 823 Jun 12 17:44 sizes.c # echo 'x - Makefile' if test -f Makefile; then echo 'shar: not overwriting Makefile'; else sed 's/^X//' << '________This_Is_The_END________' > Makefile X# Makefile for record I/O library X# Copyright (C) 1988, Marcus J. Ranum, William Welch Medical Library X# $Author: mjr $ X# X# $Header: Makefile,v 1.1 88/06/01 21:29:16 mjr rel $: Makefile X# X# $Log: Makefile,v $ X# Revision 1.1 88/06/01 21:29:16 mjr X# Initial revision X# X# X# define SYSV or BSD (only for purposes of test code) - recio.c doesn't care. X# defining BYTEORDER forces the system to maintain disk data structures X# in network byteorder. This may be more or less portable. (tested on X# VAXEN running ULTRIX and Suns) X# X#CFLAGS=-O -DBSD -DBYTEORDER XCFLAGS=-O -DBSD XLFLAGS=-s XLINTFLAGS=-h -x -u -DBSD X XLIB= librecio.a X#LIB= recio.o X X# installation stuff XLIBDIR= /usr/local/lib XHDR= recio.h XHDRDIR= /usr/include/local XMAN= recio.3 XMANDIR= /usr/man/manl X Xall: example sizes X Xlibrecio.a: recio.o X ar rcv $@ recio.o X ranlib $@ X Xexample: $(LIB) example.o X cc $(LFLAGS) -o example example.o $(LIB) X Xsizes: sizes.o X cc $(LFLAGS) -o sizes sizes.o X @sizes X Xrecio.o: $(HDR) Makefile Xexample.o: $(HDR) Makefile X Xinstall: $(LIB) $(MAN) X cp $(LIB) $(LIBDIR)/$(LIB) X chmod 644 $(LIBDIR)/$(LIB) X cp $(HDR) $(HDRDIR)/$(HDR) X chmod 644 $(HDRDIR)/$(HDR) X ranlib $(LIBDIR)/$(LIB) X cp $(MAN) $(MANDIR)/$(MAN) X chmod 644 $(MANDIR)/$(MAN) X Xclean: X rm -f $(LIB) *.o example core mon.out sizes llib-lrecio.ln \ X recio.shar tags X Xlint: X lint $(LINTFLAGS) recio.c X Xdiction: X style $(MAN) X diction $(MAN) X Xlintlib: X lint -Crecio recio.c X Xshar: X shar -a README Makefile recio.c $(HDR) $(MAN) sizes.c example.c\ X > recio.shar ________This_Is_The_END________ if test `wc -c < Makefile` -ne 1512; then echo 'shar: Makefile was damaged during transit (should have been 1512 bytes)' fi fi ; : end of overwriting check echo 'x - README' if test -f README; then echo 'shar: not overwriting README'; else sed 's/^X//' << '________This_Is_The_END________' > README XThe poor man's record management library: X XThis is partII of the poor man's data management library. It's intended Xfor use with the btree index library, but can also be used stand alone. XThe recio library is designed with the following in mind: X X1) a simple low-level package to support inserting, modifying and Xretrieving records of (not necessarily fixed) length into a file. This Xis doubtless less efficient than simply keeping a whole bunch of UNIX Xfiles around, but that is a pain. The recio package, then, trades the Xspeed and elegance of UNIX files for a chunk-oriented structure that Xresides in a single UNIX file. X X2) a read/write/lseek-like interface, with a rudimentary higher level Xthat looks somewhat like stdio (at least puts/gets) X X3) reasonable performance and stability. It'll never be as fast as UNIX Xfiles, since the files are stored in 'linked' chunks, which are parsed Xin succession. There are a lot of trade offs. X XThe recio files are stored as a pair of UNIX files, one of which Xcontains a map of records by number to pages in the actual data file. XThe map file also contains information about which blocks in the data Xfile are free, etc. This file can contain 'holes', since a record is Xcreated at a specific offset per record number. The structures are as Xsmall as possible to minimize side-effects. The data file consists of Xpages of variable size (set when the data file is created). Each page Xcontains some small amount of information in a header, including Xpointers to a chain of offspring nodes, if needed, as well as Xadditional information about whether that block is free, in case the Xfile needs to be reconstructed. The recio.h header file lays it all Xout, somewhat. X XAs with the poor man's btree library, there are no concurrency controls Xin effect. The design of the whole package is with an eye to having it Xbe serviced by a deamon across a network, anyway, so this problem is Xleft as an excercise for the reader :-) Also, like the poor man's btree Xlibrary, this code is not public domain, though it is freely modifiable Xand distributable as long as the copyright notices are retained. If Xyou make any nifty improvements, please let me know, (E-mail, I am not Xon usenet) bugfixes, etc, are welcome. X XByte-order and portability: There are #ifdefs for BYTEORDER, which make Xthe program store data in network byte order (at least the data Xstructures that drive the library - user data is the user's problem.) XThere are several unsolved problems with this approach. It works fine Xbetween my Sun and my VAX, but the way the structures are written to Xdisk is also going to depend on your compiler. (the deleted element of Xstruct ripag is a long to force alignment in the VAX compiler, which Xthought a ripag was 16 bytes, while the Sun thought it was 15, with XNASTY results) The BYTEORDER code is not guaranteed to work, and if it Xdoesn't, your best bet is to look at the output of sizes.c and to check Xto see if your compiler assembles the structures in the same ORDER. X X Marcus J. Ranum, William Welch Medical Library, 1988 X mjr@jhuigf.BITNET || uunet!mimsy!aplcen!osiris!welchvax!mjr ________This_Is_The_END________ if test `wc -c < README` -ne 3119; then echo 'shar: README was damaged during transit (should have been 3119 bytes)' fi fi ; : end of overwriting check echo 'x - example.c' if test -f example.c; then echo 'shar: not overwriting example.c'; else sed 's/^X//' << '________This_Is_The_END________' > example.c X/* X * Copyright (C) 1988, Marcus J. Ranum, William Welch Medical Library X * $Author: mjr $ X */ X X#ifndef lint Xstatic char *RCSid="$Header: example.c,v 1.1 88/06/01 21:27:06 mjr Rel $: example.c"; X#endif X X/* X * $Log: example.c,v $ X * Revision 1.1 88/06/01 21:27:06 mjr X * Initial revision X * X */ X X/* sample page size to use - obviously 20 is quite small, but it makes */ X/* testing things really a lot easier than having to enter 1024 bytes of */ X/* junk data for testing input */ X X#define SAMPLESIZE 20 X X X/* nobody ever writes nice test programs. in fact, this one is pretty gross */ X/* basically, this allows exercising all the various functions of the rifd */ X/* library */ X X#ifdef BSD X#include <sys/file.h> X#endif X#ifdef SYSV X#include <sys/fcntl.h> X#endif X#include <stdio.h> X#include "recio.h" X Xvoid Xhelp() X{ X (void)printf("commands are:\n"); X (void)printf("Read:\treads a chunk of the requested record.\n"); X (void)printf("\tNote that this is artificially constrained in this\n"); X (void)printf("\texample, to show how the consecutive reads work\n"); X (void)printf("\nMore\tprints the next chunk in consecutive reads.\n"); X (void)printf("\nGets\treturns pointer to next line.\n"); X (void)printf("\nPuts\tappends a newline terminated string to rec.\n"); X (void)printf("\nDel\tdeletes the named record by placing its blocks\n"); X (void)printf("\ton the free block list\n"); X (void)printf("\nLabel\tsets the database comment block that resides\n"); X (void)printf("\tin page #0 of the page file.\n"); X (void)printf("\nWrite\tallows input of text to a specified record.\n"); X (void)printf("\nAppen\tallows appending text to a specified record.\n"); X (void)printf("\nHelp\tthis.\n"); X} X Xmain(ac,av) Xint ac; Xchar *av[]; X{ X RIFD *h1; X char instr[BUFSIZ]; X char buf[BUFSIZ]; X char *files[2]; X long rec; X int ret; X int size = SAMPLESIZE;/* page block size - we use a small */ X /* size to give things a real workout */ X int num; X extern long atol(); X extern char *gets(); X extern char *strcat(); X X if(ac < 3) { X (void)fprintf(stderr,"(using foo.map foo.pag)\n"); X files[0] = "foo.map"; X files[1] = "foo.pag"; X } else { X /* there are keener ways of doing this, but why ? */ X files[0] = av[1]; X files[1] = av[2]; X } X X (void)printf("record page size to use:(<return> = %d)",size); X (void)gets(instr); X if(strlen(instr) != 0) X size = atoi(instr); X X /* args passed in order: textfile,mapfile */ X if((h1 = riopen(files,O_CREAT|O_RDWR,0600,size)) ==NULL) { X perror(av[1]); X exit(1); X } X X X while (1) { X (void)printf("Read More Gets Puts Del Label Write Appen Help Quit:"); X if(gets(instr) == NULL) X exit(riclose(h1)); X X switch (*instr) { X X case 'q': X case 'Q': X exit(riclose(h1)); X X case 'd': X case 'D': X (void)printf("delete record:"); X (void)gets(instr); X rec = atol(instr); X ret = riunlink(h1,rec); X (void)printf("...returns %d\n",ret); X break; X X case 'g': X case 'G': X (void)printf("gets from record:(<return> = continue)"); X (void)gets(instr); X if(strlen(instr) != 0) X num = atol(instr); X X buf[0] = '\0'; X if(rigets(h1,rec,buf) != NULL) X (void)printf("\"%s\"\n",buf); X else X (void)printf("returns NULL\n"); X break; X X case 'r': X case 'R': X /* this is just an example - not pretty */ X (void)printf("read record:"); X (void)gets(instr); X rec = atol(instr); X (void)printf("how many bytes (max:%d <return> = 20):",BUFSIZ); X (void)gets(instr); X num = atoi(instr); X if(num ==0) X num = 20; X if(num > BUFSIZ) X num = BUFSIZ; X X buf[0] = '\0'; X ret = riread(h1,rec,buf,num,0); X X if(ret >= 0) X (void)printf("\"%s\"\n",buf); X else X (void)printf("returns %d\n",ret); X break; X X X case 'm': X case 'M': X (void)printf("how many bytes (max:%d <return>=20):",BUFSIZ); X (void)gets(instr); X num = atoi(instr); X if(num ==0) X num = 20; X if(num > BUFSIZ) X num = BUFSIZ; X buf[0] = '\0'; X ret = riread(h1,rec,buf,num,1); X if(ret > 0) X (void)printf("\"%s\"\n",buf); X else X (void)printf("returns %d\n",ret); X break; X X case 'p': X case 'P': X (void)printf("puts to record: (<return> = same record)"); X (void)gets(instr); X if(strlen(instr) != 0) X rec = atol(instr); X X (void)printf("enter data: "); X (void)gets(buf); X if(riputs(h1,rec,buf) == EOF) X (void)printf("returns EOF\n"); X else X (void)printf(" ok\n"); X X break; X X case 'w': X case 'W': X /* this is just an example - not pretty */ X (void)printf("write to record:"); X (void)gets(instr); X rec = atol(instr); X (void)printf("enter data: "); X (void)gets(buf); X ret = riwrite(h1,rec,buf,strlen(buf),0); X X (void)printf("....returns %d-",ret); X if(ret == strlen(buf)) X (void)printf(" ok\n"); X else X (void)printf(" count does not match\n"); X X break; X X case 'a': X case 'A': X (void)printf("append to record:"); X (void)gets(instr); X (void)printf("enter data:"); X (void)gets(buf); X ret = riwrite(h1,rec,buf,strlen(buf),1); X X (void)printf("....returns %d-",ret); X if(ret == strlen(buf)) X (void)printf(" ok\n"); X else X (void)printf(" count does not match\n"); X X break; X X case 'h': X case 'H': X (void)help(); X break; X X case 'l': X case 'L': X (void)printf("enter label:"); X *buf = '\0'; X X (void)gets(buf); X ret = risetlab(h1,buf,strlen(buf)); X if(ret == 0) { X *buf = '\0'; X if(rigetlab(h1,buf,BUFSIZ) <0) { X (void)printf("error reading label\n"); X } else { X (void)printf("label is:\"%s\"\n",buf); X } X } else { X (void)printf("error writing label\n"); X } X break; X X X default: X (void)printf("huh?\n"); X } X } X} ________This_Is_The_END________ if test `wc -c < example.c` -ne 5585; then echo 'shar: example.c was damaged during transit (should have been 5585 bytes)' fi fi ; : end of overwriting check echo 'x - recio.3' if test -f recio.3; then echo 'shar: not overwriting recio.3'; else sed 's/^X//' << '________This_Is_The_END________' > recio.3 X.\" recio.3l (C)1988 Marcus J. Ranum, Welch Medical Library X.\" $Header: recio.3,v 1.1 88/06/01 21:28:40 mjr rel $ X.TH RECIO 3l X.SH NAME Xriopen, riclose, riwrite, riread, riunlink, rigets, riputs, risetlab, rigetlab X.br X\- the poor man's record management library X.SH SYNTAX X.B #include <local/recio.h> X.PP X.B RIFD *riopen(paths,flags,mode,size) X.br X.SM X.B char *paths[2]; X.br X.B int flags, mode, size; X.PP X.B int riclose(rifd) X.br X.SM X.B RIFD *rifd; X.PP X.B int riwrite(rifd,recno,buf,num,appen) X.br X.SM X.B RIFD *rifd; X.br X.B long recno; X.br X.B char *buf; X.br X.B int num, appen; X.PP X.B int riread(rifd,recno,buf,num,cont) X.br X.SM X.B RIFD *rifd; X.br X.B long recno; X.br X.B char *buf; X.br X.B int num, cont; X.PP X.B int riunlink(rifd,recno) X.br X.SM X.B RIFD *rifd; X.br X.B long recno; X.PP X.B char *rigets(rifd,recno,buf) X.br X.SM X.B RIFD *rifd; X.br X.B long recno; X.br X.B char *buf; X.PP X.B int riputs(rifd,recno,buf) X.br X.SM X.B RIFD *rifd; X.br X.B long recno; X.br X.B char *buf; X.sp X.PP X.B int risetlab(rifd,buf,size) X.br X.SM X.B RIFD *rifd; X.br X.B char *buf; X.br X.B int size; X.sp X.PP X.B int rigetlab(rifd,buf,size) X.br X.SM X.B RIFD *rifd; X.br X.B char *buf; X.br X.B int size; X.sp X.SH DESCRIPTION X.PP XThe poor man's record management library is a set of routines to manage Xfiles containing linked chunks of data that can be either fixed or Xvariable length. Records are designed to be stored based on the Xrecord number (presumably indexed someplace else, hashed, or whatever). XEach data file consists of two UNIX files, the "record" file and the "map" Xfile. The "map" file contains very little, holding one entry per Xrecord number, consisting of the numbers of the first and last data Xrecords. The "record" file contains a linked list of records, with a small Xheader preceeding each, with information about which record it belongs Xto, whether it is deleted, which record is the successor, etc. Some of Xthe information is redundant, to make consistency checking possible. XA free chain of records is maintained, with a pointer to the first Xstored in a superblock at the head of the "map" file. Deleted records are Xreclaimed as needed, though not in any order that prevents fragmentation. X.PP XThere are several extraneous elements in the RIFD data structure that Xkeep track of the current record, record offset, and so forth. This is used Xin the X.B rigets() Xand X.B riputs() Xfunctions, which are designed to provide a familiar interface, as well Xas allowing a user to append to existing records, or read them a line Xat a time, with some minimally intelligent buffering. The X.B riwrite() Xand X.B riread() Xfunctions also contain means for appending writes, or performing Xa sequential series of partial reads. These are useful kludges. X.PP XThe X.B riopen Xsubroutine allocates a control structure, using the path names Xprovided in X.B paths. XThe first string in paths is assumed to be the name of the "map" file Xto use, and the second the "record" file. Needless to say, they should Xboth be the appropriate type of file, or trouble may ensue. XThe X.B flags Xand X.B mode Xarguments are passed to open(2) to control creation and permissions. XThe X.B size Xargument is used to set the record size if the file needs to be Xcreated. (If the file already exists, the X.B size Xargument is ignored.) XIf the data file is created (either through Xopen(2) flags O_TRUNC or O_CREAT on a nonexistent file) a new header Xis initialized automatically, and the record size is set as Xrequested. A X.B size Xof 0 will result in the file being initialized with a record Xsize of BUFSIZ less the size of a record header. XThe size of the records should be chosen carefully, since it will Xaffect either disk usage or performance considerably. X.B Riopen Xchecks a magic number stored in the "map" file superblock, and will fail Xunless the magic number is correct, to prevent accidentally using a Xdamaged file, or an incorrect file. No such check is performed on Xthe "record" file. If X.B riopen Xcannot open the requested file, or encounters other problems, Xit returns NULL. X.PP XThe X.B riclose Xsubroutine closes opened files, and deallocates the memory that Xwas allocated in riopen. X.PP XThe X.B riwrite Xfunction writes X.B num Xbytes of data to the specified record from X.B buf. XIf the X.B appen Xflag is nonzero, the data is written to the end of the record. If the X.B appen Xflag is zero, any old data is overwritten. X.B riwrite Xreturns -1 on error, or the number of bytes written. X.PP XThe X.B riread Xfunction reads data from the record file into X.B buf, Xstopping when it has read X.B num Xbytes, or when it has run out of data to read. X.B Riread Xreturns the number of bytes read, or -1 in the event of error. If X.B cont Xis nonzero, X.B riread Xwill continue reading from where it left off after the last read, Xor from the beginning of the record. Due to the implementation, if Xa different record is read, the X.B cont Xflag is ignored, and reads are performed from the beginning of the Xrecord. If the current record is deleted with X.B riunlink Xthe position is invalidated. Thus, continuous reads work unless Xyou read a different record, delete the current one, or read from Xthe beginning of the same record. Since X.B rigets Xis built on X.B riread Xthe same restrictions apply. Performing a X.B rigets Xof a different record from the one you were last Xreading will reset the current record pointer to the Xnew record, which may produce some confusion. XThis implementation is rather Xodd, but it seems to be fairly useful, and is certainly more useful Xthan not supporting continuous reads at all. X.PP XThe X.B riunlink Xfunction marks record X.B recno Xas deleted, and adds all of its record blocks to the free list. X.PP XThe X.B rigets Xfunction reads a newline-terminated string from record X.B recno Xinto X.B buf, Xreplacing the final newline with a null terminator. X.B Buf Xis assumed to be large enough to contain the returned data. Some internal Xbuffering is performed, and sequential calls to X.B rigets Xwill return sequential strings from the record, unless the current Xrecord is reset (see X.B riread Xabove). XIf there is an error, or the end of a record is reached, X.B rigets Xreturns NULL, otherwise it returns X.B buf. X.PP XThe X.B riputs Xfunction acts like its counterpart X.B puts, Xby appending the contents of X.B buf Xfollowed with a newline to record X.B recno. X.B Riputs Xreturns 0 if successful, or EOF in the event of a failure. X.PP XThe X.B risetlab Xand X.B riputlab Xroutines allow access to record #0 of the record file (which is otherwise Xinaccessible). This can be used to store any miscellaneous data that Xcan fit. If the X.B size Xis larger than the size of a "record", the label will not be set, Xand -1 is returned. Otherwise X.B risetlab Xreturns 0. The function X.B rigetlab Xretrieves X.B size Xbytes from the label into X.B buf, Xreturning 0 for success, and -1 to indicate failure. X.PP X.SH RESTRICTIONS X.PP X.SH DIAGNOSTICS X.PP X.SH AUTHOR X.PP XMarcus J. Ranum, Welch Medical Library. X.br X.ti 1i Xuunet!aplcen!osiris!welchvax!mjr X.br X.ti 1i Xmjr@jhuigf.BITNET X.SH SEE ALSO ________This_Is_The_END________ if test `wc -c < recio.3` -ne 7001; then echo 'shar: recio.3 was damaged during transit (should have been 7001 bytes)' fi fi ; : end of overwriting check echo 'x - recio.c' if test -f recio.c; then echo 'shar: not overwriting recio.c'; else sed 's/^X//' << '________This_Is_The_END________' > recio.c X/* X * Copyright (C) 1988, Marcus J. Ranum, William Welch Medical Library X * $Author: mjr $ X */ X X#ifndef lint Xstatic char *RCSid="$Header: recio.c,v 1.4 88/06/06 13:44:58 mjr Exp $: recio.c"; X#endif X X/* X * $Log: recio.c,v $ X * Revision 1.4 88/06/06 13:44:58 mjr X * fixed the drop of occasional chars in rigets() X * X * Revision 1.3 88/06/06 09:58:58 mjr X * *** empty log message *** X * X * Revision 1.2 88/06/02 15:35:32 mjr X * added rigetcurpag and risetcurpag X * X * Revision 1.1 88/06/01 21:27:20 mjr X * Initial revision X * X */ X X#include <stdio.h> X#include "recio.h" X X/* if we wish to store our disk data in network byte order */ X#ifdef BYTEORDER X#include <sys/types.h> X#include <netinet/in.h> X#endif X X/* size of a page header (without buffer) */ X#define RI_PHSIZ sizeof(struct ripag) X X/* size of a page (with buffer) */ X#define RI_PSIZ(rf) (sizeof(struct ripag) + (rf)->sblk.reclen) X X/* size of a rifd superblock */ X#define RI_SSIZ (sizeof(struct risuper)) X X/* size of a map file entry */ X#define RI_MSIZ (sizeof(struct rimap)) X X/* write the rifd mapfile superblock to disk */ Xstatic int Xwsuper(rf) Xstruct rifd *rf; X{ X extern long lseek(); X#ifdef BYTEORDER X struct risuper boge; X#endif X X X if (lseek(rf->mfd, 0L, 0) < 0) X return (-1); X X#ifdef BYTEORDER X boge.magic = htonl(rf->sblk.magic); X boge.reclen = htonl(rf->sblk.reclen); X boge.free = htonl(rf->sblk.free); X boge.last = htonl(rf->sblk.last); X X if (write(rf->mfd, (char *) &boge, RI_SSIZ) != RI_SSIZ) X return (-1); X#else X if (write(rf->mfd, (char *) &rf->sblk, RI_SSIZ) != RI_SSIZ) X return (-1); X#endif X X X return (0); X} X X X X/* read the rifd mapfile superblock from disk */ Xstatic int Xrsuper(rf) Xstruct rifd *rf; X{ X extern long lseek(); X#ifdef BYTEORDER X struct risuper boge; X#endif X X if (lseek(rf->mfd, 0L, 0) < 0) X return (-1); X X#ifdef BYTEORDER X if (read(rf->mfd, (char *) &boge, RI_SSIZ) != RI_SSIZ) X return (-1); X X rf->sblk.magic = ntohl(boge.magic); X rf->sblk.reclen = ntohl(boge.reclen); X rf->sblk.free = ntohl(boge.free); X rf->sblk.last = ntohl(boge.last); X#else X if (read(rf->mfd, (char *) &rf->sblk, RI_SSIZ) != RI_SSIZ) X return (-1); X#endif X X return (0); X} X X X X/* dynamically allocate a control structure for an open rifd */ X/* including opening and checking the mapfile and textfile */ Xstruct rifd * Xriopen(path, flags, mode, reclen) Xchar *path[2]; Xint flags; Xint mode; Xint reclen; X{ X struct rifd *rf; X int r; X extern char *malloc(); X X /* lets be dynamic, shall we ? */ X#ifndef lint X /* this to avoid the possible pointer alignment lint message */ X if ((rf = (struct rifd *) malloc(sizeof(struct rifd))) == NULL) X return (NULL); X#else X rf = (struct rifd *)0; X#endif X X /* open and check the mapfile - the long part */ X if ((rf->mfd = open(path[0], flags, mode)) > -1) { X X r = read(rf->mfd, (char *) &rf->sblk, RI_SSIZ); X X /* if read nothing, must be a new guy, right ? */ X if (r == 0) { X rf->sblk.magic = RI_MAGIC; X if(reclen == 0) X rf->sblk.reclen = BUFSIZ - sizeof(struct rimap); X else X rf->sblk.reclen = reclen; X rf->sblk.free = 0L; X X /* this keeps page 0 as a comment page */ X rf->sblk.last = 1L; X X if (wsuper(rf) == 0) X r = RI_SSIZ; X } X#ifdef BYTEORDER X else { X /* read something, decode the numbers */ X rf->sblk.magic = ntohl(rf->sblk.magic); X rf->sblk.reclen = ntohl(rf->sblk.reclen); X rf->sblk.free = ntohl(rf->sblk.free); X rf->sblk.last = ntohl(rf->sblk.last); X } X#endif X X X /* cleverly check ret value from either read or write */ X if (r != RI_SSIZ) { X (void) close(rf->mfd); X (void) free((char *) rf); X return (NULL); X } X X /* check that ole magic number */ X if (rf->sblk.magic != RI_MAGIC) { X (void) close(rf->mfd); X (void) free((char *) rf); X return (NULL); X } X } else { X /* couldnt even open the bloody file */ X (void) free((char *) rf); X return (NULL); X } X X /* allocate buffer memory */ X if((rf->dat = malloc((unsigned)rf->sblk.reclen)) == NULL) { X (void) close(rf->mfd); X (void) free((char *) rf); X return (NULL); X } X X /* now open the text file */ X if ((rf->fd = open(path[1], flags, mode)) < 0) { X (void) close(rf->mfd); X (void) free((char *) rf->dat); X (void) free((char *) rf); X return (NULL); X } X X rf->pagoff = 0; X rf->curpag = 0L; X X return (rf); X} X X X X/* close and deallocate the control structure */ Xriclose(rf) Xstruct rifd *rf; X{ X int t; X X t = wsuper(rf); X (void) close(rf->fd); X (void) close(rf->mfd); X (void) free((char *) rf->dat); X (void) free((char *) rf); X return (t); X} X X X X/* allocate a page block, either by using the free page chain or */ X/* by creating a new page at the end of the file */ Xstatic long Xpagalloc(rf,rec) Xstruct rifd *rf; Xlong rec; X{ X struct ripag pag; X long nfree; X extern long lseek(); X X X /* read superblock in case someone else has changed it */ X /* this is not totally bulletproof, but doubtless helps */ X if(rsuper(rf) < 0) X return(-1); X X /* if there are free blocks, use them, otherwise use the end of file */ X if(rf->sblk.free != 0L) { X /* allocate the block, and then reset the free pointer */ X if(rpaghed(rf,rf->sblk.free,&pag) < 0) X return(-1); X X /* should never happen ! this block is marked as in */ X /* use but is on the free list ! */ X /* (the other alternative is to just eat through the */ X /* file, using allocated blocks instead of the free */ X /* list :-)) */ X if(pag.deleted != RI_DELETED) { X /* AIIIEEEEEE!!!! */ X return(-1); X } X X /* remember it */ X nfree = rf->sblk.free; X X /* the new free pointer is the free block successor */ X rf->sblk.free = pag.next; X } else { X /* use page after current highest page in file */ X /* this is dependent on your OS being UNIX-like */ X /* in that 'holes' in files are filled with 0s */ X /* rather than something else - hence we expect */ X /* that data blocks in a 'hole' will read as all 0 */ X /* this should blow up MS-DOS */ X nfree = rf->sblk.last++; X } X X /* all is OK - rewrite our superblock - new free head */ X if(wsuper(rf) < 0) X return(-1); X X /* we now have removed the free entry from the list */ X /* and mark it as active */ X pag.deleted = RI_ACTIVE; X pag.next = 0L; X pag.recno = rec; X pag.high = 0; X if(wpaghed(rf,nfree,&pag) < 0) X return(-1); X X return(nfree); X} X X X X/* write a map block from the given struct */ Xstatic int Xwmapblk(rf,rec,blk) Xstruct rifd *rf; Xlong rec; Xstruct rimap *blk; X{ X extern long lseek(); X#ifdef BYTEORDER X struct rimap boge; X#endif X X X if(lseek(rf->mfd,(rec * RI_MSIZ) + RI_SSIZ,0) < 0) X return(-1); X X#ifdef BYTEORDER X boge.page = htonl(blk->page); X boge.tail = htonl(blk->tail); X X if (write(rf->mfd, (char *) &boge, RI_MSIZ) != RI_MSIZ) X return (-1); X#else X if(write(rf->mfd,(char *)blk, RI_MSIZ) != RI_MSIZ) X return(-1); X#endif X return(0); X} X X X X/* read a map block into the given struct */ Xstatic int Xrmapblk(rf,rec,blk) Xstruct rifd *rf; Xlong rec; Xstruct rimap *blk; X{ X int ret; X extern long lseek(); X#ifdef BYTEORDER X struct rimap boge; X#endif X X if(lseek(rf->mfd,(rec * RI_MSIZ) + RI_SSIZ,0) <0) X return(-1); X X#ifdef BYTEORDER X ret = read(rf->mfd,(char *)&boge, RI_MSIZ); X blk->page = ntohl(boge.page); X blk->tail = ntohl(boge.tail); X#else X ret = read(rf->mfd,(char *)blk, RI_MSIZ); X#endif X X /* if we EOFfed we are still (maybe) OK */ X if(ret == 0) { X blk->page = 0L; X blk->tail = 0L; X ret = RI_MSIZ; /* kluge */ X } X X if(ret != RI_MSIZ) X return(-1); X X return(0); X} X X X X/* read a page header into the given struct */ Xstatic int Xrpaghed(rf,rec,blk) Xstruct rifd *rf; Xlong rec; Xstruct ripag *blk; X{ X int r; X extern long lseek(); X#ifdef BYTEORDER X struct ripag boge; X#endif X X /* seek the distance, and read a page header */ X if(lseek(rf->fd,(rec * RI_PSIZ(rf)),0) < 0) X return(-1); X X#ifdef BYTEORDER X r = read(rf->fd,(char *)&boge, RI_PHSIZ); X blk->recno = ntohl(boge.recno); X blk->next = ntohl(boge.next); X blk->high = ntohl(boge.high); X blk->deleted = ntohl(boge.deleted); X#else X /* try to read a page */ X r = read(rf->fd,(char *)blk, RI_PHSIZ); X#endif X X /* if we EOFfed, we are still (maybe) OK */ X if(r == 0) { X blk->recno = rec; X blk->next = 0L; X blk->high = 0; X blk->deleted = RI_DELETED; X r = RI_PHSIZ; /* kluge */ X } X X if(r != RI_PHSIZ) X return(-1); X X return(0); X} X X X X/* write a page header from the given struct */ Xstatic int Xwpaghed(rf,rec,blk) Xstruct rifd *rf; Xlong rec; Xstruct ripag *blk; X{ X extern long lseek(); X#ifdef BYTEORDER X struct ripag boge; X#endif X X if(lseek(rf->fd,(rec * RI_PSIZ(rf)),0) < 0) X return(-1); X#ifdef BYTEORDER X boge.recno = htonl(blk->recno); X boge.next = htonl(blk->next); X boge.high = htonl(blk->high); X boge.deleted = htonl(blk->deleted); X if(write(rf->fd,(char *)&boge, RI_PHSIZ) != RI_PHSIZ) X return(-1); X#else X if(write(rf->fd,(char *)blk, RI_PHSIZ) != RI_PHSIZ) X return(-1); X#endif X return(0); X} X X X Xriwrite(rf,rec,buf,num,appen) Xstruct rifd *rf; Xlong rec; Xchar *buf; Xint num; Xint appen; X{ X long j; /* junk */ X int k; /* junk */ X int wrote =0; /* number of char written */ X long this =0L; /* current page */ X struct rimap map; /* buffer for map entries */ X struct ripag pag; /* buffer for page entries */ X extern long lseek(); X X /* no matter what, we will need our map block */ X if(rmapblk(rf,rec,&map) < 0) X return(-1); X X /* turn off read continuation for this record */ X rf->curpag = -1L; X X /* if no page at start, write map to disk now - new page */ X if(map.page == 0L) { X map.page = pagalloc(rf,rec); X if(map.page == -1L) X return(-1); X if(wmapblk(rf,rec,&map) < 0) X return(-1); X } X X X /* if we are appending, we start at the last page and continue */ X if(appen) { X if(map.tail != 0) X this = map.tail; X else X this = map.page; X } else { X /* start at the first page */ X this = map.page; X } X X X /* main loop, in which we spit the bytes to disk */ X while(num > 0 ) { X X /* read our page header */ X if(rpaghed(rf,this,&pag) < 0) X return(-1); X X /* seek to head of page data segment possibly */ X /* skipping over previously written bytes */ X if(appen) X j = (this * (long)RI_PSIZ(rf))+RI_PHSIZ+pag.high; X else X j = (this * (long)RI_PSIZ(rf))+RI_PHSIZ; X X /* actually do the thing */ X if(lseek(rf->fd,j,0) <0) X return(-1); X X /* figure how much to write */ X /* is this devo ?? - my head hurtz */ X if(appen) { X if(num > rf->sblk.reclen - pag.high) X k = rf->sblk.reclen - pag.high; X else X k = num; X } else { X if(num > rf->sblk.reclen) X k = rf->sblk.reclen; X else X k = num; X } X X /* actually do the thing */ X if(write(rf->fd,&buf[wrote],k) != k) X return(-1); X X /* adjust our notion of where we are and so on */ X wrote += k; X num -= k; X X if(appen) { X pag.high += k; X } else { X /* this prevents us from accidentally losing the */ X /* rest of the page in overwrites */ X if( k > pag.high) X pag.high = k; X } X X /* remember our new last page in chain is this one */ X map.tail = this; X X if( num > 0 ) { X /* the current block has no associated page */ X if(pag.next == 0L) { X pag.next = pagalloc(rf,rec); X if(pag.next == -1L) X return(-1); X } X X /* write our page header with next pointer */ X if(wpaghed(rf,this,&pag) < 0) X return(-1); X X /* move along to next page */ X this = pag.next; X } X } X X /* write our page header */ X if(wpaghed(rf,this,&pag) < 0) X return(-1); X X /* write our map block */ X if(wmapblk(rf,rec,&map) < 0) X return(-1); X X return(wrote); X} X X X X/* drop a list of pages, and pop them onto the free list */ Xriunlink(rf,rec) Xstruct rifd *rf; Xlong rec; X{ X long head =0L; /* first page - we build a chain */ X long this =0L; /* current page */ X struct rimap map; /* buffer for map entries */ X struct ripag pag; /* buffer for page entries */ X extern long lseek(); X X /* read superblock in case someone else has changed it */ X if(rsuper(rf) < 0) X return(-1); X X /* turn off read continuation if this record */ X if(rec == rf->currec) X rf->curpag = -1L; X X /* now read the mapblock to be deleted */ X if(rmapblk(rf,rec,&map) < 0) X return(-1); X X /* the map block has no associated page block - life is easy */ X if(map.page == 0L) X return(0); X X /* remember our page */ X this = map.page; X head = map.page; X X map.page = 0L; X map.tail = 0L; X X /* first off, mark the thing as gone */ X if(wmapblk(rf,rec,&map) < 0) X return(-1); X X while(1) { X /* read our page header */ X if(rpaghed(rf,this,&pag) < 0) X return(-1); X X /* if no next page, we are mostly done */ X /* drop to bottom, which links us into the free chain */ X if(pag.next == 0L) X break; X X /* mark it and go to next */ X pag.deleted = RI_DELETED; X if(wpaghed(rf,this,&pag) < 0) X return(-1); X X this = pag.next; X } X X /* this now points at the end of our list */ X /* head points at the first one. so we: */ X X /* make the superblock next free block pointer the next of our */ X /* last node - if we crash here we lose our free list - they just */ X /* sit there */ X pag.next = rf->sblk.free; X X pag.deleted = RI_DELETED; X if(wpaghed(rf,this,&pag) < 0) X return(-1); X X /* now the superblock free list points to our list head */ X rf->sblk.free = head; X if(wsuper(rf) < 0) X return(-1); X X return(0); X} X Xriread(rf,rec,buf,num,cont) Xstruct rifd *rf; Xlong rec; Xchar *buf; Xint num; Xint cont; X{ X long j; /* junk */ X int k; /* junk */ X long this; X int nread =0; /* number of char read */ X struct rimap map; /* buffer for map entries */ X struct ripag pag; /* buffer for page entries */ X extern long lseek(); X X /* these check to make sure our current record and such are */ X /* all still more or less valid - a kludge, but it works... */ X if(rec != rf->currec) { X cont = 0; X rf->currec = rec; X } X X if(cont) { X if(rf->curpag != -1L) X this = rf->curpag; X else X cont = 0; X } X X if(!cont) { X /* we are starting a new record - read the map block */ X if(rmapblk(rf,rec,&map) < 0) X return(-1); X X this = map.page; X rf->curpag = 0L; X rf->pagoff = 0; X } X X while(num > 0 ) { X X /* we have run out of pages to read */ X if(this == 0L) X break; X X /* read our page header */ X if(rpaghed(rf,this,&pag) < 0) X return(-1); X X /* seek to head of page data segment */ X if(cont) X j = (this * (long)RI_PSIZ(rf))+RI_PHSIZ + rf->pagoff; X else X j = (this * (long)RI_PSIZ(rf))+RI_PHSIZ; X X if(lseek(rf->fd,j,0) <0) X return(-1); X X /* read a page worth OR whatever is left in that page */ X if(cont) { X if(num >= (pag.high - rf->pagoff)) { X /* read rest of page, go to next */ X k = pag.high - rf->pagoff; X this = pag.next; X rf->curpag = this; X } else { X k = num; X } X } else { X if(num < pag.high) { X k = num; X } else { X this = pag.next; X rf->curpag = this; X k = pag.high; X } X } X X if(read(rf->fd,&buf[nread],k) < 0) X return(-1); X X /* adjust our notion of where we are */ X nread += k; X num -= k; X rf->pagoff += k; X if(rf->pagoff >= rf->sblk.reclen) { X rf->pagoff -= rf->sblk.reclen; X } X } X X rf->curpag = this; X buf[nread] ='\0'; X X return(nread); X} X X/* set the record file label (page 0) from the contents of buf */ Xrisetlab(rf,buf,num) Xstruct rifd *rf; Xchar *buf; Xint num; X{ X extern long lseek(); X X /* will it fit ? */ X if(num > rf->sblk.reclen) X return(-1); X X if(lseek(rf->fd,(long)RI_PHSIZ,0) <0) X return(-1); X if(write(rf->fd,buf,num) != num) X return(-1); X X return(0); X} X X X/* get the record file label (page 0) into buf */ Xrigetlab(rf,buf,num) Xstruct rifd *rf; Xchar *buf; Xint num; X{ X extern long lseek(); X X if(lseek(rf->fd,(long)RI_PHSIZ,0) <0) X return(-1); X X if(read(rf->fd,buf,num) < 0) X return(-1); X X buf[rf->sblk.reclen] = '\0'; X X return(0); X} X Xchar * Xrigets(rf,rec,buf) Xstruct rifd *rf; Xlong rec; Xchar *buf; X{ X char *bptr = buf; X static int cont; X static char *rptr = NULL; X X /* this is kind of wasteful, but necessary */ X /* we re-check to make sure that the record has not changed */ X /* in riread() - but have to do it here, too, to make sure */ X /* that we can take from our buffer, instead of reading */ X cont = rec == rf->currec; X X /* if the record number has changed, or the buffer is empty */ X /* we must needs fill us a buffer */ X if(!cont || rptr == 0 || *rptr == '\0') { X if(riread(rf,rec,rf->dat,rf->sblk.reclen,1) == 0) X return(NULL); X rptr = rf->dat; X } X X while(1) { X switch(*rptr) { X X case '\0': X /* out of stuff, load another buffer */ X if(riread(rf,rec,rf->dat,rf->sblk.reclen,1) == 0) { X /* record empty, finish buffer */ X *bptr = '\0'; X return(buf); X } X rptr = rf->dat; X *bptr++ = *rptr; X break; X X case '\n': X *bptr = '\0'; X rptr++; X return(buf); X X default: X *bptr++ = *rptr; X X } X rptr++; X } X} X X X Xriputs(rf,rec,buf) Xstruct rifd *rf; Xlong rec; Xchar *buf; X{ X char wbuf[BUFSIZ]; X char *bptr = wbuf; X char *rptr = buf; X int count =0; X X /* combined strcpy() and strlen() */ X while(rptr && *rptr) { X *bptr++ = *rptr++; X count++; X } X *bptr++ = '\n'; X count++; X X *bptr = '\0'; X if(riwrite(rf,rec,wbuf,count,1) != count) X return(EOF); X X return(0); X} X X X Xrigetcurpag(rf,buf) Xstruct rifd *rf; Xchar *buf; X{ X if(rf->curpag <= 0L) X return(-1); X if(lseek(rf->fd,(rf->curpag * (long)RI_PSIZ(rf))+RI_PHSIZ,0) < 0) X return(-1); X if(read(rf->fd,buf,rf->sblk.reclen) != rf->sblk.reclen) X return(-1); X return(0); X} X X X Xrisetcurpag(rf,buf) Xstruct rifd *rf; Xchar *buf; X{ X if(rf->curpag <= 0L) X return(-1); X if(lseek(rf->fd,(rf->curpag * (long)RI_PSIZ(rf))+RI_PHSIZ,0) < 0) X return(-1); X if(write(rf->fd,buf,rf->sblk.reclen) != rf->sblk.reclen) X return(-1); X return(0); X} ________This_Is_The_END________ if test `wc -c < recio.c` -ne 17221; then echo 'shar: recio.c was damaged during transit (should have been 17221 bytes)' fi fi ; : end of overwriting check echo 'x - recio.h' if test -f recio.h; then echo 'shar: not overwriting recio.h'; else sed 's/^X//' << '________This_Is_The_END________' > recio.h X/* X * Copyright (C) 1988, Marcus J. Ranum, William Welch Medical Library X * $Author: mjr $ X */ X X/* X * $Header: recio.h,v 1.1 88/06/01 21:28:06 mjr rel $: recio.h X * X * $Log: recio.h,v $ X * Revision 1.1 88/06/01 21:28:06 mjr X * Initial revision X * X */ X X#ifndef _INCL_RECIO_H X X X/* this is actually highly dependent on a few major assumptions: */ X/* 1) you can lseek() past EOF and extend with 'holes' */ X/* (which fails under MSDOS, I think, but who cares about DOS) */ X/* 2) when you read data from a 'hole' in a file, it will */ X/* come back zeros - hence we can check the flags to */ X/* see if it is free based on that. If your system does */ X/* not work in this way, you're out of luck, or you have */ X/* to construct large blank databases beforehand */ X/* this indicates a node deleted */ X#define RI_DELETED 0 X#define RI_ACTIVE -1L X X/* as a result of using 0L as an indicator of being a free block or a */ X/* null block, we have a left-over page at page #0. This can be used to */ X/* store invariant information */ X X/* our magic number - just to be safe */ X#define RI_MAGIC 0x72252 X X/* record file superblock (one at head of map file) */ Xstruct risuper { X long magic; /* magic number */ X int reclen; /* record chunk length (variable) */ X long free; /* head of free chain */ X long last; /* last page in file (alternate free list) */ X}; X X X/* an individual record header - one per record in map file */ X/* the map file consists of nothing but the superblock and one */ X/* of these critters per record */ Xstruct rimap { X long page; /* assigned page if any, else 0L for free */ X long tail; /* tail of assigned page list */ X}; X X X/* a page header - one at the head of each page in page file. */ X/* for now, quite rudimentary - room for more data if needed */ X/* otherwise it would not be worth defining a structure */ X/* - note - the use of longs here is a bit unecessary, but this */ X/* way its 4 longs, and there are no alignment problems between */ X/* my Sun and my VAX. Its a pain - feel free to fix it */ Xstruct ripag { X long recno; /* extra to help reconstruct crashes */ X long next; /* next page pointer 0L = no next */ X long deleted; /* 0 is free, anything else is not */ X long high; /* this page highwater mark (EOF) */ X}; X X X/* the actual control structure. Includes a buffer for reading pages */ X/* and a few other handy buffers for map entries, etc */ Xstruct rifd { X int fd; /* record file descriptor */ X int mfd; /* free map file descriptor */ X struct risuper sblk; /* superblock */ X /* all used to emulate stdio, sort of */ X char *dat; /* page record buffer (malloced in riopen()) */ X long curpag; /* used to simulate consecutive reads */ X long currec; /* used to simulate consecutive reads */ X int pagoff; /* offset within current page */ X}; X#define RIFD struct rifd X Xextern struct rifd *riopen(); Xextern long riseek(); Xextern char *rigets(); X X#define _INCL_RECIO_H X#endif ________This_Is_The_END________ if test `wc -c < recio.h` -ne 2905; then echo 'shar: recio.h was damaged during transit (should have been 2905 bytes)' fi fi ; : end of overwriting check echo 'x - sizes.c' if test -f sizes.c; then echo 'shar: not overwriting sizes.c'; else sed 's/^X//' << '________This_Is_The_END________' > sizes.c X/* X * Copyright (C) 1988, Marcus J. Ranum, William Welch Medical Library X * $Author: mjr $ X */ X X#ifndef lint Xstatic char *RCSid="$Header: sizes.c,v 1.1 88/06/01 21:27:31 mjr Rel $: sizes.c"; X#endif X X/* X * $Log: sizes.c,v $ X * Revision 1.1 88/06/01 21:27:31 mjr X * Initial revision X * X */ X X/* provides some minimally useful info about how big your */ X/* data structures are */ X X#include "recio.h" X Xmain() X{ X (void)printf("a record I/O control structure is %d bytes\n",sizeof(RIFD)); X (void)printf("(that is without the dynamically allocated page buffer)\n"); X (void)printf("\na map block is %d bytes\n",sizeof(struct rimap)); X (void)printf("\na page header is %d bytes\n",sizeof(struct ripag)); X (void)printf("make sure these sizes match, if you're running on\n"); X (void)printf("several different types of CPUs.\n"); X} ________This_Is_The_END________ if test `wc -c < sizes.c` -ne 823; then echo 'shar: sizes.c was damaged during transit (should have been 823 bytes)' fi fi ; : end of overwriting check exit 0