Simtel MSDOS 1992 December/simtel1292_SIMTEL_1292_Walnut_Creek.iso/msdos/pcmag/extend_4.arc/EXTEND.DOC

Text File | 1987-10-26 | 17KB | 295 lines

(*
                                  EXTEND.PAS
                                 Version 3.0
                               October 26, 1987
                              by Scott Bussinger
                             Compuserve 72247,2671


  MS-DOS and PC-DOS Turbo Pascal 3.0 and higher only allow up to 15 files to
be open at the same time, due to limitations in DOS.  This file shows you how
to have up to 96 files open simultaneously under DOS 2.0 or 2.1, or 252 files
open simultaneously under DOS 3.0 or higher.  Below is a description of how to
use this technique, followed by a technical explanation of the implementation,
for those who are interested.  Note this documentation is liberally based on
the original EXTEND.PAS documentation by Randy Forgaard, but has been renamed
to EXTEND.DOC and modified by Scott Bussinger to reflect the changes to
involved in versions 2 and 3.

TO USE THIS TECHNIQUE:

  You simply include the EXTEND unit in the USES statement in the source code
of your main program.  You will have to either compile the EXTEND unit
separately or use Turbo Pascal 4.0's project compiling features to Make or
Build your program (which will compile the EXTEND unit automatically).  The
EXTEND unit should be the first unit in your USES statement, following only
the DOS unit (which is USEd by the EXTEND unit).  The unit automatically
installs itself and will uninstall itself when your program ends or an error
is detected.  See the demo program below for an example of using the EXTEND
unit.

  The EXTEND.PAS file will need to be available available to be compiled if
the Make or Build features are used to compile the program.  The EXTEND.ASM
is the source code for the assembly language interrupt handler used when
running under DOS 2.  The EXTEND.OBJ file is the assembled object code for
EXTEND.ASM and was compiled using A86, a shareware compiler available on
Compuserve and other bulletin boards.

  You must edit your CONFIG.SYS file (see the DOS manual for details), so
that it includes a line that says "FILES=XXX".  XXX should be a number that is
at least 3 greater than the maximum number of files that will be in use
(opened by RESET, REWRITE or APPEND) at the same time during the execution of
your program (larger values will provide no additional benefit, with respect
to your individual program), and should not exceed 99 (for DOS 2.0/2.1) or 255
(for DOS 3.0 and higher).  Under any version of DOS, the minimum allowable
value for XXX is 8.  Then, reboot your computer so that the FILES=XXX
parameter takes hold.  This same change to the CONFIG.SYS file will have to be
made on any machine in which your program is to run.

  Running the sample program at the bottom of this file will tell you the
maximum number of files you can have open at once (usually 3 less than the
number of files specified in the CONFIG.SYS file as discussed above, unless a
resident program has opened some files and hasn't closed them yet).


THE TECHNICAL DETAILS:

  Much of the following information is not documented in the DOS Technical
Reference manual.

  Under DOS 1.0 and 1.1, all files were accessed via File Control Blocks
(FCB's).  There was no limit to the number of FCB's that a program could use,
so there was no limit to the number of files open simultaneously.

  Under DOS 2.0 and higher, an alternate (and preferable) method of accessing
files was introduced, using a 2-byte integer called a "handle" to refer to a
file.  A "handle" file is described using a data structure called a Device
Control Block (DCB).  However, DOS provides the storage space for all DCB's,
rather than having the application program store the DCB's, so the number of
available DCB's is limited to the amount of space that DOS has set aside for
them.  The maximum number of files/devices that can be open simultaneously, on
the whole computer, is the number of slots available in the DCB Table created
by DOS.  The DCB's in the DCB Table are consecutively numbered, starting with
0.  DCB's 0, 1, and 2 are predefined by DOS to correspond to the AUX, CON, and
PRN devices, respectively.  All remaining DCB's in the DCB Table are available
for files or devices used by application programs.

  So that I/O redirection can be supported, the DCB numbers are not used
directly when accessing files.  Instead, a file "handle" is used.  A "handle"
is an index into a 20-byte array, called the Handle Table, located at offset
18H of the Program Segment Prefix (PSP) for a program (for a general
discussion of the PSP, see the DOS Technical Reference manual).  Each element
of the Handle Table is the DCB number of a file or device.  The value at index
"handle" in the Handle Table is the DCB number of the file or device that
implements that file handle.  Thus, if the value 8 is in the 6th byte of the
Handle Table, the handle "6" refers to the file (or device) described by the
DCB in slot 8 of the DCB Table.  If a handle is not currently being used, its
entry in the Handle Table is FFH.  DOS predefines the first 5 handles to be
primary input, primary output, error, auxiliary, and printer, so the first 5
entries in the Handle Table are 1, 1, 1, 0, and 2, corresponding to the DCB
numbers for the CON (1), AUX (0), and PRN (2) devices.  This leaves only 15
available handles for opening files (or new devices).

  Every time a new handle file is opened, a new handle gets used.  Since there
are only 20 slots available in the Handle Table for a program, DOS only allows
a "process" to have a maximum of 20 file handles in use simultaneously (and the
first 5 entries are predefined, as just noted, unless those handles get closed
and reused).  Every new handle file requires a unique handle, so only 20
files/devices can be open at the same time by a single process (unless FCB's
are used).  (A "process" is any program spawned using the DOS EXEC function
call.  A process can be invoked by COMMAND.COM, or by another program.)  There
can be many more than 20 DCB's in the DCB Table, so the real limitation is in
the size of the Handle Table in the PSP.

  The size of the DCB Table (i.e., the maximum number of files/devices that
can be open simultaneously in the whole computer) is controlled by the
FILES=XXX entry in the CONFIG.SYS file.  The minimum number of slots is 8.
Under DOS 2.0/2.1, the maximum number is 99, and under DOS 3.0 and higher,
the maximum is 255.  As previously mentioned, the first three of these DCB
slots are occupied by the AUX, CON, and PRN devices.

  A single program can use all of the DCB's in the DCB Table (except for the 3
reserved by DOS) all on its own, by effectively bypassing the Handle Table in
the PSP, except on a temporary basis.  The program can accomplish this feat by
using, say, only one entry in the Handle Table for all of its files.  Instead
of allowing DOS to store the DCB numbers in the Handle Table, the program can
store these numbers elsewhere.  Then, to manipulate a file using DOS, the
program can temporarily put the DCB number of that file into a designated slot
in the Handle Table, pass the index of that table slot (i.e., that "handle")
to DOS, and DOS will operate on that handle/DCB number.  After the DOS call,
the program can remove that DCB number from the designated Handle Table slot,
freeing up that handle for use in another DOS call for another file.  In this
way, DOS can be fooled into accessing up to 96 (or 252) different files/devices
using a single handle entry in the Handle Table.

  To obtain the address of the Handle Table, which is at offset 18H in the
PSP, the program needs to find the address of its PSP.  Under Turbo Pascal
4.0, the segment address of the PSP is stored in a pre-defined variable called
PrefixSeg setup by Turbo's standard initialization code.  All of the
manipulation of the DCB's and handles is dealt with by ExtendHandler which is
an interrupt handler which intercepts all DOS functions calls (INT 21) and
checks to see if the function involves a handle number.  If the function
returns a handle (e.g. an Open File) it then returns the DCB to the calling
program in place of a handle.  It adds an constant offset of 5 to the DCB so
that ExHandler can distinguish between DCB's and calls refering to the 5
automatically assigned standard handles.  If the function invoked passes a
handle to DOS (e.g. Read File) it checks to see if it refers to a standard
handle and if so it passes control normally.  If not, it subtracts the
constant offset of 5 and temporarily usurps the use of handle 19.  It places
the DCB into the the slot in the handle table for handle 19 and changes the
handle in the registers to 19.  The previous DCB in handle 19 is saved and
restored afterwards (this probably isn't necessary, but made me feel better).

  Note that these routines support all DOS functions up through DOS 3.3
including those not normally used by Turbo.  The exception is function
$46 (FORCEDUP). This call is rarely used, but incompatible with the methods
used by ExHandler.  The only use that I know of for this function is to
redirect the standard handles and since the standard handles are effectively
skipped by these routines, there is no problem in using handles 0-4 as the
second handle in a FORCEDUP call.  No other handles are allowed and will
automatically result in an error being returned.  In other words, go ahead and
redirect the standard handles, but don't try it with anything else.

  As of version 2.4, the installation routine now checks the DOS version that
the program is being run under.  Under 2.x, the unit functions as described
above.  Under DOS 3.x or higher, the EXTEND unit now uses a slightly
documented capability to install a new, larger handle table in the later
versions of the operating system.  Essentially, under DOS 3.x Microsoft
started to permit a standard way of creating a new handle table.  This
information is has been seen in at least one Microsoft document and that is
probably a sign that it will continue to work in future versions of DOS.
There are now two fields in the PSP which contain a pointer to the beginning
of the handle table and a byte specifying the number of available handles.
This method is much less kludgey and solves some problems with running
programs under debuggers.  The space for the new handle table is allocated off
of the heap to save space in the data segment.  When Exit routine is called,
the handle table is DISPOSEd.  According to the Turbo manual DISPOSE is not
compatible with RELEASE, so be aware that if you are using RELEASE in you
program that you will need to modify the UnExtendHandles routine to be
compatible.  Possibly the easiest thing to do is simply remove the DISPOSE
statement, but it will depend on your program.

  As of version 2.3, the interrupt handler checks to make sure that the
program calling the DOS function has the same code segment as the interrupt
handler.  This solves a problem with EXEC'ing out to a DOS shell and trying
to execute batch files.  For some reason, the methods used by EXTEND
completely screw up the DOS batch file processor, so it just ignores calls
from outside programs.  This may also prevent some problems from already
loaded in-memory TSR programs like SideKick that also use DOS calls.  Note
that because the internal file handle table is largely bypassed, any child
process which is invoked using EXEC will not be able to inherit open handles
other than the standard handles.  Version 3.0 modifies this logic slightly to
account for Turbo 4.0's new memory map.  It checks that the program calling
INT 21 has a code segment between the PSP and the DS of the installing
program.  Because of how Turbo 4.0 initializes its registers, this guarantees
the the calling program is the same as installed the interrupt handler.

  There is one incompatibility that I know of with EXTEND.  That is
sub-functions 4 and 5 of DOS function $44 (IOCTL) when running under DOS 2.
These calls send and receive control strings from block devices.  These two
sub-functions use registers differently than all other sub-functions of IOCTL.
I can't imagine a use for these two functions in a Pascal program, so I
haven't bothered to modify the interrupt handler to identify this case.  If it
is a problem, let me know and I may fix it.  The only other call that could
potentially collide with EXTEND is the new function $67 (Set Handle Count)
that was added in DOS 3.3.  This function serves the same purpose as the
EXTEND unit, so I can't imagine why you would use both in one program anyway.
Though I haven't tested it, I suspect the only effect would be to waste some
memory (and not much at that).

  Many thanks to Randy Forgaard (Compuserve 70307,521) for writing the
version 1 of this program, Bela Lubkin (CompuServe 76703,3015) for
masterminding this idea, and to Kim Kokkonen (CompuServe 72457,2131) for
helping me debug it.  For more discussion of Handle Tables and the
implementation of DOS redirection, please see Stan Mitchell, "Command
Line Redirection," PC Tech Journal, January 1986, Page 44.

Change Log:

  Version 1.1: Now invokes a DOS function to get the PSP address, so that
               programs using this technique can be run in memory under Turbo.

  Version 1.2: Replaced some outdated comments.

  Version 1.3: Changed example program so that it erases the files it creates.

  Version 2.0: Changed program from explicit calls to extend files into an
               INT21 handler which intercepts all DOS calls.  This makes use
               of extended files generally invisible to the source code and
               simplifies its use.  First version by Scott Bussinger.

  Version 2.1: Changed it so that handle passed to DOS is reconverted back
               into a DCB since some of the runtime library code assumes that
               registers are useful after calling DOS.

  Version 2.2: Fixed some problems with EXEC and allowed freedom to use
               FORCEDUP with the standard handles.

  Version 2.3: EXTEND now only affects file handling calls made from within
               its own code segment (i.e. the installing Pascal program). This
               solves problems with using EXEC with batch files and already
               resident TSR (terminate and stay-resident) programs.

  Version 2.4: Fixed a problem with the DUP function under DOS 2.x.  EXTEND
               now allocates new handle table under DOS 3.x on heap to be
               compatible with TDebug Plus.

  Version 2.5: Work done by Kim Kokkonen and Brian Foley to work with Turbo
               Extender and whittle off a few clock cycles

  Version 3.0: Reworked as a UNIT for use with Turbo Pascal 4
               EXTEND.ASM reworked to be compatible with A86 assembler
               Added support for DOS 3.3 (Commit File call)
*)

{  Example program  --  This program opens as many Text files as it can, until
 DOS runs out of room in its DCB Table.  It then reports how many files were
 successfully opened, writes a line to each of them, then closes and erases
 each of them.  Note: The value of the FILES=XXX parameter in the CONFIG.SYS
 file must be set to an appropriate value (see above).  If you change the
 FILES=XXX value, be sure to reboot before running this program.

 This program takes a while to run, due to the heavy disk I/O, so running it on
 a hard disk (or, even better, a RAM disk) is recommended.  Make sure that you
 are running the program in a subdirectory, so that you don't run up against
 the DOS limit on the number of allowable files in the root directory of a
 drive. }

program Test;

uses Extend;       { <--- This is the magic line that does everything }

const MaxCount = 255;

type FileArray = array[1..MaxCount] of text;

var Count: integer;
    F: ^FileArray;
    I: integer;
    Num: string[6];

begin
new(F);            { Use heap because of large size of this array }
writeln('Opening files...');
I := 0;
repeat
  inc(I);
  str(I,Num);
  assign(F^[I],'junk' + num + '.txt');
  {$I-}
  rewrite(F^[I])
  {$I+}
until ioresult <> 0;
Count := I - 1;
writeln('Successfully opened ', Count, ' files at the same time.  Writing to each file...');
for I := 1 to Count do
  writeln(F^[I], 'This is a test.');
writeln('Closing and erasing each file...');
for I := 1 to Count do
  begin
  close(F^[I]);
  erase(F^[I])
  end;
writeln('Done.')
end.