home *** CD-ROM | disk | FTP | other *** search
- Compressdisk.device 37.14 (release 1.0)
-
- by Antonio Schifano
-
-
- 1 - Introduction.
-
- Data compression is not a novelty in the Amiga scene: since many years
- there are many commercial and public domain utilities that give to all
- Amiga users the possibility to save disk space. We can roughly divide data
- compressors in two main classes: the class of archivers (like ZOO, LHARC,
- etc.), and the class of "file oriented" compressors (like COMPRESS,
- POWERPACKER, IMPLODER, etc.). Archivers' main feature is their ability to
- preserve the files hierarchy; the price to pay is the impossibility to use
- archived files in the compressed form. On the contrary, file oriented
- compressors sacrifice this feature to achieve a better usability of
- compressed files: programs compressed by PowerPacker, for instance, remain
- executable; moreover, the package includes some useful tools (like PPMORE)
- that compensate for the versatility lost because of the compression (for
- instance, you can still read compressed text files, but you cannot edit
- them). Compressdisk.device tries to resolve the defects of both types of
- packers with a different approach to the problem.
-
-
-
- 2 - What is compressdisk.device?
-
- As its name probably suggests, it is an Exec device, similar in the way of
- operating to Matt Dillon's fmsdisk.device. Like that, in fact, it allows
- you to create virtual disks, whose tracks are saved on a file. Unlike that,
- however, it uses a different file for each track, and compresses the
- tracks. This allows you to create virtual partitions on your hard-disk,
- usable like normal partitions, but having the great advantage that every
- file you write is automatically compressed (and obviously, it is decompressed
- whenever you read it back).
-
-
-
-
- 3 -Installation.
-
- The device can be installed only on Amigas with kickstart V37+. The
- installation is (almost) totally performed by the programs "CDInstall" and
- "CDPartition", who take care of copying all necessary files in the right
- directories:
-
- - The file "compressdisk.device" is copied in the DEVS: directory.
- - For each virtual partition desired, an opportune entry is created in the
- DEVS:mountlist file. If you are using Workbench V38+, all entries will
- be created in the DEVS:DOSDrivers directory.
- - The preference editor "CDPrefs" is copied in the SYS:Prefs directory.
-
- For the device to work properly, you should add the following line to your
- s:user-startup file, just after the command SetPatch:
-
- assign CDISK: <directory>
-
- where <directory> is the complete path of the directory in which you want
- the device to store the compressed tracks.
- Otherwise, the device will store the tracks in the root directory of the
- boot partition.
- Once mounted, each virtual partition must be initialized; this can be
- easily accomplished with the system command "format". Since the device
- stores only the tracks written at least once, it is advisable to use the
- QUICK option: this will let you save disk space and time (formatting a 40M
- partition can take more than thirty minutes on a standard Amiga 500).
- To speed up read/write operations, the device features a built-in cache in
- which it stores the most recently used tracks; the cache size can range
- from one to thirtytwo tracks. The default configuration is eight tracks per
- unit; from tests the device proved to be usable with four or, if you want
- to save your memory, even with two tracks, whereas with only one track the
- performance reduced significantly, expecially in writing.
-
-
-
- 4 - How to use compressdisk.device.
-
- You can use virtual partitions created with compressdisk.device like
- usual ones; nevertheless, it is better to follow some guidelines. First of
- all, it is recommendable not to write any files into any virtual partition
- if the device work disk (CDISK:) is full: if the device cannot store some
- tracks, you lose some data. Moreover, you should not use virtual partitions
- for storing temporary files; remember that deleting a file (on the standard
- file system), does not clear the blocks that it occupied (in fact, you can
- recover it using proper utilities), until the file system does not
- overwrite them. This means that when you delete a file you will NOT free
- any space on the device work disk. Virtual partitions created with
- CDPartition use the Fast File System; however you can use any file system
- you want. Since Standard File System and the correspondent International
- version reserve part of the data blocks for internal informations, they can
- reduce the tracks compression; for this reason their use is not
- recommended.
-
-
-
- 5 - The compression algorithm.
-
- The more decisive choice in the device realization was the selection of the
- compression algorithm: it is clear that the device performance depends
- tightly on the data compressor and decompressor ones. The algorithm had to
- be as fast as possible, to use little memory, and reasonably good at
- compressing. The used algorithm was ideated by Ross Williams, who named it
- LZRW3-A; it requires only 16K of work memory, and has an average
- compression of 54%. The version I wrote and optimized in assembler has a
- compression speed of 16K/sec and a decompression speed of 64K/sec (on a
- 68000 at 7.16MHz). The following table gives a rough idea of the
- performances, obtained by compressing two text files, an executable, a link
- library and a soundtracker module:
-
- -------------------+------------+-----------+--------+----------+----------
- file | dim. ante | dim. post | % comp | K/sec c. | K/sec d.
- -------------------+------------+-----------+--------+----------+----------
- intuition.doc | 237916 | 98950 | 58.4 | 19.07 | 84.80
- dkbffp | 144648 | 81606 | 44.6 | 13.17 | 60.36
- dkb212.doc | 157458 | 74523 | 52.7 | 17.43 | 76.12
- amiga.lib | 157880 | 43866 | 72.2 | 26.40 | 118.60
- mod.trial | 108946 | 86847 | 20.3 | 8.35 | 39.99
-
-
- For further informations about LZRW3-A refer to the original documentation
- given by Ross Williams (directory "pub/compression" on the FTP site
- sirius.itd.adelaide.edu.au [IP=129.127.40.3]).
-
-
-
- 6 - Implementation.
-
- From a technical viewpoint, compressdisk.device may look a bit strange,
- since it seems to break the hierarchy of the Amiga OS. In fact it is
- unusual for a device to use the DOS, since this one is at an higher level
- in the hierarchy. The hierarchy is purely functional: devices are used
- for management of physical resources (eventually virtual ones) and
- handlers for O.S. logical resources. For this reason a device is allowed
- to implement its primitives using a logical resource, provided that the
- tasks which use the device are not aware of the implementation of the
- primitives.
- The device core is a process which satisfies all I/O requests: whenever it
- receives a CMD_READ command, it loads and decompresses all the needed
- tracks; in a similar way, whenever it receives a CMD_WRITE or CMD_FORMAT
- command, it compresses and stores the tracks. If possible, the process uses
- the cache both in reading and writing: in reading, if a track is already in
- the cache, there is no need to reload and decompress it; in writing, tracks
- are compressed and stored only if there is no space left in the cache.
- Tracks are updated upon receiving a CMD_UPDATE command; data consistency is
- thus ensured. The track buffers can range from 1 to 32. Every track has 64
- sectors of 512 bytes each, for a total of 32K. The device needs about 60K
- of working memory, plus about 32K for each track buffer.
-
- The highest cylinder number supported by the device is 65000. The largest
- size of a virtual partition is thus 2031.25M.
-
-
-
- 7 - Acknowledgements.
-
- I wish to thank the following people, in no particular order:
-
- - Raffaele Dell'Aversana, for the academic discussions prior to the
- realization of the device, as well as for testing my brand new 2MB chip
- ram expansion on his A500+ (still frightened of seeing your caps lock
- key flashing? );
-
- - Ross N. Williams, for the data compression algorithm;
-
- - Jan van den Baard, for the (useful!) utility "GadToolsBox";
-
- - Giuseppe Sacco, for the utility "ZeroUno" (see the DISCLAIMER!! ;-);
-
- - The organizers of I.P.I.S.A. '92;
-
- - my sister Letizia, for no particular reason.
-
-
-
- 8 - Copyright/distribution.
-
- Compressdisk.device, Copyright © 1992 Antonio Schifano.
- All rights reserved. This program is SHAREWARE; if you use it, send
- Lit. 15000 if you live in Italy, 20DM if you live in the European
- Community, or 20USD if you live somewhere else.
- The program can be distributed in its original archive format; can be
- included in collection disks of public domain software. It can also be
- distributed in electronic form via BBS, FTP, and similar. You may not
- charge more than Fred Fish does for a single library disk. Also no profit
- is to be made by selling this program. The commercial use and distribution
- of this program is NOT allowed without written permission from the author.
- The author cannot be held liable for any damage or data loss directly or
- indirectly caused by the use or misuse of this program; by deciding to use
- it the user assumes all the responsibilities.
- Suggestions, bug reports and comments are welcome.
-
- Send the registration form to:
-
- Antonio Schifano
- via Don Bosco, 3/28
- 56100 Pisa, Italy
- (until 31 January 1993);
-
- Antonio Schifano
- via dell'Oca, 11
- 91100 Trapani, Italy
- (always);
-
- The FidoNet address is: 2:332/607.666
-
-
-
-
-