Windows NT Super Tune-Up Kit/PIE-WindowsNTSuperTuneUpKit-1997.iso/COMPRESS/NTUNEZ/BUGS.LON

Text File | 1994-01-23 | 18KB | 388 lines

Bogus bugs (not our fault!):
---------------------------

 By far THE BIGGEST source of bug reports to Info-ZIP/zip-bugs is the
 incorrect transfer of zipfiles (or of the UnZip executable itself).
 ALWAYS TRANSFER IN BINARY MODE!  This includes ftp transfers and *both*
 ends of a Kermit connection ("set file type binary").  If your copy
 isn't exactly the same size as the original, you made a mistake.

 Another common source of errors such as "compression method 8 not sup-
 ported" is the existence of an old version of UnZip somewhere in your
 path.  Make sure you're using the version you think you're using; give
 the full path explicitly if necessary.  Executing "unzip" without any
 options will print a help screen, at the top of which is the UnZip
 version number and release date.


Bugs (real and/or imagined):
---------------------------

 - DEC Ultrix:  on long zipfiles, unzip will sometimes fail (bad CRC, not always
    reproducible); this is apparently due either to a hardware bug (cache mem)
    or OS bug (page faults?) [Igor, Jean-loup, bottom of BUGS.long]
 - Pyramid:  USE_FWRITE causes CRC errors (???) [Kevin]
 - possible problems with unzip -a on binary file in compact-model-MSDOS-Zip-
    created archive? (showed up with MSDOS zip 1.9l and various unzips; inflate
    crashes(?)) [Piet]
 - funzip/more/decryption/no-echo bug:  race condition(?) causes terminal
    to be "reset" to no-echo state
 - directory dates/times (special Unix perms?) not restored
 - Macintosh (100200), Atari (020000) external file attributes not interpreted
    correctly (both unzip and zipinfo)
 - pkbug error:  zipfile with incorrect csize and/or ucsize--check for end of
    compressed (csize) data in uncompression routines:
      unimplod.c:    while ((!zipeof) && ((outpos + outcnt) < ucsize)) {
      unreduce.c:    while (((outpos + outcnt) < ucsize) && (!zipeof)) {
    [James Birdsall, Mark, bottom of BUGS.long]
 - OS/2:  directory EAs not restored if directory exists [Kai Uwe, KG27515@uark]
    (subsequent note:  no way to determine which EAs are newer ==> cannot
    restore without user input)
 - MS-DOS:  Borland executables don't allow other than 80-column, 25/43/50-line
    screen modes (Borland bug) [Michael Stillwell]


Features (possible and/or definite):
-----------------------------------

 - put man pages in more "proper" nroff format
 - ignore case for internal filename match on non-Unix systems, unless file-
    specs enclosed in single quotes
 - save/extract Unix mtime/ctime/atime/UID/GID info (Unix extra field)
 - modify to decompress input stream if part of a pipe, but continue using
    central directory if not (BIG job!)--extended local header capability
 - add zipinfo option(s) to sort alphabetically, by date/time, in reverse, etc.
 - add "near" to global vars [Steve Salisbury, 92.4.21]
 - modify set_file_time routines to share common code (macro?)
 - when listing filenames, use '?' for non-printables? [Thomas Wolff, 92.6.1]
 - add zipinfo "in-depth" option? (check local vs. central filenames, etc.)
 - create zipcat program to concatenate zipfiles
 - assembly-language routines?
 - VM/CMS version (Walter Moore, Phil Howard, Chua Kong Sian, others)
 - add -oo option (overwrite and override)?  no user queries (if bad password,
    skip file; if disk full, take default action; if VMS special on non-VMS,
    unpack anyway; etc.)
 - add -Q[Q[Q]] option (quiet mode on comments, cautions, warnings and errors)?
    forget -oo, or make synonym?  Default level -Q?
 - add OS/2 .INF format helpfiles for UnZip and ZipInfo?




Additional thoughts/reports/arguments/issues (many VMS):
-------------------------------------------------------

From: [e-mail] (Terry Poot)
Newsgroups: comp.os.vms
Subject: Re: Speeding up VAX C writes
Date: 22 Oct 91 11:48:59 GMT

In article <[e-mail]>, [e-mail] (Paul Hardy)
writes:
>Some months ago, I remember seeing an item here about how to speed up
>writes from VAX C, in the case where the data being written was binary.

Actually, the only trick I know about works no matter what the data is. I'm
talking here about the normal C stream_lf files. You can use other RMS file
types by specifying parameters to fopen, open, or creat, and they may be faster
or slower depending on what you are doing, and how you are doing it. However,
there is a way to speed up most I/O on stream files, in many cases
dramatically.

You want to add the argument "mbc=nnn" where nnn is the number of pages of
memory you can spare for I/O buffering. mbc is the RMS field name for the
Multi-Block Count. It tells the C rtl to use a buffer big enough to hold nnn
blocks (blocks are the same size as pages for a little while longer at least,
512 bytes). Thus rather than reading or writing a block at a time, you can have
it do it 200 blocks at a time, if you can spare 100Kb of memory, or any other
value. (I'm sure there's an upper limit, but I don't know what it is. In
practical terms, for most people it'll probably be PGFLQUO.)

BTW, note that I said the C rtl and not RMS. C I/O uses RMS block mode I/O,
which doesn't actually support mbc. It does, however, support whatever buffer
size you choose, as long as it is an integral number of blocks. The designers
of the RTL decided to check for mbc even on normal C stream files and allocate
the buffer size accordingly (Thanks, guys!). This is why specifying mbf, the
multi-buffer size, doesn't do anything. They didn't do multi-buffering, and RMS
doesn't support that either for block mode I/O. (Anyone wanna submit an
enhancement request SPR?)

These little tidbits of info are courtesy one of the C RTL developers, from
discussion after a session on C I/O at the DECUS symposium in Las Vegas.


------------------------------
From: [e-mail] (Bob Techentin)
Newsgroups: comp.os.vms
Subject: Re: Speeding up VAX C writes

Paul Hardy, <[e-mail]> asked:

>  Some months ago, I remember seeing an item here about how to speed up
>  writes from VAX C, in the case where the data being written was binary.
>
>  We now have an application which is using fopen, then fseek and fwrite,
>  and is taking forever to do the writes. I'm sure that by putting the right
>  magic words in the fopen, we can cut out a lot of the unneccessary work
>  that the C RTL is doing on our behalf.

The VAX C RTL function fopen() creates files that are stream-lf by default.
This is not a natural kind of file for RMS, which prefers record oriented
structures.  You can use some non-portable options in the fopen() command to
create record oriented files.   The call:

    fp = fopen(filename, access, "ctx=rec");

will force the RTL to use record access for read or write - even on a streamlf
file.  This improves performance significantly.  The call:

    fp = fopen(filename, "w", "rfm=var", "rat=cr");

will create a variable length record/carriage return carriage control file
(instead of a stream-lf/no carriage control file), which is what your typical
text editor creates.  Read and write performance to record structure files is
even better than just specifying record access.

You can use other options documented in the VAX C Run-Time Library Reference
Manual, creat() function.  We set multibuffer counts, default extensions, and
read-ahead and write-behind buffering to improve our file performance.

Bob Techentin                              Internet: [e-mail]
Mayo Foundation, Rochester MN, 55905 USA             [phone]


------------------------------
Date: Sun, 12 May 91 11:30:40 PDT
From: [e-mail] (Mark Adler)

James Birdsall cautions:

>>    Warning! Do NOT trust PKUNZIP -t. I have a ZIP file which is badly
>> corrupted -- missing about 25K from the end. PKUNZIP -t claims it is OK.

Rather alarming.  It is a problem, but it's not as bad as it sounds
initially.

I took a look at his zoo201.zip (which has the single entry zoo201.tar)
and here's what I found:

1. Unzip fails with a crc error.
2. PKUNZIP succeeds, but the resulting file is too short (444416
   compared to the 532480 reported in the zip file).
3. If I PKZIP that 444416 byte file, I get exactly zoo201.zip back.
   (Except the length entries are 444416 in the local and central
   headers--a total difference of four bytes between the two.)
4. Unzip thinks that new zip file is ok.
5. 444416 is multiple of 2048.

Conclusions:

1. What's "wrong" in zoo201.zip is the length, not the data.
2. PKUNZIP doesn't notice this, since it is driven by the compressed
   "size" and not the uncompressed "length".
3. Unzip *does* notice because it is (apparently) driven by the
   length, and not the size.  (I haven't looked into this yet.)
4. What must have happened is that there was an error reading the
   original 532480 byte file at the 444417th byte (probably the
   869th 512-byte sector), but PKZIP thought is was just the end of
   file and stopped compressing.

Bugs:

1. PKZIP doesn't handle read errors properly, and doesn't check the
   number of bytes read against the size of the file.  (This is not
   a problem with Zip by the way, since it does look for read errors,
   and it sets the length from the number of bytes actually read from
   the file.)
2. PKUNZIP never looks at the length to see if it decompressed the
   right number of bytes--it only cares if the CRC is right.
3. Unzip doesn't stop when it runs out of compressed data (otherwise
   it would have gotten the right CRC, as it did with the re-PKZIPped
   version).  Both PKUNZIP and Unzip should only decompress "size"
   bytes and check that "length" bytes were generated.

Mark Adler
[e-mail]


------------------------------
Date: Sun, 12 May 91 13:57:15 PDT
From: [e-mail] (James Birdsall)

The compressed size is accurate -- for
the amount of data that was included. The uncompressed size is accurate for
the original file. ...

   I hypothesize that PKZIP, when creating the bad file, got a premature
EOF when reading the source (which was one big tar file on an NFS-mounted
drive). Assuming it was done, it proceded to button up the file as though


------------------------------
From [e-mail] Tue Apr 21 20:28:05 1992
To: [e-mail]
Cc: [e-mail]
Subject: Re: UNZIP suggestion (Buffer Size flexibility)
Date: Tue, 21 Apr 92 17:26:43 PDT

Also, Jean-loup added the "near" keyword to all of ZIP 1.6's global variables
(both the extern declarations and the actual definitions) which causes better
code generation for Microsoft C in compact model.  Shouldn't we do the same
for UNZIP 5.X?


------------------------------
Date: Wed, 18 Mar 92 06:57:28 PST
From: [e-mail] (Mark Adler)
To: [e-mail]
Subject: Re: yet another zip format extension

As for other one-pass implementation details, I have given a
little thought to that.  Unzip should write an auxilary file as
it's unzipping containing enough information to go back and
change attributes, file names, and possibly do some file
conversions, upon encountering the central directory.  I'm
figuring that it can just write all the entries to temporary file
names at the base of where it's supposed to unzip to (or a
specified temporary area), and then move, chmod, or convert files
to where and what they're supposed to be later.  When you get to
the central directory, you start reading the auxilary file for
the temporary names.  We can also use the good old tempf routines
to avoid writing small auxilary files.


------------------------------
From: [e-mail] (Phil Howard )
Newsgroups: comp.compression
Subject: Re: Wanted: ZIP on VM/CMS
Date: 20 May 92 22:31:24 GMT
Organization: Netcom - Online Communication Services  (408 241-9760 guest)

[e-mail] (Kenneth Osterberg) writes:

>The subject says it all. Has anybody ported InfoZIP (both compression
>and decompression) to the VM/CMS environment? Any estimates on the
>work involved doing this?

One of the problems is how to represent a byte-stream or byte-array
file on VM/CMS.  The conventional way is to store arbitrary sized
pieces of the byte stream as individual variable length records in
a type "V" file.  The record boundaries are arbitrary amd reconstruction
of the original stream is done simply by ignoring those boundaries.

This is the way Kermit and FTP usually transfer data in binary mode.

However it takes 2 bytes of allocated and addressable CMS file space
to store those boundaries.  Since the lengths are arbitrary, it is not
possible to randomly address a specific byte of the stream itself,
since you really have no way of knowing how many record boundaries
preceed "stream byte N".  You would need to know that to know what
record number that byte is in.

You could read the "V" file sequentially, but this is very inefficient
and certainly to be avoided.

The other method of representing a byte-stream or byte-array file is
to use a type "F" file (fixed length records) with a record length
of exactly 1.  However there are disadvantages to this as well:

1.  Even though Kermit and FTP can create and handle these files, it is
    not the default.

2.  Many CMS tools read and write files 1 record at a time (instead of
    the more sophisticated and version sensitive method of reading
    and writing many records or whole blocks).  This makes handling of
    such files very slow.  Unzipping a 1 megabyte file might take 30
    seconds whereas copying it to another minidisk could take 30 minutes.

3.  Transferring such files across networks tends to expand on the space
    requirements.  While in transit between VM/CMS systems, such files
    may take up to 3 times the normal space, apparently depending on the
    version of RSCS.

4.  Possibly no or poor support in systems like IBM TSO and DEC VMS.
    I have in the past sent such files to a VMS host, which proceeded
    to store them internally with a CR/LF between each byte (it thought
    they were records) including deleting the bytes with values of
    either 0x40 (EBCDIC space) or 0x0d (CR).  This resulted in an
    ambiguity making the original not reconstructable even if one were
    to write a tool to attempt it.

Other representation ideas include:

1.  Use larger fixed length records, with the byte count filled in as
    the first 4 bytes of the first block, or the first 4 bytes of the
    last block, or the last 4 bytes of the last block (block being the
    chosen and agreed on fixed record size, not the DASD blocksize).

2.  Agree on a non-arbitrary record length for the type "V" representation
    originally described.

Both of the above ideas would require changes to things like Kermit and
FTP in order to support representing byte-stream data in that way.  This
would require convincing IBM in one case.

Further complications to porting ZIP to VM/CMS include many aspects of
the different character code in use (EBCDIC) and what to do with the
files being added or extracted.  One already sees the minor problem of
extracting a text file from a ZIP archive on UNIX to find that it has
these nasty carriage return thingys that stick "^M" on the end of all
the lines when using the "less" program.  It gets worse, MUCH worse,
when dealing with VM/CMS (and I am sure TSO would be worse that CMS).

I originally offered to port it, but because of the large amount of work
it would require, and that at the time someone else had offered to do so,
and that I didn't (and still don't) have a C compiler I feel I can work
with (I really was considering translating it all to 370 assembler, which
is a language I can program in as fast as, or faster than, C).  Since
there is no hint of a VM/CMS version of ZIP I suspect the other person
gave up as well.  I wouldn't blame them for it.

A version of ARC was ported to VM/CMS.  It required the archive be stored
in a "F 512" file and it produced "F 512" files.  Apparently it was ported
from the CP/M version.  I've used it (a program called "VMARC" is something
else).

------------------------------

From [e-mail] Fri Jan 29 01:01:08 1993
Subject: Re: bug? in unzip 5.0p1 for VMS: "+" i directories
Date: Fri, 29 Jan 1993 07:51:16 +0100 (MET)

[One small _detail_: Now it says, for example, "Creating plus+plus/,
Inflating [.plus_plus]file...". As you can see there's a cosmetic "bug" when
unzip displays the name, because after the patch, it is plus_plus thats created.
It's really no problem, but you might want to fix it before the next release?]

Terje Sten Bjerkseth <[e-mail]>

------------------------------

From: Jean-loup Gailly <[e-mail]>
Subject: non deterministic (g)unzip on Dec Ultrix
Date: Wed, 19 Jan 94 10:09:11 +0100

> We have several DECstations with Ultrix. On *one* of these DECstations
> unzip will not extract some files from long enaugh zipfiles. It
> reports bad CRC.  This effect is not always reproducable. Sometimes
> after first unsuccessfull attempt the second one will be OK.

The exact same thing is documented in the gzip INSTALL file:

- On Mips Dec Ultrix, gunzip behaves non-deterministically with regard
  to some .gz files. The same command either succeeds or gives a CRC error.
  This problem is still being investigated. The files produced by gzip
  are correct (can reliably be extracted on other systems).

We tried using -DNOMEMCPY, and this made the problem even worse. We
reached the conclusion that there is indeed a hardware bug, probably
in the memory cache. But this may also be a bug in the OS (bad
handling of page faults?) because another user of Ultrix with a
different OS version could not reproduce the problem.

------------------------------


------------------------------
------------------------------