Creating directories with pkunzip, rar, et. al.

Creating directories with pkunzip, rar, et. al.

[Joe Ripley]

Hello!

I'm not sure if this question has been asked before, but if it has,
please let me know where I may find the answer.

The problem I have is, when pkunzip (in my case, pkunzip 2.50) is ran
with the '-d' switch (to automatically create the original directory
structure of the archive) it fails.

pkunzip output
----------------------------------------------------------------------
PKUNZIP (R)    FAST!    Extract Utility    Version 2.50   03-01-1999
Copr. 1989-1999 PKWARE Inc.  All Rights Reserved.  Shareware Version
PKUNZIP Reg. U.S. Pat. and Tm. Off.

. 80386 CPU detected.
. EMS version 4.00 detected.
. XMS version 3.00 detected.
. DPMI version 0.90 detected.

Searching ZIP: ../__UPDATE.ZIP
 Extracting: BIN/
 Extracting: DATA/
 Extracting: LANGUAGE/
 Extracting: DOCS/
 Extracting: TEXT/
 Extracting: MENUS/
PKUNZIP: (W10) Warning! can't create: BIN/INDEX.EXE
PKUNZIP: (W10) Warning! can't create: BIN/USERS.EXE
PKUNZIP: (W10) Warning! can't create: BIN/FA2FB.EXE
PKUNZIP: (W10) Warning! can't create: BIN/FB2FA.EXE
PKUNZIP: (W10) Warning! can't create: BIN/CMPRLANG.EXE
PKUNZIP: (W10) Warning! can't create: BIN/MAKELANG.EXE
PKUNZIP: (W10) Warning! can't create: DATA/TELEGARD.ADF
...
----------------------------------------------------------------------

The reason that PKUNZIP 'can't create' the files, is because the
directories (BIN, DATA, etc.) don't have the execute bit set on them.
All of the directories are set like so:

----------------------------------------------------------------------
drw-r-xr-x 2 bbs users 48 2007-05-07 10:08 bin
drw-r-xr-x 2 bbs users 48 2007-05-07 10:08 data
drw-r-xr-x 2 bbs users 48 2007-05-07 10:08 docs
drw-r-xr-x 2 bbs users 48 2007-05-07 10:08 language
drw-r-xr-x 2 bbs users 48 2007-05-07 10:08 menus
drw-r-xr-x 2 bbs users 48 2007-05-07 10:08 text
----------------------------------------------------------------------

They all have a mode of 655, which denies the bbs user (and hence
pkunzip) from accessing them.

Anyway, I'm running DOSEMU 1.3.5.  I've tried both MS-DOS 6.22 and
FreeDOS 1.0 and the problem exists in both.  Is there some kind of
umask setting that will control this behavior, or is it an internal
DOSEMU problem?

Thanks!

-- 
Joe Ripley
vitaminjoe@gmail.com

Re: Creating directories with pkunzip, rar, et. al.

[Bart Oldeman]

On 5/7/07, Joe Ripley <vitaminjoe@gmail.com> wrote:
> The problem I have is, when pkunzip (in my case, pkunzip 2.50) is ran
> with the '-d' switch (to automatically create the original directory
> structure of the archive) it fails.
...
> The reason that PKUNZIP 'can't create' the files, is because the
> directories (BIN, DATA, etc.) don't have the execute bit set on them.
...
> Anyway, I'm running DOSEMU 1.3.5.  I've tried both MS-DOS 6.22 and
> FreeDOS 1.0 and the problem exists in both.  Is there some kind of
> umask setting that will control this behavior, or is it an internal
> DOSEMU problem?

It's an internal DOSEMU problem, and an old one at that!
The problem is that DOSEMU emulates the "archive" DOS attribute
through the user x bit. Very few DOS programs attempt to set or reset
the archive attribute on directories, but pkunzip 2.50 does.

The same problem happens with dosemu 1.2.2 (which crashes unless you
use the -3 option of pkunzip).

I'm going to fix it in DOSEMU by ignoring set attribute requests on
directories but there are other workarounds, e.g. by using INFO-ZIP
(for DOS or Linux) or older versions of pkunzip.

On a mounted FAT partition the problem should not happen either.

Bart

Re: Creating directories with pkunzip, rar, et. al.

[Joe Ripley]

On 5/7/07, Bart Oldeman <bartoldeman@users.sourceforge.net> wrote:
> It's an internal DOSEMU problem, and an old one at that!
> The problem is that DOSEMU emulates the "archive" DOS attribute
> through the user x bit. Very few DOS programs attempt to set or reset
> the archive attribute on directories, but pkunzip 2.50 does.
>
> The same problem happens with dosemu 1.2.2 (which crashes unless you
> use the -3 option of pkunzip).
>
> I'm going to fix it in DOSEMU by ignoring set attribute requests on
> directories but there are other workarounds, e.g. by using INFO-ZIP
> (for DOS or Linux) or older versions of pkunzip.
>
> On a mounted FAT partition the problem should not happen either.

I thought it had to be some obscure detail... :) Thanks for the info.
I'll take a look at some of the alternatives for now.

-- 
Joe Ripley
vitaminjoe@gmail.com

Re: Creating directories with pkunzip, rar, et. al.

[Joe Ripley]

On 5/7/07, Joe Ripley <vitaminjoe@gmail.com> wrote:
> On 5/7/07, Bart Oldeman <bartoldeman@users.sourceforge.net> wrote:
> > It's an internal DOSEMU problem, and an old one at that!
> > The problem is that DOSEMU emulates the "archive" DOS attribute
> > through the user x bit. Very few DOS programs attempt to set or reset
> > the archive attribute on directories, but pkunzip 2.50 does.

Actually, I had some time to do some testing, and I found that it's
not that pkunzip is attempting to set the archive attribute.  When run
with the debug messages turned on, pkunzip tries to create the
directory with an attribute of 0x10 (i.e. a regular file) rather than
0x20 (a directory).

As a hack to fix it, all I did was add one line to the beginning of
the set_dos_attr() function in mfs.c:

if (S_ISDIR(mode)) attr &= DIRECTORY;

pkunzip extracts with directories properly now:

----------------------------------------------------------------------
drwxr-xr-x 2 bbs users 240 2007-05-07 16:05 bin
drwxr-xr-x 2 bbs users 208 2007-05-07 16:05 data
drwxr-xr-x 2 bbs users 168 2007-05-07 16:06 docs
drwxr-xr-x 2 bbs users  80 2007-05-07 16:05 language
drwxr-xr-x 2 bbs users 584 2007-05-07 16:06 menus
drwxr-xr-x 2 bbs users 144 2007-05-07 16:06 text
----------------------------------------------------------------------

-- 
Joe Ripley
vitaminjoe@gmail.com

Re: Creating directories with pkunzip, rar, et. al.

[Bart Oldeman]

On 5/7/07, Joe Ripley <vitaminjoe@gmail.com> wrote:
> On 5/7/07, Joe Ripley <vitaminjoe@gmail.com> wrote:
> Actually, I had some time to do some testing, and I found that it's
> not that pkunzip is attempting to set the archive attribute.  When run
> with the debug messages turned on, pkunzip tries to create the
> directory with an attribute of 0x10 (i.e. a regular file) rather than
> 0x20 (a directory).

No, first of all creating a directory does not involve any attributes,
and secondly
0x10=directory, and 0x20=archive.

What you see in a -D+dD log with LFN's enabled is something like this:
INT21 (0) at 02a8:3478: AX=7139, BX=caf4, CX=0000, DX=caf4, DS=0c28, ES=0c28
LFN: doing LFN!, AX=7139 DL=f4
LFN: mkdir DOC
(...)
INT21 (0) at 02a8:344b: AX=7143, BX=ca01, CX=0020, DX=caf4, DS=0c28, ES=0c28
LFN: doing LFN!, AX=7143 DL=f4
LFN: attribute DOC 1

the CX=20 here is the new attribute (ie. archive)

> As a hack to fix it, all I did was add one line to the beginning of
> the set_dos_attr() function in mfs.c:
>
> if (S_ISDIR(mode)) attr &= DIRECTORY;

This is correct, since the read-only attribute must also be ignored.
However, there is also some FAT code that does not need the clearance.

So I committed a similar check but in a different place,  get_unix_attr():

--- src/dosext/mfs/mfs.c        (revision 1784)
+++ src/dosext/mfs/mfs.c        (revision 1785)
@@ -608,6 +608,12 @@
 #if 0
 #define S_IWRITEA (S_IWUSR | S_IWGRP | S_IWOTH)
 #endif
+  /* Do not make directories read-only as this has completely different
+     semantics in DOS (mostly ignore) than in Unix.
+     Also do not reflect the archive bit as clearing the x bit as that
+     can cause inaccessible directories */
+  if (S_ISDIR(mode) || (attr & DIRECTORY))
+    attr &= DIRECTORY;
   mode &= ~(S_IFDIR | S_IWRITE | S_IEXEC);
   if (attr & DIRECTORY)
     mode |= S_IFDIR;

Re: Creating directories with pkunzip, rar, et. al.

[Mike McCarty]

Bart Oldeman wrote:
> On 5/7/07, Joe Ripley <vitaminjoe@gmail.com> wrote:
> 
>> On 5/7/07, Joe Ripley <vitaminjoe@gmail.com> wrote:
>> Actually, I had some time to do some testing, and I found that it's
>> not that pkunzip is attempting to set the archive attribute.  When run
>> with the debug messages turned on, pkunzip tries to create the
>> directory with an attribute of 0x10 (i.e. a regular file) rather than
>> 0x20 (a directory).
> 
> 
> No, first of all creating a directory does not involve any attributes,
> and secondly
> 0x10=directory, and 0x20=archive.
> 

[and gives a source patch fix]

Ok, this explains a problem I found with the Dunfield installer
being unable to install under DOSEMU. In order to work around,
I booted a computer which actually does run MSDOS, and ran there,
then zipped up the resulting directories, put the ZIP file
on a floppy, and used unzip with Linux to do the transfer.

How soon will this be folded into the standard release?

Mike
-- 
p="p=%c%s%c;main(){printf(p,34,p,34);}";main(){printf(p,34,p,34);}
Oppose globalization and One World Governments like the UN.
This message made from 100% recycled bits.
You have found the bank of Larn.
I can explain it for you, but I can't understand it for you.
I speak only for myself, and I am unanimous in that!

Re: Creating directories with pkunzip, rar, et. al.

[Joe Ripley]

On 5/7/07, Bart Oldeman <bartoldeman@users.sourceforge.net> wrote:
> No, first of all creating a directory does not involve any attributes,
> and secondly
> 0x10=directory, and 0x20=archive.

Woops... I guess that shows how much of a newbie I am. :)

> > As a hack to fix it, all I did was add one line to the beginning of
> > the set_dos_attr() function in mfs.c:
> >
> > if (S_ISDIR(mode)) attr &= DIRECTORY;
>
> This is correct, since the read-only attribute must also be ignored.
> However, there is also some FAT code that does not need the clearance.
>
> So I committed a similar check but in a different place,  get_unix_attr():

Cool, looks good.  Thanks for looking into this!

-- 
Joe Ripley
vitaminjoe@gmail.com

Re: Creating directories with pkunzip, rar, et. al.

[Bart Oldeman]

On 5/7/07, Mike McCarty <Mike.McCarty@sbcglobal.net> wrote:
>
> How soon will this be folded into the standard release?

it is already in the Subversion repository... as to when the next
release is, I don't know.
But there have been quite a few little post 1.4.0 fixes so perhaps
1.4.1 doesn't take very long.
It all depends on time, I won't make any promises.

Bart

Re: Creating directories with pkunzip, rar, et. al.

[Mike McCarty]

Bart Oldeman wrote:
> On 5/7/07, Mike McCarty <Mike.McCarty@sbcglobal.net> wrote:
> 
>>
>> How soon will this be folded into the standard release?
> 
> 
> it is already in the Subversion repository... as to when the next
> release is, I don't know.

Fair enough. Dunno yet.

> But there have been quite a few little post 1.4.0 fixes so perhaps
> 1.4.1 doesn't take very long.
> It all depends on time, I won't make any promises.

I wasn't expecting any. I was just wondering when it might be.
If you don't know yet, then that's fine.

Mike
-- 
p="p=%c%s%c;main(){printf(p,34,p,34);}";main(){printf(p,34,p,34);}
Oppose globalization and One World Governments like the UN.
This message made from 100% recycled bits.
You have found the bank of Larn.
I can explain it for you, but I can't understand it for you.
I speak only for myself, and I am unanimous in that!

Re: Creating directories with pkunzip, rar, et. al.

[Alex Mauer]

Bart Oldeman wrote:

> 
> It's an internal DOSEMU problem, and an old one at that!
> The problem is that DOSEMU emulates the "archive" DOS attribute
> through the user x bit. Very few DOS programs attempt to set or reset
> the archive attribute on directories, but pkunzip 2.50 does.
> 

Would it make sense to use extended attributes for this where XA is 
available?  Or am I misunderstanding something vital?

-Alex Mauer "hawke"