Re: dosemu and CLIPPER locks (again) (fwd)

Re: dosemu and CLIPPER locks (again) (fwd)

[Bart Oldeman]

Anderson Pereira Ataides wrote:

> I tried to hack mfs.c an saw some comments about locking. My conclusion is
> that this problem will not be solved between windows and Linux, because they
> do not "speak" same language when topic is locking. I saw there is a piece of
> code extracted from samba source an comment was "I don't know why it's
> there... Someone is playing games with lock/unlock".

locking across the network is problematic indeed. Region locking (on
the same host) can be (partly?) solved using large file support but any
locking across the network will probably remain problematic. Even local
file locking cannot completely match DOS behaviour.

But what you can check is in the following scenario:
suppose the Linux box is a samba server and the Windows client issues a
lock. Now can a DOSEMU which runs on that samba server see that lock?

That depends a lot on the options you set in the samba config file.
however using the command lsof you can see if Linux sees the file as being
locked. If there is an "R", "W", "r" or "w" behind the file descriptor
number then it is indeed and DOSEMU should not be able to read or write
the file (at least, see some lock). If that isn't the case then there must
be a bug somewhere.

If "lsof" does not report one of these letters then there is nothing you
can do about it: the kernel won't see it so DOSEMU won't see it.
The same is true for smbfs and NFS.

If you see no r,R,w or W's then this isn't a DOSEMU problem. Please
report to the maintainers of samba, smbfs or nfs instead.

> So I thought: samba team
> probably have problem too. How do they solve this problem? Do they cope with
> locks keeping track of locks themselves instead of locking linux filesystem?

see the comment in mfs.c:

    /* this is a little heuristic and does not completely
       match to DOS behaviour. That would require tracking
       how existing fd's are opened and comparing st_dev
       and st_ino fields */

This is what Samba does.

DOSEMU would need to maintain a list somewhere which consists of st_dev +
st_ino fields + locking properties (these fields uniquely determine the
file). This should be done by some "DOSEMU file lock state server" that
can talk to all the DOSEMU clients on the same machine. Now as you can
guess that would be quite a bit of work and I certainly will not be
doing this any time soon (though I would happily accept patches of
course).

Bart

Re: dosemu and CLIPPER locks (again) (fwd)

[Anderson Pereira Ataides]

Em Qua 05 Fev 2003 04:40, Bart Oldeman escreveu:
> But what you can check is in the following scenario:
> suppose the Linux box is a samba server and the Windows client issues a
> lock. Now can a DOSEMU which runs on that samba server see that lock?
Yes if I mount a samba share instead of opening file locally. Ok it is not 
the right thing to do. It should be able to open file locally and see windows 
locks, but if I could not make it work by the right way, I think it is a 
solution for now.

Opening file using mounted samba share, Windows can see dosemu locks and 
dosemu can see Windows locks. The problem is that the same application in 
dosemu is working different. I am using DBU (do you know it) to open the same 
file in dosemu and Windows ant this is what happens:

1. Open in dosem first
When I open DBU in Windows it can see file is exclusive opened in dosemu and 
show only the name of dbf file, but do not show its fields and can not open 
it to see contents of file. That is expected to happen.

2. Open file in Windows first
When I open DBU in dosemu it gives an error message: "File corruption 
detected" and abort. Ok, that means dosem can see file is already opened but 
the problem is the way it is dealing with it. DBU should work the same way as 
in Windows.

> ...Now as you can
> guess that would be quite a bit of work and I certainly will not be
> doing this any time soon (though I would happily accept patches of
> course).
And I am not the one to make this patch :-(. I do not have knowledgement 
enough in C programming language for this. If you or our dosemu partners 
could only make DBU have the same behaviour in dosemu and Windows I will be 
happy. If it happens, that means I can finally send Windows to .......

Off topic: if my english is bad and is not making problems clear enough 
please tell me and I'll try to write better next time.

[]s
Anderson Pereira Ataides

[PATCH] (locking) (was Re: dosemu and CLIPPER locks (again))

[Bart Oldeman]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 755 bytes --]

Hi,

attached is a patch to (more cleanly than before) implement 64-bit file
locking in DOSEMU. This is as far as I can see, as far as DOSEMU can
reasonably go in the locking business, but I hope I didn't introduce any
new problems. Old problems that aren't solved by this patch will probably
never be solved (on the DOSEMU side at least), I'm afraid.

You need glibc 2.2+ and kernel 2.4, otherwise the code will revert to the
old locking behaviour.

Also, you may experiment with a new option, $_full_file_locks in your
dosemu.conf or .dosemurc, which affects the way who DOSEMU locks whole
files. Default is (off), and that is the old behaviour (lock 1 byte that
DOS can't see).

patch is vs. version 1.1.4.11 (see http://www.dosemu.org/testing).

Bart

[-- Attachment #2: Type: TEXT/PLAIN, Size: 8343 bytes --]

diff -urN -X dosemu-ignore-files dosemu-1.1.4.11/etc/dosemu.conf dosemu-1.1.4.12/etc/dosemu.conf
--- dosemu-1.1.4.11/etc/dosemu.conf	Tue Feb  4 21:39:59 2003
+++ dosemu-1.1.4.12/etc/dosemu.conf	Thu Feb  6 01:10:08 2003
@@ -48,6 +48,8 @@
 #			# optionally the device may be appended such as
 #			# "threeinch:/dev/fd0"
 #$_floppy_b = ""		# ditto for B:
+#$_full_file_locks = (off) # whether to lock the full file on lredired drives 
+#			# for file locking requests or just one byte
 #
 #
 #$_debug = "-a+cw"	# same format as -D commandline option, ""="-a+cw".
diff -urN -X dosemu-ignore-files dosemu-1.1.4.11/etc/global.conf dosemu-1.1.4.12/etc/global.conf
--- dosemu-1.1.4.11/etc/global.conf	Tue Feb  4 21:39:59 2003
+++ dosemu-1.1.4.12/etc/global.conf	Thu Feb  6 01:08:51 2003
@@ -356,6 +356,8 @@
   cli_timeout $_cli_timeout
   pic_watchdog $_pic_watchdog
 
+  full_file_locks $_full_file_locks
+
   ## serial
   if (strlen($_ttylocks))
     ttylocks { directory $_ttylocks namestub LCK.. }
diff -urN -X dosemu-ignore-files dosemu-1.1.4.11/src/base/init/config.c dosemu-1.1.4.12/src/base/init/config.c
--- dosemu-1.1.4.11/src/base/init/config.c	Tue Feb  4 21:39:59 2003
+++ dosemu-1.1.4.12/src/base/init/config.c	Thu Feb  6 01:12:33 2003
@@ -228,8 +228,8 @@
         config.timers);
     (*print)("tty_lockdir \"%s\"\ntty_lockfile \"%s\"\nconfig.tty_lockbinary %d\n",
         config.tty_lockdir, config.tty_lockfile, config.tty_lockbinary);
-    (*print)("num_ser %d\nnum_lpt %d\nfastfloppy %d\n",
-        config.num_ser, config.num_lpt, config.fastfloppy);
+    (*print)("num_ser %d\nnum_lpt %d\nfastfloppy %d\nfull_file_locks %d\n",
+        config.num_ser, config.num_lpt, config.fastfloppy, config.full_file_locks);
     (*print)("emusys \"%s\"\nemuini \"%s\"\n",
         (config.emusys ? config.emusys : ""), (config.emuini ? config.emuini : ""));
     (*print)("dosbanner %d\nvbios_post %d\ndetach %d\n",
diff -urN -X dosemu-ignore-files dosemu-1.1.4.11/src/base/init/lexer.l.in dosemu-1.1.4.12/src/base/init/lexer.l.in
--- dosemu-1.1.4.11/src/base/init/lexer.l.in	Sat Feb  1 16:04:47 2003
+++ dosemu-1.1.4.12/src/base/init/lexer.l.in	Thu Feb  6 01:11:44 2003
@@ -361,6 +361,7 @@
 printer			RETURN(PRINTER);
 emusys                  RETURN(EMUSYS);
 emuini                  RETURN(EMUINI);
+full_file_locks		RETURN(FULL_FILE_LOCKS);
 ttylocks		RETURN(TTYLOCKS);
 sound_emu               RETURN(L_SOUND);
 joystick_emu		RETURN(L_JOYSTICK);
diff -urN -X dosemu-ignore-files dosemu-1.1.4.11/src/base/init/parser.y.in dosemu-1.1.4.12/src/base/init/parser.y.in
--- dosemu-1.1.4.11/src/base/init/parser.y.in	Tue Feb  4 21:39:59 2003
+++ dosemu-1.1.4.12/src/base/init/parser.y.in	Thu Feb  6 01:14:20 2003
@@ -224,7 +224,7 @@
 %token NETDEV VNET
 %token DEBUG MOUSE SERIAL COM KEYBOARD TERMINAL VIDEO EMURETRACE TIMER
 %token MATHCO CPU CPUSPEED RDTSC BOOTA BOOTB BOOTC L_XMS L_DPMI PORTS DISK DOSMEM PRINTER
-%token L_EMS L_UMB EMS_SIZE EMS_FRAME TTYLOCKS L_SOUND L_JOYSTICK
+%token L_EMS L_UMB EMS_SIZE EMS_FRAME TTYLOCKS L_SOUND L_JOYSTICK FULL_FILE_LOCKS
 %token DEXE ALLOWDISK FORCEXDOS XDOSONLY
 %token ABORT WARN
 %token BOOTDISK L_FLOPPY EMUSYS EMUINI L_X
@@ -412,6 +412,10 @@
                     {
                     config.emuini = $3;
                     c_printf("CONF: config.emuini = '%s'\n", $3);
+		    }
+		| FULL_FILE_LOCKS bool
+		    {
+		    config.full_file_locks = ($2!=0);
 		    }
 		| FASTFLOPPY floppy_bool
 			{
diff -urN -X dosemu-ignore-files dosemu-1.1.4.11/src/dosext/mfs.c dosemu-1.1.4.12/src/dosext/mfs.c
--- dosemu-1.1.4.11/src/dosext/mfs/mfs.c	Sat Feb  1 16:04:48 2003
+++ dosemu-1.1.4.12/src/dosext/mfs/mfs.c	Thu Feb  6 01:38:56 2003
@@ -153,6 +153,7 @@
 
 /* Modified by O.V.Zhirov at July 1998    */
 
+#include "config.h"
 
 #if defined(__linux__)
 #define DOSEMU 1		/* this is a port to dosemu */
@@ -197,7 +198,6 @@
 #include <signal.h>
 #include <string.h>
 #include "mfs.h"
-#include "config.h"
 /* For passing through GetRedirection Status */
 #include "memory.h"
 #include "redirect.h"
@@ -2570,6 +2570,38 @@
   return (TRUE);
 }
 
+static int lock_file_region(int fd, int cmd, struct flock *fl, long long start, unsigned long len)
+{
+  fl->l_whence = SEEK_SET;
+  fl->l_pid = 0;
+  /* first handle magic file lock value */
+  if (start == 0x100000000LL && config.full_file_locks) {
+    start = len = 0;
+  }
+#ifdef F_GETLK64
+  if (cmd == F_SETLK64 || cmd == F_GETLK64) {
+    struct flock64 fl64;
+    int result;
+    Debug0((dbg_fd, "Large file locking start=%lld, len=%lu\n", start, len));
+    fl64.l_type = fl->l_type;
+    fl64.l_whence = fl->l_whence;
+    fl64.l_pid = fl->l_pid;
+    fl64.l_start = start;
+    fl64.l_len = len;
+    result = fcntl( fd, cmd, &fl64 );
+    fl->l_type = fl64.l_type;
+    fl->l_start = (long) fl64.l_start;
+    fl->l_len = (long) fl64.l_len;
+    return result;
+  }
+#endif
+  if (start == 0x10000000LL)
+    start = 0x7fffffff;
+  fl->l_start = start;
+  fl->l_len = len;
+  return fcntl( fd, cmd, fl );
+}
+
 static boolean_t
 share(int fd, int mode, sft_t sft)
 {
@@ -2595,15 +2627,17 @@
    * and 0x7fffffff is my start position.  --Maxim Ruchko
    */
   struct flock fl;
+  int ret;
   int share_mode = ( sft_open_mode( sft ) >> 4 ) & 0x7;
-  fl.l_whence = SEEK_SET;
-  fl.l_start  = 0x7fffffff;
-  fl.l_len = 1;
-  fl.l_pid = 0;
   fl.l_type = F_WRLCK;
   /* see whatever locks are possible */
-  
-  if ( fcntl( fd, F_GETLK, &fl ) ) {
+
+#ifdef F_GETLK64
+  ret = lock_file_region( fd, F_GETLK64, &fl, 0x100000000LL, 1 );
+  if ( ret == -1 && errno == EINVAL )
+#endif
+    ret = lock_file_region( fd, F_GETLK, &fl, 0x100000000LL, 1 );
+  if ( ret == -1 ) {
   /* work around Linux's NFS locking problems (June 1999) -- sw */
     static unsigned char u[26] = { 0, };
     if(current_drive < 26) {
@@ -2713,12 +2747,11 @@
     return FALSE;
     break;
   }
-  fl.l_whence = SEEK_SET;
-  fl.l_start  = 0x7fffffff;
-  fl.l_len = 1;
-  fl.l_pid = 0;
-  fcntl( fd, F_SETLK, &fl );
-
+#ifdef F_SETLK64
+  ret = lock_file_region( fd, F_SETLK64, &fl, 0x100000000LL, 1 );
+  if ( ret == -1 && errno == EINVAL )
+#endif
+    lock_file_region( fd, F_SETLK, &fl, 0x100000000LL, 1 );
   Debug0((dbg_fd,
     "internal SHARE: locking: drive %c:, fd %d, type %d whence %d pid %d\n",
      current_drive + 'A', fd, fl.l_type, fl.l_whence, fl.l_pid
@@ -3852,7 +3885,7 @@
 		int fd = open_files[sft_fd(sft)].fd;
 		int ret;
 		struct LOCKREC{
-			long offset,size;
+			unsigned long offset,size;
 		} *pt = (struct LOCKREC*) Addr (state,ds,edx);
 		struct flock larg;
 		unsigned long mask = 0xC0000000;
@@ -3862,6 +3895,14 @@
                   return (FALSE);
                 }
 
+		Debug0((dbg_fd, "lock requested, fd=%d, is_lock=%d, start=%lx, len=%lx\n",
+			fd, is_lock, larg.l_start,larg.l_len));
+
+		if (pt->offset + pt->size < pt->offset) {
+			SETWORD(&(state->eax), ACCESS_DENIED);
+			return FALSE;
+		}
+
 #if 1   /* The kernel can't place F_WRLCK on files opened read-only and
          * FoxPro fails. IMHO the right solution is:         --Maxim Ruchko */
 		larg.l_type = is_lock ? (
@@ -3874,10 +3915,8 @@
 #else
 		larg.l_type = is_lock ? F_WRLCK : F_UNLCK;
 #endif
-		larg.l_whence = SEEK_SET;
 		larg.l_start = pt->offset;
 		larg.l_len = pt->size;
-		larg.l_pid = 0;
 
 		/*
 			This is a superdooper patch extract from the Samba project
@@ -3902,7 +3941,11 @@
 		if ((larg.l_start & mask) != 0)
 			larg.l_start = (larg.l_start & ~mask) | ((larg.l_start & mask) >> 2);
 
-		ret = fcntl (fd,F_SETLK,&larg);
+#ifdef F_SETLK64
+		ret = lock_file_region (fd,F_SETLK64,&larg,pt->offset,pt->size);
+		if (ret == -1 && errno == EINVAL)
+#endif
+			ret = lock_file_region (fd,F_SETLK,&larg,larg.l_start,larg.l_len);
 		Debug0((dbg_fd, "lock fd=%x rc=%x type=%x whence=%x start=%lx, len=%lx\n",
 			fd, ret, larg.l_type, larg.l_whence, larg.l_start,larg.l_len));
 		if (ret != -1) return TRUE; /* no error */
--- dosemu-1.1.4.11/src/include/emu.h	Tue Feb  4 21:39:59 2003
+++ dosemu-1.1.4.12/src/include/emu.h	Thu Feb  6 01:43:39 2003
@@ -316,6 +316,7 @@
        unsigned char *pre_stroke_mem;
 
        /* Lock File business */
+       boolean full_file_locks;
        char *tty_lockdir;	/* The Lock directory  */
        char *tty_lockfile;	/* Lock file pretext ie LCK.. */
        boolean tty_lockbinary;	/* Binary lock files ? */

Re: [PATCH] (locking) (was Re: dosemu and CLIPPER locks (again))

[Michal Samek]

Hi,
I can see the same old bad locking behaviour with this patch applied.
I've tested it only on local ext3 lredired drive and 2 dosemu sessions
on the same host (it's the easiest environment - no messing with
samba/windows) but even here it just don't sees the other session's
locks. No help when switched on $_full_file_locks.

Did someone succeed with it? Nothing more we can do about it? 

-- 
Michal Samek <webmaster@tony.cz>

RE: [PATCH] (locking) (was Re: dosemu and CLIPPER locks (again))

[Gregor Ibic]

Hi! I tried 1.1.4.11 with the patch, but the situation is the same. Windows
client issue a page lock (I think) and the DOSEMU klient is locked out. It
cannot read the file. The other win client can read the file without the
problem.
Also I tried to read the same file (database) with notepad on local drive,
it opens OK, but the same file from share I cant open it.
It is the same if Im using samba or W2k. Linux connected to W2k share or W2k
connected to Samba share.
Where is the problem?

Regards,
Gregor

RE: [PATCH] (locking) (was Re: dosemu and CLIPPER locks (again))

[Bart Oldeman]

On Mon, 10 Feb 2003, Gregor Ibic wrote:

> Hi! I tried 1.1.4.11 with the patch, but the situation is the same. Windows
> client issue a page lock (I think) and the DOSEMU klient is locked out. It
> cannot read the file. The other win client can read the file without the
> problem.
> Also I tried to read the same file (database) with notepad on local drive,
> it opens OK, but the same file from share I cant open it.

this is notepad on the Windows client I presume?

> It is the same if Im using samba or W2k. Linux connected to W2k share or W2k
> connected to Samba share.
> Where is the problem?

Is DOSEMU using locks locally on the Samba share? Or does it use NFS?
Or smbfs? What does "lsof filename" say on the computer you run DOSEMU on
for the filename that the lock is issued for?

Bart

RE: [PATCH] (locking) (was Re: dosemu and CLIPPER locks (again))

[Gregor Ibic]

The environment:
Samba 2.2.7a
  shared some folder and then mounted with smbfs rw, fmask=777

DOSEMU 1.1.4.11+64bit patch
dos 5.0 booted from bootdir (I tried other doses but it is the same)
lredired the mounted folder

other client is on win platform (w2k or win98)
mounted the same share with net use ...

If I start the app from win client first, the dosemu client cant open the
file
if I start the dosemu client the win client can open the file

[root@flip root]# lsof /mnt/aranea/aralj/FIN--/SKKUP-01.101
COMMAND     PID USER   FD   TYPE DEVICE   SIZE NODE NAME
dosemu.bi 17951 root   10u   REG   0,11 946176 1452
/mnt/aranea/aralj/FIN--/SKKUP-01.101
[root@flip root]#


smbstatus
....

Locked files:
Pid    DenyMode   Access      R/W        Oplock           Name
--------------------------------------------------------------
18038  DENY_NONE  0x2019f     RDWR       NONE
/home/space/araswlj/aralj/FIN--/SKKUP-01.101   Mon Feb 10 19:38:47 2003
17908  DENY_NONE  0x3         RDWR       NONE
/home/space/araswlj/aralj/FIN--/SKKUP-01.101   Mon Feb 10 19:31:02 2003

[root@flip root]#

first line is from win client and the second line is from dosemu client


Regards,
Gregor

RE: [PATCH] (locking) (was Re: dosemu and CLIPPER locks (again))

[Bart Oldeman]

On Mon, 10 Feb 2003, Gregor Ibic wrote:

> The environment:
> Samba 2.2.7a
>   shared some folder and then mounted with smbfs rw, fmask=777
>
> DOSEMU 1.1.4.11+64bit patch
> dos 5.0 booted from bootdir (I tried other doses but it is the same)
> lredired the mounted folder
>
> other client is on win platform (w2k or win98)
> mounted the same share with net use ...
>
> If I start the app from win client first, the dosemu client cant open the
> file

If I understood it correctly then this is wrong?

> if I start the dosemu client the win client can open the file

If I understood it correctly then this is OK?

> [root@flip root]# lsof /mnt/aranea/aralj/FIN--/SKKUP-01.101
> COMMAND     PID USER   FD   TYPE DEVICE   SIZE NODE NAME
> dosemu.bi 17951 root   10u   REG   0,11 946176 1452
> /mnt/aranea/aralj/FIN--/SKKUP-01.101
> [root@flip root]#
>
>
> smbstatus
> ....
>
> Locked files:
> Pid    DenyMode   Access      R/W        Oplock           Name
> --------------------------------------------------------------
> 18038  DENY_NONE  0x2019f     RDWR       NONE
> /home/space/araswlj/aralj/FIN--/SKKUP-01.101   Mon Feb 10 19:38:47 2003
> 17908  DENY_NONE  0x3         RDWR       NONE
> /home/space/araswlj/aralj/FIN--/SKKUP-01.101   Mon Feb 10 19:31:02 2003
>
> [root@flip root]#
>
> first line is from win client and the second line is from dosemu client

But what does lsof say (on the computer you would run DOSEMU) about the
file when the win client locks it, before you start the DOSEMU client?

Bart

RE: [PATCH] (locking) (was Re: dosemu and CLIPPER locks (again))

[Gregor Ibic]

Wrong is that DOSEMU client cant read the file. Win client lock it properly
but dosemu probably cant read properly the lock information and pass it to
the client. The app is written in C (normal file locking).

lsof for the file does not say anything when win client opens it
only when dosemu client opens it (like I sent it before)

Regards,
Gregor

RE: [PATCH] (locking) (was Re: dosemu and CLIPPER locks (again))

[Bart Oldeman]

On Mon, 10 Feb 2003, Gregor Ibic wrote:

> lsof for the file does not say anything when win client opens it
> only when dosemu client opens it (like I sent it before)

What happens when you try to open the file in Linux (on the same
client that DOSEMU runs on) using an editor, say "vi file"?

Bart

RE: [PATCH] (locking) (was Re: dosemu and CLIPPER locks (again))

[Gregor Ibic]

I cant, it is locked. It says resource deadlock avoided.

But I also cant open it on the win client from the editor (notepad) but from
the application yes.

Regards,
Gregor

RE: [PATCH] (locking) (was Re: dosemu and CLIPPER locks (again))

[Bart Oldeman]

On Mon, 10 Feb 2003, Gregor Ibic wrote:

> I cant, it is locked. It says resource deadlock avoided.

In that case you'd probably need to bring this issue up with the smbfs
maintainers.

DOSEMU uses normal linux commands (fcntl) to find out about file locking.
So if smbfs via the Linux virtual filesystem tells DOSEMU that the file is
locked then there is nothing that DOSEMU can do about it.

Running DOSEMU on the samba server may or not solve this issue though
(using the right samba settings). Also in this case the samba people are
much more knowledgeable than I am.

I'd guess that the best way is to just run all DOSEMU sessions on the same
machine, using the local filesystem, where the Win clients can use telnet,
putty, or Cygwin XFree86. Even the local filesystem isn't always foolproof
(as Michal Samek said) but in that case we are at least sure that it's
DOSEMU's or the Linux kernel's fault and not some network filesystem's
fault.

Bart

RE: [PATCH] (locking) (was Re: dosemu and CLIPPER locks (again))

[Gregor Ibic]

But the file is locked also for win client if I try to open it in a notepad,
and the app under win works ok.

Regards,
Gregor

RE: [PATCH] (locking) (was Re: dosemu and CLIPPER locks (again))

[Bart Oldeman]

On Mon, 10 Feb 2003, Gregor Ibic wrote:

> But the file is locked also for win client if I try to open it in a notepad,
> and the app under win works ok.

But Windows locking and Linux locking techniques are incompatible so 
translating them through smbfs transforms information.

So I don't know what locking games the Win client plays, but they may or 
may not be possible for Linux clients. And I just cannot answer that 
question -- maybe the samba people can.

Bart

Re: [PATCH] (locking) (was Re: dosemu and CLIPPER locks (again))

[Sergey Suleymanov]

>>>>> Bart Oldeman writes:

 Bart> But Windows locking and Linux locking techniques are
 Bart> incompatible so translating them through smbfs transforms
 Bart> information.

        Do you know how Samba server's _posix_ locking consist with
        Dosemu's one? I mean, is samba's mapping SMB locks to posix
        locks compatible enough to dosemu's emulation?

-- 
  Sergey Suleymanov

Re: [PATCH] (locking) (was Re: dosemu and CLIPPER locks (again))

[Bart Oldeman]

On 11 Feb 2003, Sergey Suleymanov wrote:

> >>>>> Bart Oldeman writes:
>
>  Bart> But Windows locking and Linux locking techniques are
>  Bart> incompatible so translating them through smbfs transforms
>  Bart> information.
>
>         Do you know how Samba server's _posix_ locking consist with
>         Dosemu's one? I mean, is samba's mapping SMB locks to posix
>         locks compatible enough to dosemu's emulation?

it used to be a similar mapping for byte range locks (code lifted from
samba). locking was reworked for samba 2.2 however.

There are some files you can browse here:
http://samba.org/cgi-bin/cvsweb/samba/source/locking/
and also
http://samba.org/cgi-bin/cvsweb/samba/source/smbd/open.c

and there's a very complicated function there now which requires ASCII art
to understand. Hmm.

as for file locking, Samba appears to maintain a database of what mode the
clients use -- that's possible since it's a server; multiple DOSEMU
instances don't share this information.

Also Samba always tries to open files r/w on the server whereas DOSEMU
MFS opens them r/o if DOS opens a file r/o. This works around the problem
that exclusive POSIX locks cannot be applied to r/o-opened files. This
technique could be used by DOSEMU too (DOS apps still think their open
is r/o and MFS can stop writes) but I'm not sure if it's a good idea...

Bart

WARNING: smbfs and locking (was RE: [PATCH] (locking) (was Re: dosemu and CLIPPER locks (again))))

[Bart Oldeman]

On Mon, 10 Feb 2003, Gregor Ibic wrote:

> I cant, it is locked. It says resource deadlock avoided.

ok I browsed around a bit and looked at the kernel source, here's the real
answer.

Current (linux kernel 2.4.20) smbfs doesn't support locking at all.

you may look at
http://www.hojdpunkten.ac.se/054/samba
to see some experimental patches though.
NFS *does* support locking though.

So if you see locking "working" with smbfs it is actually by accident and
not by design.

This means that it is very well possible to corrupt your DOS databases
if you access them via smbfs, and there's nothing that DOSEMU can do about
it.

Now, why does a Win client access the file when DOSEMU "locks" it? This is
because there actually is no lock at all!

And the other way around, what happens is that: DOSEMU tries to apply a
lock to the file, the filesystem fails, and the DOS app hence claims that
access is denied.

Bart

Re: WARNING: smbfs and locking -- POSIX v. CIFS locking, and how DOS doesn't do CIFS locking (bad things man)

[Bryan J. Smith]

Quoting Bart Oldeman <oldeman@math.ohio-state.edu>:
> ok I browsed around a bit and looked at the kernel source, here's the
> real answer.
> Current (linux kernel 2.4.20) smbfs doesn't support locking at all.
> you may look at
> http://www.hojdpunkten.ac.se/054/samba
> to see some experimental patches though.

As Bart pointed out, CIFS locking is _not_ supported by Linux CIFS/SMB clients.
 CIFS/SMB locking is _foreign_ to POSIX systems, so Linux will never really
support them well at all.  And _no_ other UNIX flavor (that I know of) supports
mounting CIFS/SMB shares anyway.

Understand POSIX and CIFS are completely _different_ when it comes to file
locking.  POSIX is designed for UNIX clients, CIFS is designed for Windows
clients.  DOS clients are a different breed, but will be far more like UNIX
clients than CIFS.

CIFS supports byte range locks as well as stackable locks.  POSIX systems do
not.  This makes for some interesting issues with Samba [server] emulating CIFS
atop of a POSIX filesystem (although the new Samba VFS implementation is
supposed to addresses), and even CIFS itself.  E.g., the concept of byte range
locks and stackable locks causes regular corruption on native CIFS servers (i.e.
NT/2K) themselves.  Hence why many CIFS clients use application-specific
client-side locking (e.g., Jet for MS Access).

Now let's look at DOS applications.  I don't know of _any_ DOS applications that
support CIFS file locking.  In fact, running DOS applications on multiple
systems accessing the same data over a CIFS SMB mount is almost _guaranteed_ to
cause corruptions.  Why?  Because the CIFS client system (Windows) assumes that
the application, DOS, knows how to handle its locking mechanisms when it, in
fact, doesn't.  So instead of getting nice, old "real" DOS style locks, which
only allow one application to access a file at a time, we get CIFS
"opportunistic locks" that allow multiple applications write access to the same
file.  Boom!

Oplocks are hard coded into native CIFS servers (NT/2k), although a registry
hack can disable them for the _entire_ server.  Samba allows many different
Oplock types and they can be specified on a per-share basis too.  But I'm
talking with regards to "real" CIFS/SMB clients (Windows), not "emulated" ones
(Linux SMBfs).

> NFS *does* support locking though.

Yes, and it does it in a way that is far more "DOS-like" -- one write lock per file.

> So if you see locking "working" with smbfs it is actually by accident
> and not by design.

Exactomundo.  Linux SMBfs filesystem support is a _hack_, not something you
should rely on.  Don't mount NT/2K servers on Linux clients.

_Always_ use the network filesystem that is "native" to the client OS.  For
UNIX/Linux, this is NFS.  For Windows, this is CIFS/SMB.  Unlike Samba on UNIX
for CIFS/SMB, Windows servers don't have equivalent NFS functionality (and NFS
servers available for Windows suck in my experience, although Microsoft SFU 3.0
is supposed to be better now?).  So only UNIX/Linux servers make the best
"cross-platform" file servers.

> This means that it is very well possible to corrupt your DOS databases
> if you access them via smbfs, and there's nothing that DOSEMU can do
> about it.

Exactomundo!  And even using native Windows servers and clients would do the same!

Thanx to Microsoft's "Oplocks by default".  It's not so much of a problem with
Linux or Samba, but the CIFS locking mechanism itself.  In fact, this is where
Samba shines, because it lets you disable those "multiple write locks,"
something DOS doesn't support (nor many Windows applications for that matter --
with Jet/Access being a rare exception), on a per-share basis.  But that's only
when we're talking Windows clients.

If you're using Linux clients, I have two suggestions:
 1.  Use NFS mounts from a Linux server
 2.  Even better:  Run multiple X-DOSEmu instances on one box,
     and "pump" the display to other X-Window systems

#2 is the best because now you are locking _local_.  It's an option _only_
available with DOSEmu on Linux.

> Now, why does a Win client access the file when DOSEMU "locks" it?
> This is because there actually is no lock at all!
> And the other way around, what happens is that: DOSEMU tries to apply
> a lock to the file, the filesystem fails, and the DOS app hence claims
> that access is denied.


-- 
Bryan J. Smith             |  Peace is a fruitless endeavor
http://thebs.org           |  When a defeated aggressor
Engineer, IT Professional  |  Has not, will not nor will ever
Proud American Forever     |  Adhere to terms of its surrender

Re: WARNING: smbfs and locking -- POSIX v. CIFS locking, and how DOS doesn't do CIFS locking (bad things man)

[Bart Oldeman]

On Mon, 10 Feb 2003, Bryan J. Smith wrote:

> > NFS *does* support locking though.
>
> Yes, and it does it in a way that is far more "DOS-like" -- one write lock per file.

there are still some differences between POSIX and DOS locks though:
DOS has 6 kinds of locks:

* COMPATIBILITY
* DENY_NONE
* DENY_READ
* DENY_WRITE
* DENY_ALL
* exclusive region locking

where POSIX only has two:
* shared region locking (read locks)
* exclusive region locking (write locks)

DOSEMU emulates DENY_WRITE using a shared lock and DENY_ALL/DENY_READ
using an exclusive lock if the file is opened r/w or w/o or a shared lock
if the file is opened r/o. The other modes don't cause a lock; the code
only checks if locks already exist.

File locking is emulated by either locking the whole file region or one
byte at 0x7fffffff (without large file system support) or at 0x100000000
(with large file system support).

Also, without LFS the DOS unsigned 32bit locks need to be mapped into
positive 31bit signed numbers, which is what the magic borrowed from the
Samba project does. But it's much better not having to do that if we can
and that's why I added the LFS support (63 bits to be able to access 32!).

Bart

Re: WARNING: smbfs and locking -- Mega-oversimplification ...

[Bryan J. Smith]

[-- Attachment #1: Type: text/plain, Size: 430 bytes --]

On Mon, 2003-02-10 at 19:31, Bart Oldeman wrote:
> there are still some differences between POSIX and DOS locks though:

Yeah, I mega-oversimplified in this post.  Sorry.

-- 
Bryan J. Smith             |  Peace is a fruitless endeavor
http://thebs.org           |  When a defeated aggressor
Engineer, IT Professional  |  Has not, will not nor will ever
Proud American Forever     |  Adhere to terms of its surrender
 

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: WARNING: smbfs and locking

[Anderson Pereira Ataides]

> NFS *does* support locking though.
I tried to mount a nfs resource and two linux boxes can't see locks made by 
each other. That means a file locked in a linux box is not seen by the other 
Linux box. Am I missing something?

Reading dosemu docs, it's written that I must mount nfs nolock. What does 
that mean? If nfs "does" support locking, why must I disable this? And in 
fact I really have to export it nolock because if I don't dosemu does not 
work.

In my previous messages I got almost there, because I locked a file in dosemu 
and Windows "could see" this lock. And the other side worked too, but message 
application (DBU in my case) gave was different. So that's why I wrote: "if 
you could make DBU work the same way, I'll be happy".

But now you got me worried. When I thought I was almost there you come and 
say that smbfs have locking problem and probably acessing files through smbfs 
will result in data corruption. Again: am I missing something?

Let me tell you again what I did:
1. Made a smb share
2. Mapped a drive in Windows
3. Mounted smb in Linux box
4. Used DBU to test locking

Was it an accident? I think no, because both Linux and Windows is accessing 
file through samba and so samba itself is locking file. Am I right? Will this 
result in data corruption?

> Now, why does a Win client access the file when DOSEMU "locks" it? This is
> because there actually is no lock at all!
That makes me really believe that samba keeps track of locking itself. Linux 
filesystem does not know what samba is doing.

This week I'll be upgrading my system. I'll install a new version of my 
distro and start testing dosemu again (nfs and smb). I have to do it because 
I am using a 2.2 kernel and for testing purpouses I compiled a 2.4 and boot 
it every time I receive a new dosemu patch.

[]s
Anderson Pereira Ataides

WARNING: smbfs and locking

[Reinhard Karcher]

On Tue, Feb 11, 2003 at 10:07:27AM -0200, Anderson Pereira Ataides wrote
> > NFS *does* support locking though.
> I tried to mount a nfs resource and two linux boxes can't see locks made by 
> each other. That means a file locked in a linux box is not seen by the other 
> Linux box. Am I missing something?
> 
> Reading dosemu docs, it's written that I must mount nfs nolock. What does 
> that mean? If nfs "does" support locking, why must I disable this? And in 
> fact I really have to export it nolock because if I don't dosemu does not 
> work.
> 
As far as I know, this applies to userland nfsserver. The kernelspace
nfsserver is OK with locking. I consider this part of the documention
as out of date.

Reinhard