Disable SCSI-Reservation at the driver level ?

Disable SCSI-Reservation at the driver level ?

[roland]

Hello !

i have a "problem" with VMWare ESX (not big one, but interesting one - at 
least for me).

If you don`t know VMware ESX - ESX is sort of it`s own operating system with 
Linux as a management console OS on top of a hypervisor (vmkernel), just 
similar to XEN hypervisor with and attached dom0. Besides the drivers in the 
console OS, the vmkernel loads it´s own drivers, which are derived from 
Linux drivers.

VMWare ESX refuses to create VMFS Filesystem on SATA disk, attached to a 
onBoard SAS controller (lsi1068).
When i raid1 two SATA disks, it works, if i use a single SATA disk, the 
controller seems to "expose" the disk differently to the operating system 
and creation of a VMFS fails due to missing ability to issue SCSI 
reservation command.

Ok there is no official SATA support for now, but some controllers work with 
SATA,  because you can either attach SAS and SATA disks to that (LSI1068). 
Furthermore there are SATA-controllers which can be used with the driver of 
some SCSI-model (LSI Megaraid 150-4)

Afaik, SCSI reservations being used/needed in shared storage environment 
(SAN, Shared local attached storage), so i wonder, why it's also being used 
on local attached disk. Only vmware may give an answer here.

Maybe i can can "workaround" this somehow ?
I couldn`t find a way to stop the vmkfstools (which is used to format 
blockdevice with VMFS) issueing a SCSI reservation command, but i assume 
this can probably be done via filtering some syscall (ioctl?) via LD_PRELOAD 
library - i.e. wrapping vmkfstools - or maybe i can hack this at the driver 
level, since source of the ESX storage drivers is available for download and 
can be recompiled. :)

if somebody like to help hacking this -  there is a thread at:
http://www.vmware.com/community/thread.jspa?messageID=518182&#518182
thread begins with some related iSCSI issues and discusses SATA issues later 
on.

regards
Roland

ps:
yes, i know this is somhow "evil" and i shouldn`t do that, because something 
could badly fail -  but it`s for private purpose, for proof of concept and 
for personal satisfaction. :) i´m not going to use such hack on any 
production system. 

-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Disable SCSI-Reservation at the driver level ?

[James Bottomley]

On Sun, 2006-11-26 at 17:31 +0100, roland wrote:
> VMWare ESX refuses to create VMFS Filesystem on SATA disk, attached to a 
> onBoard SAS controller (lsi1068).
> When i raid1 two SATA disks, it works, if i use a single SATA disk, the 
> controller seems to "expose" the disk differently to the operating system 
> and creation of a VMFS fails due to missing ability to issue SCSI 
> reservation command.

There's no SCSI fix for this ... the SAT has no translation for the SCSI
reservation commands, largely because there's no corresponding ATA
equivalent and even for SCSI devices they may fail anyway.  The
application should cope with such a failure, so in this case it's the
application that needs fixing.


> Maybe i can can "workaround" this somehow ?
> I couldn`t find a way to stop the vmkfstools (which is used to format 
> blockdevice with VMFS) issueing a SCSI reservation command, but i assume 
> this can probably be done via filtering some syscall (ioctl?) via LD_PRELOAD 
> library - i.e. wrapping vmkfstools - or maybe i can hack this at the driver 
> level, since source of the ESX storage drivers is available for download and 
> can be recompiled. :)

It's most likely done via a SG_IO ioctl() but you should trace the
application to be sure.

James

Re: Disable SCSI-Reservation at the driver level ?

[roland]

> There's no SCSI fix for this ... the SAT has no translation for the SCSI
> reservation commands, largely because there's no corresponding ATA
> equivalent and even for SCSI devices they may fail anyway.  The
> application should cope with such a failure, so in this case it's the
> application that needs fixing.

yes, sure i cannot translate a SCSI reservation command into SATA equivalent 
because SATA doesn`t support.
but i`m unsure, what sense does SCSI reservation does make with local 
storage, anyway - because there is no concurrent access in this scenario.
ESX is very SAN oriented, and in nearly all cases SCSI is being used as 
storage, so SCSI reservation will ususally "just work", regardless if being 
needed or not.
I assume it`s not really needed with local storage, but i`m not sure here.
So if the application issues SCSI reservation and i cannot fix the 
application due to mising source, the only way to fix this is to "fake" a 
successful SCSI reservation, maybe via syscall modification (LD_PRELOAD) or 
via driver reporting success on every SCSI reservation command.

> It's most likely done via a SG_IO ioctl() but you should trace the
> application to be sure.

thanks, this is a good hint for what i should search for. i will see if i 
can find a suitable strace package for ESX (ESX doesn`t come with strace) 
and investigate

regards
roland


----- Original Message ----- 
From: "James Bottomley" <James.Bottomley@SteelEye.com>
To: "roland" <devzero@web.de>
Cc: <linux-scsi@vger.kernel.org>
Sent: Sunday, November 26, 2006 6:46 PM
Subject: Re: Disable SCSI-Reservation at the driver level ?


> On Sun, 2006-11-26 at 17:31 +0100, roland wrote:
>> VMWare ESX refuses to create VMFS Filesystem on SATA disk, attached to a
>> onBoard SAS controller (lsi1068).
>> When i raid1 two SATA disks, it works, if i use a single SATA disk, the
>> controller seems to "expose" the disk differently to the operating system
>> and creation of a VMFS fails due to missing ability to issue SCSI
>> reservation command.
>
> There's no SCSI fix for this ... the SAT has no translation for the SCSI
> reservation commands, largely because there's no corresponding ATA
> equivalent and even for SCSI devices they may fail anyway.  The
> application should cope with such a failure, so in this case it's the
> application that needs fixing.
>
>
>> Maybe i can can "workaround" this somehow ?
>> I couldn`t find a way to stop the vmkfstools (which is used to format
>> blockdevice with VMFS) issueing a SCSI reservation command, but i assume
>> this can probably be done via filtering some syscall (ioctl?) via 
>> LD_PRELOAD
>> library - i.e. wrapping vmkfstools - or maybe i can hack this at the 
>> driver
>> level, since source of the ESX storage drivers is available for download 
>> and
>> can be recompiled. :)
>
> It's most likely done via a SG_IO ioctl() but you should trace the
> application to be sure.
>
> James
>
> 

Re: Disable SCSI-Reservation at the driver level ?

[Douglas Gilbert]

James Bottomley wrote:
> On Sun, 2006-11-26 at 17:31 +0100, roland wrote:
>> VMWare ESX refuses to create VMFS Filesystem on SATA disk, attached to a 
>> onBoard SAS controller (lsi1068).
>> When i raid1 two SATA disks, it works, if i use a single SATA disk, the 
>> controller seems to "expose" the disk differently to the operating system 
>> and creation of a VMFS fails due to missing ability to issue SCSI 
>> reservation command.
> 
> There's no SCSI fix for this ... the SAT has no translation for the SCSI
> reservation commands, largely because there's no corresponding ATA
> equivalent and even for SCSI devices they may fail anyway.  The
> application should cope with such a failure, so in this case it's the
> application that needs fixing.

SAT originally did have persistent reservations and it
was dropped and is back on the agenda for SAT-2. A SAT
layer (such as the one found in libata) can do more
that just translate command, it may also emulate SCSI
commands.

And PERSISTENT RESERVE IN and OUT (and maybe the older
RESERVE and RELEASE) would be very good candidates for
emulation. To do this however libata would need to be
a lot more transport aware than it is now. To do such
an emulation a SAT layer needs to know:
  a) whether it has full control over the SATA device
     (i.e. there is no other path to it) and failing
     that, it has some other mechanism such as
     affiliations in SAS with SMP available to control
     them
  b) the identity of the initiator (port) asking for
     the reservation.

If libata could do this it would add a lot of value
over and above simple command translation.

Doug Gilbert

Re: Disable SCSI-Reservation at the driver level ?

[Christoph Hellwig]

On Sun, Nov 26, 2006 at 05:31:26PM +0100, roland wrote:
> Hello !
> 
> i have a "problem" with VMWare ESX (not big one, but interesting one - at 
> least for me).
> 
> If you don`t know VMware ESX - ESX is sort of it`s own operating system 
> with Linux as a management console OS on top of a hypervisor (vmkernel), 

And more.  It's in fact one of the most blatant violations of the
Linux 2.4 kernel copyrights.  Don't expect support if you use such
an illegal product.

Re: Disable SCSI-Reservation at the driver level ?

[devzero]

> And more.  It's in fact one of the most blatant violations of the
> Linux 2.4 kernel copyrights.  Don't expect support if you use such
> an illegal product.

really?
this is the first time i hear that ESX is an "illegal" product from the point of view of the kernel community.
afaik, the esx kernel (vmnix) is a completely different product and for example, the source of the linux drivers they have modified to run under ESX are made available for download, as GPL tells them to do.

what`s the problem here?
can you provide a link to some thread where this has been discussed?
something to be resolved with the help of gpl-violations.org ?




> -----Ursprüngliche Nachricht-----
> Von: Christoph Hellwig <hch@infradead.org>
> Gesendet: 27.11.06 06:56:24
> An: roland <devzero@web.de>
> CC: linux-scsi@vger.kernel.org
> Betreff: Re: Disable SCSI-Reservation at the driver level ?


> On Sun, Nov 26, 2006 at 05:31:26PM +0100, roland wrote:
> > Hello !
> > 
> > i have a "problem" with VMWare ESX (not big one, but interesting one - at 
> > least for me).
> > 
> > If you don`t know VMware ESX - ESX is sort of it`s own operating system 
> > with Linux as a management console OS on top of a hypervisor (vmkernel), 
> 
> And more.  It's in fact one of the most blatant violations of the
> Linux 2.4 kernel copyrights.  Don't expect support if you use such
> an illegal product.
> 


_______________________________________________________________________
Viren-Scan für Ihren PC! Jetzt für jeden. Sofort, online und kostenlos.
Gleich testen! http://www.pc-sicherheit.web.de/freescan/?mc=022222

-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Disable SCSI-Reservation at the driver level ?

[Christoph Hellwig]

On Mon, Nov 27, 2006 at 10:45:42AM +0100, devzero@web.de wrote:
> > And more.  It's in fact one of the most blatant violations of the
> > Linux 2.4 kernel copyrights.  Don't expect support if you use such
> > an illegal product.
> 
> really?
> this is the first time i hear that ESX is an "illegal" product from the point of view of the kernel community.
> afaik, the esx kernel (vmnix) is a completely different product and for example, the source of the linux drivers they have modified to run under ESX are made available for download, as GPL tells them to do.
> 
> what`s the problem here?

It's a modified linux kernel with hooks for a big binary module that takes
over part of the control.  It's rather hard to argue that it's not a derived
work.  Especially as they reuse various I/O subsystem of the kernel
that has been parasited.

> can you provide a link to some thread where this has been discussed?

I don't think so.

> something to be resolved with the help of gpl-violations.org ?

I haven't found any evidence that they are using enough code that
gpl-violations.org has the right to defend.

Re: Disable SCSI-Reservation at the driver level ?

[devzero]

Hi !

> It's a modified linux kernel with hooks for a big binary module that takes
> over part of the control.  It's rather hard to argue that it's not a derived
> work.  Especially as they reuse various I/O subsystem of the kernel
> that has been parasited.


They have put their modified sources for download at http://www.vmware.com/download/vi/open_source.html

For recent version of ESX this is  http://download3.vmware.com/software/vi/VMware-esx-public-source-3.0.1-32039.tar.gz

which is:

VMware-esx-drivers-public-source-3.0.1-32039.tar.gz
VMware-esx-gpl-public-source-3.0.1-32039.tar.gz
VMware-esx-server-public-source-3.0.1-32039.tar.gz


which is:
# ls -la VMware-esx-drivers-public-source-3.0.1-32039/src/drivers/* |wc -l
41

# ls -la VMware-esx-public-source-3.0.1-32039 
total 54
drwxr-xr-x   11 root root   360 Sep 27 01:51 .
drwxrwxrwt  188 root root 23896 Nov 27 19:18 ..
-r--r--r--    1 root root 18003 Sep 27 01:51 COPYING
-r--r--r--    1 root root   832 Sep 27 01:51 README
drwxr-xr-x    2 root root  3656 Sep 27 01:51 include
drwxr-xr-x    4 root root   120 Sep 27 01:51 installer
drwxr-xr-x    2 root root  2144 Sep 27 01:51 linux-iscsi
drwxr-xr-x    2 root root    88 Sep 27 01:51 pixops-2.0
drwxr-xr-x    2 root root   136 Sep 27 01:51 ramcheck
drwxr-xr-x    2 root root    88 Sep 27 01:51 uwglibc-2.2.5
drwxr-xr-x    2 root root  1008 Sep 27 01:51 vmkiscsi-util
drwxr-xr-x    2 root root   624 Sep 27 01:51 vmklinux
drwxr-xr-x    2 root root   128 Sep 27 01:51 vmkload_mod

# tar tzvf VMware-esx-gpl-public-source-3.0.1-32039.tar.gz
booty-0.31.1-5.src.rpm
busybox-0.60.5-11.src.rpm
initscripts-7.31.30.EL-13vmw.src.rpm
iputils-20020927-11.30.4vmw.src.rpm
kudzu-1.1.22.14-4.1.4.src.rpm
mingetty-1.06-2vmw.src.rpm
mkinitrd-3.5.13-16.src.rpm
openssh-3.6.1p2-33.30.9vmw.src.rpm
pam_passwdqc-1.0.2-1.src.rpm
parted-1.6.3-48vmw.src.rpm
pixops-2.0.tar.gz
procmail-3.22-10vmw.src.rpm
redhat-config-keyboard-1.1.4-2.src.rpm
rhpl-0.110.6-2.src.rpm
rpm-4.2.3-26vmw.src.rpm
samba-3.0.9-1.3E.10vmw.src.rpm
shadow-utils-4.0.3-22.03.src.rpm
util-linux-2.11y-31.12vmw.src.rpm
util-linux-2.11y-31.14vmw.src.rpm
vmware-hwdata-1.00-6vmw.src.rpm
vmware-release-3-3vmw.src.rpm
yum-2.0.7-3vmw.src.rpm


So they have re-distributed the sources of all kernel-code/tools they have modified for their product.

I cannot judge, if this is really complete download or if there is missing some or some thousands of lines, but if there is no proof that there is missing a line, i don`t really understand why there is any issue with this.

The GPL allows modification and even sale of  GPL`ed protected code or. It just enforces derived code being GPL`ed, too.

Ok, maybe half of their product is standing on the shoulders of giants (i.e kernel hackers and not people working for vmware) and they don`t tell everyone - but this is the same with half of the world`s DSL-routers.

i`m not sure, but do you just mean:

"they take everything from us, making millions of dollars, but they don`t give anything (real) back to us because the modified sources are more or less useless for the rest of the world...."

???
 
Hey, come on - even if you are right with this - all i wanted to know is how to modify their product so it works for me as i want it to work. That`s why i like freedom of open-source and why i dislike closed-source. I`d happily switch to XEN, but unfortunately XEN is no option for now....

Am i sort of a traitor now , becoming ostracized ?

roland





> -----Ursprüngliche Nachricht-----
> Von: Christoph Hellwig <hch@infradead.org>
> Gesendet: 27.11.06 18:19:24
> An: devzero@web.de
> CC: Christoph Hellwig <hch@infradead.org>, linux-scsi@vger.kernel.org
> Betreff: Re: Disable SCSI-Reservation at the driver level ?


> On Mon, Nov 27, 2006 at 10:45:42AM +0100, devzero@web.de wrote:
> > > And more.  It's in fact one of the most blatant violations of the
> > > Linux 2.4 kernel copyrights.  Don't expect support if you use such
> > > an illegal product.
> > 
> > really?
> > this is the first time i hear that ESX is an "illegal" product from the point of view of the kernel community.
> > afaik, the esx kernel (vmnix) is a completely different product and for example, the source of the linux drivers they have modified to run under ESX are made available for download, as GPL tells them to do.
> > 
> > what`s the problem here?
> 
> It's a modified linux kernel with hooks for a big binary module that takes
> over part of the control.  It's rather hard to argue that it's not a derived
> work.  Especially as they reuse various I/O subsystem of the kernel
> that has been parasited.
> 
> > can you provide a link to some thread where this has been discussed?
> 
> I don't think so.
> 
> > something to be resolved with the help of gpl-violations.org ?
> 
> I haven't found any evidence that they are using enough code that
> gpl-violations.org has the right to defend.
> 


______________________________________________________________________________
"Ein Herz für Kinder" - Ihre Spende hilft! Aktion: www.deutschlandsegelt.de
Unser Dankeschön: Ihr Name auf dem Segel der 1. deutschen America's Cup-Yacht!

-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html