Re: pv_ops kernel and nvidia binary driver

Re: pv_ops kernel and nvidia binary driver

[Boris Derzhavets]

[-- Attachment #1.1: Type: text/plain, Size: 1660 bytes --]

A while ago Jeremy responded the same question.
Per Jeremy:-  unmodified binary Nvidia driver would crash under Xen.
Modified driver should in the pvops kernel.

It's also my major concern as well.

Boris

--- On Sun, 7/12/09, Michael Ralston <michael@ralston.id.au> wrote:

From: Michael Ralston <michael@ralston.id.au>
Subject: [Xen-users] pv_ops kernel and nvidia binary driver
To: xen-users@lists.xensource.com
Date: Sunday, July 12, 2009, 1:48 AM

Just wondering what it will take to get the nvidia binary driver
working on a pv_ops kernel.

It makes it difficult to debug without the source to the nvidia
driver, but I think it should be possible to get it to work without
changing the binary driver. If the dom0 kernel had access to all the
resources that a bare metal kernel did, then it should work right?

I'm using Jeremy's current master branch, and hypervisor 3.4.1-rc6 on
Ubuntu 9.04 64 bit.

Attached is my kern.log and Xorg.0.log when I attempt to start Xorg.
The screen goes blank and my monitor enters powersave mode, but I am
able to run an xterm and it appears to connect to the X server. Also
when the X server exits, it returns the console to a sane state.

I've also tried to get the nvidia driver to work with rebased suse
kernel 2.6.29.6 and hypervisor 3.3 and 3.4. With that version kernel
it blacks the screen. I can remote ssh in to my pc, but I can't reboot
the pc. It needs a hard reset.

-- Michael

-----Inline Attachment Follows-----

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users


      

[-- Attachment #1.2: Type: text/html, Size: 2238 bytes --]

[-- Attachment #2: Type: text/plain, Size: 137 bytes --]

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Re: Re: [Xen-users] pv_ops kernel and nvidia binary driver

[Jeremy Fitzhardinge]

On 07/11/09 23:03, Boris Derzhavets wrote:
> A while ago Jeremy responded the same question.
> Per Jeremy:-  unmodified binary Nvidia driver would crash under Xen.
> Modified driver should in the pvops kernel.
>
> It's also my major concern as well.
>
> Boris
>
> --- On *Sun, 7/12/09, Michael Ralston /<michael@ralston.id.au>/* wrote:
>
>
>     From: Michael Ralston <michael@ralston.id.au>
>     Subject: [Xen-users] pv_ops kernel and nvidia binary driver
>     To: xen-users@lists.xensource.com
>     Date: Sunday, July 12, 2009, 1:48 AM
>
>     Just wondering what it will take to get the nvidia binary driver
>     working on a pv_ops kernel.
>
>     It makes it difficult to debug without the source to the nvidia
>     driver, but I think it should be possible to get it to work without
>     changing the binary driver. If the dom0 kernel had access to all the
>     resources that a bare metal kernel did, then it should work right?
>

The problem comes down to whether the nvidia driver assumes the kernel's
(pseudo-)physical addresses are really machine physical or not.  If it
doesn't do the appropriate conversions between physical and machine
addresses using the standard Linux DMA API (or similar), then it will
end up misprogramming the hardware and reading/writing random memory. 
There's not a lot we can do about that if that happens within the binary
part of the nvidia driver.  If the binary code calls out to the
source-available parts of the driver to do those conversions, then it
would be possible to fix there.

There are other potential pitfalls the nvidia driver could be falling
into, but that's the most likely.

    J

Re: Re: [Xen-users] pv_ops kernel and nvidia binary driver

[Ian Campbell]

On Tue, 2009-07-14 at 20:52 +0100, Jeremy Fitzhardinge wrote:
> The problem comes down to whether the nvidia driver assumes the kernel's
> (pseudo-)physical addresses are really machine physical or not.  If it
> doesn't do the appropriate conversions between physical and machine
> addresses using the standard Linux DMA API (or similar), then it will
> end up misprogramming the hardware and reading/writing random memory. 
> There's not a lot we can do about that if that happens within the binary
> part of the nvidia driver.  If the binary code calls out to the
> source-available parts of the driver to do those conversions, then it
> would be possible to fix there.

I've been running the Nvidia driver on an old style Xen kernel for quite
a while now so I guess it will be possible to make it work for pvops
too. The glue layer contains:
        /*
         * Traditionally, CONFIG_XEN indicated that the target kernel was
         * built exclusively for use under a Xen hypervisor, requiring
         * modifications to or disabling of a variety of NVIDIA graphics
         * driver code paths. As of the introduction of CONFIG_PARAVIRT
         * and support for Xen hypervisors within the CONFIG_PARAVIRT_GUEST
         * architecture, CONFIG_XEN merely indicates that the target
         * kernel can run under a Xen hypervisor, but not that it will.
         *
         * If CONFIG_XEN and CONFIG_PARAVIRT are defined, the old Xen
         * specific code paths are disabled. If the target kernel executes
         * stand-alone, the NVIDIA graphics driver will work fine. If the
         * kernels executes under a Xen (or other) hypervisor, however, the
         * NVIDIA graphics driver has no way of knowing and is unlikely
         * to work correctly.
         */
        #if defined(CONFIG_XEN) && !defined(CONFIG_PARAVIRT)
        #include <asm/maddr.h>
        #include <xen/interface/memory.h>
        #define NV_XEN_SUPPORT_OLD_STYLE_KERNEL
        #endif
which suggests it doesn't currently work but I would guess that the
issue could be fixed in the glue layer and the usages of
NV_XEN_SUPPORT_OLD_STYLE_KERNEL will point towards the areas which need
consideration, there aren't too many of them...

Ian.

Re: Re: [Xen-users] pv_ops kernel and nvidia binary driver

[Jeremy Fitzhardinge]

On 07/15/09 01:14, Ian Campbell wrote:
> On Tue, 2009-07-14 at 20:52 +0100, Jeremy Fitzhardinge wrote:
>   
>> The problem comes down to whether the nvidia driver assumes the kernel's
>> (pseudo-)physical addresses are really machine physical or not.  If it
>> doesn't do the appropriate conversions between physical and machine
>> addresses using the standard Linux DMA API (or similar), then it will
>> end up misprogramming the hardware and reading/writing random memory. 
>> There's not a lot we can do about that if that happens within the binary
>> part of the nvidia driver.  If the binary code calls out to the
>> source-available parts of the driver to do those conversions, then it
>> would be possible to fix there.
>>     
>
> I've been running the Nvidia driver on an old style Xen kernel for quite
> a while now so I guess it will be possible to make it work for pvops
> too. The glue layer contains:
>         /*
>          * Traditionally, CONFIG_XEN indicated that the target kernel was
>          * built exclusively for use under a Xen hypervisor, requiring
>          * modifications to or disabling of a variety of NVIDIA graphics
>          * driver code paths. As of the introduction of CONFIG_PARAVIRT
>          * and support for Xen hypervisors within the CONFIG_PARAVIRT_GUEST
>          * architecture, CONFIG_XEN merely indicates that the target
>          * kernel can run under a Xen hypervisor, but not that it will.
>          *
>          * If CONFIG_XEN and CONFIG_PARAVIRT are defined, the old Xen
>          * specific code paths are disabled. If the target kernel executes
>          * stand-alone, the NVIDIA graphics driver will work fine. If the
>          * kernels executes under a Xen (or other) hypervisor, however, the
>          * NVIDIA graphics driver has no way of knowing and is unlikely
>          * to work correctly.
>   

The whole point of pvops and the other changes is that they don't need
to care whether its running under Xen.  They must be going around the
standard APIs, and doing something special for the Xen case.   Of
course, they can always use the xen_domain() predicates to make that
code runtime conditional if they want to do it that way...

    J

Re: Re: [Xen-users] pv_ops kernel and nvidia binary driver

[Boris Derzhavets]

[-- Attachment #1.1: Type: text/plain, Size: 4637 bytes --]

Seems to build and fails  to install :-

root@SeverUbuntuJaunty:/usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv# make SYSSRC=/usr/src/linux-2.6-xen  install
NVIDIA: calling KBUILD...
make CC=cc  KBUILD_VERBOSE=1 -C /usr/src/linux-2.6-xen SUBDIRS=/usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv modules
test -e include/linux/autoconf.h -a -e include/config/auto.conf || (        \
    echo;                                \
    echo "  ERROR: Kernel configuration is invalid.";        \
    echo "         include/linux/autoconf.h or include/config/auto.conf are missing.";    \
    echo "         Run 'make oldconfig && make prepare' on kernel src to fix it.";    \
    echo;                                \
    /bin/false)
mkdir -p /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/.tmp_versions ; rm -f /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/.tmp_versions/*
make -f scripts/Makefile.build obj=/usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv
(cat /dev/null;   echo kernel//usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/nvidia.ko;) > /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/modules.order
make -f /usr/src/linux-2.6-xen/scripts/Makefile.modpost
  scripts/mod/modpost -m -a -i /usr/src/linux-2.6-xen/Module.symvers -I /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/Module.symvers  -o /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/Module.symvers -S -K /usr/src/linux-2.6-xen/Module.markers -M /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/Module.markers -w  -s
WARNING: could not find /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/.nv-kernel.o.cmd for /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/nv-kernel.o
NVIDIA: left KBUILD.

PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/bin:/sbin" modprobe nvidia && \
    echo "nvidia.ko installed successfully.";
FATAL: Module nvidia not found.
make[1]: *** [package-install] Error 1


Boris
--- On Tue, 7/14/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote:

From: Jeremy Fitzhardinge <jeremy@goop.org>
Subject: Re: [Xen-devel] Re: [Xen-users] pv_ops kernel and nvidia binary driver
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Michael Ralston" <michael@ralston.id.au>, xen-devel@lists.xensource.com, xen-users@lists.xensource.com
Date: Tuesday, July 14, 2009, 5:21 PM

On 07/14/09 14:17, Boris Derzhavets wrote:
> Following  README i attemped :-
>
> root@SeverUbuntuJaunty:/usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv#
> make SYSSRC=/usr/src/linux-2.6-xen module
> . . . . . . . .
>   ld -r -m elf_x86_64  --build-id -o
> /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/nvidia.ko
> /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/nvidia.o
> /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/nvidia.mod.o
> NVIDIA: left KBUILD.
> root@SeverUbuntuJaunty:/usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv#
> make SYSSRC=/usr/src/linux-2.6-xen module
> root@SeverUbuntuJaunty:/usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv#
> ls -l
> total 40416
> -rw-r--r-- 1 root root     1148 2009-07-15 01:05 conftest.h
> -rw-r--r-- 1 root root    49341 2009-05-27 13:02 conftest.sh
> -rw-r--r-- 1 root root     8517 2009-05-27 13:02 cpuopsys.h
> -rw-r--r-- 1 root root     9077 2009-05-27 13:02 gcc-version-check.c
> -rw-r--r-- 1 root root     1834 2009-05-27 13:02 makefile
> lrwxrwxrwx 1 root root       15 2009-07-15 01:05 Makefile ->
> Makefile.kbuild
> -rw-r--r-- 1 root root     9174 2009-05-27 13:02 Makefile.kbuild
> -rw-r--r-- 1 root root     3726 2009-05-27 13:02 Makefile.nvidia
> -rw-r--r-- 1 root root       45 2009-07-15 01:05 Module.markers
> -rw-r--r-- 1 root root       72 2009-07-15 01:05 modules.order
> -rw-r--r-- 1 root root      104 2009-07-15 01:05 Module.symvers
> -rw-r--r-- 1 root root    33436 2009-05-27 13:02 nvacpi.c
> -rw-r--r-- 1 root root   178616 2009-07-15 01:05 nvacpi.o
> -rw-r--r-- 1 root root   131773 2009-05-27 13:02 nv.c
> -rw-r--r-- 1 root root       65 2009-07-15 01:05 nv_compiler.h
> -rw-r--r-- 1 root root    23476 2009-05-27 13:02 nv.h
> -rw-r--r-- 1 root root     9980 2009-05-27 13:02 nv-i2c.c
> -rw-r--r-- 1 root root   161448 2009-07-15 01:05 nv-i2c.o
> ++++++++++++++++++++++++++++++++++++++
> -rw-r--r-- 1 root root 13577047 2009-07-15 01:05 nvidia.ko
> ++++++++++++++++++++++++++++++++++++++
>

What are you saying here?  It built?  It works?

Either way, its up to nvidia to incorporate the support into their
driver.  (Or for someone to maintain it separately.)

    J



      

[-- Attachment #1.2: Type: text/html, Size: 6184 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel