public inbox for kvm@vger.kernel.org
 help / color / mirror / Atom feed
* [GIT PULL] Native Linux KVM tool for 3.1
@ 2011-07-24 20:37 Pekka Enberg
  2011-07-24 23:12 ` Jan Kiszka
  2011-07-25  1:19 ` Anthony Liguori
  0 siblings, 2 replies; 70+ messages in thread
From: Pekka Enberg @ 2011-07-24 20:37 UTC (permalink / raw)
  To: torvalds
  Cc: mingo, avi, akpm, linux-kernel, kvm, gorcunov, levinsasha928,
	asias.hejun, prasadjoshi124

[-- Attachment #1: Type: TEXT/PLAIN, Size: 59041 bytes --]

Hi Linus,

Please consider pulling from

   ssh://master.kernel.org/pub/scm/linux/kernel/git/penberg/linux.git kvm-tool-for-linus

to merge the Native Linux KVM tool to Linux 3.1.

[ The changes to 9p headers were already merged but show up in the pull request. ]

The goal of this tool is to provide a clean, from-scratch, lightweight KVM host
tool implementation that can boot Linux guest images with no BIOS dependencies
and with only the minimal amount of legacy device emulation. The primary focus
of the tool is to Linux but there are already people on working on supporting
GRUB and other operating systems.

We want the tool to be part of Linux kernel source tree because we believe that
‘perf’ clearly showed the benefits of a single repository for both kernel and
userspace components. See Ingo Molnar’s email that started the project for
details:

   http://thread.gmane.org/gmane.linux.kernel/962051/focus=962620

The tool has been in -tip for months and Ingo tells me they’ve been using it
for testing and I use it for all kernel development I’ve do. I’ve asked Avi
Kivity privately on his opinion and he doesn’t object to merging the tool.

Features:

- Zero-config userspace networking for guests that have DHCP enabled

- SMP support

- No external BIOS required

- Experimental GUI support with SDL and VNC

- Host filesystem access via 9P/virtio in guest OS

- Raw image support

- Experimental QCOW2 image support (writing is disabled for now, compressed or
   copy-on-write images are not supported)

- Host block devices as in-memory copy-on-write guest images on 64-bit hosts

1. To try out the tool, clone the git repository:

   git clone git://github.com/penberg/linux-kvm.git

or alternatively, if you already have a kernel source tree:

   git remote add kvm-tool git://github.com/penberg/linux-kvm.git
   git remote update
   git checkout -b kvm-tool kvm-tool/master

2. Compile the tool:

   cd tools/kvm && make

3. Download a raw userspace image:

   Minimal:

     wget http://wiki.qemu.org/download/linux-0.2.img.bz2 && bunzip2 linux-0.2.img.bz2

   Debian Squeeze:

     wget http://people.debian.org/~aurel32/qemu/i386/debian_squeeze_i386_standard.qcow2

     [ as we don’t support writes to QCOW images, you need to convert the image to raw
       image with ‘qemu-img’ as follows: ]

     qemu-img convert -O raw \
       debian_squeeze_i386_standard.qcow2 \
       debian_squeeze_i386_standard.raw

4. The guest kernel has to be built with the following configuration:

  - For the default console output:
 	CONFIG_SERIAL_8250=y
 	CONFIG_SERIAL_8250_CONSOLE=y

  - For running 32bit images on 64bit hosts:
 	CONFIG_IA32_EMULATION=y

  - Proper FS options according to image FS (e.g. CONFIG_EXT2_FS, CONFIG_EXT4_FS).

  - For all virtio devices listed below:
 	CONFIG_VIRTIO=y
 	CONFIG_VIRTIO_RING=y
 	CONFIG_VIRTIO_PCI=y

  - For virtio-blk devices (--disk, -d):
 	CONFIG_VIRTIO_BLK=y

  - For virtio-net devices ([--network, -n] virtio):
 	CONFIG_VIRTIO_NET=y

  - For virtio-9p devices (--virtio-9p):
 	CONFIG_NET_9P=y
 	CONFIG_NET_9P_VIRTIO=y
 	CONFIG_9P_FS=y

  - For virtio-balloon device (--balloon):
 	CONFIG_VIRTIO_BALLOON=y

  - For virtio-console device (--console virtio):
 	CONFIG_VIRTIO_CONSOLE=y

  - For virtio-rng device (--rng):
 	CONFIG_HW_RANDOM_VIRTIO=y

5. And finally, launch the hypervisor:

./kvm run -d linux-0.2.img

or

  ./kvm run -p “root=/dev/vda1” -d debian_squeeze_i386_standard.raw

[ keymap loading takes a while, use ctlr-c if you are inpatient.
   You can login with root/root to the guest and use ‘reboot’ to exit. ]

                         Pekka

The following changes since commit b6844e8f64920cdee620157252169ba63afb0c89:
   Linus Torvalds (1):
         Merge branch 'for-linus' of master.kernel.org:/home/rmk/linux-2.6-arm

are available in the git repository at:

   ssh://master.kernel.org/pub/scm/linux/kernel/git/penberg/linux.git kvm-tool-for-linus

Amerigo Wang (2):
       kvm tools: build rbtree.o from source
       kvm tools: implement "help xxx" command

Amos Kong (4):
       kvm tools: Make virt_queue__available return false if queue is not initialized
       kvm tools: Add a script to setup private bridge
       kvm tools: Add a script to setup tap device
       kvm tools: Setup bridged network by a script

Aneesh Kumar K.V (17):
       tools/kvm/9p: Add support for multiple 9p export dirs
       tools/kvm/9p: Always include system header before kernel headers
       tools/kvm/9p: Use the same #define as the kernel
       tools/kvm/virtio: Add new iov helper
       tools/kvm/9p: Don't follow symlink on server
       tools/kvm/9p: Fix the pdu len.
       tools/kvm/9p: Simplify the handler
       tools/kvm/9p: check the iov count with the read/write count
       tools/kvm/9p: Add error protocol reply
       tools/kvm/9p: Make the 9p handler void return
       tools/kvm/9p: Add error handling
       tools/kvm/9p: Add encode/decode routines for protocol data
       tools/kvm/9p: return EOPNOTSUPP if the handler is not implemented
       kvm tools, 9p: Add error handling to protocol handlers
       kvm tools, 9p: WSTAT don't use open fid
       net/9p: Remove structure not used in the code
       tools/kvm: Use kernel header version of net/9p/9p.h

Anton Vorontsov (1):
       kvm tools: Fix broken terminal when kvm exits because of a signal

Asias He (115):
       bios: No new line at EOF nit fix
       kvm: remove unneccessary iotcl parameter
       kvm__init: the kernel should support irq and pit
       kvm: reimplement kvm__setup_sregs
       kvm: kvm__setup_sregs initialize more registers
       early_printk.c: add rep/outsb support
       kvm__setup_sregs: fix indentation
       kvm, test: add PIT 8254 and PIC 8259 test code
       kvm, test: add a counter to tick test in ISR
       kvm: Check vmx capability using cpuid
       kvm__setup_cpuid: use KVM_GET_SUPPORTED_CPUID to simplify cpuid setup
       kvm, pci: fill up virtio device configuration header
       kvm: Add cscope target to Makefile
       virtio: capacity should be in 512-byte sectors
       kvm,virtio: move SECTOR_SHIFT and SECTOR_SIZE to disk-image.h
       kvm,virtio: do not publish read only feature to guest
       kvm: Add rw to default kernel command line parameters
       kvm,virtio: add scatter-gather support
       kvm,virtio: add scatter-gather support v2
       kvm tools: add README
       kvm tools: remove KVM_EXIT_INTERNAL_ERROR
       kvm tools: Cleanup IO space and PCI magic numbers
       kvm tools: Add helper functions for virtqueue code
       kvm tools: Move terminal related code to a new file term.c
       kvm tools: Make 8250 serial use infrastructure provided by term.c
       kvm tools: Virtio console support
       kvm tools: Introduce --enable-virtio-console option
       kvm tools: Rename struct device to struct blk_device for block devices
       kvm tools: Introduce virtio.c and virtio.h
       kvm tools: Unify virtio code file names
       kvm tools: Drop virt_queue__get_used_elem virtio helper
       kvm tools: Use virt_queue__get_iov to simpify virtio blk IO handler
       kvm tools: Make virtio console device code thread-safe
       kvm tools: Remove unnecessary goto label out_unlock
       kvm tools: Exit KVM session on Ctrl+a and 'x'
       kvm tools: Replace option --enable-virtio-console with --console
       kvm tools: Fix virtio console PCI class number and device ID number
       kvm tools: Fix lock imbalance in virtio_console_pci_io_out()
       kvm tools: Use IRQ pin 2 for virtio console
       kvm tools: Implement virtio network device
       kvm tools: Fix virtio console input problem
       kvm tools: Implement virtio net TSO/UFO support
       kvm tools: Respect ISR status in virtio header
       kvm tools: Respect VRING_AVAIL_F_NO_INTERRUPT
       kvm tools: Use virt_queue__trigger_irq() to trigger IRQ for virtio console
       kvm tools: Use virt_queue__trigger_irq() to trigger IRQ for virtio blk
       kvm tools: Use virt_queue__trigger_irq() to trigger IRQ for virtio rng
       kvm tools: Fix virtio console hangs by removing IRQ injection for tx path
       kvm tools: Bring VIRTIO_BLK_F_SEG_MAX feature back to virtio blk
       kvm tools: Tune the command-line option
       kvm tools: Move disk image related code under disk directory
       kvm tools: Rename disk-image.c to core.c
       kvm tools: Split raw image and blk device code from disk/core.c
       kvm tools: Rename disk_image__{read, write}_sector_iov
       kvm tools: Remove dead coe disk_image__{read, write}_sector
       kvm tools: Consolidate disk_image__{new, new_readonly}
       kvm tools: Split blk device code from raw.c to blk.c
       kvm tools: Tune up ops in 'struct disk_image_operations'
       kvm tools: Rename struct disk_image_operations ops name for raw image
       kvm tools: Rename raw_image_ops to blk_dev_ops
       kvm tools: Remove unnecessary S_ISBLK check
       kvm tools: Do not use 'inline' for disk_image__flush
       kvm tools: Add debug info for disk_image__{read, write}
       kvm tools: Print debug info for qcow1_nowrite_sector
       kvm tools: Update README
       kvm tools: Introduce ethernet frame buffer system for uip
       kvm tools: Add ARP support for uip
       kvm tools: Add IPV4 support for uip
       kvm tools: Implement IP checksum for uip
       kvm tools: Add ICMP support for uip
       kvm tools: Introduce struct uip_udp to present UDP package
       kvm tools: Introduce struct uip_pseudo_hdr to present UDP pseudo header
       kvm tools: Introduce struct uip_udp_socket
       kvm tools: Add two helpers to return UDP {header, total} length
       kvm tools: Add helper to return ethernet header length
       kvm tools: Implement uip_csum_udp() to calculate UDP checksum
       kvm tools: Add UDP support for uip
       kvm tools: Introduce struct uip_tcp to present TCP package.
       kvm tools: Introduce struct uip_tcp_socket
       kvm tools: Add helpers to return TCP {header, total, payload} length
       kvm tools: Add helper to return start address of TCP payload
       kvm tools: Add helpers to test whether SYN or FIN bit is set.
       kvm tools: Add helper to allocate and get TCP initial sequence number
       kvm tools: Implement uip_csum_tcp() to calculate TCP checksum
       kvm tools: Add TCP support for uip
       kvm tools: Introduce uip_init() for uip
       kvm tools: Introduce uip_tx() for uip
       kvm tools: Introduce uip_rx() for uip
       kvm tools: Add MACRO for user and tap mode for virtio net
       kvm tools: Reanme net_device to net_dev
       kvm tools: Introduce -net {user, tap, none} options for virtio net
       kvm tools: Change default guest MAC address to 00:15:15:15:15:15
       kvm tools: Make virtio net work with user mode network
       kvm tools: Make default network mode to user mode
       kvm tools: Make default host ip address to 192.168.33.1
       kvm tools: Introduce struct net_dev_operations
       kvm tools: Make virtio net work on older kernels
       kvm tools: Move uip to net directory
       kvm tools: Introduce uip_udp_make_pkg()
       kvm tools: Introduce struct uip_dhcp
       kvm tools: Add helper to tell if a UDP package is a DHCP package
       kvm tools: Add helpers to tell the type of a DHCP message
       kvm tools: Get domain name and nameserver from host
       kvm tools: Fill DHCP options with domain name and DNS server IP
       kvm tools: Fill all DHCP options
       kvm tools: Introduce uip_dhcp_make_pkg()
       kvm tools: Introduce uip_tx_do_ipv4_udp_dhcp()
       kvm tools: Get DNS information from host in uip_init()
       kvm tools: Handle DHCP package in gernal UDP processing path
       kvm tools: Introduce --guest-ip option
       kvm tools: Introduce --host-mac option
       kvm tools: Rename --host-ip-addr to --host-ip
       kvm tools: Initialize MAC address for virtio net properly
       kvm tools: Initialize MAC and IP address for uip properly
       kvm tools: Add '-Wunused-result' to WARNINGS

Cyrill Gorcunov (108):
       Add utility functions
       Don't die on old kernels without single-step debug support
       Makefile: Add CPPFLAGS handling
       Add info() helper
       kvm: load_bzimage: Proper check for number of setup sectors
       kvm: Use 64 bit names when showing registers
       kvm: Fixup boot selector
       kvm: Add --kernel parameter handling
       kvm: Add --params option
       kvm: Introduce IVT handling
       kvm: Setup fake IVT table
       kvm: Fill up fake IVT
       kvm: Rename ivt_ prefixed entities
       kvm: Put fake bios interrupt handlers into known memory area
       bios: Add simple bin2c converter
       interrupt: Add interrupt_table__set helper
       kvm: Introduce kvm__dump_mem helper
       interrupt: Move bios related constants to kvm/bios.h
       bios: Add bios stubs generated from assembly files
       kvm: Add copying of kernel command line into guest memory
       kvm: Update command line related enitites in boot parameters
       kvm: Reference to undefined variable
       kvm: Make command line to be placed at predefined address
       ioport: Make all PIT channels being dummy
       kvm: Add more default kernel command line parameters
       kvm: Kill the bin2c helper
       Makefile: Add cpu specific definition
       kvm: Command line update must honor ABI
       kvm: Set more fields in boot protocol
       kvm: Don't touch 64 bit specific MSR registers on 32 bit
       Change __x86_x__ to CONFIG_X86_X
       kvm: Print error code on unknown error
       kvm: Use PRIx qualificator for printf
       kvm: Use strlcat helper for copying cmdline params
       kvm: Check for required KVM extensions in one place
       kvm: Append missing ending space for built-in command line
       kvm: The command line storage should not be too short
       early_printk: Simplify the code
       tests/pit: Issue cli on ISR handling
       test/pit: Fill up IDT with noop handler
       kvm: Check for HTL extension granted
       tests/pit: Increase PIT frequency up to 1KHz
       kvm: Add force exit from tests
       kvm, tests: Use motherboard specific ports for testing
       kvm, cpuid: Fix XSTOR and VMX compatibility in cpuid
       kvm: KVM_CAP_EXT_CPUID is required
       kvm, cpuid: Fill up func 0 and 4
       kvm, cpuid: Use logical AND on bit ops
       kvm: Check for SVM extension being supported for AMD cpus
       kvm, bios: Get rid of buggy int10 handler
       Exclude .cscope from .gitignore
       kvm, bios: Rework BIOS setup
       bios: Don't clobber 32bit registers
       bios: No need for sti at int15 irq exit
       kvm: Add --mem= option
       kvm, bios: BIOS code redesign
       kvm, bios: Make e820 map to look similar the real one
       bios: clean predefined bios area early
       pci: Use proper PCI port names
       pci: Fix typo in previous commit
       kvm, pci: Return "no device" by default
       kvm, pci: Add dummy virtio device for probing
       kvm, pci: Add bus forwarding register
       kvm, pci: Fix offset for pci CONFIG_DATA ioport
       kvm, pci: Claim virtio device ioport in pci header
       kvm: Support loading initrd image
       kvm: Add virtio_ring.h to tree
       kvm: Add virtio_blk.h to tree
       kvm: Don't forget to close initrd file
       bios: Get rid of redundant ops
       bios: Use proper grep syntax
       kvm: Setup disk geometry if needed
       virtio-blk: Leave disk geometry to compute in kernel
       make: Add TAGS, tags targets and tune cscope
       kvm: save/restore std ttys
       8250-serial: Simplify is_readable function
       8250-serial: Use linux/serial-reg.h with predefined constants
       8250-serial: Make transmitter being always ready to send data
       8250-serial: Emulate 8250 behaviour on autoprobing
       virtio, block: Rename IOPORT_VIRTIO to IOPORT_VIRTIO_BLK
       kvm tools: Setup BIOS for both bzImage and flat image
       kvm tools: Remove useless empty lines for a call series in kvm__reset_vcpu
       kvm tools: Separate BIOS specifics into own file
       kvm tools, bios: Introduce bioscall specificator
       kvm tools: Use static assignment for default RAM size
       kvm tools: Add --cpus parameter
       kvm tools: Update documentation
       kvm tools: Add missing space before root= option
       kvm tools: Use non shared pin/irqs for virtio devices
       kvm tools: Add MP tables support
       kvm tools: Use system wide msr-index.h instead of own definitions
       kvm tools: Add NR_CPUS definition in case of non-configured kernel sources
       kvm tools: Gather Virtio-PCI constants into one place
       kvm tools: Fix up PCI pin assignment to conform specification
       kvm tools: Fix up mtable srcbusirq assignment for PCI devices
       kvm tools: Add debug() helper
       kvm tools: Prefix error() and friends helpers with pr_
       kvm tools: Add conditional compilation of symbol resolving
       kvm tools: Fix alignment for mpf_intel table
       kvm tools, 9p: Test for tuncation result
       kvm tools: Print out a warning on io-port re-registration
       kvm tools: Drop unused vars from int10.c code
       kvm tools: Options parser to handle hex numbers
       kvm tools: Introduce vidmode parmeter
       kvm tools: Delete dangling cursor from int10
       kvm tools: Get rid of spaces in ld script
       kvm tools: Reform bios make rules
       kvm tools, bios: Make sure IRQ handlers segment is properly set

David Ahern (1):
       kvm tools: do not append root=/dev/vda if passed via command line

Emil Renner Berthing (1):
       kvm tools: Update README with instructions for Arch Linux

Giuseppe Calderaro (2):
       kvm tools: Fix compilation issue in virtio-blk.c
       kvm tools: Remove unused handle_sigint() function

Ingo Molnar (11):
       kvm tools: Emit a more informative error message when /dev/kvm does not open
       kvm tools: Fix segfault when running 'kvm' without a disk image
       kvm tools: Indicate the end of a KVM session
       kvm tools: Fix 64-bit assumptions and type uglinesses
       kvm tools: Fix 32-bit build of the asm/system.h include
       kvm tools: Enable earlyprintk=serial by default
       kvm tools: Fix and improve the CPU register dump debug output code
       kvm tools: Dump vCPUs in order
       kvm tools: Use standardized style for the virtio/net.c driver
       kvm tools: Fix type mismatches on GCC 4.4 on 32-bit systems
       kvm tools: Fix virtio net build breakage on 32-bit

John Floren (6):
       kvm tools: Add BIOS INT10 handler
       kvm tools: Add video mode to kernel initialization
       kvm tools: Add VESA device
       kvm tools: Update makefile and feature tests
       kvm tools: Initialize and use VESA and VNC
       kvm tools: Add support for PS/2 keyboard system

Konstantin Khlebnikov (1):
       kvm tools: fix instances enumeration

Paul Bolle (1):
       kvm tools: Fix compilation on x86_64

Pekka Enberg (309):
       Initial commit
       Check for KVM_API_VERSION at init
       Add .gitignore file
       Create a VM using ioctl(KVM_CREATE_VM)
       Move KVM fds into 'struct kvm'
       Check for KVM_CAP_USER_MEMORY
       Clean up KVM initialization
       Initialize KVM some more
       Set up TSS address to make vcpu creation work
       Clean up struct kvm file descriptor names
       Mmap the kvm_run structure
       Move CPU emulation code to cpu.c
       Load kernel image to the VM
       Initial version of kernel image loading
       Initialize guest memory region properly
       Load bzImage to guest RAM
       Update .gitignore to include object files
       Dump registers after KVM_RUN returns
       Initialize RIP before starting to run the guest
       Print out constant names for KVM exit reasons
       Show dump of executed code at KVM exit
       Fix guest code printout
       Setup rflags properly to fix EXIT_UNKNOWN
       Fix up rflags printing
       Dump KVM_EXIT_IO details
       Move KVM_SET_REGS to kvm__reset_vcpu()
       Enable KVM guest single-stepping
       Separate KVM code from the main loop
       Setup segmentation for the guest
       Fix LDT descriptor type for KVM_SET_SREG
       Improve KVM register dump output
       Fix up 'ip' in kvm__reset_vcpu() to take real mode into account
       Print 'avl' bits of a segment
       Add a simple "kernel" binary that can be used for testing
       Add support for loading flat binaries
       kvm: Load bzImage real-mode boot sector and setup code
       kvm: Fix bzImage setup sector default count
       kvm: Read the whole bzImage to memory
       kvm: Hardcode a "reset vector" at f000:fff0
       kvm: Add some single-step debugging and don't stop running
       kvm: Clean up reset vector setup code
       kvm: Document the 'reset vector' hack
       kvm: Enable 'interrupt request window'
       Revert "kvm: Enable 'interrupt request window'"
       kvm: Fix setup size calculation
       kvm: Remove 'reset vector' hack
       kvm: Fix kernel boot IP
       kvm: Load kernel at 0x1000
       kvm: Setup stack for the kernel
       kvm: Introduce host_real_to_guest() helper function
       kvm: Rename interrupt emulation source files
       kvm: Clean up kvm/interrupt.h a bit
       kvm: Remove unnecessary includes from interrupt.c
       kvm: Move interrupt table to struct kvm
       kvm: Clean up interrupt_table__setup()
       kvm: Move util.h to include/kvm directory
       kvm: Introduce --single-step command line option
       kvm: Setup CPUID for the guest
       kvm: Enable GCC debugging information
       kvm: Extract CPUID code into its own file
       kvm: Extract ioport emulation code to its own file
       kvm: Don't abort guest on KVM_EXIT_IO
       kvm: Add simple NMI enable/disable support
       kvm: Remove unused cpu.c file
       kvm: Setup guest FPU state
       kvm: Ignore math co-processor reset I/O writes
       kvm: Ignore PIC init during boot
       kvm: Fix protected mode code dump
       kvm: Free memory on exit
       kvm: Clean up Makefile output
       kvm: Add test binaries to .gitignore
       kvm: Clean up kvm__reset_vcpu() function
       kvm: Setup VCPU MSRs
       kvm: Add .gitignore for bios build files
       kvm: Show gdt and idt registers
       kvm: Dump stack on KVM oops
       kvm: Dump page tables at exit
       kvm: Set up MAXPHYADDR guest CPUID
       kvm: Don't crash debug dump if address isn't in guest memory
       kvm: Ignore PCI config space accesses
       kvm: Ignore MMIO accesses
       kvm: Ignore PIC accesses
       kvm: Ignore CMOS RAM/RTC ioports
       kvm: Ignore CRT control ioports
       kvm: Ignore dummy delay ioport
       kvm: Ignore PIT ioports
       kvm: Ignore keyboard controller ioports
       kvm: Fix off-by-one bugs in kvm__setup_cpuid()
       kvm: Fix minor formatting glitch in cpuid.c
       kvm: Add support for early serial printk
       kvm: Force 'notsc' and 'earlyprintk' kernel parameters
       kvm: Clean up early serial code
       Merge branch 'master' of git://github.com/cyrillos/vm
       kvm: Dump state on SIGQUIT
       kvm: Flush stdout for early printk
       kvm: Fix .gitignore patterns
       kvm, test: Move PIT tests to separate directory
       kvm: Add Makefile for compiling all test cases
       kvm: Add .gitignore for tests
       kvm, test: Improve PIT test output
       kvm: Add 'check' target to Makefile
       kvm: Use 'nopci pci=off' kernel parameters by default
       kvm: Cleanup command line parsing
       kvm: Cleanup ioport_ops table
       kvm: Add proper 00E0 ioport handling
       kvm: Add '--params' to usage
       kvm: Don't stop on ioport errors by default
       kvm: Fix compilation warning
       kvm: Introduce 'make devices' target
       kvm: Introduce '--kvm-dev' command line option
       kvm: Cleanup kvm__cpu_supports_vm()
       kvm: Cleanup struct boot_params setup
       kvm: BIOS E820 memory map emulation
       kvm: Use -Os optimization for GCC
       kvm: Fix VM support check for unrecognized CPUs
       kvm: Check for read() return value in load_bzimage()
       kvm: Fix usage command line format
       kvm: Fix load_bzimage for flat binaries
       kvm: Use 32-bit operand size prefix for rdfs8()
       kvm: Use -Werror for building the thing
       kvm: Remove unused outb() function from e820.c
       kvm: Cleanup BIOS setup code
       kvm: Use sizeof instead of magic number
       kvm: Clear performance monitoring CPUID flags
       kvm: Update bios .gitignore list
       kvm: Fix Makefile header dependencies
       kvm: Clean header dependency files
       kvm: Exclude BIOS object files from header deps
       kvm: Use proper segment + offset in e820 bios
       kvm: Add some more ioport emulation
       kvm: Enable Linux kernel PCI probing
       kvm: Force PCI Configuration Mechanism 1
       kvm: Initial PCI probe emulation
       kvm: Add PCI CONFIG_ADDRESS register layout
       kvm, pci: Fix device probe accesses
       kvm: Fix PCI config space emulation
       kvm, pci: Don't calculate offset twice
       kvm, pci: Fix unaligned PCI accesses
       kvm: Kill virtio debugging code
       kvm: Add KVM ABI headers
       Merge branch 'master' of git://github.com/cyrillos/vm
       kvm: Fix unaligned PCI_CONFIG_ADDRESS accesses
       kvm: Extract virtio-blk driver to separate file
       kvm: Update .gitignore to ignore tags
       kvm: Add port range to ioport__register()
       kvm: Add virtio PCI ioport handler stubs
       kvm: Fail virtio blk ioports by default
       kvm: Virtio block device emulation
       kvm: Fix kernel loading error messages
       kvm: Fix virtio block PCI device IRQ configuration
       kvm: Implement "host features" for virtio blk device
       kvm: Cleanup virtio block device I/O functions
       kvm: Don't use VIRTIO_BLK_F_SEG_MAX in the hypervisor
       kvm: Remove bogus #include directive
       Revert "kvm: Fix virtio block PCI device IRQ configuration"
       kvm: Implement virtio blk device config space
       kvm: Make virtio block device read-only
       kvm: Add ->queue_selector and ->queue_pfn to struct device
       kvm: Implement virtqueues for virtio block device
       kvm: Implement virtio blk device vring lookup
       kvm: Kill stale interrupt configuration comment
       kvm: Add support for disk images
       kvm: Use disk image API from virtio block device
       kvm: Cleanup disk geometry setup code
       kvm: Introduce a list of known BUGS
       Merge branch 'master' of git://github.com/cyrillos/vm
       kvm: Enable virtio block device if disk image specified
       kvm: Improve kvm__init() error message
       kvm: Fail if user specifies a QCOW disk image
       kvm: Fix virtqueue ring index check
       kvm: Fix virtio block device support some more
       kvm: Support MSR register for serial console
       kvm: Kill debug logging from blk-virtio.c
       kvm: Implement virtio block device write support
       kvm: Redirect serial console to stdout, not stderr
       kvm: Improve 8250 serial console support
       kvm: Fix magic numbers in 8250 support
       kvm,serial: Enable THRI bit in IIR register
       kvm: Implement support interrupt injection
       kvm: Rename early_printk.c to 8250-serial.c
       kvm,8250: Inject interrupts to guest
       kvm: Use serial console by default
       kvm,8250: Implement serial input support
       kvm: Put terminal in canonical mode
       Merge branch 'master' of /home/penberg/kvm into kvm/core
       kvm: Use disk name for mounting root
       kvm: Improve 8250 serial console interrupt handling
       kvm: Support for more than one serial ttys
       kvm,8250: Make ttyS1 and ttyS2 disabled
       kvm,8250: Fix "too much work for irq4" problems
       kvm,8250: Fix ->counter clearing
       kvm,8250: Cleanup receive emulation
       kvm,8250: Implement missing register emulation
       kvm,serial: Implement 16550A FIFO support
       kvm: Cleanup interrupt timer logic
       Revert "kvm,virtio: add scatter-gather support"
       kvm: Remove duplicate headers
       kvm tools: Cleanup termios handling
       kvm,8250: Fix device initial state
       kvm tools: Cleanup virtqueue handling
       tools/kvm: Exit gracefully upon KVM_EXIT_SHUTDOWN
       kvm tools, 8250: Don't set the UART_LSR_OE bit
       kvm tools: Fix large disk images on 32-bit
       kvm tools: Fix E820 map on x86-64
       kvm tools: Fix required boot protocol version
       tools kvm: Fix missing ARCH define on 32-bit in Makefile
       kvm tools: Cleanup strstr() in expression
       kvm tools: Cleanup virtio block device configuration
       kvm tools: Cleanup disk image code
       kvm tools: Drop hard-coded 'nosmp' kernel parameter
       kvm tools: Use STDIN_FILENO instead of fileno(stdin)
       kvm tools: Use pread() and pwrite() in disk-image.c
       kvm tools: Don't exit() on SIGQUIT
       kvm tools: Send SysRq-P to guest kernel on SIGQUIT
       Merge branch 'kvm/gitish' into kvm/core
       kvm tools: Fix .gitignore after Gitification merge
       kvm tools: Cleanup virtio_blk_do_io_request I/O error handling
       kvm tools: Make code mostly checkpatch clean
       kvm tools: Introduce KVM VCPU data structure
       kvm tools: Make 8250 code thread-safe
       kvm tools: Make virtio block device code thread-safe
       kvm tools: Extract kvm_cpu__start() function
       kvm tools: Move CPU initialization to kvm_cpu__start()
       kvm tools: Fix pthread mutex error checks
       kvm tools: Use per-VCPU threads for execution
       kvm tools: Fix 'kill -3' to send SysRq-P to the guest
       kvm tools: Use mutex_lock() and mutex_unlock() wrappers
       kvm tools: Remove the BUGS file
       kvm tools: Use ioport__register() for legacy devices
       Revert "kvm tools: Use mmap by default for disk images"
       kvm tools: Use host kernel image by default
       kvm tools: Don't override kernel image passed by user
       kvm tools: Add read-only support for block devices
       kvm tools: Cleanup qcow1_read_sector() function
       kvm tools: Untangle qcow1_read_sector() function
       kvm tools: Kill redundant assignment in qcow1_read_cluster()
       kvm tools: Fix qcow1_read_cluster() return value on error
       kvm tools: Add read-only support for QCOW2 images
       kvm tools: Show KVM state on SIGQUIT
       kvm tools: Fix disk image double-free on KVM panic
       kvm tools: Emulate RTC to fix system time in guests
       Revert "kvm tools: Use threadpool for virtio-net"
       kvm tools: Fix virtio rng build breakage
       kvm tools: Move virtio drivers under virtio directory
       kvm tools: Cleanup virtio code some more
       kvm tools: Scale guest RAM size by CPU count
       kvm tools: Limit CPU count by KVM_CAP_NR_VCPUS
       kvm tools: Don't use all of host RAM for guests by default
       kvm tools: Warn if guest RAM size exceeds host RAM size
       kvm tools: Fix 'kill -3' hangs
       kvm tools: QCOW code cleanups
       kvm tools: Use '-c' for '--cpus', not '--console'
       kvm tools: Print out important command line options at startup
       kvm tools: Fix read-only support in QCOW
       kvm tools: Fix typo in converting bytes to MBs
       kvm tools: Lookup symbol based on RIP for 'kill -3'
       kvm tools: Fix includes for preadv/pwritev
       kvm tools: Make host_ram_size() more robust
       kvm tools: Fail if passed initrd is not really an initrd
       kvm tools: Move hardware drivers to hw directory
       kvm tools, serial: Register 0x2e8 ioport
       kvm tools: Don't register dummy ops for serial ports
       kvm tools: Drop dummy PCI ioport registrations
       kvm tools: Code cleanups to hw/vesa.c
       kvm tools: Use more readable name for ioport mutex
       kvm tools: Fix up common QCOW function names
       kvm tools: Add 'kvm debug' command
       kvm tools: Use constants for i8042 register numbers
       kvm tools, i8042: Sort status register bits
       kmv tools, i8042: Use kernel status register bit names
       kvm tools, i8042: Use kernel command names
       kvm tools, i8042: Fix device init failure
       kvm tools, ui: Add framebuffer infrastructure
       kvm tools, ui: Move VNC specific framebuffer code to ui/vnc.c
       kvm tools, ui: Add support for SDL framebuffer output target
       kvm tools, vesa: Cleanup code in bios/int10.c
       kvm tools, vesa: Fix 'ah' access in int10_vesa()
       kvm tools, vesa: Use guest-mapped memory for framebuffer
       kvm tools: Kill libvnc dependency from hw/i8042.c
       kvm tools, ui: Add simple keyboard support to SDL UI
       kvm, ui: Kill fb_write() function
       kvm tools: Makefile cleanups
       kvm tools: Add WERROR to Makefile for disabling -Werror
       kvm tools: Fix 'kvm run' on 32-bit machines with >2 GB of RAM
       kvm tools, qcow: Use fdatasync() instead of sync_file_range()
       kvm run: Add 'panic=1' to default kernel options
       Merge commit 'v3.0-rc5' into kvm/core
       kvm tools: Fix guest single-stepping setup
       kvm tools: Don't sort command-list.txt for help text
       kvm tools: Add KVMTOOLS-VERSION-FILE to .gitignore
       kvm tools, qcow: Rename struct qcow_l2_cache to struct qcow_l2_table
       kvm tools, qcow: Use 'struct qcow_l2_table' instead of untyped array
       kvm tools, qcow: Fix locking issues
       kvm tools, qcow: Introduce qcow_disk_flush()
       kvm tools, qcow: Delayed L1 table writeout
       kvm tools, qcow: Don't fdatasync() L2 table writeout
       kvm tools, qcow: Use big endian order for L2 table entries
       kvm tools, qcow: Delayed L2 table writeout
       kvm tools, qcow: Flush only dirty L2 tables
       kvm tools, qcow: I/O error on compressed sectors
       kvm tools: Use writev() in xwritev()
       kvm tools, qcow: Fix copy-on-write image corruption
       kvm tools: Rename 'struct qcow_table' to 'struct qcow_l1_table'
       kvm tools, qcow: Unify L1 and L2 variable names
       kvm tools: Fix formatting in include/kvm/qcow.h
       kvm tools, qcow: Move L2 cache to 'struct qcow_l1_table'
       kvm tools, qcow: Rename L2 table lookup functions
       kvm tools, qcow: Force read-only mode for QCOW images
       Merge branch 'kvm/core' into kvm-tool-for-linus

Prasad Joshi (29):
       kvm tools: Fix KVM problem on SuSe 2.6.37 kernel
       kvm tools: Generate list of common kvm tool commands
       kvm tools: Use code from perf for argument processing
       kvm tools: Provide the basic Gitish framework
       kvm tools: Use the Gitish freamwork to run the virtual machine
       kvm tools: Fix a possible segfault if raw_image__probe returns NULL
       kvm tools: Avoid using disk_image->priv member in disk_image__new()
       kvm tools: add a close method for raw_image_ro_mmap_ops
       kvm tools: Add QCOW version 1 read-only support
       kvm tools: Use the readily built kernel with 'kvm run' if not specified explicitly
       kvm tools: Fix function names in qcow.c
       kvm tools: Avoid byte-order conversion during each read operation
       kvm tools: Free l1_table in qcow1_disk_close() and in error path of qcow1_probe()
       kvm tool: Remove the __stringify*() definition from util.h
       kvm tool: Change the type of QCOW table index and offset variables to u64
       kvm tools: check the cluster boundary in the qcow read code
       kvm tools: Correct a variable naming spelling mistake
       kvm tools: Use the root partition of the host to boot the guest machine
       kvm tools: check read permission before using the root partition of the host to boot VM
       kvm tools: display appropriate error message when default kernel image could not be found
       kvm tools: fix a memory leak in qcow2_read_header
       kvm tools: Add QCOW write support
       kvm tools: Add VIRTIO_BLK_T_FLUSH feature to handle flush operation from VM
       kvm tools: Add a wrapper function to open disk images
       kvm tools: Close the disk images after the guest shuts down
       kvm tools: Add a wrapper function to initialize all virtio block devices
       kvm tools: Release memory allocated during virtio block initialization
       kvm tools: Add QCOW level2 caching support
       kvm tools: Add IO delay option

Sasha Levin (127):
       kvm tools: Free memory and FDs on exit
       kvm tools: Fix input hang when compiling without optimization
       kvm tools: Close kernel FD after loading it into memory
       kvm tools: Use mmap by default for disk images
       kvm tools: Add option to load disk image read only
       kvm tools: Add OPT_GROUP to cmdline parser
       kvm tools: Arrange cmdline help menu
       kvm tools: Set up tun interface using ioctls
       kvm tools: Make host side IP configurable
       kvm tools: Organize net parameters into struct
       kvm tools: Enable network by default
       kvm tools: Add option to specify guest MAC
       kvm tools: Rename raw_image__close_sector_ro_mmap
       kvm tools: Fix leak in QCOW
       kvm tools: Add scatter-gather variants of IO functions
       kvm tools: Add scatter-gather support for disk images
       kvm tools: Rename _sg to _iov and remove dead code
       kvm tools: Fix iov shifting
       kvm tools: Fix possible leak in disk_image
       kvm tools: Use threading for virtio block devices
       kvm tools: Use mmap to allocate guest RAM
       kvm tools: Fix possible leak in qcow
       kvm tools: Prevent duplicate definitions of ALIGN
       kvm tools: Add kernel headers required for using list
       kvm tools: Introduce generic I/O thread pool
       kvm tools: Use threadpool for virtio-blk
       kvm tools: Use threadpool for virtio-console
       kvm tools: Use threadpool for virtio-net
       kvm tools: Modify thread pool API
       kvm tools: Lock job_mutex before signalling
       kvm tools: Introduce virtio-rng
       kvm tools: Add cmdline switch to enable virtio-rng
       kvm tools: Drop ALIGN from bios.h
       kvm tools: Fix virt_queue__set_used_elem
       kvm tools: Move disk_image into virtio-blk
       kvm tools: Add support for multiple virtio-blk
       kvm tools: Add cmdline options for loading multiple images
       kvm tools: Abolishment of uint*_t types
       kvm tools: virtio-blk code cleanup
       kvm tools: virtio-console code cleanup
       kvm tools: virtio-net code cleanup
       kvm tools: virtio-rng code cleanup
       kvm tools: Fix loading root device as image
       kvm tools: Enable SMP support
       kvm tools: Simplify search for root device
       kvm tools: Introduce IRQ registry
       kvm tools: Dynamically add devices when creating mptable
       kvm tools: Convert virtio devices to use IRQ registry
       kvm tools: Rename pci_device to pci_hdr for clarity
       kvm tools: Add missing space after kernel params
       kvm tools: Add memory gap for larger RAM sizes
       kvm tools: Prevent PFN wraparound
       kvm tools: Use definitions from kernel headers
       kvm tools: Rename 'self' variables
       kvm tools: Use constants for commonly used mmap flags
       kvm tools: Add boot test to checks
       kvm tools: Return correct values from disk IOV functions
       kvm tools: Add interval red-black tree helper
       kvm tools: Add MMIO address mapper
       kvm tools: Use virtio IDs from <linux/virtio_ids.h>
       kvm tools: Exit properly on SMP guests
       kvm tools: Default guest cpu count to host cpu count
       kvm tools: Copy net/9p/9p.h
       kvm tools: Add virtio-9p
       kvm tools: Cleanup e820 code
       kvm tools: Fix rbtree-interval balancing
       kvm tools: Modify ioport to use interval rbtree
       kvm tools: Add optional parameter used in ioport callbacks
       kvm tools: Add basic ioport dynamic allocation
       kvm tools: Use ioport context to control blk devices
       kvm tools: Add support for multiple virtio-rng devices
       kvm tools: Use dynamic IO port allocation in vesa driver
       kvm tools: Use dynamic IO port allocation in 9p driver
       kvm tools: Use dynamic IO port allocation in virtio-console
       kvm tools: Use dynamic IO port allocation in virtio-net
       kvm tools: Prevent double assignment of guest memory info
       kvm tools: Exit VCPU thread only when SIGKVMEXIT is received
       kvm tools: Add ioeventfd support
       kvm tools: Use ioeventfd in virtio-blk
       kvm tools: Use ioeventfd in virtio-net
       kvm tools: Use ioeventfd in virtio-rng
       kvm tools: Use correct value for user signal base
       kvm tools: Remove wrong global definition of kvm
       kvm tools: Add APIs to allow pausing guests
       kvm tools: Pause/resume guest using SIGUSR2
       kvm tools: Add a brlock
       kvm tools: Add rwlock wrapper
       kvm tools: Add debug mode to brlock
       kvm tools: Use brlock in MMIO and IOPORT
       kvm tools: Use vesa reserved space for strings and modes
       kvm tools: Add MMIO coalescing support
       kvm tools: Fix some SDL keyboard translations
       kvm tools: Use double buffering with SDL
       kvm tools: Add 'kvm pause' command
       kvm tools: Signal only one thread when injecting console interrupts
       kvm tools: Fix attempt to free mmaped memory
       kvm tools: Implement keyboard reset method
       kvm tools: Use ioeventfd in virtio-9p
       kvm tools: Don't dynamically allocate threadpool jobs
       kvm tools: Process virtio-blk requests in parallel
       kvm tools: Allow giving instance names
       kvm tools: Provide instance name when running 'kvm debug'
       kvm tools: Provide instance name when running 'kvm pause'
       kvm tools: Add virtio-balloon device
       kvm tools: Advise memory allocated for guest RAM as KSM mergable
       kvm tools: Add 'kvm balloon' command
       kvm tools: Stop VCPUs before freeing struct kvm
       kvm tools: Fix home dir resolution
       kvm tools: Give guest instances default name
       kvm tools: Add instance enumerator
       kvm tools: Add 'kvm list' command
       kvm tools: Allow pausing and debugging all running instances
       kvm tools: Export kvm__remove_pidfile
       kvm tools: Clean ghost pid files in 'kvm list'
       kvm tools: Add 'kvm version' command
       kvm tools: Properly add 'kvm list' to command list
       kvm tools: Properly add 'kvm debug' to command list
       kvm tools: Update 'kvm pause' documentation
       kvm tools: Properly add 'kvm balloon' to command list
       kvm tools: Rename command source files
       kvm tools: Rename debug options under 'kvm run'
       kvm tools: Fix leaked descriptors
       kvm tools: Update README
       kvm tools: Implement VIRTIO_BLK_T_GET_ID
       kvm tools: Update max VCPU limit
       kvm tools: Fix warning on 32bit
       kvm tools: Set hardcoded MAC to local administered

  tools/kvm/.gitignore                    |    9 +
  tools/kvm/CREDITS-Git                   |   30 ++
  tools/kvm/Documentation/kvm-balloon.txt |   24 +
  tools/kvm/Documentation/kvm-debug.txt   |   16 +
  tools/kvm/Documentation/kvm-list.txt    |   16 +
  tools/kvm/Documentation/kvm-pause.txt   |   16 +
  tools/kvm/Documentation/kvm-run.txt     |   63 +++
  tools/kvm/Documentation/kvm-version.txt |   21 +
  tools/kvm/Makefile                      |  269 ++++++++++
  tools/kvm/README                        |  109 ++++
  tools/kvm/bios.c                        |  157 ++++++
  tools/kvm/bios/.gitignore               |    3 +
  tools/kvm/bios/bios-rom.S               |   12 +
  tools/kvm/bios/bios.S                   |   89 ++++
  tools/kvm/bios/e820.c                   |   69 +++
  tools/kvm/bios/gen-offsets.sh           |   14 +
  tools/kvm/bios/int10.c                  |  163 ++++++
  tools/kvm/bios/local.S                  |    7 +
  tools/kvm/bios/macro.S                  |   25 +
  tools/kvm/bios/rom.ld.S                 |   17 +
  tools/kvm/builtin-balloon.c             |   34 ++
  tools/kvm/builtin-debug.c               |   32 ++
  tools/kvm/builtin-help.c                |   61 +++
  tools/kvm/builtin-list.c                |   43 ++
  tools/kvm/builtin-pause.c               |   32 ++
  tools/kvm/builtin-run.c                 |  757 +++++++++++++++++++++++++++
  tools/kvm/builtin-version.c             |   15 +
  tools/kvm/code16gcc.h                   |   15 +
  tools/kvm/command-list.txt              |   10 +
  tools/kvm/config/feature-tests.mak      |  148 ++++++
  tools/kvm/config/utilities.mak          |  188 +++++++
  tools/kvm/cpuid.c                       |   51 ++
  tools/kvm/disk/blk.c                    |   39 ++
  tools/kvm/disk/core.c                   |  222 ++++++++
  tools/kvm/disk/qcow.c                   |  845 +++++++++++++++++++++++++++++++
  tools/kvm/disk/raw.c                    |   82 +++
  tools/kvm/framebuffer.c                 |   68 +++
  tools/kvm/hw/i8042.c                    |  348 +++++++++++++
  tools/kvm/hw/rtc.c                      |   87 ++++
  tools/kvm/hw/serial.c                   |  355 +++++++++++++
  tools/kvm/hw/vesa.c                     |   76 +++
  tools/kvm/include/asm/hweight.h         |    8 +
  tools/kvm/include/kvm/8250-serial.h     |   10 +
  tools/kvm/include/kvm/apic.h            |   17 +
  tools/kvm/include/kvm/assembly.h        |   24 +
  tools/kvm/include/kvm/barrier.h         |   15 +
  tools/kvm/include/kvm/bios-export.h     |   13 +
  tools/kvm/include/kvm/bios.h            |   59 +++
  tools/kvm/include/kvm/boot-protocol.h   |   16 +
  tools/kvm/include/kvm/brlock.h          |   41 ++
  tools/kvm/include/kvm/builtin-balloon.h |    6 +
  tools/kvm/include/kvm/builtin-debug.h   |    6 +
  tools/kvm/include/kvm/builtin-help.h    |    6 +
  tools/kvm/include/kvm/builtin-list.h    |    6 +
  tools/kvm/include/kvm/builtin-pause.h   |    6 +
  tools/kvm/include/kvm/builtin-run.h     |    7 +
  tools/kvm/include/kvm/builtin-version.h |    6 +
  tools/kvm/include/kvm/cpufeature.h      |   41 ++
  tools/kvm/include/kvm/disk-image.h      |   73 +++
  tools/kvm/include/kvm/e820.h            |   18 +
  tools/kvm/include/kvm/framebuffer.h     |   34 ++
  tools/kvm/include/kvm/i8042.h           |   12 +
  tools/kvm/include/kvm/interrupt.h       |   26 +
  tools/kvm/include/kvm/ioeventfd.h       |   27 +
  tools/kvm/include/kvm/ioport.h          |   64 +++
  tools/kvm/include/kvm/irq.h             |   24 +
  tools/kvm/include/kvm/kvm-cmd.h         |   17 +
  tools/kvm/include/kvm/kvm-cpu.h         |   44 ++
  tools/kvm/include/kvm/kvm.h             |  105 ++++
  tools/kvm/include/kvm/mptable.h         |    8 +
  tools/kvm/include/kvm/mutex.h           |   33 ++
  tools/kvm/include/kvm/parse-options.h   |  211 ++++++++
  tools/kvm/include/kvm/pci.h             |   55 ++
  tools/kvm/include/kvm/qcow.h            |   98 ++++
  tools/kvm/include/kvm/rbtree-interval.h |   27 +
  tools/kvm/include/kvm/read-write.h      |   32 ++
  tools/kvm/include/kvm/rtc.h             |    6 +
  tools/kvm/include/kvm/rwsem.h           |   39 ++
  tools/kvm/include/kvm/sdl.h             |   17 +
  tools/kvm/include/kvm/segment.h         |   16 +
  tools/kvm/include/kvm/strbuf.h          |    6 +
  tools/kvm/include/kvm/symbol.h          |   22 +
  tools/kvm/include/kvm/term.h            |   17 +
  tools/kvm/include/kvm/threadpool.h      |   37 ++
  tools/kvm/include/kvm/types.h           |    7 +
  tools/kvm/include/kvm/uip.h             |  356 +++++++++++++
  tools/kvm/include/kvm/util.h            |   78 +++
  tools/kvm/include/kvm/vesa.h            |   19 +
  tools/kvm/include/kvm/virtio-9p.h       |   73 +++
  tools/kvm/include/kvm/virtio-balloon.h  |    8 +
  tools/kvm/include/kvm/virtio-blk.h      |   12 +
  tools/kvm/include/kvm/virtio-console.h  |    9 +
  tools/kvm/include/kvm/virtio-net.h      |   21 +
  tools/kvm/include/kvm/virtio-pci-dev.h  |   24 +
  tools/kvm/include/kvm/virtio-rng.h      |    9 +
  tools/kvm/include/kvm/virtio.h          |   59 +++
  tools/kvm/include/kvm/vnc.h             |   14 +
  tools/kvm/include/linux/bitops.h        |   33 ++
  tools/kvm/include/linux/byteorder.h     |    7 +
  tools/kvm/include/linux/kernel.h        |   39 ++
  tools/kvm/include/linux/module.h        |    6 +
  tools/kvm/include/linux/prefetch.h      |    6 +
  tools/kvm/include/linux/types.h         |   49 ++
  tools/kvm/interrupt.c                   |   27 +
  tools/kvm/ioeventfd.c                   |  128 +++++
  tools/kvm/ioport.c                      |  187 +++++++
  tools/kvm/irq.c                         |  112 ++++
  tools/kvm/kvm-cmd.c                     |   74 +++
  tools/kvm/kvm-cpu.c                     |  505 ++++++++++++++++++
  tools/kvm/kvm.c                         |  741 +++++++++++++++++++++++++++
  tools/kvm/main.c                        |   14 +
  tools/kvm/mmio.c                        |  130 +++++
  tools/kvm/mptable.c                     |  284 +++++++++++
  tools/kvm/net/uip/arp.c                 |   30 ++
  tools/kvm/net/uip/buf.c                 |  114 +++++
  tools/kvm/net/uip/core.c                |  190 +++++++
  tools/kvm/net/uip/csum.c                |   92 ++++
  tools/kvm/net/uip/dhcp.c                |  195 +++++++
  tools/kvm/net/uip/icmp.c                |   29 +
  tools/kvm/net/uip/ipv4.c                |   29 +
  tools/kvm/net/uip/tcp.c                 |  317 ++++++++++++
  tools/kvm/net/uip/udp.c                 |  236 +++++++++
  tools/kvm/pci.c                         |  115 +++++
  tools/kvm/read-write.c                  |  318 ++++++++++++
  tools/kvm/symbol.c                      |   98 ++++
  tools/kvm/term.c                        |  131 +++++
  tools/kvm/tests/Makefile                |   19 +
  tools/kvm/tests/boot/Makefile           |   13 +
  tools/kvm/tests/boot/init.c             |   11 +
  tools/kvm/tests/kernel/.gitignore       |    2 +
  tools/kvm/tests/kernel/Makefile         |   20 +
  tools/kvm/tests/kernel/README           |   16 +
  tools/kvm/tests/kernel/kernel.S         |    8 +
  tools/kvm/tests/pit/.gitignore          |    2 +
  tools/kvm/tests/pit/Makefile            |   20 +
  tools/kvm/tests/pit/README              |   16 +
  tools/kvm/tests/pit/tick.S              |  109 ++++
  tools/kvm/threadpool.c                  |  146 ++++++
  tools/kvm/ui/sdl.c                      |  150 ++++++
  tools/kvm/ui/vnc.c                      |  218 ++++++++
  tools/kvm/util.c                        |  101 ++++
  tools/kvm/util/KVMTOOLS-VERSION-GEN     |   40 ++
  tools/kvm/util/generate-cmdlist.sh      |   23 +
  tools/kvm/util/kvm-ifup-vbr0            |    6 +
  tools/kvm/util/parse-options.c          |  631 +++++++++++++++++++++++
  tools/kvm/util/rbtree-interval.c        |   90 ++++
  tools/kvm/util/set_private_br.sh        |   51 ++
  tools/kvm/util/strbuf.c                 |   13 +
  tools/kvm/virtio/9p-pdu.c               |  242 +++++++++
  tools/kvm/virtio/9p.c                   |  835 ++++++++++++++++++++++++++++++
  tools/kvm/virtio/balloon.c              |  265 ++++++++++
  tools/kvm/virtio/blk.c                  |  345 +++++++++++++
  tools/kvm/virtio/console.c              |  257 ++++++++++
  tools/kvm/virtio/core.c                 |  102 ++++
  tools/kvm/virtio/net.c                  |  485 ++++++++++++++++++
  tools/kvm/virtio/rng.c                  |  251 +++++++++
  156 files changed, 15404 insertions(+), 0 deletions(-)
  create mode 100644 tools/kvm/.gitignore
  create mode 100644 tools/kvm/CREDITS-Git
  create mode 100644 tools/kvm/Documentation/kvm-balloon.txt
  create mode 100644 tools/kvm/Documentation/kvm-debug.txt
  create mode 100644 tools/kvm/Documentation/kvm-list.txt
  create mode 100644 tools/kvm/Documentation/kvm-pause.txt
  create mode 100644 tools/kvm/Documentation/kvm-run.txt
  create mode 100644 tools/kvm/Documentation/kvm-version.txt
  create mode 100644 tools/kvm/Makefile
  create mode 100644 tools/kvm/README
  create mode 100644 tools/kvm/bios.c
  create mode 100644 tools/kvm/bios/.gitignore
  create mode 100644 tools/kvm/bios/bios-rom.S
  create mode 100644 tools/kvm/bios/bios.S
  create mode 100644 tools/kvm/bios/e820.c
  create mode 100644 tools/kvm/bios/gen-offsets.sh
  create mode 100644 tools/kvm/bios/int10.c
  create mode 100644 tools/kvm/bios/local.S
  create mode 100644 tools/kvm/bios/macro.S
  create mode 100644 tools/kvm/bios/rom.ld.S
  create mode 100644 tools/kvm/builtin-balloon.c
  create mode 100644 tools/kvm/builtin-debug.c
  create mode 100644 tools/kvm/builtin-help.c
  create mode 100644 tools/kvm/builtin-list.c
  create mode 100644 tools/kvm/builtin-pause.c
  create mode 100644 tools/kvm/builtin-run.c
  create mode 100644 tools/kvm/builtin-version.c
  create mode 100644 tools/kvm/code16gcc.h
  create mode 100644 tools/kvm/command-list.txt
  create mode 100644 tools/kvm/config/feature-tests.mak
  create mode 100644 tools/kvm/config/utilities.mak
  create mode 100644 tools/kvm/cpuid.c
  create mode 100644 tools/kvm/disk/blk.c
  create mode 100644 tools/kvm/disk/core.c
  create mode 100644 tools/kvm/disk/qcow.c
  create mode 100644 tools/kvm/disk/raw.c
  create mode 100644 tools/kvm/framebuffer.c
  create mode 100644 tools/kvm/hw/i8042.c
  create mode 100644 tools/kvm/hw/rtc.c
  create mode 100644 tools/kvm/hw/serial.c
  create mode 100644 tools/kvm/hw/vesa.c
  create mode 100644 tools/kvm/include/asm/hweight.h
  create mode 100644 tools/kvm/include/kvm/8250-serial.h
  create mode 100644 tools/kvm/include/kvm/apic.h
  create mode 100644 tools/kvm/include/kvm/assembly.h
  create mode 100644 tools/kvm/include/kvm/barrier.h
  create mode 100644 tools/kvm/include/kvm/bios-export.h
  create mode 100644 tools/kvm/include/kvm/bios.h
  create mode 100644 tools/kvm/include/kvm/boot-protocol.h
  create mode 100644 tools/kvm/include/kvm/brlock.h
  create mode 100644 tools/kvm/include/kvm/builtin-balloon.h
  create mode 100644 tools/kvm/include/kvm/builtin-debug.h
  create mode 100644 tools/kvm/include/kvm/builtin-help.h
  create mode 100644 tools/kvm/include/kvm/builtin-list.h
  create mode 100644 tools/kvm/include/kvm/builtin-pause.h
  create mode 100644 tools/kvm/include/kvm/builtin-run.h
  create mode 100644 tools/kvm/include/kvm/builtin-version.h
  create mode 100644 tools/kvm/include/kvm/cpufeature.h
  create mode 100644 tools/kvm/include/kvm/disk-image.h
  create mode 100644 tools/kvm/include/kvm/e820.h
  create mode 100644 tools/kvm/include/kvm/framebuffer.h
  create mode 100644 tools/kvm/include/kvm/i8042.h
  create mode 100644 tools/kvm/include/kvm/interrupt.h
  create mode 100644 tools/kvm/include/kvm/ioeventfd.h
  create mode 100644 tools/kvm/include/kvm/ioport.h
  create mode 100644 tools/kvm/include/kvm/irq.h
  create mode 100644 tools/kvm/include/kvm/kvm-cmd.h
  create mode 100644 tools/kvm/include/kvm/kvm-cpu.h
  create mode 100644 tools/kvm/include/kvm/kvm.h
  create mode 100644 tools/kvm/include/kvm/mptable.h
  create mode 100644 tools/kvm/include/kvm/mutex.h
  create mode 100644 tools/kvm/include/kvm/parse-options.h
  create mode 100644 tools/kvm/include/kvm/pci.h
  create mode 100644 tools/kvm/include/kvm/qcow.h
  create mode 100644 tools/kvm/include/kvm/rbtree-interval.h
  create mode 100644 tools/kvm/include/kvm/read-write.h
  create mode 100644 tools/kvm/include/kvm/rtc.h
  create mode 100644 tools/kvm/include/kvm/rwsem.h
  create mode 100644 tools/kvm/include/kvm/sdl.h
  create mode 100644 tools/kvm/include/kvm/segment.h
  create mode 100644 tools/kvm/include/kvm/strbuf.h
  create mode 100644 tools/kvm/include/kvm/symbol.h
  create mode 100644 tools/kvm/include/kvm/term.h
  create mode 100644 tools/kvm/include/kvm/threadpool.h
  create mode 100644 tools/kvm/include/kvm/types.h
  create mode 100644 tools/kvm/include/kvm/uip.h
  create mode 100644 tools/kvm/include/kvm/util.h
  create mode 100644 tools/kvm/include/kvm/vesa.h
  create mode 100644 tools/kvm/include/kvm/virtio-9p.h
  create mode 100644 tools/kvm/include/kvm/virtio-balloon.h
  create mode 100644 tools/kvm/include/kvm/virtio-blk.h
  create mode 100644 tools/kvm/include/kvm/virtio-console.h
  create mode 100644 tools/kvm/include/kvm/virtio-net.h
  create mode 100644 tools/kvm/include/kvm/virtio-pci-dev.h
  create mode 100644 tools/kvm/include/kvm/virtio-rng.h
  create mode 100644 tools/kvm/include/kvm/virtio.h
  create mode 100644 tools/kvm/include/kvm/vnc.h
  create mode 100644 tools/kvm/include/linux/bitops.h
  create mode 100644 tools/kvm/include/linux/byteorder.h
  create mode 100644 tools/kvm/include/linux/kernel.h
  create mode 100644 tools/kvm/include/linux/module.h
  create mode 100644 tools/kvm/include/linux/prefetch.h
  create mode 100644 tools/kvm/include/linux/types.h
  create mode 100644 tools/kvm/interrupt.c
  create mode 100644 tools/kvm/ioeventfd.c
  create mode 100644 tools/kvm/ioport.c
  create mode 100644 tools/kvm/irq.c
  create mode 100644 tools/kvm/kvm-cmd.c
  create mode 100644 tools/kvm/kvm-cpu.c
  create mode 100644 tools/kvm/kvm.c
  create mode 100644 tools/kvm/main.c
  create mode 100644 tools/kvm/mmio.c
  create mode 100644 tools/kvm/mptable.c
  create mode 100644 tools/kvm/net/uip/arp.c
  create mode 100644 tools/kvm/net/uip/buf.c
  create mode 100644 tools/kvm/net/uip/core.c
  create mode 100644 tools/kvm/net/uip/csum.c
  create mode 100644 tools/kvm/net/uip/dhcp.c
  create mode 100644 tools/kvm/net/uip/icmp.c
  create mode 100644 tools/kvm/net/uip/ipv4.c
  create mode 100644 tools/kvm/net/uip/tcp.c
  create mode 100644 tools/kvm/net/uip/udp.c
  create mode 100644 tools/kvm/pci.c
  create mode 100644 tools/kvm/read-write.c
  create mode 100644 tools/kvm/symbol.c
  create mode 100644 tools/kvm/term.c
  create mode 100644 tools/kvm/tests/Makefile
  create mode 100644 tools/kvm/tests/boot/Makefile
  create mode 100644 tools/kvm/tests/boot/init.c
  create mode 100644 tools/kvm/tests/kernel/.gitignore
  create mode 100644 tools/kvm/tests/kernel/Makefile
  create mode 100644 tools/kvm/tests/kernel/README
  create mode 100644 tools/kvm/tests/kernel/kernel.S
  create mode 100644 tools/kvm/tests/pit/.gitignore
  create mode 100644 tools/kvm/tests/pit/Makefile
  create mode 100644 tools/kvm/tests/pit/README
  create mode 100644 tools/kvm/tests/pit/tick.S
  create mode 100644 tools/kvm/threadpool.c
  create mode 100644 tools/kvm/ui/sdl.c
  create mode 100644 tools/kvm/ui/vnc.c
  create mode 100644 tools/kvm/util.c
  create mode 100755 tools/kvm/util/KVMTOOLS-VERSION-GEN
  create mode 100755 tools/kvm/util/generate-cmdlist.sh
  create mode 100755 tools/kvm/util/kvm-ifup-vbr0
  create mode 100644 tools/kvm/util/parse-options.c
  create mode 100644 tools/kvm/util/rbtree-interval.c
  create mode 100755 tools/kvm/util/set_private_br.sh
  create mode 100644 tools/kvm/util/strbuf.c
  create mode 100644 tools/kvm/virtio/9p-pdu.c
  create mode 100644 tools/kvm/virtio/9p.c
  create mode 100644 tools/kvm/virtio/balloon.c
  create mode 100644 tools/kvm/virtio/blk.c
  create mode 100644 tools/kvm/virtio/console.c
  create mode 100644 tools/kvm/virtio/core.c
  create mode 100644 tools/kvm/virtio/net.c
  create mode 100644 tools/kvm/virtio/rng.c

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-24 20:37 [GIT PULL] Native Linux KVM tool for 3.1 Pekka Enberg
@ 2011-07-24 23:12 ` Jan Kiszka
  2011-07-25  7:37   ` Pekka Enberg
  2011-07-25  7:50   ` Sasha Levin
  2011-07-25  1:19 ` Anthony Liguori
  1 sibling, 2 replies; 70+ messages in thread
From: Jan Kiszka @ 2011-07-24 23:12 UTC (permalink / raw)
  To: Pekka Enberg
  Cc: torvalds, mingo, avi, akpm, linux-kernel, kvm, gorcunov,
	levinsasha928, asias.hejun, prasadjoshi124

[-- Attachment #1: Type: text/plain, Size: 1953 bytes --]

On 2011-07-24 22:37, Pekka Enberg wrote:
> Hi Linus,
> 
> Please consider pulling from
> 
>   ssh://master.kernel.org/pub/scm/linux/kernel/git/penberg/linux.git
> kvm-tool-for-linus
> 
> to merge the Native Linux KVM tool to Linux 3.1.
> 
> [ The changes to 9p headers were already merged but show up in the pull
> request. ]
> 
> The goal of this tool is to provide a clean, from-scratch, lightweight
> KVM host
> tool implementation that can boot Linux guest images with no BIOS
> dependencies
> and with only the minimal amount of legacy device emulation. The primary
> focus
> of the tool is to Linux but there are already people on working on
> supporting
> GRUB and other operating systems.
> 
> We want the tool to be part of Linux kernel source tree because we
> believe that
> ‘perf’ clearly showed the benefits of a single repository for both
> kernel and
> userspace components. See Ingo Molnar’s email that started the project for
> details:
> 
>   http://thread.gmane.org/gmane.linux.kernel/962051/focus=962620

I've read several times now that developing in a single tree leads to
better results. Can you provide some example from the QEMU/KVM projects
where the split is preventing innovation, optimizations, or some other
kind of progress?

Let's assume this gets merged. What will change for people working on
generic or x86 KVM features? Will they sooner or later be asked to
provide corresponding bits also for the tools folder? Having worked on
two userlands over the past years (QEMU and qemu-kvm), still
contributing to finally unify them, I'm wondering if this will end up
repeating the painful history.

That said, I definitely appreciate the bug fixes as well as code and
documentation improvements for KVM that originate from this effort! I'm
just not convinced that writing a new userland and merging it into the
kernel is the most efficient way to achieve that.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-24 20:37 [GIT PULL] Native Linux KVM tool for 3.1 Pekka Enberg
  2011-07-24 23:12 ` Jan Kiszka
@ 2011-07-25  1:19 ` Anthony Liguori
  2011-07-25  7:27   ` Pekka Enberg
  1 sibling, 1 reply; 70+ messages in thread
From: Anthony Liguori @ 2011-07-25  1:19 UTC (permalink / raw)
  To: Pekka Enberg
  Cc: torvalds, mingo, avi, akpm, linux-kernel, kvm, gorcunov,
	levinsasha928, asias.hejun, prasadjoshi124

On 07/24/2011 03:37 PM, Pekka Enberg wrote:
> Hi Linus,
>
> Please consider pulling from
>
> ssh://master.kernel.org/pub/scm/linux/kernel/git/penberg/linux.git
> kvm-tool-for-linus
>
> to merge the Native Linux KVM tool to Linux 3.1.
>
> [ The changes to 9p headers were already merged but show up in the pull
> request. ]
>
> The goal of this tool is to provide a clean, from-scratch, lightweight
> KVM host
> tool implementation that can boot Linux guest images with no BIOS
> dependencies
> and with only the minimal amount of legacy device emulation. The primary
> focus
> of the tool is to Linux but there are already people on working on
> supporting
> GRUB and other operating systems.

lguest already does this and lives in the kernel.

So purely from a kernel perspective, why have two tools in the tree that 
do the same thing?  Shouldn't you at least unify the userspace with the 
lguest userspace?

Regards,

Anthony Liguori

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  1:19 ` Anthony Liguori
@ 2011-07-25  7:27   ` Pekka Enberg
  2011-07-25  7:36     ` Avi Kivity
  0 siblings, 1 reply; 70+ messages in thread
From: Pekka Enberg @ 2011-07-25  7:27 UTC (permalink / raw)
  To: Anthony Liguori
  Cc: torvalds, mingo, avi, akpm, linux-kernel, kvm, gorcunov,
	levinsasha928, asias.hejun, prasadjoshi124

Hi Anthony,

On Mon, Jul 25, 2011 at 4:19 AM, Anthony Liguori <anthony@codemonkey.ws> wrote:
> lguest already does this and lives in the kernel.

Does Lguest have SMP, usermode networking, and GUI support?

On Mon, Jul 25, 2011 at 4:19 AM, Anthony Liguori <anthony@codemonkey.ws> wrote:
> So purely from a kernel perspective, why have two tools in the tree that do
> the same thing?  Shouldn't you at least unify the userspace with the lguest
> userspace?

Are you talking about Documentation/lguest/lguest.c? How would you
suggest we unify our code with that?

                                Pekka

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  7:27   ` Pekka Enberg
@ 2011-07-25  7:36     ` Avi Kivity
  2011-07-25  7:39       ` Pekka Enberg
                         ` (2 more replies)
  0 siblings, 3 replies; 70+ messages in thread
From: Avi Kivity @ 2011-07-25  7:36 UTC (permalink / raw)
  To: Pekka Enberg
  Cc: Anthony Liguori, torvalds, mingo, akpm, linux-kernel, kvm,
	gorcunov, levinsasha928, asias.hejun, prasadjoshi124

On 07/25/2011 10:27 AM, Pekka Enberg wrote:
> Hi Anthony,
>
> On Mon, Jul 25, 2011 at 4:19 AM, Anthony Liguori<anthony@codemonkey.ws>  wrote:
> >  lguest already does this and lives in the kernel.
>
> Does Lguest have SMP, usermode networking, and GUI support?
>

IIRC, yes, no, and no.

> On Mon, Jul 25, 2011 at 4:19 AM, Anthony Liguori<anthony@codemonkey.ws>  wrote:
> >  So purely from a kernel perspective, why have two tools in the tree that do
> >  the same thing?  Shouldn't you at least unify the userspace with the lguest
> >  userspace?
>
> Are you talking about Documentation/lguest/lguest.c? How would you
> suggest we unify our code with that?

It should be easy to have tools/kvm drive lguest - they're both virtio 
based.  All you need to do is provide yet another ops structure to drive 
the two ABIs.

I guess lguest.c has to remain, as point of lguest was a simple teaching 
aid for virtualization (which doesn't work very well, as the techniques 
it uses are obsolete).

-- 
error compiling committee.c: too many arguments to function

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-24 23:12 ` Jan Kiszka
@ 2011-07-25  7:37   ` Pekka Enberg
  2011-07-25  7:53     ` Ingo Molnar
  2011-07-25 18:24     ` Jan Kiszka
  2011-07-25  7:50   ` Sasha Levin
  1 sibling, 2 replies; 70+ messages in thread
From: Pekka Enberg @ 2011-07-25  7:37 UTC (permalink / raw)
  To: Jan Kiszka
  Cc: torvalds, mingo, avi, akpm, linux-kernel, kvm, gorcunov,
	levinsasha928, asias.hejun, prasadjoshi124

Hi Jan,

On Mon, Jul 25, 2011 at 2:12 AM, Jan Kiszka <jan.kiszka@web.de> wrote:
> I've read several times now that developing in a single tree leads to
> better results. Can you provide some example from the QEMU/KVM projects
> where the split is preventing innovation, optimizations, or some other
> kind of progress?

I really don't follow the Qemu project well enough to comment on what
your biggest pain points are there.

As for tools/kvm, it's pretty obvious by now that we want tighter
integration with perf and the tracing facilities (and share code!),
for example so for us merging to mainline is important.

On Mon, Jul 25, 2011 at 2:12 AM, Jan Kiszka <jan.kiszka@web.de> wrote:
> Let's assume this gets merged. What will change for people working on
> generic or x86 KVM features? Will they sooner or later be asked to
> provide corresponding bits also for the tools folder? Having worked on
> two userlands over the past years (QEMU and qemu-kvm), still
> contributing to finally unify them, I'm wondering if this will end up
> repeating the painful history.

I'm not a KVM maintainer so I really don't know how tools/kvm will
change any of the above things. I'm certainly not in a position to
*require* anyone to update tools/kvm when they add new KVM features
but it'd sure be great.

On Mon, Jul 25, 2011 at 2:12 AM, Jan Kiszka <jan.kiszka@web.de> wrote:
> That said, I definitely appreciate the bug fixes as well as code and
> documentation improvements for KVM that originate from this effort! I'm
> just not convinced that writing a new userland and merging it into the
> kernel is the most efficient way to achieve that.

Just to make this crystal clear for everyone: if it weren't for
tools/kvm, I wouldn't be hacking on KVM at all. I've looked at Qemu in
the past (and a lot recently!) and I simply don't see myself
contributing to it, sorry. So 'most efficient' or not, I think
tools/kvm is a net win for Linux and KVM in general.

                        Pekka

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  7:36     ` Avi Kivity
@ 2011-07-25  7:39       ` Pekka Enberg
  2011-07-25  7:55       ` Sasha Levin
       [not found]       ` <CAFO3S428qqUAu19QjPxDDAVv+eJSX0MEfYp5y03znNi6XbEtTg@mail.gmail.com>
  2 siblings, 0 replies; 70+ messages in thread
From: Pekka Enberg @ 2011-07-25  7:39 UTC (permalink / raw)
  To: Avi Kivity
  Cc: Anthony Liguori, torvalds, mingo, akpm, linux-kernel, kvm,
	gorcunov, levinsasha928, asias.hejun, prasadjoshi124

Hi Avi,

On Mon, Jul 25, 2011 at 10:36 AM, Avi Kivity <avi@redhat.com> wrote:
>> Are you talking about Documentation/lguest/lguest.c? How would you
>> suggest we unify our code with that?
>
> It should be easy to have tools/kvm drive lguest - they're both virtio
> based.  All you need to do is provide yet another ops structure to drive the
> two ABIs.

Right, we could do that. But for what purpose? I thought Lguest ABI
was mostly for educational and testing purposes.

On Mon, Jul 25, 2011 at 10:36 AM, Avi Kivity <avi@redhat.com> wrote:
> I guess lguest.c has to remain, as point of lguest was a simple teaching aid
> for virtualization (which doesn't work very well, as the techniques it uses
> are obsolete).

Yup, I don't see point in merging lguest.c with tools/kvm.

                        Pekka

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-24 23:12 ` Jan Kiszka
  2011-07-25  7:37   ` Pekka Enberg
@ 2011-07-25  7:50   ` Sasha Levin
  2011-07-25  8:34     ` Alexander Graf
  2011-07-25 12:59     ` Paolo Bonzini
  1 sibling, 2 replies; 70+ messages in thread
From: Sasha Levin @ 2011-07-25  7:50 UTC (permalink / raw)
  To: Jan Kiszka
  Cc: Pekka Enberg, torvalds, mingo, avi, akpm, linux-kernel, kvm,
	gorcunov, asias.hejun, prasadjoshi124

On Mon, 2011-07-25 at 01:12 +0200, Jan Kiszka wrote:
> On 2011-07-24 22:37, Pekka Enberg wrote:
> > Hi Linus,
> > 
> > Please consider pulling from
> > 
> >   ssh://master.kernel.org/pub/scm/linux/kernel/git/penberg/linux.git
> > kvm-tool-for-linus
> > 
> > to merge the Native Linux KVM tool to Linux 3.1.
> > 
> > [ The changes to 9p headers were already merged but show up in the pull
> > request. ]
> > 
> > The goal of this tool is to provide a clean, from-scratch, lightweight
> > KVM host
> > tool implementation that can boot Linux guest images with no BIOS
> > dependencies
> > and with only the minimal amount of legacy device emulation. The primary
> > focus
> > of the tool is to Linux but there are already people on working on
> > supporting
> > GRUB and other operating systems.
> > 
> > We want the tool to be part of Linux kernel source tree because we
> > believe that
> > ‘perf’ clearly showed the benefits of a single repository for both
> > kernel and
> > userspace components. See Ingo Molnar’s email that started the project for
> > details:
> > 
> >   http://thread.gmane.org/gmane.linux.kernel/962051/focus=962620
> 
> I've read several times now that developing in a single tree leads to
> better results. Can you provide some example from the QEMU/KVM projects
> where the split is preventing innovation, optimizations, or some other
> kind of progress?
> 

Anthony had a talk on last years KVM forum regarding the QEMU threading
model (slide:
http://www.linux-kvm.org/wiki/images/7/70/2010-forum-threading-qemu.pdf) .

It was suggested that the KVM part of QEMU is having a hard time
achieving the ideal threading model due to its need to support TCG -
something which has nothing to do with KVM itself.


-- 

Sasha.


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  7:37   ` Pekka Enberg
@ 2011-07-25  7:53     ` Ingo Molnar
  2011-07-25  8:14       ` Alexander Graf
  2011-07-25 18:24     ` Jan Kiszka
  1 sibling, 1 reply; 70+ messages in thread
From: Ingo Molnar @ 2011-07-25  7:53 UTC (permalink / raw)
  To: Pekka Enberg
  Cc: Jan Kiszka, torvalds, avi, akpm, linux-kernel, kvm, gorcunov,
	levinsasha928, asias.hejun, prasadjoshi124


* Pekka Enberg <penberg@kernel.org> wrote:

> On Mon, Jul 25, 2011 at 2:12 AM, Jan Kiszka <jan.kiszka@web.de> wrote:
> > That said, I definitely appreciate the bug fixes as well as code and
> > documentation improvements for KVM that originate from this effort! I'm
> > just not convinced that writing a new userland and merging it into the
> > kernel is the most efficient way to achieve that.
> 
> Just to make this crystal clear for everyone: if it weren't for 
> tools/kvm, I wouldn't be hacking on KVM at all. I've looked at Qemu 
> in the past (and a lot recently!) and I simply don't see myself 
> contributing to it, sorry. So 'most efficient' or not, I think 
> tools/kvm is a net win for Linux and KVM in general.

Same here - in fact i first asked Qemu to be put into tools/qemu/ so 
that it all becomes more hackable and more usable - that suggestion 
was rebuked very strongly.

So i wanted to have a lightweight tool that allows me to test KVM and 
tools/kvm/ does that very nicely: i type './kvm run' and i can test a 
native bzImage (which has some virtualization options enabled as 
well) on the _host_ distro i am running, booting to a text shell 
prompt.

I can do that without downloading any (inevitably outdated) 
virtualization images or maintaining my own ones. Maintaining host 
userspace is more than enough for me.

So, since we already have the lguest tool in the kernel tree, why 
cannot we have the much more capable tools/kvm/ in the tree?

So while it is the Qemu folks' right to oppose tools/qemu/, i don't 
see why they are opposing tools/kvm/ ...

Wrt. integration with lguest - this is a new argument that was not 
brought up before (i wish people would not come up with new 
requirements on the day of the pull request) - i don't see how it's 
relevant really: lguest was designed for legacy CPUs and tools/kvm/ 
is precisely about being simple and not doing legacy stuff.

If then Qemu should be the project that integrates lguest. Is anyone 
on the Qemu side looking at lguest integration?

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  7:36     ` Avi Kivity
  2011-07-25  7:39       ` Pekka Enberg
@ 2011-07-25  7:55       ` Sasha Levin
       [not found]       ` <CAFO3S428qqUAu19QjPxDDAVv+eJSX0MEfYp5y03znNi6XbEtTg@mail.gmail.com>
  2 siblings, 0 replies; 70+ messages in thread
From: Sasha Levin @ 2011-07-25  7:55 UTC (permalink / raw)
  To: Avi Kivity
  Cc: Pekka Enberg, Anthony Liguori, torvalds, mingo, akpm,
	linux-kernel, kvm, gorcunov, asias.hejun, prasadjoshi124

On Mon, 2011-07-25 at 10:36 +0300, Avi Kivity wrote:
> On 07/25/2011 10:27 AM, Pekka Enberg wrote:
> > Hi Anthony,
> >
> > On Mon, Jul 25, 2011 at 4:19 AM, Anthony Liguori<anthony@codemonkey.ws>  wrote:
> > >  lguest already does this and lives in the kernel.
> >
> > Does Lguest have SMP, usermode networking, and GUI support?
> >
> 
> IIRC, yes, no, and no.

Theres no SMP support in lguest.

-- 

Sasha.


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
       [not found]       ` <CAFO3S428qqUAu19QjPxDDAVv+eJSX0MEfYp5y03znNi6XbEtTg@mail.gmail.com>
@ 2011-07-25  8:11         ` Ingo Molnar
  2011-07-25  8:17           ` Pekka Enberg
  0 siblings, 1 reply; 70+ messages in thread
From: Ingo Molnar @ 2011-07-25  8:11 UTC (permalink / raw)
  To: Asias He
  Cc: Avi Kivity, Pekka Enberg, Anthony Liguori, torvalds, akpm,
	linux-kernel, kvm, gorcunov, levinsasha928, prasadjoshi124


* Asias He <asias.hejun@gmail.com> wrote:

> On Mon, Jul 25, 2011 at 3:36 PM, Avi Kivity <avi@redhat.com> wrote:
> 
> > On 07/25/2011 10:27 AM, Pekka Enberg wrote:
> >
> >> Hi Anthony,
> >>
> >> On Mon, Jul 25, 2011 at 4:19 AM, Anthony Liguori<anthony@codemonkey.ws>
> >>  wrote:
> >> >  lguest already does this and lives in the kernel.
> >>
> >> Does Lguest have SMP, usermode networking, and GUI support?
> >>
> >>
> > IIRC, yes, no, and no.
> >
> >
> And Lguest only supports 32-bit kernel.

The answer to SMP is 'no' as well, as lguest supports only a single 
CPU:

   [Documentation/virtual/lguest/lguest.c]

        /* We're CPU 0.  In fact, that's the only CPU possible right now. */
        cpu_id = 0;

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  7:53     ` Ingo Molnar
@ 2011-07-25  8:14       ` Alexander Graf
  2011-07-25  8:23         ` Pekka Enberg
                           ` (3 more replies)
  0 siblings, 4 replies; 70+ messages in thread
From: Alexander Graf @ 2011-07-25  8:14 UTC (permalink / raw)
  To: Ingo Molnar
  Cc: Pekka Enberg, Jan Kiszka, torvalds, avi, akpm, linux-kernel, kvm,
	gorcunov, levinsasha928, asias.hejun, prasadjoshi124


On 25.07.2011, at 09:53, Ingo Molnar wrote:

> 
> * Pekka Enberg <penberg@kernel.org> wrote:
> 
>> On Mon, Jul 25, 2011 at 2:12 AM, Jan Kiszka <jan.kiszka@web.de> wrote:
>>> That said, I definitely appreciate the bug fixes as well as code and
>>> documentation improvements for KVM that originate from this effort! I'm
>>> just not convinced that writing a new userland and merging it into the
>>> kernel is the most efficient way to achieve that.
>> 
>> Just to make this crystal clear for everyone: if it weren't for 
>> tools/kvm, I wouldn't be hacking on KVM at all. I've looked at Qemu 
>> in the past (and a lot recently!) and I simply don't see myself 
>> contributing to it, sorry. So 'most efficient' or not, I think 
>> tools/kvm is a net win for Linux and KVM in general.
> 
> Same here - in fact i first asked Qemu to be put into tools/qemu/ so 
> that it all becomes more hackable and more usable - that suggestion 
> was rebuked very strongly.

So instead of thinking a bit and trying to realize that there might be a reason people don't want all their user space in the kernel tree you go ahead and start your own crusade of creating a new user space. Great. That's how I always hoped Linux would be :(.

> So i wanted to have a lightweight tool that allows me to test KVM and 
> tools/kvm/ does that very nicely: i type './kvm run' and i can test a 
> native bzImage (which has some virtualization options enabled as 
> well) on the _host_ distro i am running, booting to a text shell 
> prompt.

I do that all the time.

  $ qemu-kvm -nographic -kernel arch/x86/boot/bzImage -append console=ttyS0

does the exact same thing. If that's too much typing for you, make it a bash alias.

> I can do that without downloading any (inevitably outdated) 
> virtualization images or maintaining my own ones. Maintaining host 
> userspace is more than enough for me.

Who would need images? I usually only run -kernel and -initrd directly to test out things. Or if I really want to boot into a real system I do -snapshot /dev/sda.

> So, since we already have the lguest tool in the kernel tree, why 
> cannot we have the much more capable tools/kvm/ in the tree?

Lguest is in Documentation/ for a reason. It's not meant as a user space tool that you take as-is and use. It's meant for documenting how lguest works in general. I admit though, that that's also the reason people don't use it :).

> So while it is the Qemu folks' right to oppose tools/qemu/, i don't 
> see why they are opposing tools/kvm/ ...

In your logic, you would put all of the GNU utils into tools/. This is just plain insane. There's a reason we have the split between kernel and user space. What does putting them into the same tree bring you? Fame? Glorious stats on kernel commits? Seriously, it's a separate project. It's not the kernel.

> Wrt. integration with lguest - this is a new argument that was not 
> brought up before (i wish people would not come up with new 
> requirements on the day of the pull request) - i don't see how it's 
> relevant really: lguest was designed for legacy CPUs and tools/kvm/ 
> is precisely about being simple and not doing legacy stuff.
> 
> If then Qemu should be the project that integrates lguest. Is anyone 
> on the Qemu side looking at lguest integration?

I don't think lguest integration makes sense for anyone or anything. Lguest is a toy, so let it be the toy it wants to be.


Alex


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  8:11         ` Ingo Molnar
@ 2011-07-25  8:17           ` Pekka Enberg
  0 siblings, 0 replies; 70+ messages in thread
From: Pekka Enberg @ 2011-07-25  8:17 UTC (permalink / raw)
  To: Ingo Molnar
  Cc: Asias He, Avi Kivity, Anthony Liguori, torvalds, akpm,
	linux-kernel, kvm, gorcunov, levinsasha928, prasadjoshi124

On Mon, Jul 25, 2011 at 4:19 AM, Anthony Liguori<anthony@codemonkey.ws> wrote:
>> >> >  lguest already does this and lives in the kernel.

On 07/25/2011 10:27 AM, Pekka Enberg wrote:
>> >> Does Lguest have SMP, usermode networking, and GUI support?

On Mon, Jul 25, 2011 at 3:36 PM, Avi Kivity <avi@redhat.com> wrote:
>> > IIRC, yes, no, and no.

* Asias He <asias.hejun@gmail.com> wrote:
>> And Lguest only supports 32-bit kernel.

On Mon, Jul 25, 2011 at 11:11 AM, Ingo Molnar <mingo@elte.hu> wrote:
> The answer to SMP is 'no' as well, as lguest supports only a single
> CPU:
>
>   [Documentation/virtual/lguest/lguest.c]
>
>        /* We're CPU 0.  In fact, that's the only CPU possible right now. */
>        cpu_id = 0;

Uhm - so why bring up Lguest at all in this pull request? Anthony,
what am I missing here?

                        Pekka

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  8:14       ` Alexander Graf
@ 2011-07-25  8:23         ` Pekka Enberg
  2011-07-25  8:31           ` Alexander Graf
  2011-07-25  9:14           ` Ingo Molnar
  2011-07-25  8:30         ` Pekka Enberg
                           ` (2 subsequent siblings)
  3 siblings, 2 replies; 70+ messages in thread
From: Pekka Enberg @ 2011-07-25  8:23 UTC (permalink / raw)
  To: Alexander Graf
  Cc: Ingo Molnar, Jan Kiszka, torvalds, avi, akpm, linux-kernel, kvm,
	gorcunov, levinsasha928, asias.hejun, prasadjoshi124

Hi Alexander,

On Mon, Jul 25, 2011 at 11:14 AM, Alexander Graf <agraf@suse.de> wrote:
>> Same here - in fact i first asked Qemu to be put into tools/qemu/ so
>> that it all becomes more hackable and more usable - that suggestion
>> was rebuked very strongly.
>
> So instead of thinking a bit and trying to realize that there might be a reason people don't want all their user space in the kernel tree you go ahead and start your own crusade of creating a new user space. Great. That's how I always hoped Linux would be :(.

Damn you Ingo Molnar, I knew you'd somehow get all the credit for our
hard work! ;-)

More seriously, though, I fail to see what's bothering you Alexander.
I and Ingo already mentioned we wouldn't be hacking on Qemu even if
there wasn't no tools/kvm. It's not as if we're putting *your* user
space code into the kernel tree - we wrote our own! What's wrong with
that?

                        Pekka

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  8:14       ` Alexander Graf
  2011-07-25  8:23         ` Pekka Enberg
@ 2011-07-25  8:30         ` Pekka Enberg
  2011-07-25  8:37           ` Alexander Graf
  2011-07-25 12:24           ` Kevin Wolf
  2011-07-25  8:54         ` Ingo Molnar
  2011-07-25 10:38         ` Christoph Hellwig
  3 siblings, 2 replies; 70+ messages in thread
From: Pekka Enberg @ 2011-07-25  8:30 UTC (permalink / raw)
  To: Alexander Graf
  Cc: Ingo Molnar, Jan Kiszka, torvalds, avi, akpm, linux-kernel, kvm,
	gorcunov, levinsasha928, asias.hejun, prasadjoshi124

Hi Alexander,

On Mon, Jul 25, 2011 at 11:14 AM, Alexander Graf <agraf@suse.de> wrote:
>> So i wanted to have a lightweight tool that allows me to test KVM and
>> tools/kvm/ does that very nicely: i type './kvm run' and i can test a
>> native bzImage (which has some virtualization options enabled as
>> well) on the _host_ distro i am running, booting to a text shell
>> prompt.
>
> I do that all the time.
>
>  $ qemu-kvm -nographic -kernel arch/x86/boot/bzImage -append console=ttyS0
>
> does the exact same thing. If that's too much typing for you, make it a bash alias.

You know, they said the same thing about oprofile. All you needed to do was to
write few simple shell scripts to make it work. One of the key
features of tools/kvm
is 'as little configuration as possible' and I can assure you that
bash alias is really
not a solution for that.

[ Yes, yes, I know people apparently use virtio-manager for lauching Qemu so
  they don't need to figure out the setup themselves. But now you have *three*
  separate components (KVM, Qemu, virtio-manager) which is a completely
  different direction we're taking. Hell, we even went ahead and wrote our own
  mini-BIOS just to keep things in one unified tree. ]

                                Pekka

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  8:23         ` Pekka Enberg
@ 2011-07-25  8:31           ` Alexander Graf
  2011-07-25  8:35             ` Avi Kivity
  2011-07-25  8:37             ` Pekka Enberg
  2011-07-25  9:14           ` Ingo Molnar
  1 sibling, 2 replies; 70+ messages in thread
From: Alexander Graf @ 2011-07-25  8:31 UTC (permalink / raw)
  To: Pekka Enberg
  Cc: Ingo Molnar, Jan Kiszka, torvalds, avi, akpm, linux-kernel, kvm,
	gorcunov, levinsasha928, asias.hejun, prasadjoshi124


On 25.07.2011, at 10:23, Pekka Enberg wrote:

> Hi Alexander,
> 
> On Mon, Jul 25, 2011 at 11:14 AM, Alexander Graf <agraf@suse.de> wrote:
>>> Same here - in fact i first asked Qemu to be put into tools/qemu/ so
>>> that it all becomes more hackable and more usable - that suggestion
>>> was rebuked very strongly.
>> 
>> So instead of thinking a bit and trying to realize that there might be a reason people don't want all their user space in the kernel tree you go ahead and start your own crusade of creating a new user space. Great. That's how I always hoped Linux would be :(.
> 
> Damn you Ingo Molnar, I knew you'd somehow get all the credit for our
> hard work! ;-)

Well, IIUC he's the one initiating the whole thing, no?

> More seriously, though, I fail to see what's bothering you Alexander.
> I and Ingo already mentioned we wouldn't be hacking on Qemu even if
> there wasn't no tools/kvm. It's not as if we're putting *your* user
> space code into the kernel tree - we wrote our own! What's wrong with
> that?

Nothing. I like competition. But why push it into the kernel? It's not a kernel, it's not a library the kernels needs for internal stuff. So why would it have to be in there?

In Ingo's reasoning, the next step would be to rewrite glibc and put it into the kernel tree, because we end up adding syscalls so adding them to the in-kernel libc with the same commit would be a lot easier and cleaner.


Alex


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  7:50   ` Sasha Levin
@ 2011-07-25  8:34     ` Alexander Graf
  2011-07-25 12:59     ` Paolo Bonzini
  1 sibling, 0 replies; 70+ messages in thread
From: Alexander Graf @ 2011-07-25  8:34 UTC (permalink / raw)
  To: Sasha Levin
  Cc: Jan Kiszka, Pekka Enberg, torvalds, mingo, avi, akpm,
	linux-kernel, kvm, gorcunov, asias.hejun, prasadjoshi124


On 25.07.2011, at 09:50, Sasha Levin wrote:

> On Mon, 2011-07-25 at 01:12 +0200, Jan Kiszka wrote:
>> On 2011-07-24 22:37, Pekka Enberg wrote:
>>> Hi Linus,
>>> 
>>> Please consider pulling from
>>> 
>>>  ssh://master.kernel.org/pub/scm/linux/kernel/git/penberg/linux.git
>>> kvm-tool-for-linus
>>> 
>>> to merge the Native Linux KVM tool to Linux 3.1.
>>> 
>>> [ The changes to 9p headers were already merged but show up in the pull
>>> request. ]
>>> 
>>> The goal of this tool is to provide a clean, from-scratch, lightweight
>>> KVM host
>>> tool implementation that can boot Linux guest images with no BIOS
>>> dependencies
>>> and with only the minimal amount of legacy device emulation. The primary
>>> focus
>>> of the tool is to Linux but there are already people on working on
>>> supporting
>>> GRUB and other operating systems.
>>> 
>>> We want the tool to be part of Linux kernel source tree because we
>>> believe that
>>> ‘perf’ clearly showed the benefits of a single repository for both
>>> kernel and
>>> userspace components. See Ingo Molnar’s email that started the project for
>>> details:
>>> 
>>>  http://thread.gmane.org/gmane.linux.kernel/962051/focus=962620
>> 
>> I've read several times now that developing in a single tree leads to
>> better results. Can you provide some example from the QEMU/KVM projects
>> where the split is preventing innovation, optimizations, or some other
>> kind of progress?
>> 
> 
> Anthony had a talk on last years KVM forum regarding the QEMU threading
> model (slide:
> http://www.linux-kvm.org/wiki/images/7/70/2010-forum-threading-qemu.pdf) .
> 
> It was suggested that the KVM part of QEMU is having a hard time
> achieving the ideal threading model due to its need to support TCG -
> something which has nothing to do with KVM itself.

Sure, but TCG is a really great part of Qemu. And I'm actually one of the few people who believe that in the KVM community ;).

Recently, I sat down for a couple of months and implemented s390x emulation with TCG in Qemu. You know why?

We had KVM support (device emulation, KVM backend) for s390x for more than a year at that point. People constantly broke stuff in the s390x code paths, because they couldn't test it - people seemed to not have too many mainframes lying around at home. With the emulation target, I could make sure that everyone's able to at least compile and run the s390x specific parts and make sure he can do regression testing.

So yes, while TCG is keeping us back in certain areas, it's tremendously useful in others.


Alex


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  8:31           ` Alexander Graf
@ 2011-07-25  8:35             ` Avi Kivity
  2011-07-25 13:10               ` Joerg Roedel
  2011-07-25  8:37             ` Pekka Enberg
  1 sibling, 1 reply; 70+ messages in thread
From: Avi Kivity @ 2011-07-25  8:35 UTC (permalink / raw)
  To: Alexander Graf
  Cc: Pekka Enberg, Ingo Molnar, Jan Kiszka, torvalds, akpm,
	linux-kernel, kvm, gorcunov, levinsasha928, asias.hejun,
	prasadjoshi124

On 07/25/2011 11:31 AM, Alexander Graf wrote:
> In Ingo's reasoning, the next step would be to rewrite glibc and put it into the kernel tree, because we end up adding syscalls so adding them to the in-kernel libc with the same commit would be a lot easier and cleaner.

That actually makes a ton of sense.  One immediate win would be that 
klibc can be tuned to the kernel it ships with (the dynamic loader will 
pick the correct object), so less #ifdef trees.  Another would be to 
make klibc the formal kernel interface, which allows us to reimplement 
an older interface in terms of the one that supercedes it.

-- 
error compiling committee.c: too many arguments to function


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  8:31           ` Alexander Graf
  2011-07-25  8:35             ` Avi Kivity
@ 2011-07-25  8:37             ` Pekka Enberg
  2011-07-25  8:44               ` Alexander Graf
  2011-07-25  9:16               ` Ingo Molnar
  1 sibling, 2 replies; 70+ messages in thread
From: Pekka Enberg @ 2011-07-25  8:37 UTC (permalink / raw)
  To: Alexander Graf
  Cc: Ingo Molnar, Jan Kiszka, torvalds, avi, akpm, linux-kernel, kvm,
	gorcunov, levinsasha928, asias.hejun, prasadjoshi124

Hi Alexander,

On Mon, Jul 25, 2011 at 11:31 AM, Alexander Graf <agraf@suse.de> wrote:
>> Damn you Ingo Molnar, I knew you'd somehow get all the credit for our
>> hard work! ;-)
>
> Well, IIUC he's the one initiating the whole thing, no?

As much as I appreciate Ingo's help and support with the project, no,
I don't consider him to have initiated the whole thing. Yes, it's his
idea and that's what got me into hacking on the thing in the first
place. But calling this Ingo's crusade is somewhat missing the point.
It's really people like Asias, Sasha, Cyrill, and Prashad who have
made all the heavy lifting.

On Mon, Jul 25, 2011 at 11:31 AM, Alexander Graf <agraf@suse.de> wrote:
>> More seriously, though, I fail to see what's bothering you Alexander.
>> I and Ingo already mentioned we wouldn't be hacking on Qemu even if
>> there wasn't no tools/kvm. It's not as if we're putting *your* user
>> space code into the kernel tree - we wrote our own! What's wrong with
>> that?
>
> Nothing. I like competition. But why push it into the kernel? It's not a kernel, it's not a library the kernels needs for internal stuff. So why would it have to be in there?

For the same reasons we want tools/perf to be there.

On Mon, Jul 25, 2011 at 11:31 AM, Alexander Graf <agraf@suse.de> wrote:
> In Ingo's reasoning, the next step would be to rewrite glibc and put it into the kernel tree, because we end up adding syscalls so adding them to the in-kernel libc with the same commit would be a lot easier and cleaner.

It's called klibc, btw. :-)

                                Pekka

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  8:30         ` Pekka Enberg
@ 2011-07-25  8:37           ` Alexander Graf
  2011-07-25  8:47             ` Pekka Enberg
  2011-07-25 10:47             ` Ingo Molnar
  2011-07-25 12:24           ` Kevin Wolf
  1 sibling, 2 replies; 70+ messages in thread
From: Alexander Graf @ 2011-07-25  8:37 UTC (permalink / raw)
  To: Pekka Enberg
  Cc: Ingo Molnar, Jan Kiszka, torvalds, avi, akpm, linux-kernel, kvm,
	gorcunov, levinsasha928, asias.hejun, prasadjoshi124


On 25.07.2011, at 10:30, Pekka Enberg wrote:

> Hi Alexander,
> 
> On Mon, Jul 25, 2011 at 11:14 AM, Alexander Graf <agraf@suse.de> wrote:
>>> So i wanted to have a lightweight tool that allows me to test KVM and
>>> tools/kvm/ does that very nicely: i type './kvm run' and i can test a
>>> native bzImage (which has some virtualization options enabled as
>>> well) on the _host_ distro i am running, booting to a text shell
>>> prompt.
>> 
>> I do that all the time.
>> 
>>  $ qemu-kvm -nographic -kernel arch/x86/boot/bzImage -append console=ttyS0
>> 
>> does the exact same thing. If that's too much typing for you, make it a bash alias.
> 
> You know, they said the same thing about oprofile. All you needed to do was to
> write few simple shell scripts to make it work. One of the key
> features of tools/kvm
> is 'as little configuration as possible' and I can assure you that
> bash alias is really
> not a solution for that.

I like perf. Really. But I still don't see why it wouldn't be able to live in its own tree either.

> [ Yes, yes, I know people apparently use virtio-manager for lauching Qemu so
>  they don't need to figure out the setup themselves. But now you have *three*
>  separate components (KVM, Qemu, virtio-manager) which is a completely

Having the split between KVM and user space is necessary. One lives in kernel space, the other one in user space. The split between qemu and virt-manager is something I don't like either. But then again I don't like virt-manager :). But what does this have to do with pulling something into the kernel? And reimplementing something that's already been there?

>  different direction we're taking. Hell, we even went ahead and wrote our own
>  mini-BIOS just to keep things in one unified tree. ]

Yes, making sure that you have even more non-working non-Linux OSs.


Alex


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  8:37             ` Pekka Enberg
@ 2011-07-25  8:44               ` Alexander Graf
  2011-07-25  8:51                 ` Pekka Enberg
  2011-07-25  9:16               ` Ingo Molnar
  1 sibling, 1 reply; 70+ messages in thread
From: Alexander Graf @ 2011-07-25  8:44 UTC (permalink / raw)
  To: Pekka Enberg
  Cc: Ingo Molnar, Jan Kiszka, torvalds, avi, akpm, linux-kernel, kvm,
	gorcunov, levinsasha928, asias.hejun, prasadjoshi124


On 25.07.2011, at 10:37, Pekka Enberg wrote:

> Hi Alexander,
> 
> On Mon, Jul 25, 2011 at 11:31 AM, Alexander Graf <agraf@suse.de> wrote:
>>> Damn you Ingo Molnar, I knew you'd somehow get all the credit for our
>>> hard work! ;-)
>> 
>> Well, IIUC he's the one initiating the whole thing, no?
> 
> As much as I appreciate Ingo's help and support with the project, no,
> I don't consider him to have initiated the whole thing. Yes, it's his
> idea and that's what got me into hacking on the thing in the first
> place. But calling this Ingo's crusade is somewhat missing the point.
> It's really people like Asias, Sasha, Cyrill, and Prashad who have
> made all the heavy lifting.
> 
> On Mon, Jul 25, 2011 at 11:31 AM, Alexander Graf <agraf@suse.de> wrote:
>>> More seriously, though, I fail to see what's bothering you Alexander.
>>> I and Ingo already mentioned we wouldn't be hacking on Qemu even if
>>> there wasn't no tools/kvm. It's not as if we're putting *your* user
>>> space code into the kernel tree - we wrote our own! What's wrong with
>>> that?
>> 
>> Nothing. I like competition. But why push it into the kernel? It's not a kernel, it's not a library the kernels needs for internal stuff. So why would it have to be in there?
> 
> For the same reasons we want tools/perf to be there.

Yeah, I want a pony too.


Alex


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  8:37           ` Alexander Graf
@ 2011-07-25  8:47             ` Pekka Enberg
  2011-07-25  9:06               ` Alexander Graf
  2011-07-25  9:26               ` Ingo Molnar
  2011-07-25 10:47             ` Ingo Molnar
  1 sibling, 2 replies; 70+ messages in thread
From: Pekka Enberg @ 2011-07-25  8:47 UTC (permalink / raw)
  To: Alexander Graf
  Cc: Ingo Molnar, Jan Kiszka, torvalds, avi, akpm, linux-kernel, kvm,
	gorcunov, levinsasha928, asias.hejun, prasadjoshi124

Hi Alexander,

On Mon, Jul 25, 2011 at 11:37 AM, Alexander Graf <agraf@suse.de> wrote:
>>  different direction we're taking. Hell, we even went ahead and wrote our own
>>  mini-BIOS just to keep things in one unified tree. ]
>
> Yes, making sure that you have even more non-working non-Linux OSs.

You know, I've been a Linux kernel hacker for more than five years now
and I've spent way too much of my spare time to improve it. So yes, I
care about Linux. I care about it a lot, actually. It's fair to say I
care about Linux more than I care about it more than any other
operating system out there.

[ I thought the 'native Linux' part in 'native Linux KVM tool' was a
dead giveaway, really. ]

Now if people want to support other operating systems, that's cool and
I'm happy to help out where I can. But I don't understand why people
keep bringing non-Linux OSs as an argument for not merging tools/kvm
into the Linux kernel tree. I mean really, did someone actually expect
that a Linux kernel developer spends his weekends improving the state
of Windows virtualization?

And don't get this the wrong way either, I'm not hostile against other
operating systems, but I simply am not interested enough in them to
spend my time improving them.

                        Pekka

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  8:44               ` Alexander Graf
@ 2011-07-25  8:51                 ` Pekka Enberg
  2011-07-25  8:55                   ` Alexander Graf
  0 siblings, 1 reply; 70+ messages in thread
From: Pekka Enberg @ 2011-07-25  8:51 UTC (permalink / raw)
  To: Alexander Graf
  Cc: Ingo Molnar, Jan Kiszka, torvalds, avi, akpm, linux-kernel, kvm,
	gorcunov, levinsasha928, asias.hejun, prasadjoshi124

On Mon, Jul 25, 2011 at 11:44 AM, Alexander Graf <agraf@suse.de> wrote:
>> For the same reasons we want tools/perf to be there.
>
> Yeah, I want a pony too.

I can contact the Linux Foundation to see if we can arrange that.

Seriously, though, I don't understand your point. What is it? Do you
not agree that perf benefited from being part of the Linux kernel
tree? Or do you think it does not apply here?

                        Pekka

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  8:14       ` Alexander Graf
  2011-07-25  8:23         ` Pekka Enberg
  2011-07-25  8:30         ` Pekka Enberg
@ 2011-07-25  8:54         ` Ingo Molnar
  2011-07-25  8:59           ` Ingo Molnar
  2011-07-25  9:03           ` Alexander Graf
  2011-07-25 10:38         ` Christoph Hellwig
  3 siblings, 2 replies; 70+ messages in thread
From: Ingo Molnar @ 2011-07-25  8:54 UTC (permalink / raw)
  To: Alexander Graf
  Cc: Pekka Enberg, Jan Kiszka, torvalds, avi, akpm, linux-kernel, kvm,
	gorcunov, levinsasha928, asias.hejun, prasadjoshi124


* Alexander Graf <agraf@suse.de> wrote:

> On 25.07.2011, at 09:53, Ingo Molnar wrote:
> 
> > 
> > * Pekka Enberg <penberg@kernel.org> wrote:
> > 
> >> On Mon, Jul 25, 2011 at 2:12 AM, Jan Kiszka <jan.kiszka@web.de> wrote:
> >>>
> >>> That said, I definitely appreciate the bug fixes as well as 
> >>> code and documentation improvements for KVM that originate from 
> >>> this effort! I'm just not convinced that writing a new userland 
> >>> and merging it into the kernel is the most efficient way to 
> >>> achieve that.
> >> 
> >> Just to make this crystal clear for everyone: if it weren't for 
> >> tools/kvm, I wouldn't be hacking on KVM at all. I've looked at 
> >> Qemu in the past (and a lot recently!) and I simply don't see 
> >> myself contributing to it, sorry. So 'most efficient' or not, I 
> >> think tools/kvm is a net win for Linux and KVM in general.
> > 
> > Same here - in fact i first asked Qemu to be put into tools/qemu/ 
> > so that it all becomes more hackable and more usable - that 
> > suggestion was rebuked very strongly.
> 
> So instead of thinking a bit and trying to realize that there might 
> be a reason people don't want all their user space in the kernel 
> tree you go ahead and start your own crusade of creating a new user 
> space. Great. That's how I always hoped Linux would be :(.

Not "all their user space" but the ones that are tightly integrated 
with the kernel to begin with. How hard is that concept to 
understand?

Virtualization is very tightly bound to the kernel, like it or not. 
So is profiling, power management and a few other things.

And when you do 'ls tools/' you'll see exactly those topics 
populated:

 earth5:~/tip> ls tools/
 firewire  kvm  perf  power  slub  testing  usb  virtio

[ In fact tools/virtio/ was merged upstream yesterday and putting 
  that code there was a good call IMO. ]

So just as there are good reasons to keep some user-space projects 
out of the kernel tree (while i'd love to see a usable mail client 
for Linux i dont think it belongs into tools/) there are similarly 
good reasons to keep some of them in the kernel tree.

tools/kvm/ is an excellent example for that, just look at the code - 
it shares code with the kernel, uses the kernel coding style, is in 
significant part developed by kernel developers and it is being used 
by kernel developers.

I wish there was a hackable tools/qemu/ base but there isnt and won't 
be any. The thing i *can* do is to help create a hackable 
virtualization tool.

> > So i wanted to have a lightweight tool that allows me to test KVM 
> > and tools/kvm/ does that very nicely: i type './kvm run' and i 
> > can test a native bzImage (which has some virtualization options 
> > enabled as well) on the _host_ distro i am running, booting to a 
> > text shell prompt.
> 
> I do that all the time.
> 
>   $ qemu-kvm -nographic -kernel arch/x86/boot/bzImage -append console=ttyS0
> 
> does the exact same thing. If that's too much typing for you, make 
> it a bash alias.

This only boots the bzImage itself and panics when it would like to 
hit user-space.

> > I can do that without downloading any (inevitably outdated) 
> > virtualization images or maintaining my own ones. Maintaining 
> > host userspace is more than enough for me.
> 
> Who would need images? I usually only run -kernel and -initrd 
> directly to test out things. Or if I really want to boot into a 
> real system I do -snapshot /dev/sda.

that too panics here, while with tools/kvm/ i get to a prompt:

 [root@aldebaran ~]# uptime
  08:42:13 up 0 min,  0 users,  load average: 0.00, 0.00, 0.00

But i agree with you that obviously Qemu (or my usage of parameters, 
whichever is the problem) could be fixed to do this correctly.

> > So, since we already have the lguest tool in the kernel tree, why 
> > cannot we have the much more capable tools/kvm/ in the tree?
> 
> Lguest is in Documentation/ for a reason. It's not meant as a user 
> space tool that you take as-is and use. It's meant for documenting 
> how lguest works in general. I admit though, that that's also the 
> reason people don't use it :).

There's an obvious contrast with the request to merge tools/kvm/ to 
lguest (not brought up by me) and your description of lguest only 
being for documentation purposes, not to be used really, etc.

I wanted to highlight this contrast, so if you thought we disagree 
much about lguest i don't think we do.

> > So while it is the Qemu folks' right to oppose tools/qemu/, i 
> > don't see why they are opposing tools/kvm/ ...
> 
> In your logic, you would put all of the GNU utils into tools/. This 
> is just plain insane. There's a reason we have the split between 
> kernel and user space. What does putting them into the same tree 
> bring you? Fame? Glorious stats on kernel commits? Seriously, it's 
> a separate project. It's not the kernel.

Where did i claim that *all* user-space projects need to move into 
the kernel tree? It's all case by case. Before you argue against a 
position and ridicule it you need to understand it.

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  8:51                 ` Pekka Enberg
@ 2011-07-25  8:55                   ` Alexander Graf
  0 siblings, 0 replies; 70+ messages in thread
From: Alexander Graf @ 2011-07-25  8:55 UTC (permalink / raw)
  To: Pekka Enberg
  Cc: Ingo Molnar, Jan Kiszka, torvalds, avi, akpm, linux-kernel, kvm,
	gorcunov, levinsasha928, asias.hejun, prasadjoshi124


On 25.07.2011, at 10:51, Pekka Enberg wrote:

> On Mon, Jul 25, 2011 at 11:44 AM, Alexander Graf <agraf@suse.de> wrote:
>>> For the same reasons we want tools/perf to be there.
>> 
>> Yeah, I want a pony too.
> 
> I can contact the Linux Foundation to see if we can arrange that.
> 
> Seriously, though, I don't understand your point. What is it? Do you
> not agree that perf benefited from being part of the Linux kernel
> tree? Or do you think it does not apply here?

I disagree with the assumption that putting something into the kernel tree benefits the project (except for maybe bumping its visibility, so it potentially kills competition again). If however it's true, then we should analyze why exactly projects benefit from being in the kernel tree, so we can maybe find a good solution to achieve the same differently.

The approach "let's move everyting into tools/" simply doesn't scale.


Alex


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  8:54         ` Ingo Molnar
@ 2011-07-25  8:59           ` Ingo Molnar
  2011-07-25  9:03           ` Alexander Graf
  1 sibling, 0 replies; 70+ messages in thread
From: Ingo Molnar @ 2011-07-25  8:59 UTC (permalink / raw)
  To: Alexander Graf
  Cc: Pekka Enberg, Jan Kiszka, torvalds, avi, akpm, linux-kernel, kvm,
	gorcunov, levinsasha928, asias.hejun, prasadjoshi124


* Ingo Molnar <mingo@elte.hu> wrote:

> Virtualization is very tightly bound to the kernel, like it or not. 
> So is profiling, power management and a few other things.
> 
> And when you do 'ls tools/' you'll see exactly those topics 
> populated:
> 
>  earth5:~/tip> ls tools/
>  firewire  kvm  perf  power  slub  testing  usb  virtio
> 
> [ In fact tools/virtio/ was merged upstream yesterday and putting 
>   that code there was a good call IMO. ]

Correction: tools/virtio/ was merged much earlier than that - in 
v2.6.37.

Still the point remains: you cannot ridicule us moving user-space 
code to tools/ while virtio itself is moving code there ...

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  8:54         ` Ingo Molnar
  2011-07-25  8:59           ` Ingo Molnar
@ 2011-07-25  9:03           ` Alexander Graf
  2011-07-25  9:41             ` Ingo Molnar
  2011-07-25 11:26             ` Olivier Galibert
  1 sibling, 2 replies; 70+ messages in thread
From: Alexander Graf @ 2011-07-25  9:03 UTC (permalink / raw)
  To: Ingo Molnar
  Cc: Pekka Enberg, Jan Kiszka, torvalds, avi, akpm, linux-kernel, kvm,
	gorcunov, levinsasha928, asias.hejun, prasadjoshi124


On 25.07.2011, at 10:54, Ingo Molnar wrote:

> 
> * Alexander Graf <agraf@suse.de> wrote:
> 
>> On 25.07.2011, at 09:53, Ingo Molnar wrote:
>> 
>>> 
>>> * Pekka Enberg <penberg@kernel.org> wrote:
>>> 
>>>> On Mon, Jul 25, 2011 at 2:12 AM, Jan Kiszka <jan.kiszka@web.de> wrote:
>>>>> 
>>>>> That said, I definitely appreciate the bug fixes as well as 
>>>>> code and documentation improvements for KVM that originate from 
>>>>> this effort! I'm just not convinced that writing a new userland 
>>>>> and merging it into the kernel is the most efficient way to 
>>>>> achieve that.
>>>> 
>>>> Just to make this crystal clear for everyone: if it weren't for 
>>>> tools/kvm, I wouldn't be hacking on KVM at all. I've looked at 
>>>> Qemu in the past (and a lot recently!) and I simply don't see 
>>>> myself contributing to it, sorry. So 'most efficient' or not, I 
>>>> think tools/kvm is a net win for Linux and KVM in general.
>>> 
>>> Same here - in fact i first asked Qemu to be put into tools/qemu/ 
>>> so that it all becomes more hackable and more usable - that 
>>> suggestion was rebuked very strongly.
>> 
>> So instead of thinking a bit and trying to realize that there might 
>> be a reason people don't want all their user space in the kernel 
>> tree you go ahead and start your own crusade of creating a new user 
>> space. Great. That's how I always hoped Linux would be :(.
> 
> Not "all their user space" but the ones that are tightly integrated 
> with the kernel to begin with. How hard is that concept to 
> understand?

It's very hard to understand. It's similar to religion - I could easily apply your point to every reasonably low-level user space project out there. X for example. X needs to interact with KMS and DRI and whatdoiknow. So it'd be a perfect fit to pull into tools/, no?

> Virtualization is very tightly bound to the kernel, like it or not. 

I don't see why. The whole point of virtualization is to model simple, with KVM preferably even very-close-to-real-hardware interfaces that allow you to keep things separate.

> So is profiling, power management and a few other things.

I'm also failing to see the reasoning here TBH.

> And when you do 'ls tools/' you'll see exactly those topics 
> populated:
> 
> earth5:~/tip> ls tools/
> firewire  kvm  perf  power  slub  testing  usb  virtio
> 
> [ In fact tools/virtio/ was merged upstream yesterday and putting 
>  that code there was a good call IMO. ]
> 
> So just as there are good reasons to keep some user-space projects 
> out of the kernel tree (while i'd love to see a usable mail client 
> for Linux i dont think it belongs into tools/) there are similarly 
> good reasons to keep some of them in the kernel tree.
> 
> tools/kvm/ is an excellent example for that, just look at the code - 
> it shares code with the kernel, uses the kernel coding style, is in 
> significant part developed by kernel developers and it is being used 
> by kernel developers.

Then make a separate tree, add the kernel as git submodule and simply pull in your library/header dependencies from there. Shouldn't be too hard, no?

> I wish there was a hackable tools/qemu/ base but there isnt and won't 
> be any. The thing i *can* do is to help create a hackable 
> virtualization tool.

What's the problem with a code base that's hackable, but not in tools/?

> 
>>> So i wanted to have a lightweight tool that allows me to test KVM 
>>> and tools/kvm/ does that very nicely: i type './kvm run' and i 
>>> can test a native bzImage (which has some virtualization options 
>>> enabled as well) on the _host_ distro i am running, booting to a 
>>> text shell prompt.
>> 
>> I do that all the time.
>> 
>>  $ qemu-kvm -nographic -kernel arch/x86/boot/bzImage -append console=ttyS0
>> 
>> does the exact same thing. If that's too much typing for you, make 
>> it a bash alias.
> 
> This only boots the bzImage itself and panics when it would like to 
> hit user-space.

Well, sure.

> 
>>> I can do that without downloading any (inevitably outdated) 
>>> virtualization images or maintaining my own ones. Maintaining 
>>> host userspace is more than enough for me.
>> 
>> Who would need images? I usually only run -kernel and -initrd 
>> directly to test out things. Or if I really want to boot into a 
>> real system I do -snapshot /dev/sda.
> 
> that too panics here, while with tools/kvm/ i get to a prompt:
> 
> [root@aldebaran ~]# uptime
>  08:42:13 up 0 min,  0 users,  load average: 0.00, 0.00, 0.00
> 
> But i agree with you that obviously Qemu (or my usage of parameters, 
> whichever is the problem) could be fixed to do this correctly.

Well, you need to make sure to pass the right partition into -append. No idea what ./kvm run does, but it's certainly something easily scriptable with any other virtualization user land.

> 
>>> So, since we already have the lguest tool in the kernel tree, why 
>>> cannot we have the much more capable tools/kvm/ in the tree?
>> 
>> Lguest is in Documentation/ for a reason. It's not meant as a user 
>> space tool that you take as-is and use. It's meant for documenting 
>> how lguest works in general. I admit though, that that's also the 
>> reason people don't use it :).
> 
> There's an obvious contrast with the request to merge tools/kvm/ to 
> lguest (not brought up by me) and your description of lguest only 
> being for documentation purposes, not to be used really, etc.
> 
> I wanted to highlight this contrast, so if you thought we disagree 
> much about lguest i don't think we do.

I don't think we disagree about lguest :).

> 
>>> So while it is the Qemu folks' right to oppose tools/qemu/, i 
>>> don't see why they are opposing tools/kvm/ ...
>> 
>> In your logic, you would put all of the GNU utils into tools/. This 
>> is just plain insane. There's a reason we have the split between 
>> kernel and user space. What does putting them into the same tree 
>> bring you? Fame? Glorious stats on kernel commits? Seriously, it's 
>> a separate project. It's not the kernel.
> 
> Where did i claim that *all* user-space projects need to move into 
> the kernel tree? It's all case by case. Before you argue against a 
> position and ridicule it you need to understand it.

I'm slightly exaggerating, but that's the trend I'm seeing.


Alex


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  8:47             ` Pekka Enberg
@ 2011-07-25  9:06               ` Alexander Graf
  2011-07-25 12:45                 ` Pekka Enberg
  2011-07-25  9:26               ` Ingo Molnar
  1 sibling, 1 reply; 70+ messages in thread
From: Alexander Graf @ 2011-07-25  9:06 UTC (permalink / raw)
  To: Pekka Enberg
  Cc: Ingo Molnar, Jan Kiszka, torvalds, avi, akpm, linux-kernel, kvm,
	gorcunov, levinsasha928, asias.hejun, prasadjoshi124


On 25.07.2011, at 10:47, Pekka Enberg wrote:

> Hi Alexander,
> 
> On Mon, Jul 25, 2011 at 11:37 AM, Alexander Graf <agraf@suse.de> wrote:
>>>  different direction we're taking. Hell, we even went ahead and wrote our own
>>>  mini-BIOS just to keep things in one unified tree. ]
>> 
>> Yes, making sure that you have even more non-working non-Linux OSs.
> 
> You know, I've been a Linux kernel hacker for more than five years now
> and I've spent way too much of my spare time to improve it. So yes, I
> care about Linux. I care about it a lot, actually. It's fair to say I
> care about Linux more than I care about it more than any other
> operating system out there.
> 
> [ I thought the 'native Linux' part in 'native Linux KVM tool' was a
> dead giveaway, really. ]
> 
> Now if people want to support other operating systems, that's cool and
> I'm happy to help out where I can. But I don't understand why people
> keep bringing non-Linux OSs as an argument for not merging tools/kvm
> into the Linux kernel tree. I mean really, did someone actually expect
> that a Linux kernel developer spends his weekends improving the state
> of Windows virtualization?
> 
> And don't get this the wrong way either, I'm not hostile against other
> operating systems, but I simply am not interested enough in them to
> spend my time improving them.

Then kvm-tool is about as useful as Mac-on-Linux. Why don't we have MoL user land in the kernel? I even added support for KVM to it about a year ago. So all I need to do is change it to the kernel coding style, add some dependencies on kernel headers and I'm good for a pull request?


Alex


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  8:23         ` Pekka Enberg
  2011-07-25  8:31           ` Alexander Graf
@ 2011-07-25  9:14           ` Ingo Molnar
  1 sibling, 0 replies; 70+ messages in thread
From: Ingo Molnar @ 2011-07-25  9:14 UTC (permalink / raw)
  To: Pekka Enberg
  Cc: Alexander Graf, Jan Kiszka, torvalds, avi, akpm, linux-kernel,
	kvm, gorcunov, levinsasha928, asias.hejun, prasadjoshi124


* Pekka Enberg <penberg@kernel.org> wrote:

> Hi Alexander,
> 
> On Mon, Jul 25, 2011 at 11:14 AM, Alexander Graf <agraf@suse.de> wrote:

> >> Same here - in fact i first asked Qemu to be put into 
> >> tools/qemu/ so that it all becomes more hackable and more usable 
> >> - that suggestion was rebuked very strongly.
> >
> > So instead of thinking a bit and trying to realize that there 
> > might be a reason people don't want all their user space in the 
> > kernel tree you go ahead and start your own crusade of creating a 
> > new user space. Great. That's how I always hoped Linux would be 
> > :(.
> 
> Damn you Ingo Molnar, I knew you'd somehow get all the credit for 
> our hard work! ;-)

Yeah, i'm really just a user and code reviewer of tools/kvm/, in 
terms of actual commits i'm barely on the radar compared to the 
massive contributions of Pekka, Sasha, Asias, Cyrill and others.

> More seriously, though, I fail to see what's bothering you 
> Alexander. I and Ingo already mentioned we wouldn't be hacking on 
> Qemu even if there wasn't no tools/kvm. It's not as if we're 
> putting *your* user space code into the kernel tree - we wrote our 
> own! What's wrong with that?

That puzzles me as well.

If tools/kvm/ is really such a bad approach then it will just wither 
and die, like one of the thousand stale drivers we have in the kernel 
- or like lguest it ends up as a cool hack that didnt quite make it.

Judging by the many contributions tools/kvm/ has already attracted in 
its limited form an out of tree project i'd not bet on that outcome 
though.

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  8:37             ` Pekka Enberg
  2011-07-25  8:44               ` Alexander Graf
@ 2011-07-25  9:16               ` Ingo Molnar
  1 sibling, 0 replies; 70+ messages in thread
From: Ingo Molnar @ 2011-07-25  9:16 UTC (permalink / raw)
  To: Pekka Enberg
  Cc: Alexander Graf, Jan Kiszka, torvalds, avi, akpm, linux-kernel,
	kvm, gorcunov, levinsasha928, asias.hejun, prasadjoshi124


* Pekka Enberg <penberg@kernel.org> wrote:

> Hi Alexander,
> 
> On Mon, Jul 25, 2011 at 11:31 AM, Alexander Graf <agraf@suse.de> wrote:
> >> Damn you Ingo Molnar, I knew you'd somehow get all the credit for our
> >> hard work! ;-)
> >
> > Well, IIUC he's the one initiating the whole thing, no?
> 
> As much as I appreciate Ingo's help and support with the project, 
> no, I don't consider him to have initiated the whole thing. [...]

No, i havent - and if Alexander pulls the URI and looks at the 
changelogs he'll realize that fact too.

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  8:47             ` Pekka Enberg
  2011-07-25  9:06               ` Alexander Graf
@ 2011-07-25  9:26               ` Ingo Molnar
  2011-07-25  9:42                 ` Alexander Graf
  1 sibling, 1 reply; 70+ messages in thread
From: Ingo Molnar @ 2011-07-25  9:26 UTC (permalink / raw)
  To: Pekka Enberg
  Cc: Alexander Graf, Jan Kiszka, torvalds, avi, akpm, linux-kernel,
	kvm, gorcunov, levinsasha928, asias.hejun, prasadjoshi124


* Pekka Enberg <penberg@kernel.org> wrote:

> [ I thought the 'native Linux' part in 'native Linux KVM tool' was 
> a dead giveaway, really. ]
> 
> Now if people want to support other operating systems, that's cool 
> and I'm happy to help out where I can. But I don't understand why 
> people keep bringing non-Linux OSs as an argument for not merging 
> tools/kvm into the Linux kernel tree. I mean really, did someone 
> actually expect that a Linux kernel developer spends his weekends 
> improving the state of Windows virtualization?
> 
> And don't get this the wrong way either, I'm not hostile against 
> other operating systems, but I simply am not interested enough in 
> them to spend my time improving them.

In fact one of the problems i see with Qemu is that Qemu had to make 
many compromises to support Windows and other weird platforms that 
i'm (and i'd claim most other Linux kernel developers) are personally 
not interested in.

So here's a 14 KLOC tools/kvm/ that does everything that i need from 
virtualization (easy testing of a bzImage before rebooting the host 
system into it), is clean, readable and hackable and is 1.5% of the 
nearly 1 MLOC Qemu code base.

The whole tools/kvm/ code base is (much) smaller than Qemu's IA64 
support code for example. The size difference is startling. 

tools/kvm/ does less and in my experience does it better - is that 
such a surprising thing?

So it was a no brainer for me to pull it into -tip.

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  9:03           ` Alexander Graf
@ 2011-07-25  9:41             ` Ingo Molnar
  2011-07-25  9:46               ` Alexander Graf
  2011-07-25  9:48               ` Avi Kivity
  2011-07-25 11:26             ` Olivier Galibert
  1 sibling, 2 replies; 70+ messages in thread
From: Ingo Molnar @ 2011-07-25  9:41 UTC (permalink / raw)
  To: Alexander Graf
  Cc: Pekka Enberg, Jan Kiszka, torvalds, avi, akpm, linux-kernel, kvm,
	gorcunov, levinsasha928, asias.hejun, prasadjoshi124


* Alexander Graf <agraf@suse.de> wrote:

> On 25.07.2011, at 10:54, Ingo Molnar wrote:
> 
> > 
> > * Alexander Graf <agraf@suse.de> wrote:
> > 
> >> On 25.07.2011, at 09:53, Ingo Molnar wrote:
> >> 
> >>> 
> >>> * Pekka Enberg <penberg@kernel.org> wrote:
> >>> 
> >>>> On Mon, Jul 25, 2011 at 2:12 AM, Jan Kiszka <jan.kiszka@web.de> wrote:
> >>>>> 
> >>>>> That said, I definitely appreciate the bug fixes as well as 
> >>>>> code and documentation improvements for KVM that originate from 
> >>>>> this effort! I'm just not convinced that writing a new userland 
> >>>>> and merging it into the kernel is the most efficient way to 
> >>>>> achieve that.
> >>>> 
> >>>> Just to make this crystal clear for everyone: if it weren't for 
> >>>> tools/kvm, I wouldn't be hacking on KVM at all. I've looked at 
> >>>> Qemu in the past (and a lot recently!) and I simply don't see 
> >>>> myself contributing to it, sorry. So 'most efficient' or not, I 
> >>>> think tools/kvm is a net win for Linux and KVM in general.
> >>> 
> >>> Same here - in fact i first asked Qemu to be put into tools/qemu/ 
> >>> so that it all becomes more hackable and more usable - that 
> >>> suggestion was rebuked very strongly.
> >> 
> >> So instead of thinking a bit and trying to realize that there might 
> >> be a reason people don't want all their user space in the kernel 
> >> tree you go ahead and start your own crusade of creating a new user 
> >> space. Great. That's how I always hoped Linux would be :(.
> > 
> > Not "all their user space" but the ones that are tightly 
> > integrated with the kernel to begin with. How hard is that 
> > concept to understand?
> 
> It's very hard to understand. It's similar to religion - I could 
> easily apply your point to every reasonably low-level user space 
> project out there. X for example. X needs to interact with KMS and 
> DRI and whatdoiknow. So it'd be a perfect fit to pull into tools/, 
> no?

If the graphics folks feel comfortable with that approach then yes, i 
think graphics would be a good example as well - in my experience a 
lot of the user-space pain related to graphics development comes from 
ugly version friction between various graphics components, and all 
the APIs are pretty fluid and heavily developed - which is easier to 
do within a single code repository.

But if the Xorg/graphics folks want it separate it's their decision 
and they've said it in the past that they like the current 
modularization.

If someone comes with tools/X11/ that actually *works* and provides a 
usable X11 replacement i sure would try it out and would likely 
contribute to it.

Basic infrastructure and tooling - many projects could apply but the 
most obvious choices are ones that relate to and depend on the 
kernel.

Browsers, email clients, GIMP, games, LibreOffice, not so much.

There's no clear line but that's not a problem - there are seldom any 
clear lines in life. It's a case by case issue and very much depends 
on the attributes of the project and the preferences of the 
developers who are driving it.

I can tell you my first hand experience: for tools/perf/ and 
tools/kvm/ it was highly beneficial to be developed in the kernel 
repo.

I don't see how you can validly bring religion into this discussion.

> > Virtualization is very tightly bound to the kernel, like it or 
> > not.
> 
> I don't see why. The whole point of virtualization is to model 
> simple, with KVM preferably even very-close-to-real-hardware 
> interfaces that allow you to keep things separate.

Look at tools/kvm/ - i cannot do anything useful without a Linux 
kernel under it. It's as deeply bound to the Linux kernel as it gets.

Then look at the actual drivers and interfaces within tools/kvm/. 
It's using the same symbols and conventions for 'guest' and 'host' 
side.

Check out tools/kvm/hw/i8042.c and match it up with 
include/linux/serio.h and drivers/input/serio/i8042.c - you can 
literally walk from one side to the other and understand how guest 
and host are tightly related not just functionality but also 
implementation wise.

This is how Qemu should be doing it as well btw., to ease the 
debugging of host/guest interaction bugs and to ease development.

> > So is profiling, power management and a few other things.
> 
> I'm also failing to see the reasoning here TBH.

You need to quote the full argument to see the reason in it:

> > Virtualization is very tightly bound to the kernel, like it or 
> > not. So is profiling, power management and a few other things.

It's a very simple point and observation: tools which integrate to 
the kernel so that they wouldnt even run on another kernel obviously 
are very natural to develop in tools/.

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  9:26               ` Ingo Molnar
@ 2011-07-25  9:42                 ` Alexander Graf
  2011-07-25 10:16                   ` Ingo Molnar
  0 siblings, 1 reply; 70+ messages in thread
From: Alexander Graf @ 2011-07-25  9:42 UTC (permalink / raw)
  To: Ingo Molnar
  Cc: Pekka Enberg, Jan Kiszka, torvalds, avi, akpm, linux-kernel, kvm,
	gorcunov, levinsasha928, asias.hejun, prasadjoshi124


On 25.07.2011, at 11:26, Ingo Molnar wrote:

> 
> * Pekka Enberg <penberg@kernel.org> wrote:
> 
>> [ I thought the 'native Linux' part in 'native Linux KVM tool' was 
>> a dead giveaway, really. ]
>> 
>> Now if people want to support other operating systems, that's cool 
>> and I'm happy to help out where I can. But I don't understand why 
>> people keep bringing non-Linux OSs as an argument for not merging 
>> tools/kvm into the Linux kernel tree. I mean really, did someone 
>> actually expect that a Linux kernel developer spends his weekends 
>> improving the state of Windows virtualization?
>> 
>> And don't get this the wrong way either, I'm not hostile against 
>> other operating systems, but I simply am not interested enough in 
>> them to spend my time improving them.
> 
> In fact one of the problems i see with Qemu is that Qemu had to make 
> many compromises to support Windows and other weird platforms that 
> i'm (and i'd claim most other Linux kernel developers) are personally 
> not interested in.

It's what makes it so powerful. Adding a new architecture for KVM for example is as easy as only implementing the CPU. All device emulation is already there. If you want something Linux only, lguest would've been enough, no?

The only real problem I see with Qemu is that it's been born in a time of uniprocessor systems. Making it fully threaded as an after-thought is insanely hard.

> So here's a 14 KLOC tools/kvm/ that does everything that i need from 
> virtualization (easy testing of a bzImage before rebooting the host 
> system into it), is clean, readable and hackable and is 1.5% of the 
> nearly 1 MLOC Qemu code base.

Sure, we'll talk again when you can run Android for ARM on kvm-tool.

> The whole tools/kvm/ code base is (much) smaller than Qemu's IA64 
> support code for example. The size difference is startling. 

That's the host code compiler. Qemu doesn't have IA64 guest support.

> tools/kvm/ does less and in my experience does it better - is that 
> such a surprising thing?

I already stated in a few mails before and also in the last discussion that I'm in general in favor of having multiple user space targets for KVM. In fact, I wrote 2 backends (both for PPC) myself. Though adding those 2 backends to their respective projects probably provided a lot more value-add (features that qemu didn't already have) than kvm-tool does atm. Kvm-tool really just smells a lot like NIH. Which is fine - that's why we have KDE and Gnome, right? ;)

> So it was a no brainer for me to pull it into -tip.

The thing I don't agree with is that it should live in the kernel tree.


Alex

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  9:41             ` Ingo Molnar
@ 2011-07-25  9:46               ` Alexander Graf
  2011-07-25 10:28                 ` Ingo Molnar
  2011-07-25  9:48               ` Avi Kivity
  1 sibling, 1 reply; 70+ messages in thread
From: Alexander Graf @ 2011-07-25  9:46 UTC (permalink / raw)
  To: Ingo Molnar
  Cc: Pekka Enberg, Jan Kiszka, torvalds, avi, akpm, linux-kernel, kvm,
	gorcunov, levinsasha928, asias.hejun, prasadjoshi124


On 25.07.2011, at 11:41, Ingo Molnar wrote:

>>> Virtualization is very tightly bound to the kernel, like it or 
>>> not. So is profiling, power management and a few other things.
> 
> It's a very simple point and observation: tools which integrate to 
> the kernel so that they wouldnt even run on another kernel obviously 
> are very natural to develop in tools/.

Ah, very good. So all we need to do to prove the point that kvm-tool doesn't belong in tools/ is port KVM to another OS and make kvm-tool compile there too? Shouldn't be too hard. People already have working ports of (old) KVM versions on FreeBSD and Windows.


Alex


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  9:41             ` Ingo Molnar
  2011-07-25  9:46               ` Alexander Graf
@ 2011-07-25  9:48               ` Avi Kivity
  2011-07-25 10:03                 ` Ingo Molnar
  1 sibling, 1 reply; 70+ messages in thread
From: Avi Kivity @ 2011-07-25  9:48 UTC (permalink / raw)
  To: Ingo Molnar
  Cc: Alexander Graf, Pekka Enberg, Jan Kiszka, torvalds, akpm,
	linux-kernel, kvm, gorcunov, levinsasha928, asias.hejun,
	prasadjoshi124

On 07/25/2011 12:41 PM, Ingo Molnar wrote:
> Look at tools/kvm/ - i cannot do anything useful without a Linux
> kernel under it. It's as deeply bound to the Linux kernel as it gets.

The actual code that interacts with the kernel is pretty small, and will 
grow smaller (as a fraction) in time.

> Then look at the actual drivers and interfaces within tools/kvm/.
> It's using the same symbols and conventions for 'guest' and 'host'
> side.
>
> Check out tools/kvm/hw/i8042.c and match it up with
> include/linux/serio.h and drivers/input/serio/i8042.c - you can
> literally walk from one side to the other and understand how guest
> and host are tightly related not just functionality but also
> implementation wise.
>
> This is how Qemu should be doing it as well btw., to ease the
> debugging of host/guest interaction bugs and to ease development.

No.  That ties the guest and host interfaces together.  If you change 
the guest to use a feature you didn't implement previously in the host, 
it's suddenly broken when virtualized.

The right thing is to implement the hardware interface exactly per spec, 
and ignore the guest(s) completely except for testing and performance.


-- 
error compiling committee.c: too many arguments to function


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  9:48               ` Avi Kivity
@ 2011-07-25 10:03                 ` Ingo Molnar
  2011-07-25 10:17                   ` Avi Kivity
  0 siblings, 1 reply; 70+ messages in thread
From: Ingo Molnar @ 2011-07-25 10:03 UTC (permalink / raw)
  To: Avi Kivity
  Cc: Alexander Graf, Pekka Enberg, Jan Kiszka, torvalds, akpm,
	linux-kernel, kvm, gorcunov, levinsasha928, asias.hejun,
	prasadjoshi124


* Avi Kivity <avi@redhat.com> wrote:

> On 07/25/2011 12:41 PM, Ingo Molnar wrote:

> > Look at tools/kvm/ - i cannot do anything useful without a Linux 
> > kernel under it. It's as deeply bound to the Linux kernel as it 
> > gets.
> 
> The actual code that interacts with the kernel is pretty small, and 
> will grow smaller (as a fraction) in time.

Look at the example i have given in my other mail, conceptual 
interaction goes beyond the strict KVM (and hardware) ABIs themselves 
- for every guest side driver there's a host side piece of 
functionality.

What we found is that there's real benefits from tools/kvm/hw/i8042.c 
using the same conventions and constants as 
drivers/input/serio/i8042.c.

The two pieces of code are obviously independent ABI-wise - but to 
developers working on both sides of the fence it's very useful to 
have the code look similar and familiar.

Some other pieces of code share library functionality between the 
kernel and tools/kvm/.

> > Then look at the actual drivers and interfaces within tools/kvm/. 
> > It's using the same symbols and conventions for 'guest' and 
> > 'host' side.
> >
> > Check out tools/kvm/hw/i8042.c and match it up with 
> > include/linux/serio.h and drivers/input/serio/i8042.c - you can 
> > literally walk from one side to the other and understand how 
> > guest and host are tightly related not just functionality but 
> > also implementation wise.
> >
> > This is how Qemu should be doing it as well btw., to ease the 
> > debugging of host/guest interaction bugs and to ease development.
> 
> No. That ties the guest and host interfaces together. [...]

Why would it tie the interfaces together? For a guest and host driver 
to be written in the same familiar conventions is useful for 
debugging and development easier but does not prevent both of those 
pieces to implement a precise interface. (Especially since in the 
i8042 case the host driver is used with real hardware as well.)

> [...]  If you change the guest to use a feature you didn't 
> implement previously in the host, it's suddenly broken when 
> virtualized.
> 
> The right thing is to implement the hardware interface exactly per 
> spec, and ignore the guest(s) completely except for testing and 
> performance.

You can certainly do that too - and nothing prevents from the guest 
and host driver to look and feel similar.

But note that for many of these things there no such thing as a 
'hardware spec' that describes all the details and quirks - what we 
have are host PC drivers that have been implemented via many cycles 
of trial and error. That's especially true of i8042, the example i 
have given.

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  9:42                 ` Alexander Graf
@ 2011-07-25 10:16                   ` Ingo Molnar
  2011-07-25 10:21                     ` Alexander Graf
  0 siblings, 1 reply; 70+ messages in thread
From: Ingo Molnar @ 2011-07-25 10:16 UTC (permalink / raw)
  To: Alexander Graf
  Cc: Pekka Enberg, Jan Kiszka, torvalds, avi, akpm, linux-kernel, kvm,
	gorcunov, levinsasha928, asias.hejun, prasadjoshi124


* Alexander Graf <agraf@suse.de> wrote:

> > In fact one of the problems i see with Qemu is that Qemu had to 
> > make many compromises to support Windows and other weird 
> > platforms that i'm (and i'd claim most other Linux kernel 
> > developers) are personally not interested in.
> 
> It's what makes it so powerful. [...]

To me and Pekka that is what made Qemu unhackable.

Really, i'm not sure why you are arguing here. We are not trying to 
merge tools/qemu/ upstream. We are trying to merge a Linux-only 
utility that lives in the kernel tree today and which we are actively 
using and developing.

> [...] Adding a new architecture for KVM for example is as easy as 
> only implementing the CPU. All device emulation is already there. 
> If you want something Linux only, lguest would've been enough, no?

That's a rather bizarre argument, we were pretty happy with the 
design of the KVM host side, what we wanted to improve was user-space 
tooling.

With lguest we'd have to write a new host implementation in essence 
...

> > [...]
> >
> > tools/kvm/ does less and in my experience does it better - is 
> > that such a surprising thing?
>
> [...] 
> 
> > So it was a no brainer for me to pull it into -tip.
> 
> The thing I don't agree with is that it should live in the kernel 
> tree.

FYI, tools/kvm/ *already* lives in the kernel tree - that is how it's 
developed and used and it also shares code with the kernel.

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25 10:03                 ` Ingo Molnar
@ 2011-07-25 10:17                   ` Avi Kivity
  2011-07-25 10:29                     ` Ingo Molnar
  0 siblings, 1 reply; 70+ messages in thread
From: Avi Kivity @ 2011-07-25 10:17 UTC (permalink / raw)
  To: Ingo Molnar
  Cc: Alexander Graf, Pekka Enberg, Jan Kiszka, torvalds, akpm,
	linux-kernel, kvm, gorcunov, levinsasha928, asias.hejun,
	prasadjoshi124

On 07/25/2011 01:03 PM, Ingo Molnar wrote:
> >  >  Then look at the actual drivers and interfaces within tools/kvm/.
> >  >  It's using the same symbols and conventions for 'guest' and
> >  >  'host' side.
> >  >
> >  >  Check out tools/kvm/hw/i8042.c and match it up with
> >  >  include/linux/serio.h and drivers/input/serio/i8042.c - you can
> >  >  literally walk from one side to the other and understand how
> >  >  guest and host are tightly related not just functionality but
> >  >  also implementation wise.
> >  >
> >  >  This is how Qemu should be doing it as well btw., to ease the
> >  >  debugging of host/guest interaction bugs and to ease development.
> >
> >  No. That ties the guest and host interfaces together. [...]
>
> Why would it tie the interfaces together? For a guest and host driver
> to be written in the same familiar conventions is useful for
> debugging and development easier but does not prevent both of those
> pieces to implement a precise interface. (Especially since in the
> i8042 case the host driver is used with real hardware as well.)

The driver is under no requirement to use all of the functionality of 
the device.  On the other hand, the device emulation has to be complete, 
or it risks not working with some past or future version of the driver, 
or with another OS (if you support that).

> >  [...]  If you change the guest to use a feature you didn't
> >  implement previously in the host, it's suddenly broken when
> >  virtualized.
> >
> >  The right thing is to implement the hardware interface exactly per
> >  spec, and ignore the guest(s) completely except for testing and
> >  performance.
>
> You can certainly do that too - and nothing prevents from the guest
> and host driver to look and feel similar.
>
> But note that for many of these things there no such thing as a
> 'hardware spec' that describes all the details and quirks - what we
> have are host PC drivers that have been implemented via many cycles
> of trial and error. That's especially true of i8042, the example i
> have given.

The device need not implement any quirks - it can implement the spec 
exactly.  As long as the driver can drive a standard device, all is fine.

Specs for the 8042 certainly exist.

-- 
error compiling committee.c: too many arguments to function


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25 10:16                   ` Ingo Molnar
@ 2011-07-25 10:21                     ` Alexander Graf
  2011-07-25 10:35                       ` Ingo Molnar
  0 siblings, 1 reply; 70+ messages in thread
From: Alexander Graf @ 2011-07-25 10:21 UTC (permalink / raw)
  To: Ingo Molnar
  Cc: Pekka Enberg, Jan Kiszka, torvalds, avi, akpm, linux-kernel, kvm,
	gorcunov, levinsasha928, asias.hejun, prasadjoshi124


On 25.07.2011, at 12:16, Ingo Molnar wrote:

>> 
>>> So it was a no brainer for me to pull it into -tip.
>> 
>> The thing I don't agree with is that it should live in the kernel 
>> tree.
> 
> FYI, tools/kvm/ *already* lives in the kernel tree - that is how it's 
> developed and used and it also shares code with the kernel.

So you're just trying to make this a self-fulfilling prophecy now? I already stated before that I could easily add those 'requirements' to any project I liked. That still wouldn't mean it's a good idea to clutter the _kernel_ repository with its code. It's not a kernel.


Alex


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  9:46               ` Alexander Graf
@ 2011-07-25 10:28                 ` Ingo Molnar
  0 siblings, 0 replies; 70+ messages in thread
From: Ingo Molnar @ 2011-07-25 10:28 UTC (permalink / raw)
  To: Alexander Graf
  Cc: Pekka Enberg, Jan Kiszka, torvalds, avi, akpm, linux-kernel, kvm,
	gorcunov, levinsasha928, asias.hejun, prasadjoshi124


* Alexander Graf <agraf@suse.de> wrote:

> On 25.07.2011, at 11:41, Ingo Molnar wrote:
> 
> >>> Virtualization is very tightly bound to the kernel, like it or 
> >>> not. So is profiling, power management and a few other things.
> > 
> > It's a very simple point and observation: tools which integrate 
> > to the kernel so that they wouldnt even run on another kernel 
> > obviously are very natural to develop in tools/.
> 
> Ah, very good. So all we need to do to prove the point that 
> kvm-tool doesn't belong in tools/ is port KVM to another OS and 
> make kvm-tool compile there too? Shouldn't be too hard. People 
> already have working ports of (old) KVM versions on FreeBSD and 
> Windows.

Firstly, tools/kvm/ itself only works on Linux and it's developed in 
the kernel repo and we see many benefits of that model and are happy 
with it.

Secondly, even your argument is rather inconsistent: by your argument 
what keeps KVM itself within the Linux kernel, if it's so portable to 
FreeBSD and Windows? By your argument all the virtio drivers should 
be moved out of the Linux kernel tree, to support both the FreeBSD, 
Windows and Linux KVM implementations. By your argument the arch/x86/ 
KVM disassembler should move out of the kernel tree, etc. etc.

You cannot have it both ways really.

So yes, i disagree with you rigid views about this, in reality what 
matters is the quality of the end result and the preferences of the 
developers. Just like i cannot (and should not) force you to develop 
in tools/qemu/, should you not try to force me to not develop in 
tools/kvm/.

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25 10:17                   ` Avi Kivity
@ 2011-07-25 10:29                     ` Ingo Molnar
  0 siblings, 0 replies; 70+ messages in thread
From: Ingo Molnar @ 2011-07-25 10:29 UTC (permalink / raw)
  To: Avi Kivity
  Cc: Alexander Graf, Pekka Enberg, Jan Kiszka, torvalds, akpm,
	linux-kernel, kvm, gorcunov, levinsasha928, asias.hejun,
	prasadjoshi124


* Avi Kivity <avi@redhat.com> wrote:

> On 07/25/2011 01:03 PM, Ingo Molnar wrote:
> >>  >  Then look at the actual drivers and interfaces within tools/kvm/.
> >>  >  It's using the same symbols and conventions for 'guest' and
> >>  >  'host' side.
> >>  >
> >>  >  Check out tools/kvm/hw/i8042.c and match it up with
> >>  >  include/linux/serio.h and drivers/input/serio/i8042.c - you can
> >>  >  literally walk from one side to the other and understand how
> >>  >  guest and host are tightly related not just functionality but
> >>  >  also implementation wise.
> >>  >
> >>  >  This is how Qemu should be doing it as well btw., to ease the
> >>  >  debugging of host/guest interaction bugs and to ease development.
> >>
> >>  No. That ties the guest and host interfaces together. [...]
> >
> > Why would it tie the interfaces together? For a guest and host 
> > driver to be written in the same familiar conventions is useful 
> > for debugging and development easier but does not prevent both of 
> > those pieces to implement a precise interface. (Especially since 
> > in the i8042 case the host driver is used with real hardware as 
> > well.)
> 
> The driver is under no requirement to use all of the functionality 
> of the device.  On the other hand, the device emulation has to be 
> complete, or it risks not working with some past or future version 
> of the driver, or with another OS (if you support that).

Sure, and nothing in tools/kvm/hw/i8042.c contradicts that and my 
points still remain.

You can have clean, harmonic code *and* have it work to spec, agreed?

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25 10:21                     ` Alexander Graf
@ 2011-07-25 10:35                       ` Ingo Molnar
  0 siblings, 0 replies; 70+ messages in thread
From: Ingo Molnar @ 2011-07-25 10:35 UTC (permalink / raw)
  To: Alexander Graf
  Cc: Pekka Enberg, Jan Kiszka, torvalds, avi, akpm, linux-kernel, kvm,
	gorcunov, levinsasha928, asias.hejun, prasadjoshi124


* Alexander Graf <agraf@suse.de> wrote:

> 
> On 25.07.2011, at 12:16, Ingo Molnar wrote:
> 
> >> 
> >>> So it was a no brainer for me to pull it into -tip.
> >> 
> >> The thing I don't agree with is that it should live in the 
> >> kernel tree.
> > 
> > FYI, tools/kvm/ *already* lives in the kernel tree - that is how 
> > it's developed and used and it also shares code with the kernel.
> 
> So you're just trying to make this a self-fulfilling prophecy now? 

No, i'm just trying to point out to you that it's already a fact that 
the tool already lives in a kernel tree and was developed there to 
the tune of 700+ commits within a few short months.

IMO you should not be arguing:

 'You should do X because that's my preference.'

IMO you should be arguing:

 'You should do X because technical advantage Y which is important to the project.'

We have listed numerous technical advantages of why we have chosen 
tools/kvm/ - and you have not countered that with any convincing 
technical disadvantage so far.

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  8:14       ` Alexander Graf
                           ` (2 preceding siblings ...)
  2011-07-25  8:54         ` Ingo Molnar
@ 2011-07-25 10:38         ` Christoph Hellwig
  2011-07-25 11:08           ` Ingo Molnar
                             ` (2 more replies)
  3 siblings, 3 replies; 70+ messages in thread
From: Christoph Hellwig @ 2011-07-25 10:38 UTC (permalink / raw)
  To: Alexander Graf
  Cc: Ingo Molnar, Pekka Enberg, Jan Kiszka, torvalds, avi, akpm,
	linux-kernel, kvm, gorcunov, levinsasha928, asias.hejun,
	prasadjoshi124

On Mon, Jul 25, 2011 at 10:14:13AM +0200, Alexander Graf wrote:
> So instead of thinking a bit and trying to realize that there might be a reason people don't want all their user space in the kernel tree you go ahead and start your own crusade of creating a new user space. Great. That's how I always hoped Linux would be :(.

It's not Linux in general yet, it's mostly a crusade of a few with a
political agenda.

> > So i wanted to have a lightweight tool that allows me to test KVM and 
> > tools/kvm/ does that very nicely: i type './kvm run' and i can test a 
> > native bzImage (which has some virtualization options enabled as 
> > well) on the _host_ distro i am running, booting to a text shell 
> > prompt.
> 
> I do that all the time.
> 
>   $ qemu-kvm -nographic -kernel arch/x86/boot/bzImage -append console=ttyS0

Same here.  I can't be bothered with all the stuid distro booting crap.

> > I can do that without downloading any (inevitably outdated) 
> > virtualization images or maintaining my own ones. Maintaining host 
> > userspace is more than enough for me.
> 
> Who would need images? I usually only run -kernel and -initrd directly to test out things. Or if I really want to boot into a real system I do -snapshot /dev/sda.

Indeed.

> 
> > So, since we already have the lguest tool in the kernel tree, why 
> > cannot we have the much more capable tools/kvm/ in the tree?
> 
> Lguest is in Documentation/ for a reason. It's not meant as a user space tool that you take as-is and use. It's meant for documenting how lguest works in general. I admit though, that that's also the reason people don't use it :).

I'd also say it's rather misplaced there, and at least in the storage
area that I know most it didn't help it from totally misunderstanding
kernel concepts and taking them into protocols (e.g. virtio barrier
support).  That for me is a reason why you don't want to couple thing
too tightly, at least you'll have to document and/or explain the
protocol to someone. 
tight

And another argument, calling toyvisor2 "kvm" is a really bad idea.  The
kvm binary has been used for the kvm-patched qemu binary for quite a
while in various distros, so re-using that name will cause utter
confusion.

I'm happy that you guys do another independent userspace for kvm, but
please:

  a) give it a useful name
  b) just develop it where it belongs, your own little git repository
     somewhere

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  8:37           ` Alexander Graf
  2011-07-25  8:47             ` Pekka Enberg
@ 2011-07-25 10:47             ` Ingo Molnar
  1 sibling, 0 replies; 70+ messages in thread
From: Ingo Molnar @ 2011-07-25 10:47 UTC (permalink / raw)
  To: Alexander Graf
  Cc: Pekka Enberg, Jan Kiszka, torvalds, avi, akpm, linux-kernel, kvm,
	gorcunov, levinsasha928, asias.hejun, prasadjoshi124


* Alexander Graf <agraf@suse.de> wrote:

> > You know, they said the same thing about oprofile. All you needed 
> > to do was to write few simple shell scripts to make it work. One 
> > of the key features of tools/kvm is 'as little configuration as 
> > possible' and I can assure you that bash alias is really not a 
> > solution for that.
> 
> I like perf. Really. But I still don't see why it wouldn't be able 
> to live in its own tree either.

Is the reason that the people who develop it prefer integration with 
the kernel tree not enough for you?

perf could possibly be ported to other OSs. Maybe some day someone 
will try that. But unless that project actually replaces the perf 
project, or perf developers move out of the kernel en masse due to 
difficulties with the development model i don't see why the project 
would want to move out of the kernel tree.

In fact my observations as a perf maintainer show the exact opposite: 
most perf developers are 100% happy to get their stuff merged and 
upstream ASAP. They do not buffer big patch-queues just to not have 
to deal with an integrated kernel tree. The integrated tree is a 
natural model of development to them and often perf tooling patches 
come mixed with kernel side patches such as new tracepoints or 
cleanups/fixes to related kernel code, so it's all very convenient.

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25 10:38         ` Christoph Hellwig
@ 2011-07-25 11:08           ` Ingo Molnar
  2011-07-25 11:24             ` Christoph Hellwig
  2011-07-25 12:09           ` Ingo Molnar
  2011-07-25 12:36           ` Pekka Enberg
  2 siblings, 1 reply; 70+ messages in thread
From: Ingo Molnar @ 2011-07-25 11:08 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: Alexander Graf, Pekka Enberg, Jan Kiszka, torvalds, avi, akpm,
	linux-kernel, kvm, gorcunov, levinsasha928, asias.hejun,
	prasadjoshi124


* Christoph Hellwig <hch@infradead.org> wrote:

> > > So, since we already have the lguest tool in the kernel tree, 
> > > why cannot we have the much more capable tools/kvm/ in the 
> > > tree?
> > 
> > Lguest is in Documentation/ for a reason. It's not meant as a 
> > user space tool that you take as-is and use. It's meant for 
> > documenting how lguest works in general. I admit though, that 
> > that's also the reason people don't use it :).
> 
> I'd also say it's rather misplaced there, and at least in the 
> storage area that I know most it didn't help it from totally 
> misunderstanding kernel concepts and taking them into protocols 
> (e.g. virtio barrier support).  That for me is a reason why you 
> don't want to couple thing too tightly, at least you'll have to 
> document and/or explain the protocol to someone.

It's funny that you bring up ABIs and virtio, as in my experience 
it's usually this pattern that creates bad ABIs in Linux:

 1) there's an out of tree user-space project, written by a group of 
    developers, proposing some new feature for which they'd need 
    kernel help

 2) they talk to another set of Linux kernel developers, who come up 
    with something that works - still out of tree

 3) they then talk to upstream Linux and the whole review process 
    starts. The ABI changes in some minor ways but often survives.

 4) nobody is risking any ABI changes because the distance from 
    upstream to the actual user-space project is 3 boundaries. 
    Whatever ad-hoc ABI was proposed initially is hammered through
    if possible, unless it's 100% unworkable.

 5) upstream often bends for pragmatic reasons, and it's at most 
    someone at the top of the maintenance hierarchy who says
    'hell NO!' and forces an (expensive!) reiteration of all 5 steps.

I've also seen a couple of good ABIs, which had a very natural 
development cycle:

 1) developers working both upstream and in a user-space project 
    propose an ABI and quickly iterate through several versions. Both 
    the kernel side and the user-space side changes frequently and 
    iteratively to perfect the ABI and the whole process is highly 
    visible on lkml and gets review at every stage of this work.

Do you recognize what i'm trying to point out?

While moving a user-space tool project to tools/ certainly does not 
*guarantee* good ABIs, it *does* guarantee friction-less iterations 
of ABIs - which are much harder to achieve with out of tree projects.

We've done this numerous times for the perf ABI and while the ABI is 
far from perfect it worked very well for us - far better than it 
would have worked as a separate project.

In such an integrated tool/kernel tree bugs are also much easier to 
debug: as we can cleanly bisect across interim versions of the ABI, 
as the tools and the kernel side ABI is developed in lock-step. We do 
not have to create cross-compatibility between every interim version 
of the ABI and any future (or past) version of the tool ...

So yes, my first hand experience shows that you are 100% wrong:

 - 'slow-play', out of tree, modularized, to-the-specification ABIs 
   tend to suck

 - 'quickly iterated', in tree, unified, specified-on-lkml ABIs tend 
   to work out much better

There's exceptions, but that's the general trend.

Fact is that developing ABIs within an integrated project is 
*amazingly* powerful. You should try it one day, instead of 
criticizing it :-)

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25 11:08           ` Ingo Molnar
@ 2011-07-25 11:24             ` Christoph Hellwig
  2011-07-25 11:32               ` Ingo Molnar
  2011-07-25 11:34               ` Olivier Galibert
  0 siblings, 2 replies; 70+ messages in thread
From: Christoph Hellwig @ 2011-07-25 11:24 UTC (permalink / raw)
  To: Ingo Molnar
  Cc: Christoph Hellwig, Alexander Graf, Pekka Enberg, Jan Kiszka,
	torvalds, avi, akpm, linux-kernel, kvm, gorcunov, levinsasha928,
	asias.hejun, prasadjoshi124

On Mon, Jul 25, 2011 at 01:08:10PM +0200, Ingo Molnar wrote:
> Fact is that developing ABIs within an integrated project is 
> *amazingly* powerful. You should try it one day, instead of 
> criticizing it :-)

I've been doing this long before you declare it the rosetta stone.  Some
of the worst ABIs I know come from that kind of development, e.g. all
the ioctls we have in xfs, inherited from the "tightly integrated" IRIX
development cycle.


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  9:03           ` Alexander Graf
  2011-07-25  9:41             ` Ingo Molnar
@ 2011-07-25 11:26             ` Olivier Galibert
  1 sibling, 0 replies; 70+ messages in thread
From: Olivier Galibert @ 2011-07-25 11:26 UTC (permalink / raw)
  To: Alexander Graf
  Cc: Ingo Molnar, Pekka Enberg, Jan Kiszka, torvalds, avi, akpm,
	linux-kernel, kvm, gorcunov, levinsasha928, asias.hejun,
	prasadjoshi124

On Mon, Jul 25, 2011 at 11:03:32AM +0200, Alexander Graf wrote:
> It's very hard to understand. It's similar to religion - I could
> easily apply your point to every reasonably low-level user space
> project out there. X for example. X needs to interact with KMS and
> DRI and whatdoiknow. So it'd be a perfect fit to pull into tools/,
> no?

The current development methodology of X11/mesa/v*api is catastrophic,
as the state of the linux graphics attest.  Having a functional
equivalent in tools would be an immense step forward, if simply by
having all the damn related code in the same place, but also because
of the interactions you cite.

  OG.


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25 11:24             ` Christoph Hellwig
@ 2011-07-25 11:32               ` Ingo Molnar
  2011-07-25 11:34               ` Olivier Galibert
  1 sibling, 0 replies; 70+ messages in thread
From: Ingo Molnar @ 2011-07-25 11:32 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: Alexander Graf, Pekka Enberg, Jan Kiszka, torvalds, avi, akpm,
	linux-kernel, kvm, gorcunov, levinsasha928, asias.hejun,
	prasadjoshi124


* Christoph Hellwig <hch@infradead.org> wrote:

> On Mon, Jul 25, 2011 at 01:08:10PM +0200, Ingo Molnar wrote:
>
> > Fact is that developing ABIs within an integrated project is 
> > *amazingly* powerful. You should try it one day, instead of 
> > criticizing it :-)
> 
> I've been doing this long before you declare it the rosetta stone.  
> Some of the worst ABIs I know come from that kind of development, 
> e.g. all the ioctls we have in xfs, inherited from the "tightly 
> integrated" IRIX development cycle.

As i said it's not a guarantee, you can mess up even under the best 
of circumstances.

Also, arguably a filesystem is constrainted wrt. ABIs: it's just one 
filesystem which pretty much pushes it towards ioctls which in turn 
is often an unstructured breeding ground for mess.

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25 11:24             ` Christoph Hellwig
  2011-07-25 11:32               ` Ingo Molnar
@ 2011-07-25 11:34               ` Olivier Galibert
  2011-07-25 11:41                 ` Christoph Hellwig
  1 sibling, 1 reply; 70+ messages in thread
From: Olivier Galibert @ 2011-07-25 11:34 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: Ingo Molnar, Alexander Graf, Pekka Enberg, Jan Kiszka, torvalds,
	avi, akpm, linux-kernel, kvm, gorcunov, levinsasha928,
	asias.hejun, prasadjoshi124

On Mon, Jul 25, 2011 at 07:24:12AM -0400, Christoph Hellwig wrote:
> On Mon, Jul 25, 2011 at 01:08:10PM +0200, Ingo Molnar wrote:
> > Fact is that developing ABIs within an integrated project is 
> > *amazingly* powerful. You should try it one day, instead of 
> > criticizing it :-)
> 
> I've been doing this long before you declare it the rosetta stone.  Some
> of the worst ABIs I know come from that kind of development, e.g. all
> the ioctls we have in xfs, inherited from the "tightly integrated" IRIX
> development cycle.

You need someone with taste in the loop.  But if you do, "evolved" is
always better than "designed before you actually know what you need".

As I'm sure you perfectly know, for the matter.

  OG.


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25 11:34               ` Olivier Galibert
@ 2011-07-25 11:41                 ` Christoph Hellwig
  0 siblings, 0 replies; 70+ messages in thread
From: Christoph Hellwig @ 2011-07-25 11:41 UTC (permalink / raw)
  To: Olivier Galibert
  Cc: Christoph Hellwig, Ingo Molnar, Alexander Graf, Pekka Enberg,
	Jan Kiszka, torvalds, avi, akpm, linux-kernel, kvm, gorcunov,
	levinsasha928, asias.hejun, prasadjoshi124

On Mon, Jul 25, 2011 at 01:34:25PM +0200, Olivier Galibert wrote:
> You need someone with taste in the loop.  But if you do, "evolved" is
> always better than "designed before you actually know what you need".
> 
> As I'm sure you perfectly know, for the matter.

Neither is actually helpful.  You reall want reference implementation
on both sides of an ABI, _and_ documentation.  And yes, usually you'll
need a few iterations over it.  In theory you can do that in a "tightly
integrated" enviroment as well, but practice shows simply boilds down
to commiting the bloody thing.

I'm also not sure why we even bother to focus with this side-line
discussion.  It's not like the kvm (as in the kernel kvm module)
developers have written the kvm tools.  It's just another userspace for
the kvm userspace (the fifth if I count correctly), and so far the
reference and often only implementation of any new kvm module feature
is for qemu-kvm.  So no matter where kvm tools lives, if you guys one
day actually start doing major kvm core features it will still evolve
discussing them with the main consumer of those interfaces.

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25 10:38         ` Christoph Hellwig
  2011-07-25 11:08           ` Ingo Molnar
@ 2011-07-25 12:09           ` Ingo Molnar
  2011-07-25 12:36           ` Pekka Enberg
  2 siblings, 0 replies; 70+ messages in thread
From: Ingo Molnar @ 2011-07-25 12:09 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: Alexander Graf, Pekka Enberg, Jan Kiszka, torvalds, avi, akpm,
	linux-kernel, kvm, gorcunov, levinsasha928, asias.hejun,
	prasadjoshi124


* Christoph Hellwig <hch@infradead.org> wrote:

> On Mon, Jul 25, 2011 at 10:14:13AM +0200, Alexander Graf wrote:

> > So instead of thinking a bit and trying to realize that there 
> > might be a reason people don't want all their user space in the 
> > kernel tree you go ahead and start your own crusade of creating a 
> > new user space. Great. That's how I always hoped Linux would be 
> > :(.
> 
> It's not Linux in general yet, it's mostly a crusade of a few with 
> a political agenda.

I find it weird (and insulting) that you are calling our patches that 
implement better tolling a 'crusade' done under a 'political agenda'.

I'd rather say that the extremist position is yours: to keep 
everything separated always, all the time and ban tools from the 
kernel repo.

You have declared it a failure from the very start and if it were up 
to you it would never get a fair chance.

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  8:30         ` Pekka Enberg
  2011-07-25  8:37           ` Alexander Graf
@ 2011-07-25 12:24           ` Kevin Wolf
  2011-07-25 12:41             ` Pekka Enberg
  1 sibling, 1 reply; 70+ messages in thread
From: Kevin Wolf @ 2011-07-25 12:24 UTC (permalink / raw)
  To: Pekka Enberg
  Cc: Alexander Graf, Ingo Molnar, Jan Kiszka, torvalds, avi, akpm,
	linux-kernel, kvm, gorcunov, levinsasha928, asias.hejun,
	prasadjoshi124

Am 25.07.2011 10:30, schrieb Pekka Enberg:
> Hi Alexander,
> 
> On Mon, Jul 25, 2011 at 11:14 AM, Alexander Graf <agraf@suse.de> wrote:
>>> So i wanted to have a lightweight tool that allows me to test KVM and
>>> tools/kvm/ does that very nicely: i type './kvm run' and i can test a
>>> native bzImage (which has some virtualization options enabled as
>>> well) on the _host_ distro i am running, booting to a text shell
>>> prompt.
>>
>> I do that all the time.
>>
>>  $ qemu-kvm -nographic -kernel arch/x86/boot/bzImage -append console=ttyS0
>>
>> does the exact same thing. If that's too much typing for you, make it a bash alias.
> 
> You know, they said the same thing about oprofile. All you needed to do was to
> write few simple shell scripts to make it work. One of the key
> features of tools/kvm
> is 'as little configuration as possible' and I can assure you that
> bash alias is really
> not a solution for that.

You've just chosen a different default. I'd argue that most users (i.e.
not developers of the tool or the kernel) actually want to run with a
disk image and graphics. You can type "qemu-kvm harddisk.img" and that's
it. This is clearly superior to something as tedious as "./kvm run -d
harddisk.img --sdl" and I can assure you that a bash alias is really not
a solution for that.

So, as always, which set of command line switches works better for you
depends entirely on your use case.

Kevin

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25 10:38         ` Christoph Hellwig
  2011-07-25 11:08           ` Ingo Molnar
  2011-07-25 12:09           ` Ingo Molnar
@ 2011-07-25 12:36           ` Pekka Enberg
  2011-07-25 19:21             ` david
  2 siblings, 1 reply; 70+ messages in thread
From: Pekka Enberg @ 2011-07-25 12:36 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: Alexander Graf, Ingo Molnar, Jan Kiszka, torvalds, avi, akpm,
	linux-kernel, kvm, gorcunov, levinsasha928, asias.hejun,
	prasadjoshi124

Hi Christoph,

On Mon, 2011-07-25 at 06:38 -0400, Christoph Hellwig wrote:
> On Mon, Jul 25, 2011 at 10:14:13AM +0200, Alexander Graf wrote:
> > So instead of thinking a bit and trying to realize that there might be a reason people don't want all their user space in the kernel tree you go ahead and start your own crusade of creating a new user space. Great. That's how I always hoped Linux would be :(.
> 
> It's not Linux in general yet, it's mostly a crusade of a few with a
> political agenda.

Would you mind sharing with the rest of us who exactly those few are and
what is their political agenda? I certainly don't recognize myself or
any of the development team from your description above. And to be
completely honest, I think that's one of the single most stupid comments
I've read in this thread.

On Mon, 2011-07-25 at 06:38 -0400, Christoph Hellwig wrote:
> And another argument, calling toyvisor2 "kvm" is a really bad idea.  The
> kvm binary has been used for the kvm-patched qemu binary for quite a
> while in various distros, so re-using that name will cause utter
> confusion.

I don't foresee 'utter confusion'. The code definitely wants to live in
tools/kvm so it's logical that the executable name is called 'kvm'. If
distros want to package it with a different name, they're free to do so.

> I'm happy that you guys do another independent userspace for kvm, but
> please:
> 
>   a) give it a useful name

I don't think toyvisor2 will do, really. I originally wanted to call the
hypervisor 'pvm' for selfish reasons because quite frankly 'penix' just
doesn't have the prestige 'linux' has. However, Ingo convinced me that
'kvm' is a good name and I agree with that.

I'm open to alternative suggestions, though.

>   b) just develop it where it belongs, your own little git repository
>      somewhere

I already explained why we want it in the kernel tree and what kind of
benefits we think there are. I also explained that we want to reuse and
share code with perf, for example. If you really want to argue against
merging, you need to cough up some actual disadvantages or refute the
suggested advantages.

			Pekka


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25 12:24           ` Kevin Wolf
@ 2011-07-25 12:41             ` Pekka Enberg
  2011-07-25 12:46               ` Avi Kivity
  0 siblings, 1 reply; 70+ messages in thread
From: Pekka Enberg @ 2011-07-25 12:41 UTC (permalink / raw)
  To: Kevin Wolf
  Cc: Alexander Graf, Ingo Molnar, Jan Kiszka, torvalds, avi, akpm,
	linux-kernel, kvm, gorcunov, levinsasha928, asias.hejun,
	prasadjoshi124

Hi Kevin,

On Mon, 2011-07-25 at 14:24 +0200, Kevin Wolf wrote:
> You've just chosen a different default. I'd argue that most users (i.e.
> not developers of the tool or the kernel) actually want to run with a
> disk image and graphics. You can type "qemu-kvm harddisk.img" and that's
> it. This is clearly superior to something as tedious as "./kvm run -d
> harddisk.img --sdl" and I can assure you that a bash alias is really not
> a solution for that.

I actually agree with you and we eventually want

  kvm run harddisk.img

to do the right thing.

On Mon, 2011-07-25 at 14:24 +0200, Kevin Wolf wrote:
> So, as always, which set of command line switches works better for you
> depends entirely on your use case.

I actually don't agree. I think Qemu requires way too much configuration
from the user and doesn't try hard enough to provide best possible
defaults. Dunno how much virt-manager changes all that, though.

			Pekka


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  9:06               ` Alexander Graf
@ 2011-07-25 12:45                 ` Pekka Enberg
  2011-07-25 12:47                   ` Avi Kivity
  0 siblings, 1 reply; 70+ messages in thread
From: Pekka Enberg @ 2011-07-25 12:45 UTC (permalink / raw)
  To: Alexander Graf
  Cc: Ingo Molnar, Jan Kiszka, torvalds, avi, akpm, linux-kernel, kvm,
	gorcunov, levinsasha928, asias.hejun, prasadjoshi124

On Mon, Jul 25, 2011 at 12:06 PM, Alexander Graf <agraf@suse.de> wrote:
>> And don't get this the wrong way either, I'm not hostile against other
>> operating systems, but I simply am not interested enough in them to
>> spend my time improving them.
>
> Then kvm-tool is about as useful as Mac-on-Linux. Why don't we have MoL user land in the kernel? I even added support for KVM to it about a year ago. So all I need to do is change it to the kernel coding style, add some dependencies on kernel headers and I'm good for a pull request?

Oh, I dunno - have you tried that? If you're interested in sending a
patch against tools/kvm that adds KVM-based support to Mac, I'm happy
to review it and consider for inclusion. ;-)

                        Pekka

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25 12:41             ` Pekka Enberg
@ 2011-07-25 12:46               ` Avi Kivity
  0 siblings, 0 replies; 70+ messages in thread
From: Avi Kivity @ 2011-07-25 12:46 UTC (permalink / raw)
  To: Pekka Enberg
  Cc: Kevin Wolf, Alexander Graf, Ingo Molnar, Jan Kiszka, torvalds,
	akpm, linux-kernel, kvm, gorcunov, levinsasha928, asias.hejun,
	prasadjoshi124

On 07/25/2011 03:41 PM, Pekka Enberg wrote:
> On Mon, 2011-07-25 at 14:24 +0200, Kevin Wolf wrote:
> >  So, as always, which set of command line switches works better for you
> >  depends entirely on your use case.
>
> I actually don't agree. I think Qemu requires way too much configuration
> from the user and doesn't try hard enough to provide best possible
> defaults.

qemu has a huge wealth of options so the command line is incredibly 
complicated.  Changing defaults is hard due to backward compatibility 
constraints.

> Dunno how much virt-manager changes all that, though.

virt-manager is a GUI interface, no command line at all.

-- 
error compiling committee.c: too many arguments to function


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25 12:45                 ` Pekka Enberg
@ 2011-07-25 12:47                   ` Avi Kivity
  2011-07-25 12:51                     ` Sasha Levin
  2011-07-25 12:51                     ` Alexander Graf
  0 siblings, 2 replies; 70+ messages in thread
From: Avi Kivity @ 2011-07-25 12:47 UTC (permalink / raw)
  To: Pekka Enberg
  Cc: Alexander Graf, Ingo Molnar, Jan Kiszka, torvalds, akpm,
	linux-kernel, kvm, gorcunov, levinsasha928, asias.hejun,
	prasadjoshi124

On 07/25/2011 03:45 PM, Pekka Enberg wrote:
> On Mon, Jul 25, 2011 at 12:06 PM, Alexander Graf<agraf@suse.de>  wrote:
> >>  And don't get this the wrong way either, I'm not hostile against other
> >>  operating systems, but I simply am not interested enough in them to
> >>  spend my time improving them.
> >
> >  Then kvm-tool is about as useful as Mac-on-Linux. Why don't we have MoL user land in the kernel? I even added support for KVM to it about a year ago. So all I need to do is change it to the kernel coding style, add some dependencies on kernel headers and I'm good for a pull request?
>
> Oh, I dunno - have you tried that? If you're interested in sending a
> patch against tools/kvm that adds KVM-based support to Mac, I'm happy
> to review it and consider for inclusion. ;-)

The patch will be a lot bigger than tools/kvm itself.

-- 
error compiling committee.c: too many arguments to function


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25 12:47                   ` Avi Kivity
@ 2011-07-25 12:51                     ` Sasha Levin
  2011-07-25 12:54                       ` Alexander Graf
  2011-07-25 12:51                     ` Alexander Graf
  1 sibling, 1 reply; 70+ messages in thread
From: Sasha Levin @ 2011-07-25 12:51 UTC (permalink / raw)
  To: Avi Kivity
  Cc: Pekka Enberg, Alexander Graf, Ingo Molnar, Jan Kiszka, torvalds,
	akpm, linux-kernel, kvm, gorcunov, asias.hejun, prasadjoshi124

On Mon, 2011-07-25 at 15:47 +0300, Avi Kivity wrote:
> On 07/25/2011 03:45 PM, Pekka Enberg wrote:
> > On Mon, Jul 25, 2011 at 12:06 PM, Alexander Graf<agraf@suse.de>  wrote:
> > >>  And don't get this the wrong way either, I'm not hostile against other
> > >>  operating systems, but I simply am not interested enough in them to
> > >>  spend my time improving them.
> > >
> > >  Then kvm-tool is about as useful as Mac-on-Linux. Why don't we have MoL user land in the kernel? I even added support for KVM to it about a year ago. So all I need to do is change it to the kernel coding style, add some dependencies on kernel headers and I'm good for a pull request?
> >
> > Oh, I dunno - have you tried that? If you're interested in sending a
> > patch against tools/kvm that adds KVM-based support to Mac, I'm happy
> > to review it and consider for inclusion. ;-)
> 
> The patch will be a lot bigger than tools/kvm itself.
> 

But it would be as a single patch series which includes kvm and
tools/kvm :)

-- 

Sasha.


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25 12:47                   ` Avi Kivity
  2011-07-25 12:51                     ` Sasha Levin
@ 2011-07-25 12:51                     ` Alexander Graf
  2011-07-25 13:09                       ` Pekka Enberg
  1 sibling, 1 reply; 70+ messages in thread
From: Alexander Graf @ 2011-07-25 12:51 UTC (permalink / raw)
  To: Avi Kivity
  Cc: Pekka Enberg, Ingo Molnar, Jan Kiszka, torvalds, akpm,
	linux-kernel, kvm, gorcunov, levinsasha928, asias.hejun,
	prasadjoshi124


On 25.07.2011, at 14:47, Avi Kivity wrote:

> On 07/25/2011 03:45 PM, Pekka Enberg wrote:
>> On Mon, Jul 25, 2011 at 12:06 PM, Alexander Graf<agraf@suse.de>  wrote:
>> >>  And don't get this the wrong way either, I'm not hostile against other
>> >>  operating systems, but I simply am not interested enough in them to
>> >>  spend my time improving them.
>> >
>> >  Then kvm-tool is about as useful as Mac-on-Linux. Why don't we have MoL user land in the kernel? I even added support for KVM to it about a year ago. So all I need to do is change it to the kernel coding style, add some dependencies on kernel headers and I'm good for a pull request?
>> 
>> Oh, I dunno - have you tried that? If you're interested in sending a
>> patch against tools/kvm that adds KVM-based support to Mac, I'm happy
>> to review it and consider for inclusion. ;-)
> 
> The patch will be a lot bigger than tools/kvm itself.

And it will be full of ugly hacks and code that needs to be compiled on a Mac. I was actually trying to pick an example of a project that you really don't want to have in the kernel tree.

One point stays true however with MoL. I added support for KVM a year ago and haven't had to touch it since. It just works. Because we have a pretty stable kernel/user space interface. So the whole thing about having user space and kernel code live together in the same tree is moot there.

And if you need to share code with perf, then that just proves the point that perf doesn't belong inside the kernel tree either.


Alex


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25 12:51                     ` Sasha Levin
@ 2011-07-25 12:54                       ` Alexander Graf
  2011-07-25 13:14                         ` Sasha Levin
  0 siblings, 1 reply; 70+ messages in thread
From: Alexander Graf @ 2011-07-25 12:54 UTC (permalink / raw)
  To: Sasha Levin
  Cc: Avi Kivity, Pekka Enberg, Ingo Molnar, Jan Kiszka, torvalds, akpm,
	linux-kernel, kvm, gorcunov, asias.hejun, prasadjoshi124


On 25.07.2011, at 14:51, Sasha Levin wrote:

> On Mon, 2011-07-25 at 15:47 +0300, Avi Kivity wrote:
>> On 07/25/2011 03:45 PM, Pekka Enberg wrote:
>>> On Mon, Jul 25, 2011 at 12:06 PM, Alexander Graf<agraf@suse.de>  wrote:
>>>>> And don't get this the wrong way either, I'm not hostile against other
>>>>> operating systems, but I simply am not interested enough in them to
>>>>> spend my time improving them.
>>>> 
>>>> Then kvm-tool is about as useful as Mac-on-Linux. Why don't we have MoL user land in the kernel? I even added support for KVM to it about a year ago. So all I need to do is change it to the kernel coding style, add some dependencies on kernel headers and I'm good for a pull request?
>>> 
>>> Oh, I dunno - have you tried that? If you're interested in sending a
>>> patch against tools/kvm that adds KVM-based support to Mac, I'm happy
>>> to review it and consider for inclusion. ;-)
>> 
>> The patch will be a lot bigger than tools/kvm itself.
>> 
> 
> But it would be as a single patch series which includes kvm and
> tools/kvm :)

Not sure I get what you mean here. We already have KVM support for Macs ;). And pSeries. And the PS3. It even works on the Wii.


Alex


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  7:50   ` Sasha Levin
  2011-07-25  8:34     ` Alexander Graf
@ 2011-07-25 12:59     ` Paolo Bonzini
  2011-07-25 13:16       ` Anthony Liguori
  1 sibling, 1 reply; 70+ messages in thread
From: Paolo Bonzini @ 2011-07-25 12:59 UTC (permalink / raw)
  To: Sasha Levin
  Cc: Jan Kiszka, Pekka Enberg, mingo, avi, linux-kernel, kvm, gorcunov,
	asias.hejun, prasadjoshi124

On 07/25/2011 09:50 AM, Sasha Levin wrote:
> Anthony had a talk on last years KVM forum regarding the QEMU threading
> model (slide:
> http://www.linux-kvm.org/wiki/images/7/70/2010-forum-threading-qemu.pdf) .
>
> It was suggested that the KVM part of QEMU is having a hard time
> achieving the ideal threading model due to its need to support TCG -
> something which has nothing to do with KVM itself.

No, it is not having a hard time.  The "foot in the door" that Anthony 
mentions has been part of QEMU and qemu-kvm for a long time 
(multi-threading is necessary to support SMP!) and works quite well.


Historically, there were three main loops:

1) QEMU single-threaded;

2) QEMU multi-threaded; clean, but buggy, untested and bitrotting;

3) qemu-kvm multi-threaded, forked from (1), ugly but robust and widely 
deployed.

In 0.15 the two multi-threaded versions have been unified by Jan Kiszka. 
  I have even ported (2) to Windows with little or no pain; porting to 
Mac OS X interestingly is harder than Windows, because non-portable 
Linux assumptions about signal handling have crept in the code (to 
preempt the objections: they weren't just non-portabilities, they were 
latent bugs).  Windows just does not have signals. :)

So, right now, the only difference is that QEMU is still defaulting to 
the single-threaded main loop, while qemu-kvm enables multi-threading by 
default.  In some time even QEMU will switch.

Yes, this is of course worse than getting it right in the first place; 
Nobody is saying the opposite.

Paolo

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25 12:51                     ` Alexander Graf
@ 2011-07-25 13:09                       ` Pekka Enberg
  2011-07-25 13:29                         ` Alexander Graf
  0 siblings, 1 reply; 70+ messages in thread
From: Pekka Enberg @ 2011-07-25 13:09 UTC (permalink / raw)
  To: Alexander Graf
  Cc: Avi Kivity, Ingo Molnar, Jan Kiszka, torvalds, akpm, linux-kernel,
	kvm, gorcunov, levinsasha928, asias.hejun, prasadjoshi124

Hi Alexander,

On Mon, 2011-07-25 at 14:51 +0200, Alexander Graf wrote:
> And if you need to share code with perf, then that just proves the
> point that perf doesn't belong inside the kernel tree either.

You seem to want to prove something here but our reasoning is based on
experience and practicalities - not some theoretical idea what should
and what shouldn't be in the Linux kernel tree. I really don't think
that perf would have turned out to be as useful as it is if it was
developed out-of-the-tree. That's also why we think tools/kvm is such a
good idea.

			Pekka


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  8:35             ` Avi Kivity
@ 2011-07-25 13:10               ` Joerg Roedel
  2011-07-25 15:05                 ` Avi Kivity
  0 siblings, 1 reply; 70+ messages in thread
From: Joerg Roedel @ 2011-07-25 13:10 UTC (permalink / raw)
  To: Avi Kivity
  Cc: Alexander Graf, Pekka Enberg, Ingo Molnar, Jan Kiszka, torvalds,
	akpm, linux-kernel, kvm, gorcunov, levinsasha928, asias.hejun,
	prasadjoshi124

On Mon, Jul 25, 2011 at 11:35:52AM +0300, Avi Kivity wrote:
> On 07/25/2011 11:31 AM, Alexander Graf wrote:
>> In Ingo's reasoning, the next step would be to rewrite glibc and put
>> it into the kernel tree, because we end up adding syscalls so adding
>> them to the in-kernel libc with the same commit would be a lot easier
>> and cleaner.
>
> That actually makes a ton of sense.  One immediate win would be that  
> klibc can be tuned to the kernel it ships with (the dynamic loader will  
> pick the correct object), so less #ifdef trees.  Another would be to  
> make klibc the formal kernel interface, which allows us to reimplement  
> an older interface in terms of the one that supercedes it.

A libc in the kernel tree would indeed make a lot of sense and I am all
for it :) It would also help to get new interfaces to the users faster.
But it should probably not be the primary exported interface. This would
just move the compatability problem from the kernel into user-space.

	Joerg


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25 12:54                       ` Alexander Graf
@ 2011-07-25 13:14                         ` Sasha Levin
  0 siblings, 0 replies; 70+ messages in thread
From: Sasha Levin @ 2011-07-25 13:14 UTC (permalink / raw)
  To: Alexander Graf
  Cc: Avi Kivity, Pekka Enberg, Ingo Molnar, Jan Kiszka, torvalds, akpm,
	linux-kernel, kvm, gorcunov, asias.hejun, prasadjoshi124

On Mon, 2011-07-25 at 14:54 +0200, Alexander Graf wrote:
> On 25.07.2011, at 14:51, Sasha Levin wrote:
> 
> > On Mon, 2011-07-25 at 15:47 +0300, Avi Kivity wrote:
> >> On 07/25/2011 03:45 PM, Pekka Enberg wrote:
> >>> On Mon, Jul 25, 2011 at 12:06 PM, Alexander Graf<agraf@suse.de>  wrote:
> >>>>> And don't get this the wrong way either, I'm not hostile against other
> >>>>> operating systems, but I simply am not interested enough in them to
> >>>>> spend my time improving them.
> >>>> 
> >>>> Then kvm-tool is about as useful as Mac-on-Linux. Why don't we have MoL user land in the kernel? I even added support for KVM to it about a year ago. So all I need to do is change it to the kernel coding style, add some dependencies on kernel headers and I'm good for a pull request?
> >>> 
> >>> Oh, I dunno - have you tried that? If you're interested in sending a
> >>> patch against tools/kvm that adds KVM-based support to Mac, I'm happy
> >>> to review it and consider for inclusion. ;-)
> >> 
> >> The patch will be a lot bigger than tools/kvm itself.
> >> 
> > 
> > But it would be as a single patch series which includes kvm and
> > tools/kvm :)
> 
> Not sure I get what you mean here. We already have KVM support for Macs ;). And pSeries. And the PS3. It even works on the Wii.

I assumed from Avi's comment that we don't have KVM for Macs. Ignore me
then.

-- 

Sasha.


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25 12:59     ` Paolo Bonzini
@ 2011-07-25 13:16       ` Anthony Liguori
  0 siblings, 0 replies; 70+ messages in thread
From: Anthony Liguori @ 2011-07-25 13:16 UTC (permalink / raw)
  To: Paolo Bonzini
  Cc: Sasha Levin, Jan Kiszka, Pekka Enberg, mingo, avi, linux-kernel,
	kvm, gorcunov, asias.hejun, prasadjoshi124

On 07/25/2011 07:59 AM, Paolo Bonzini wrote:
> On 07/25/2011 09:50 AM, Sasha Levin wrote:
>> Anthony had a talk on last years KVM forum regarding the QEMU threading
>> model (slide:
>> http://www.linux-kvm.org/wiki/images/7/70/2010-forum-threading-qemu.pdf)
>> .
>>
>> It was suggested that the KVM part of QEMU is having a hard time
>> achieving the ideal threading model due to its need to support TCG -
>> something which has nothing to do with KVM itself.
>
> No, it is not having a hard time. The "foot in the door" that Anthony
> mentions has been part of QEMU and qemu-kvm for a long time
> (multi-threading is necessary to support SMP!) and works quite well.
>
>
> Historically, there were three main loops:
>
> 1) QEMU single-threaded;
>
> 2) QEMU multi-threaded; clean, but buggy, untested and bitrotting;
>
> 3) qemu-kvm multi-threaded, forked from (1), ugly but robust and widely
> deployed.
>
> In 0.15 the two multi-threaded versions have been unified by Jan Kiszka.
> I have even ported (2) to Windows with little or no pain; porting to Mac
> OS X interestingly is harder than Windows, because non-portable Linux
> assumptions about signal handling have crept in the code (to preempt the
> objections: they weren't just non-portabilities, they were latent bugs).
> Windows just does not have signals. :)
>
> So, right now, the only difference is that QEMU is still defaulting to
> the single-threaded main loop, while qemu-kvm enables multi-threading by
> default. In some time even QEMU will switch.
>
> Yes, this is of course worse than getting it right in the first place;
> Nobody is saying the opposite.

In all fairness, the fact that TCG requires signals to break execution 
does complicate the QEMU code a fair bit.  But that's because signals 
are tricky to get right, not that the model is fundamentally complicated.

Regards,

Anthony Liguori

>
> Paolo
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25 13:09                       ` Pekka Enberg
@ 2011-07-25 13:29                         ` Alexander Graf
  0 siblings, 0 replies; 70+ messages in thread
From: Alexander Graf @ 2011-07-25 13:29 UTC (permalink / raw)
  To: Pekka Enberg
  Cc: Avi Kivity, Ingo Molnar, Jan Kiszka, torvalds, akpm, linux-kernel,
	kvm, gorcunov, levinsasha928, asias.hejun, prasadjoshi124


On 25.07.2011, at 15:09, Pekka Enberg wrote:

> Hi Alexander,
> 
> On Mon, 2011-07-25 at 14:51 +0200, Alexander Graf wrote:
>> And if you need to share code with perf, then that just proves the
>> point that perf doesn't belong inside the kernel tree either.
> 
> You seem to want to prove something here but our reasoning is based on
> experience and practicalities - not some theoretical idea what should
> and what shouldn't be in the Linux kernel tree. I really don't think
> that perf would have turned out to be as useful as it is if it was
> developed out-of-the-tree.

IIUC that was mainly because the interface is very very closely coupled with kernel code.

> That's also why we think tools/kvm is such a good idea.

I like pears, so I like apples too?


Alex

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25 13:10               ` Joerg Roedel
@ 2011-07-25 15:05                 ` Avi Kivity
  0 siblings, 0 replies; 70+ messages in thread
From: Avi Kivity @ 2011-07-25 15:05 UTC (permalink / raw)
  To: Joerg Roedel
  Cc: Alexander Graf, Pekka Enberg, Ingo Molnar, Jan Kiszka, torvalds,
	akpm, linux-kernel, kvm, gorcunov, levinsasha928, asias.hejun,
	prasadjoshi124

On 07/25/2011 04:10 PM, Joerg Roedel wrote:
> >  That actually makes a ton of sense.  One immediate win would be that
> >  klibc can be tuned to the kernel it ships with (the dynamic loader will
> >  pick the correct object), so less #ifdef trees.  Another would be to
> >  make klibc the formal kernel interface, which allows us to reimplement
> >  an older interface in terms of the one that supercedes it.
>
> A libc in the kernel tree would indeed make a lot of sense and I am all
> for it :) It would also help to get new interfaces to the users faster.
> But it should probably not be the primary exported interface. This would
> just move the compatability problem from the kernel into user-space.

It's a better place for it, IMO.  For example, if we dump xnotify in 
favour of ynotify, we can implement xnotify using ynotify in klibc, and 
so avoid increasing the kernel's footprint.  As another example, there's 
an OS that implements its 32-bit ABI in userspace.

(of course that only works for new interfaces; the old ones have to be 
implemented in the kernel unless we thunk them somehow)

-- 
error compiling committee.c: too many arguments to function


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25  7:37   ` Pekka Enberg
  2011-07-25  7:53     ` Ingo Molnar
@ 2011-07-25 18:24     ` Jan Kiszka
  2011-07-25 19:43       ` Pekka Enberg
  1 sibling, 1 reply; 70+ messages in thread
From: Jan Kiszka @ 2011-07-25 18:24 UTC (permalink / raw)
  To: Pekka Enberg
  Cc: torvalds, mingo, avi, akpm, linux-kernel, kvm, gorcunov,
	levinsasha928, asias.hejun, prasadjoshi124

[-- Attachment #1: Type: text/plain, Size: 1241 bytes --]

On 2011-07-25 09:37, Pekka Enberg wrote:
> Hi Jan,
> 
> On Mon, Jul 25, 2011 at 2:12 AM, Jan Kiszka <jan.kiszka@web.de> wrote:
>> I've read several times now that developing in a single tree leads to
>> better results. Can you provide some example from the QEMU/KVM projects
>> where the split is preventing innovation, optimizations, or some other
>> kind of progress?
> 
> I really don't follow the Qemu project well enough to comment on what
> your biggest pain points are there.

Mmh, you can solve problems you do not even need to know about, just by
merging them into the kernel? Wait, will send you some more! :)

> 
> As for tools/kvm, it's pretty obvious by now that we want tighter
> integration with perf and the tracing facilities (and share code!),
> for example so for us merging to mainline is important.

Tracing&profiling are important topics, but in the end just small pieces
of the virtualization problem space. If that was the strongest reason,
it would be like asking gdb guys to merge qemu because it contains a
gdbserver.

Anyway. Thanks for all the fish! Found nothing new, not much concrete,
but many really entertaining answers. So why not repeat this on the next
merge window?

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25 12:36           ` Pekka Enberg
@ 2011-07-25 19:21             ` david
  0 siblings, 0 replies; 70+ messages in thread
From: david @ 2011-07-25 19:21 UTC (permalink / raw)
  To: Pekka Enberg
  Cc: Christoph Hellwig, Alexander Graf, Ingo Molnar, Jan Kiszka,
	torvalds, avi, akpm, linux-kernel, kvm, gorcunov, levinsasha928,
	asias.hejun, prasadjoshi124

On Mon, 25 Jul 2011, Pekka Enberg wrote:

> I don't think toyvisor2 will do, really. I originally wanted to call the
> hypervisor 'pvm' for selfish reasons because quite frankly 'penix' just
> doesn't have the prestige 'linux' has. However, Ingo convinced me that
> 'kvm' is a good name and I agree with that.
>
> I'm open to alternative suggestions, though.

I don't have a suggestions, but KVM is already a common acronym in 
datacenters, referring to 'Keyboard Video Mouse" consoles (and associated 
switching systems)

it would be nice to to have this sort of conflict with a new name.

David Lang

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [GIT PULL] Native Linux KVM tool for 3.1
  2011-07-25 18:24     ` Jan Kiszka
@ 2011-07-25 19:43       ` Pekka Enberg
  0 siblings, 0 replies; 70+ messages in thread
From: Pekka Enberg @ 2011-07-25 19:43 UTC (permalink / raw)
  To: Jan Kiszka
  Cc: torvalds, mingo, avi, akpm, linux-kernel, kvm, gorcunov,
	levinsasha928, asias.hejun, prasadjoshi124

On Mon, Jul 25, 2011 at 9:24 PM, Jan Kiszka <jan.kiszka@web.de> wrote:
>> On Mon, Jul 25, 2011 at 2:12 AM, Jan Kiszka <jan.kiszka@web.de> wrote:
>>> I've read several times now that developing in a single tree leads to
>>> better results. Can you provide some example from the QEMU/KVM projects
>>> where the split is preventing innovation, optimizations, or some other
>>> kind of progress?
>>
>> I really don't follow the Qemu project well enough to comment on what
>> your biggest pain points are there.
>
> Mmh, you can solve problems you do not even need to know about, just by
> merging them into the kernel? Wait, will send you some more! :)

Don't be an idiot.

Qemu's problems are pretty well understood and many of them explained
in this particular thread. I'm not trying to fix Qemu's problems nor
am I trying to fix your Windows virtualization issues. I'm trying to
implement tooling to support Linux on KVM well enough that I don't
have to use VirtualBox for virtualization myself.

>> As for tools/kvm, it's pretty obvious by now that we want tighter
>> integration with perf and the tracing facilities (and share code!),
>> for example so for us merging to mainline is important.
>
> Tracing&profiling are important topics, but in the end just small pieces
> of the virtualization problem space. If that was the strongest reason,
> it would be like asking gdb guys to merge qemu because it contains a
> gdbserver.

I think that just shows that you don't really understand the benefits
of integrating KVM and perf.

> Anyway. Thanks for all the fish! Found nothing new, not much concrete,
> but many really entertaining answers. So why not repeat this on the next
> merge window?

While it's entertaining for you, it's not so much for me. I think we
have something pretty useful and cool in tools/kvm but all you seem to
want to talk about is Qemu which I personally find pretty
uninteresting.

Anyway, it's really up to Linus. FWIW, Avi doesn't oppose to the
merging and Ingo seems to think it's useful enough to have in -tip.

                        Pekka

^ permalink raw reply	[flat|nested] 70+ messages in thread

end of thread, other threads:[~2011-07-25 19:43 UTC | newest]

Thread overview: 70+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-07-24 20:37 [GIT PULL] Native Linux KVM tool for 3.1 Pekka Enberg
2011-07-24 23:12 ` Jan Kiszka
2011-07-25  7:37   ` Pekka Enberg
2011-07-25  7:53     ` Ingo Molnar
2011-07-25  8:14       ` Alexander Graf
2011-07-25  8:23         ` Pekka Enberg
2011-07-25  8:31           ` Alexander Graf
2011-07-25  8:35             ` Avi Kivity
2011-07-25 13:10               ` Joerg Roedel
2011-07-25 15:05                 ` Avi Kivity
2011-07-25  8:37             ` Pekka Enberg
2011-07-25  8:44               ` Alexander Graf
2011-07-25  8:51                 ` Pekka Enberg
2011-07-25  8:55                   ` Alexander Graf
2011-07-25  9:16               ` Ingo Molnar
2011-07-25  9:14           ` Ingo Molnar
2011-07-25  8:30         ` Pekka Enberg
2011-07-25  8:37           ` Alexander Graf
2011-07-25  8:47             ` Pekka Enberg
2011-07-25  9:06               ` Alexander Graf
2011-07-25 12:45                 ` Pekka Enberg
2011-07-25 12:47                   ` Avi Kivity
2011-07-25 12:51                     ` Sasha Levin
2011-07-25 12:54                       ` Alexander Graf
2011-07-25 13:14                         ` Sasha Levin
2011-07-25 12:51                     ` Alexander Graf
2011-07-25 13:09                       ` Pekka Enberg
2011-07-25 13:29                         ` Alexander Graf
2011-07-25  9:26               ` Ingo Molnar
2011-07-25  9:42                 ` Alexander Graf
2011-07-25 10:16                   ` Ingo Molnar
2011-07-25 10:21                     ` Alexander Graf
2011-07-25 10:35                       ` Ingo Molnar
2011-07-25 10:47             ` Ingo Molnar
2011-07-25 12:24           ` Kevin Wolf
2011-07-25 12:41             ` Pekka Enberg
2011-07-25 12:46               ` Avi Kivity
2011-07-25  8:54         ` Ingo Molnar
2011-07-25  8:59           ` Ingo Molnar
2011-07-25  9:03           ` Alexander Graf
2011-07-25  9:41             ` Ingo Molnar
2011-07-25  9:46               ` Alexander Graf
2011-07-25 10:28                 ` Ingo Molnar
2011-07-25  9:48               ` Avi Kivity
2011-07-25 10:03                 ` Ingo Molnar
2011-07-25 10:17                   ` Avi Kivity
2011-07-25 10:29                     ` Ingo Molnar
2011-07-25 11:26             ` Olivier Galibert
2011-07-25 10:38         ` Christoph Hellwig
2011-07-25 11:08           ` Ingo Molnar
2011-07-25 11:24             ` Christoph Hellwig
2011-07-25 11:32               ` Ingo Molnar
2011-07-25 11:34               ` Olivier Galibert
2011-07-25 11:41                 ` Christoph Hellwig
2011-07-25 12:09           ` Ingo Molnar
2011-07-25 12:36           ` Pekka Enberg
2011-07-25 19:21             ` david
2011-07-25 18:24     ` Jan Kiszka
2011-07-25 19:43       ` Pekka Enberg
2011-07-25  7:50   ` Sasha Levin
2011-07-25  8:34     ` Alexander Graf
2011-07-25 12:59     ` Paolo Bonzini
2011-07-25 13:16       ` Anthony Liguori
2011-07-25  1:19 ` Anthony Liguori
2011-07-25  7:27   ` Pekka Enberg
2011-07-25  7:36     ` Avi Kivity
2011-07-25  7:39       ` Pekka Enberg
2011-07-25  7:55       ` Sasha Levin
     [not found]       ` <CAFO3S428qqUAu19QjPxDDAVv+eJSX0MEfYp5y03znNi6XbEtTg@mail.gmail.com>
2011-07-25  8:11         ` Ingo Molnar
2011-07-25  8:17           ` Pekka Enberg

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox