* [PATCH 2/7] lguest documentation:Chapter II [not found] <1181272257.24958.12.camel@localhost.localdomain> @ 2007-06-08 3:11 ` Rusty Russell [not found] ` <1181272316.24958.13.camel@localhost.localdomain> ` (2 subsequent siblings) 3 siblings, 0 replies; 8+ messages in thread From: Rusty Russell @ 2007-06-08 3:11 UTC (permalink / raw) To: Andrew Morton; +Cc: lkml - Kernel Mailing List, virtualization Documentation: The Guest Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> --- drivers/lguest/lguest.c | 456 ++++++++++++++++++++++++++++++++++++++++--- drivers/lguest/lguest_asm.S | 57 +++-- include/linux/lguest.h | 47 +++- 3 files changed, 510 insertions(+), 50 deletions(-) =================================================================== --- a/drivers/lguest/lguest.c +++ b/drivers/lguest/lguest.c @@ -68,6 +68,12 @@ #include <asm/io.h> #include <asm/sched-clock.h> +/*G:010 Welcome to the Guest! + * + * The Guest in our tale is a simple creature: identical to the Host but + * behaving in simplified but equivalent ways. In particular, the Guest is the + * same kernel as the Host (or at least, built from the same source code). :*/ + /* Declarations for definitions in lguest_guest.S */ extern char lguest_noirq_start[], lguest_noirq_end[]; extern const char lgstart_cli[], lgend_cli[]; @@ -85,7 +91,26 @@ struct lguest_data lguest_data = { }; struct lguest_device_desc *lguest_devices; -static enum paravirt_lazy_mode lazy_mode; +/*G:035 Notice the lazy_hcall() above, rather than hcall(). This is our first + * real optimization trick! + * + * When lazy_mode is set, it means we're allowed to defer all hypercalls and do + * them as a batch when lazy_mode is eventually turned off. Because hypercalls + * are reasonably expensive, batching them up makes sense. For example, a + * large mmap might update dozens of page table entries: that code calls + * lguest_lazy_mode(PARAVIRT_LAZY_MMU), does the dozen updates, then calls + * lguest_lazy_mode(PARAVIRT_LAZY_NONE). + * + * So, when we're in lazy mode, we call async_hypercall() to store the call for + * future processing. When lazy mode is turned off we issue a hypercall to + * flush the stored calls. + * + * There's also a hack where "mode" is set to "PARAVIRT_LAZY_FLUSH" which + * indicates we're to flush any outstanding calls immediately. This is used + * when an interrupt handler does a kmap_atomic(): the page table changes must + * happen immediately even if we're in the middle of a batch. Usually we're + * not, though, so there's nothing to do. */ +static enum paravirt_lazy_mode lazy_mode; /* Note: not SMP-safe! */ static void lguest_lazy_mode(enum paravirt_lazy_mode mode) { if (mode == PARAVIRT_LAZY_FLUSH) { @@ -109,6 +134,16 @@ static void lazy_hcall(unsigned long cal async_hcall(call, arg1, arg2, arg3); } +/* async_hcall() is pretty simple: I'm quite proud of it really. We have a + * ring buffer of stored hypercalls which the Host will run though next time we + * do a normal hypercall. Each entry in the ring has 4 slots for the hypercall + * arguments, and a "hcall_status" word which is 0 if the call is ready to go, + * and 255 once the Host has finished with it. + * + * If we come around to a slot which hasn't been finished, then the table is + * full and we just make the hypercall directly. This has the nice side + * effect of causing the Host to run all the stored calls in the ring buffer + * which empties it for next time! */ void async_hcall(unsigned long call, unsigned long arg1, unsigned long arg2, unsigned long arg3) { @@ -116,6 +151,9 @@ void async_hcall(unsigned long call, static unsigned int next_call; unsigned long flags; + /* Disable interrupts if not already disabled: we don't want an + * interrupt handler making a hypercall while we're already doing + * one! */ local_irq_save(flags); if (lguest_data.hcall_status[next_call] != 0xFF) { /* Table full, so do normal hcall which will flush table. */ @@ -125,7 +163,7 @@ void async_hcall(unsigned long call, lguest_data.hcalls[next_call].edx = arg1; lguest_data.hcalls[next_call].ebx = arg2; lguest_data.hcalls[next_call].ecx = arg3; - /* Make sure host sees arguments before "valid" flag. */ + /* Arguments must all be written before we mark it to go */ wmb(); lguest_data.hcall_status[next_call] = 0; if (++next_call == LHCALL_RING_SIZE) @@ -133,9 +171,14 @@ void async_hcall(unsigned long call, } local_irq_restore(flags); } - +/*:*/ + +/* Wrappers for the SEND_DMA and BIND_DMA hypercalls. This is mainly because + * Jeff Garzik complained that __pa() should never appear in drivers, and this + * helps remove most of them. But also, it wraps some ugliness. */ void lguest_send_dma(unsigned long key, struct lguest_dma *dma) { + /* The hcall might not write this if something goes wrong */ dma->used_len = 0; hcall(LHCALL_SEND_DMA, key, __pa(dma), 0); } @@ -143,11 +186,16 @@ int lguest_bind_dma(unsigned long key, s int lguest_bind_dma(unsigned long key, struct lguest_dma *dmas, unsigned int num, u8 irq) { + /* This is the only hypercall which actually wants 5 arguments, and we + * only support 4. Fortunately the interrupt number is always less + * than 256, so we can pack it with the number of dmas in the final + * argument. */ if (!hcall(LHCALL_BIND_DMA, key, __pa(dmas), (num << 8) | irq)) return -ENOMEM; return 0; } +/* Unbinding is the same hypercall as binding, but with 0 num & irq. */ void lguest_unbind_dma(unsigned long key, struct lguest_dma *dmas) { hcall(LHCALL_BIND_DMA, key, __pa(dmas), 0); @@ -165,35 +213,65 @@ void lguest_unmap(void *addr) iounmap((__force void __iomem *)addr); } +/*G:033 + * Here are our first native-instruction replacements: four functions for + * interrupt control. + * + * The simplest way of implementing these would be to have "turn interrupts + * off" and "turn interrupts on" hypercalls. Unfortunately, this is too slow: + * these by far the most commonly called functions of those we override. + * + * So instead we keep an "irq_enabled" field inside our "struct lguest_data", + * which the Guest can update with a single instruction. The Host knows to + * check there when it wants to deliver an interrupt. + */ + +/* save_flags() is expected to return the processor state (ie. "eflags"). The + * eflags word contains all kind of stuff, but in practice Linux only cares + * about the interrupt flag. Our "save_flags()" just returns that. */ static unsigned long save_fl(void) { return lguest_data.irq_enabled; } +/* "restore_flags" just sets the flags back to the value given. */ static void restore_fl(unsigned long flags) { - /* FIXME: Check if interrupt pending... */ lguest_data.irq_enabled = flags; } +/* Interrupts go off... */ static void irq_disable(void) { lguest_data.irq_enabled = 0; } +/* Interrupts go on... */ static void irq_enable(void) { - /* FIXME: Check if interrupt pending... */ lguest_data.irq_enabled = X86_EFLAGS_IF; } +/*G:034 + * The Interrupt Descriptor Table (IDT). + * + * The IDT tells the processor what to do when an interrupt comes in. Each + * entry in the table is a 64-bit descriptor: this holds the privilege level, + * address of the handler, and... well, who cares? The Guest just asks the + * Host to make the change anyway, because the Host controls the real IDT. + */ static void lguest_write_idt_entry(struct desc_struct *dt, int entrynum, u32 low, u32 high) { + /* Keep the local copy up to date. */ write_dt_entry(dt, entrynum, low, high); + /* Tell Host about this new entry. */ hcall(LHCALL_LOAD_IDT_ENTRY, entrynum, low, high); } +/* Changing to a different IDT is very rare: we keep the IDT up-to-date every + * time it is written, so we can simply loop through all entries and tell the + * Host about them. */ static void lguest_load_idt(const struct Xgt_desc_struct *desc) { unsigned int i; @@ -203,12 +281,29 @@ static void lguest_load_idt(const struct hcall(LHCALL_LOAD_IDT_ENTRY, i, idt[i].a, idt[i].b); } +/* + * The Global Descriptor Table. + * + * The Intel architecture defines another table, called the Global Descriptor + * Table (GDT). You tell the CPU where it is (and its size) using the "lgdt" + * instruction, and then several other instructions refer to entries in the + * table. There are three entries which the Switcher needs, so the Host simply + * controls the entire thing and the Guest asks it to make changes using the + * LOAD_GDT hypercall. + * + * This is the opposite of the IDT code where we have a LOAD_IDT_ENTRY + * hypercall and use that repeatedly to load a new IDT. I don't think it + * really matters, but wouldn't it be nice if they were the same? + */ static void lguest_load_gdt(const struct Xgt_desc_struct *desc) { BUG_ON((desc->size+1)/8 != GDT_ENTRIES); hcall(LHCALL_LOAD_GDT, __pa(desc->address), GDT_ENTRIES, 0); } +/* For a single GDT entry which changes, we do the lazy thing: alter our GDT, + * then tell the Host to reload the entire thing. This operation is so rare + * that this naive implementation is reasonable. */ static void lguest_write_gdt_entry(struct desc_struct *dt, int entrynum, u32 low, u32 high) { @@ -216,19 +311,58 @@ static void lguest_write_gdt_entry(struc hcall(LHCALL_LOAD_GDT, __pa(dt), GDT_ENTRIES, 0); } +/* OK, I lied. There are three "thread local storage" GDT entries which change + * on every context switch (these three entries are how glibc implements + * __thread variables). So we have a hypercall specifically for this case. */ static void lguest_load_tls(struct thread_struct *t, unsigned int cpu) { lazy_hcall(LHCALL_LOAD_TLS, __pa(&t->tls_array), cpu, 0); } - +/*:*/ + +/*G:038 That's enough excitement for now, back to ploughing through each of + * the paravirt_ops (we're about 1/3 of the way through). + * + * This is the Local Descriptor Table, another weird Intel thingy. Linux only + * uses this for some strange applications like Wine. We don't do anything + * here, so they'll get an informative and friendly Segmentation Fault. */ static void lguest_set_ldt(const void *addr, unsigned entries) { } +/* This loads a GDT entry into the "Task Register": that entry points to a + * structure called the Task State Segment. Some comments scattered though the + * kernel code indicate that this used for task switching in ages past, along + * with blood sacrifice and astrology. + * + * Now there's nothing interesting in here that we don't get told elsewhere. + * But the native version uses the "ltr" instruction, which makes the Host + * complain to the Guest about a Segmentation Fault and it'll oops. So we + * override the native version with a do-nothing version. */ static void lguest_load_tr_desc(void) { } +/* The "cpuid" instruction is a way of querying both the CPU identity + * (manufacturer, model, etc) and its features. It was introduced before the + * Pentium in 1993 and keeps getting extended by both Intel and AMD. As you + * might imagine, after a decade and a half this treatment, it is now a giant + * ball of hair. Its entry in the current Intel manual runs to 28 pages. + * + * This instruction even it has its own Wikipedia entry. The Wikipedia entry + * has been translated into 4 languages. I am not making this up! + * + * We could get funky here and identify ourselves as "GenuineLguest", but + * instead we just use the real "cpuid" instruction. Then I pretty much turned + * off feature bits until the Guest booted. (Don't say that: you'll damage + * lguest sales!) Shut up, inner voice! (Hey, just pointing out that this is + * hardly future proof.) Noone's listening! They don't like you anyway, + * parenthetic weirdo! + * + * Replacing the cpuid so we can turn features off is great for the kernel, but + * anyone (including userspace) can just use the raw "cpuid" instruction and + * the Host won't even notice since it isn't privileged. So we try not to get + * too worked up about it. */ static void lguest_cpuid(unsigned int *eax, unsigned int *ebx, unsigned int *ecx, unsigned int *edx) { @@ -241,21 +375,43 @@ static void lguest_cpuid(unsigned int *e *ecx &= 0x00002201; /* SSE, SSE2, FXSR, MMX, CMOV, CMPXCHG8B, TSC, FPU. */ *edx &= 0x07808111; - /* Host wants to know when we flush kernel pages: set PGE. */ + /* The Host can do a nice optimization if it knows that the + * kernel mappings (addresses above 0xC0000000 or whatever + * PAGE_OFFSET is set to) haven't changed. But Linux calls + * flush_tlb_user() for both user and kernel mappings unless + * the Page Global Enable (PGE) feature bit is set. */ *edx |= 0x00002000; break; case 0x80000000: /* Futureproof this a little: if they ask how much extended - * processor information, limit it to known fields. */ + * processor information there is, limit it to known fields. */ if (*eax > 0x80000008) *eax = 0x80000008; break; } } +/* Intel has four control registers, imaginatively named cr0, cr2, cr3 and cr4. + * I assume there's a cr1, but it hasn't bothered us yet, so we'll not bother + * it. The Host needs to know when the Guest wants to change them, so we have + * a whole series of functions like read_cr0() and write_cr0(). + * + * We start with CR0. CR0 allows you to turn on and off all kinds of basic + * features, but Linux only really cares about one: the horrifically-named Task + * Switched (TS) bit at bit 3 (ie. 8) + * + * What does the TS bit do? Well, it causes the CPU to trap (interrupt 7) if + * the floating point unit is used. Which allows us to restore FPU state + * lazily after a task switch, and Linux uses that gratefully, but wouldn't a + * name like "FPUTRAP bit" be a little less cryptic? + * + * We store cr0 (and cr3) locally, because the Host never changes it. The + * Guest sometimes wants to read it and we'd prefer not to bother the Host + * unnecessarily. */ static unsigned long current_cr0, current_cr3; static void lguest_write_cr0(unsigned long val) { + /* 8 == TS bit. */ lazy_hcall(LHCALL_TS, val & 8, 0, 0); current_cr0 = val; } @@ -265,17 +421,25 @@ static unsigned long lguest_read_cr0(voi return current_cr0; } +/* Intel provided a special instruction to clear the TS bit for people too cool + * to use write_cr0() to do it. This "clts" instruction is faster, because all + * the vowels have been optimized out. */ static void lguest_clts(void) { lazy_hcall(LHCALL_TS, 0, 0, 0); current_cr0 &= ~8U; } +/* CR2 is the virtual address of the last page fault, which the Guest only ever + * reads. The Host kindly writes this into our "struct lguest_data", so we + * just read it out of there. */ static unsigned long lguest_read_cr2(void) { return lguest_data.cr2; } +/* CR3 is the current toplevel pagetable page: the principle is the same as + * cr0. Keep a local copy, and tell the Host when it changes. */ static void lguest_write_cr3(unsigned long cr3) { lazy_hcall(LHCALL_NEW_PGTABLE, cr3, 0, 0); @@ -287,7 +451,7 @@ static unsigned long lguest_read_cr3(voi return current_cr3; } -/* Used to enable/disable PGE, but we don't care. */ +/* CR4 is used to enable and disable PGE, but we don't care. */ static unsigned long lguest_read_cr4(void) { return 0; @@ -297,6 +461,59 @@ static void lguest_write_cr4(unsigned lo { } +/* + * Page Table Handling. + * + * Now would be a good time to take a rest and grab a coffee or similarly + * relaxing stimulant. The easy parts are behind us, and the trek gradually + * winds uphill from here. + * + * Quick refresher: memory is divided into "pages" of 4096 bytes each. The CPU + * maps virtual addresses to physical addresses using "page tables". We could + * use one huge index of 1 million entries: each address is 4 bytes, so that's + * 1024 pages just to hold the page tables. But since most virtual addresses + * are unused, we use a two level index which saves space. The CR3 register + * contains the physical address of the top level "page directory" page, which + * contains physical addresses of up to 1024 second-level pages. Each of these + * second level pages contains up to 1024 physical addresses of actual pages, + * or Page Table Entries (PTEs). + * + * Here's a diagram, where arrows indicate physical addresses: + * + * CR3 ---> +---------+ + * | --------->+---------+ + * | | | PADDR1 | + * Top-level | | PADDR2 | + * (PMD) page | | | + * | | Lower-level | + * | | (PTE) page | + * | | | | + * .... .... + * + * So to convert a virtual address to a physical address, we look up the top + * level, which points us to the second level, which gives us the physical + * address of that page. If the top level entry was not present, or the second + * level entry was not present, then the virtual address is invalid (we + * say "the page was not mapped"). + * + * Put another way, a 32-bit virtual address is divided up like so: + * + * 1 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 + * |<---- 10 bits ---->|<---- 10 bits ---->|<------ 12 bits ------>| + * Index into top Index into second Offset within page + * page directory page pagetable page + * + * The kernel spends a lot of time changing both the top-level page directory + * and lower-level pagetable pages. The Guest doesn't know physical addresses, + * so while it maintains these page tables exactly like normal, it also needs + * to keep the Host informed whenever it makes a change: the Host will create + * the real page tables based on the Guests'. + */ + +/* The Guest calls this to set a second-level entry (pte), ie. to map a page + * into a process' address space. We set the entry then tell the Host the + * toplevel and address this corresponds to. The Guest uses one pagetable per + * process, so we need to tell the Host which one we're changing (mm->pgd). */ static void lguest_set_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pteval) { @@ -304,7 +521,9 @@ static void lguest_set_pte_at(struct mm_ lazy_hcall(LHCALL_SET_PTE, __pa(mm->pgd), addr, pteval.pte_low); } -/* We only support two-level pagetables at the moment. */ +/* The Guest calls this to set a top-level entry. Again, we set the entry then + * tell the Host which top-level page we changed, and the index of the entry we + * changed. */ static void lguest_set_pmd(pmd_t *pmdp, pmd_t pmdval) { *pmdp = pmdval; @@ -312,7 +531,15 @@ static void lguest_set_pmd(pmd_t *pmdp, (__pa(pmdp)&(PAGE_SIZE-1))/4, 0); } -/* FIXME: Eliminate all callers of this. */ +/* There are a couple of legacy places where the kernel sets a PTE, but we + * don't know the top level any more. This is useless for us, since we don't + * know which pagetable is changing or what address, so we just tell the Host + * to forget all of them. Fortunately, this is very rare. + * + * ... except in early boot when the kernel sets up the initial pagetables, + * which makes booting astonishingly slow. So we don't even tell the Host + * anything changed until we've done the first page table switch. + */ static void lguest_set_pte(pte_t *ptep, pte_t pteval) { *ptep = pteval; @@ -321,22 +548,51 @@ static void lguest_set_pte(pte_t *ptep, lazy_hcall(LHCALL_FLUSH_TLB, 1, 0, 0); } +/* Unfortunately for Lguest, the paravirt_ops for page tables were based on + * native page table operations. On native hardware you can set a new page + * table entry whenever you want, but if you want to remove one you have to do + * a TLB flush (a TLB is a little cache of page table entries kept by the CPU). + * + * So the lguest_set_pte_at() and lguest_set_pmd() functions above are only + * called when a valid entry is written, not when it's removed (ie. marked not + * present). Instead, this is where we come when the Guest wants to remove a + * page table entry: we tell the Host to set that entry to 0 (ie. the present + * bit is zero). */ static void lguest_flush_tlb_single(unsigned long addr) { - /* Simply set it to zero, and it will fault back in. */ + /* Simply set it to zero: if it was not, it will fault back in. */ lazy_hcall(LHCALL_SET_PTE, current_cr3, addr, 0); } +/* This is what happens after the Guest has removed a large number of entries. + * This tells the Host that any of the page table entries for userspace might + * have changed, ie. virtual addresses below PAGE_OFFSET. */ static void lguest_flush_tlb_user(void) { lazy_hcall(LHCALL_FLUSH_TLB, 0, 0, 0); } +/* This is called when the kernel page tables have changed. That's not very + * common (unless the Guest is using highmem, which makes the Guest extremely + * slow), so it's worth separating this from the user flushing above. */ static void lguest_flush_tlb_kernel(void) { lazy_hcall(LHCALL_FLUSH_TLB, 1, 0, 0); } +/* + * The Unadvanced Programmable Interrupt Controller. + * + * This is an attempt to implement the simplest possible interrupt controller. + * I spent some time looking though routines like set_irq_chip_and_handler, + * set_irq_chip_and_handler_name, set_irq_chip_data and set_phasers_to_stun and + * I *think* this is as simple as it gets. + * + * We can tell the Host what interrupts we want blocked ready for using the + * lguest_data.interrupts bitmap, so disabling (aka "masking") them is as + * simple as setting a bit. We don't actually "ack" interrupts as such, we + * just mask and unmask them. I wonder if we should be cleverer? + */ static void disable_lguest_irq(unsigned int irq) { set_bit(irq, lguest_data.blocked_interrupts); @@ -345,9 +601,9 @@ static void enable_lguest_irq(unsigned i static void enable_lguest_irq(unsigned int irq) { clear_bit(irq, lguest_data.blocked_interrupts); - /* FIXME: If it's pending? */ -} - +} + +/* This structure describes the lguest IRQ controller. */ static struct irq_chip lguest_irq_controller = { .name = "lguest", .mask = disable_lguest_irq, @@ -355,6 +611,10 @@ static struct irq_chip lguest_irq_contro .unmask = enable_lguest_irq, }; +/* This sets up the Interrupt Descriptor Table (IDT) entry for each hardware + * interrupt (except 128, which is used for system calls), and then tells the + * Linux infrastructure that each interrupt is controlled by our level-based + * lguest interrupt controller. */ static void __init lguest_init_IRQ(void) { unsigned int i; @@ -367,9 +627,17 @@ static void __init lguest_init_IRQ(void) handle_level_irq); } } + /* This call is required to set up for 4k stacks, where we have + * separate stacks for hard and soft interrupts. */ irq_ctx_init(smp_processor_id()); } +/* + * Time. + * + * It would be far better for everyone if the Guest had its own clock, but + * until then it must ask the Host for the time. + */ static unsigned long lguest_get_wallclock(void) { return hcall(LHCALL_GET_WALLCLOCK, 0, 0, 0); @@ -439,7 +707,9 @@ static void lguest_time_irq(unsigned int { unsigned long flags; - /* Check in case host TSC has changed rate. */ + /* Host tells us TSC has changed rate, and we frob our clock. This + * isn't perfect, but it seems there's no perfect way and this is + * simple. */ if (unlikely(tsc_khz != lguest_data.tsc_khz)) { tsc_khz = lguest_data.tsc_khz; lguest_clock.mult = clocksource_khz2mult(tsc_khz, 22); @@ -453,26 +723,50 @@ static void lguest_time_irq(unsigned int local_irq_restore(flags); } +/* At some point in the boot process, we get asked to set up our timing + * infrastructure. The kernel doesn't expect timer interrupts before this, but + * we cleverly initialized the "blocked_interrupts" field of "struct + * lguest_data" so that timer interrupts were blocked until now. */ static void lguest_time_init(void) { + /* We set up the timer interrupt (0) to go to our simple timer + * routine */ set_irq_handler(0, lguest_time_irq); + /* The "mult" field of the lguest_clock can be determined now. */ lguest_clock.mult = clocksource_khz2mult(tsc_khz, 22); clocksource_register(&lguest_clock); - /* We can't set cpumask in the initializer: damn C limitations! */ + /* We can't set cpumask in the initializer: damn C limitations! Set it + * here and register our timer device. */ lguest_clockevent.cpumask = cpumask_of_cpu(0); clockevents_register_device(&lguest_clockevent); + /* We can't set cpumask in the initializer: damn C limitations! */ enable_lguest_irq(0); } +/* The Host tells us the rate the Time Stamp Counter runs at. */ static unsigned long lguest_get_cpu_khz(void) { - /* The Host tells us the TSC speed */ return lguest_data.tsc_khz; } +/* + * Miscellaneous bits and pieces. + * + * Here is an oddball collection of functions which the Guest needs for things + * to work. They're pretty simple. + */ + +/* The Guest needs to tell the host what stack it expects traps to use. For + * native hardware, this is part of the Task State Segment mentioned above in + * lguest_load_tr_desc(), but to help hypervisors there's this special call. + * + * We tell the Host the segment we want to use (__KERNEL_DS is the kernel data + * segment), the privilege level (we're privilege level 1, the Host is 0 and + * will not tolerate us trying to use that), the stack pointer, and the number + * of pages in the stack. */ static void lguest_load_esp0(struct tss_struct *tss, struct thread_struct *thread) { @@ -480,15 +774,31 @@ static void lguest_load_esp0(struct tss_ THREAD_SIZE/PAGE_SIZE); } +/* Let's just say, I wouldn't do debugging under a Guest. */ static void lguest_set_debugreg(int regno, unsigned long value) { /* FIXME: Implement */ } +/* There are times when the kernel wants to make sure that no memory writes are + * caught in the cache (that they've all reached real hardware devices). This + * doesn't matter for the Guest which has virtual hardware. + * + * On the Pentium 4 and above, cpuid() indicates that the Cache Line Flush + * (clflush) instruction is available and the kernel uses that. Otherwise, it + * uses the older "Write Back and Invalidate Cache" (wbinvd) instruction. + * Unlike clflush, wbinvd can only be run at privilege level 0. So we can + * ignore clflush, but replace wbinvd. + */ static void lguest_wbinvd(void) { } +/* If the Guest expects to have an Advanced Programmable Interrupt Controller, + * we play dumb by ignoring writes and returning 0 for reads. So it's no + * longer Programmable nor Controlling anything, and I don't think 8 lines of + * code qualifies for Advanced. It will also never interrupt anything. It + * does, however, allow us to get through the Linux boot code. */ #ifdef CONFIG_X86_LOCAL_APIC static void lguest_apic_write(unsigned long reg, unsigned long v) { @@ -500,19 +810,32 @@ static unsigned long lguest_apic_read(un } #endif +/* STOP! Until an interrupt comes in. */ static void lguest_safe_halt(void) { hcall(LHCALL_HALT, 0, 0, 0); } +/* Perhaps CRASH isn't the best name for this hypercall, but we use it to get a + * message out when we're crashing as well as elegant termination like powering + * off. + * + * Note that the Host always prefers that the Guest speak in physical addresses + * rather than virtual addresses, so we use __pa() here. */ static void lguest_power_off(void) { hcall(LHCALL_CRASH, __pa("Power down"), 0, 0); } +/* + * Panicing. + * + * Don't. But if you did, this is what happens. + */ static int lguest_panic(struct notifier_block *nb, unsigned long l, void *p) { hcall(LHCALL_CRASH, __pa(p), 0, 0); + /* The hcall won't return, but to keep gcc happy, we're "done". */ return NOTIFY_DONE; } @@ -520,15 +843,45 @@ static struct notifier_block paniced = { .notifier_call = lguest_panic }; +/* Setting up memory is fairly easy. */ static __init char *lguest_memory_setup(void) { - /* We do this here because lockcheck barfs if before start_kernel */ + /* We do this here and not earlier because lockcheck barfs if we do it + * before start_kernel() */ atomic_notifier_chain_register(&panic_notifier_list, &paniced); + /* The Linux bootloader header contains an "e820" memory map: the + * Launcher populated the first entry with our memory limit. */ add_memory_region(E820_MAP->addr, E820_MAP->size, E820_MAP->type); + + /* This string is for the boot messages. */ return "LGUEST"; } +/*G:050 + * Patching (Powerfully Placating Performance Pedants) + * + * We have already seen that "struct paravirt_ops" lets us replace simple + * native instructions with calls to the appropriate back end all throughout + * the kernel. This allows the same kernel to run as a Guest and as a native + * kernel, but it's slow because of all the indirect branches. + * + * Remember that David Wheeler quote about "Any problem in computer science can + * be solved with another layer of indirection"? The rest of that quote is + * "... But that usually will create another problem." This is the first of + * those problems. + * + * Our current solution is to allow the paravirt back end to optionally patch + * over the indirect calls to replace them with something more efficient. We + * patch the four most commonly called functions: disable interrupts, enable + * interrupts, restore interrupts and save interrupts. We usually have 10 + * bytes to patch into: the Guest versions of these operations are small enough + * that we can fit comfortably. + * + * First we need assembly templates of each of the patchable Guest operations, + * and these are in lguest_asm.S. */ + +/*G:060 We construct a table from the assembler templates: */ static const struct lguest_insns { const char *start, *end; @@ -538,35 +891,52 @@ static const struct lguest_insns [PARAVIRT_PATCH(restore_fl)] = { lgstart_popf, lgend_popf }, [PARAVIRT_PATCH(save_fl)] = { lgstart_pushf, lgend_pushf }, }; + +/* Now our patch routine is fairly simple (based on the native one in + * paravirt.c). If we have a replacement, we copy it in and return how much of + * the available space we used. */ static unsigned lguest_patch(u8 type, u16 clobber, void *insns, unsigned len) { unsigned int insn_len; - /* Don't touch it if we don't have a replacement */ + /* Don't do anything special if we don't have a replacement */ if (type >= ARRAY_SIZE(lguest_insns) || !lguest_insns[type].start) return paravirt_patch_default(type, clobber, insns, len); insn_len = lguest_insns[type].end - lguest_insns[type].start; - /* Similarly if we can't fit replacement. */ + /* Similarly if we can't fit replacement (shouldn't happen, but let's + * be thorough). */ if (len < insn_len) return paravirt_patch_default(type, clobber, insns, len); + /* Copy in our instructions. */ memcpy(insns, lguest_insns[type].start, insn_len); return insn_len; } +/*G:030 Once we get to lguest_init(), we know we're a Guest. The paravirt_ops + * structure in the kernel provides a single point for (almost) every routine + * we have to override to avoid privileged instructions. */ __init void lguest_init(void *boot) { - /* Copy boot parameters first. */ + /* Copy boot parameters first: the Launcher put the physical location + * in %esi, and head.S converted that to a virtual address and handed + * it to us. */ memcpy(&boot_params, boot, PARAM_SIZE); + /* The boot parameters also tell us where the command-line is: save + * that, too. */ memcpy(boot_command_line, __va(boot_params.hdr.cmd_line_ptr), COMMAND_LINE_SIZE); + /* We're under lguest, paravirt is enabled, and we're running at + * privilege level 1, not 0 as normal. */ paravirt_ops.name = "lguest"; paravirt_ops.paravirt_enabled = 1; paravirt_ops.kernel_rpl = 1; + /* We set up all the lguest overrides for sensitive operations. These + * are detailed with the operations themselves. */ paravirt_ops.save_fl = save_fl; paravirt_ops.restore_fl = restore_fl; paravirt_ops.irq_disable = irq_disable; @@ -610,18 +980,41 @@ __init void lguest_init(void *boot) paravirt_ops.set_lazy_mode = lguest_lazy_mode; paravirt_ops.wbinvd = lguest_wbinvd; paravirt_ops.get_cpu_khz = lguest_get_cpu_khz; - + /* Now is a good time to look at the implementations of these functions + * before returning to the rest of lguest_init(). */ + + /*G:070 Now we've seen all the paravirt_ops, we return to + * lguest_init() where the rest of the fairly chaotic boot setup + * occurs. + * + * The Host expects our first hypercall to tell it where our "struct + * lguest_data" is, so we do that first. */ hcall(LHCALL_LGUEST_INIT, __pa(&lguest_data), 0, 0); - /* We use top of mem for initial pagetables. */ + /* The native boot code sets up initial page tables immediately after + * the kernel itself, and sets init_pg_tables_end so they're not + * clobbered. The Launcher places our initial pagetables somewhere at + * the top of our physical memory, so we don't need extra space: set + * init_pg_tables_end to the end of the kernel. */ init_pg_tables_end = __pa(pg0); + /* Load the %fs segment register (the per-cpu segment register) with + * the normal data segment to get through booting. */ asm volatile ("mov %0, %%fs" : : "r" (__KERNEL_DS) : "memory"); + /* The Host uses the top of the Guest's virtual address space for the + * Host<->Guest Switcher, and it tells us how much it needs in + * lguest_data.reserve_mem, set up on the LGUEST_INIT hypercall. */ reserve_top_address(lguest_data.reserve_mem); + /* The IDE code spends about 3 seconds probing for disks: if we reserve + * all the I/O ports up front it can't get them and so doesn't probe. + * Other device drivers are similar (but less severe). This cuts the + * kernel boot time on my machine from 4.1 seconds to 0.45 seconds. */ paravirt_disable_iospace(); + /* This is messy CPU setup stuff which the native boot code does before + * start_kernel, so we have to do, too: */ cpu_detect(&new_cpu_data); /* Math is always hard! */ new_cpu_data.hard_math = 1; @@ -627,14 +1016,27 @@ __init void lguest_init(void *boot) #ifdef CONFIG_X86_MCE mce_disabled = 1; #endif - #ifdef CONFIG_ACPI acpi_disabled = 1; acpi_ht = 0; #endif + /* We set the perferred console to "hvc". This is the "hypervisor + * virtual console" driver written by the PowerPC people, which we also + * adapted for lguest's use. */ add_preferred_console("hvc", 0, NULL); + /* Last of all, we set the power management poweroff hook to point to + * the Guest routine to power off. */ pm_power_off = lguest_power_off; + + /* Now we're set up, call start_kernel() in init/main.c and we proceed + * to boot as normal. It never returns. */ start_kernel(); } +/* + * This marks the end of stage II of our journey, The Guest. + * + * It is now time for us to explore the nooks and crannies of the three Guest + * devices and complete our understanding of the Guest in "make Drivers". + */ =================================================================== --- a/drivers/lguest/lguest_asm.S +++ b/drivers/lguest/lguest_asm.S @@ -6,15 +6,15 @@ /* FIXME: Once asm/processor-flags.h goes in, include that */ #define X86_EFLAGS_IF 0x00000200 -/* - * This is where we begin: we have a magic signature which the launcher looks - * for. The plan is that the Linux boot protocol will be extended with a +/*G:020 This is where we begin: we have a magic signature which the launcher + * looks for. The plan is that the Linux boot protocol will be extended with a * "platform type" field which will guide us here from the normal entry point, - * but for the moment this suffices. We pass the virtual address of the boot - * info to lguest_init(). + * but for the moment this suffices. The normal boot code uses %esi for the + * boot header, so we do too. We convert it to a virtual address by adding + * PAGE_OFFSET, and hand it to lguest_init() as its argument (ie. %eax). * - * We put it in .init.text will be discarded after boot. - */ + * The .section line puts this code in .init.text so it will be discarded after + * boot. */ .section .init.text, "ax", @progbits .ascii "GenuineLguest" /* Set up initial stack. */ @@ -23,7 +23,9 @@ addl $__PAGE_OFFSET, %eax jmp lguest_init -/* The templates for inline patching. */ +/*G:055 We create a macro which puts the assembler code between lgstart_ and + * lgend_ markers. These templates end up in the .init.text section, so they + * are discarded after boot. */ #define LGUEST_PATCH(name, insns...) \ lgstart_##name: insns; lgend_##name:; \ .globl lgstart_##name; .globl lgend_##name @@ -32,24 +34,47 @@ LGUEST_PATCH(sti, movl $X86_EFLAGS_IF, l LGUEST_PATCH(sti, movl $X86_EFLAGS_IF, lguest_data+LGUEST_DATA_irq_enabled) LGUEST_PATCH(popf, movl %eax, lguest_data+LGUEST_DATA_irq_enabled) LGUEST_PATCH(pushf, movl lguest_data+LGUEST_DATA_irq_enabled, %eax) +/*:*/ .text /* These demark the EIP range where host should never deliver interrupts. */ .global lguest_noirq_start .global lguest_noirq_end -/* - * We move eflags word to lguest_data.irq_enabled to restore interrupt state. - * For page faults, gpfs and virtual interrupts, the hypervisor has saved - * eflags manually, otherwise it was delivered directly and so eflags reflects - * the real machine IF state, ie. interrupts on. Since the kernel always dies - * if it takes such a trap with interrupts disabled anyway, turning interrupts - * back on unconditionally here is OK. - */ +/*G:045 There is one final paravirt_op that the Guest implements, and glancing + * at it you can see why I left it to last. It's *cool*! It's in *assembler*! + * + * The "iret" instruction is used to return from an interrupt or trap. The + * stack looks like this: + * old address + * old code segment & privilege level + * old processor flags ("eflags") + * + * The "iret" instruction pops those values off the stack and restores them all + * at once. The only problem is that eflags includes the Interrupt Flag which + * the Guest can't change: the CPU will simply ignore it when we do an "iret". + * So we have to copy eflags from the stack to lguest_data.irq_enabled before + * we do the "iret". + * + * There are two problems with this: firstly, we need to use a register to do + * the copy and secondly, the whole thing needs to be atomic. The first + * problem is easy to solve: push %eax on the stack so we can use it, and then + * restore it at the end just before the real "iret". + * + * The second is harder: copying eflags to lguest_data.irq_enabled will turn + * interrupts on before we're finished, so we could be interrupted before we + * return to userspace or wherever. Our solution to this is to surround the + * code with lguest_noirq_start: and lguest_noirq_end: labels. We tell the + * Host that it is *never* to interrupt us there, even if interrupts seem to be + * enabled. */ ENTRY(lguest_iret) pushl %eax movl 12(%esp), %eax lguest_noirq_start: + /* Note the %ss: segment prefix here. Normal data accesses use the + * "ds" segment, but that will have already been restored for whatever + * we're returning to (such as userspace): we can't trust it. The %ss: + * prefix makes sure we use the stack segment, which is still valid. */ movl %eax,%ss:lguest_data+LGUEST_DATA_irq_enabled popl %eax iret =================================================================== --- a/include/linux/lguest.h +++ b/include/linux/lguest.h @@ -27,18 +27,38 @@ #define LG_CLOCK_MIN_DELTA 100UL #define LG_CLOCK_MAX_DELTA ULONG_MAX +/*G:031 First, how does our Guest contact the Host to ask for privileged + * operations? There are two ways: the direct way is to make a "hypercall", + * to make requests of the Host Itself. + * + * Our hypercall mechanism uses the highest unused trap code (traps 32 and + * above are used by real hardware interrupts). Seventeen hypercalls are + * available: the hypercall number is put in the %eax register, and the + * arguments (when required) are placed in %edx, %ebx and %ecx. If a return + * value makes sense, it's returned in %eax. + * + * Grossly invalid calls result in Sudden Death at the hands of the vengeful + * Host, rather than returning failure. This reflects Winston Churchill's + * definition of a gentleman: "someone who is only rude intentionally". */ #define LGUEST_TRAP_ENTRY 0x1F static inline unsigned long hcall(unsigned long call, unsigned long arg1, unsigned long arg2, unsigned long arg3) { + /* "int" is the Intel instruction to trigger a trap. */ asm volatile("int $" __stringify(LGUEST_TRAP_ENTRY) + /* The call is in %eax (aka "a"), and can be replaced */ : "=a"(call) + /* The other arguments are in %eax, %edx, %ebx & %ecx */ : "a"(call), "d"(arg1), "b"(arg2), "c"(arg3) + /* "memory" means this might write somewhere in memory. + * This isn't true for all calls, but it's safe to tell + * gcc that it might happen so it doesn't get clever. */ : "memory"); return call; } +/*:*/ void async_hcall(unsigned long call, unsigned long arg1, unsigned long arg2, unsigned long arg3); @@ -52,31 +72,40 @@ struct hcall_ring u32 eax, edx, ebx, ecx; }; -/* All the good stuff happens here: guest registers it with LGUEST_INIT */ +/*G:032 The second method of communicating with the Host is to via "struct + * lguest_data". The Guest's very first hypercall is to tell the Host where + * this is, and then the Guest and Host both publish information in it. :*/ struct lguest_data { -/* Fields which change during running: */ - /* 512 == enabled (same as eflags) */ + /* 512 == enabled (same as eflags in normal hardware). The Guest + * changes interrupts so often that a hypercall is too slow. */ unsigned int irq_enabled; - /* Interrupts blocked by guest. */ + /* Fine-grained interrupt disabling by the Guest */ DECLARE_BITMAP(blocked_interrupts, LGUEST_IRQS); - /* Virtual address of page fault. */ + /* The Host writes the virtual address of the last page fault here, + * which saves the Guest a hypercall. CR2 is the native register where + * this address would normally be found. */ unsigned long cr2; - /* Async hypercall ring. 0xFF == done, 0 == pending. */ + /* Async hypercall ring. Instead of directly making hypercalls, we can + * place them in here for processing the next time the Host wants. + * This batching can be quite efficient. */ + + /* 0xFF == done (set by Host), 0 == pending (set by Guest). */ u8 hcall_status[LHCALL_RING_SIZE]; + /* The actual registers for the hypercalls. */ struct hcall_ring hcalls[LHCALL_RING_SIZE]; -/* Fields initialized by the hypervisor at boot: */ +/* Fields initialized by the Host at boot: */ /* Memory not to try to access */ unsigned long reserve_mem; - /* ID of this guest (used by network driver to set ethernet address) */ + /* ID of this Guest (used by network driver to set ethernet address) */ u16 guestid; /* KHz for the TSC clock. */ u32 tsc_khz; -/* Fields initialized by the guest at boot: */ +/* Fields initialized by the Guest at boot: */ /* Instruction range to suppress interrupts even if enabled */ unsigned long noirq_start, noirq_end; }; ^ permalink raw reply [flat|nested] 8+ messages in thread
[parent not found: <1181272316.24958.13.camel@localhost.localdomain>]
* [PATCH 3/7] lguest documentation:Chapter III [not found] ` <1181272316.24958.13.camel@localhost.localdomain> @ 2007-06-08 3:12 ` Rusty Russell [not found] ` <1181272355.24958.15.camel@localhost.localdomain> 1 sibling, 0 replies; 8+ messages in thread From: Rusty Russell @ 2007-06-08 3:12 UTC (permalink / raw) To: Andrew Morton; +Cc: lkml - Kernel Mailing List, virtualization Documentation: The Drivers Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> --- drivers/block/lguest_blk.c | 171 +++++++++++++++++++++++++++--- drivers/char/hvc_lguest.c | 77 +++++++++++++ drivers/lguest/lguest_bus.c | 72 ++++++++++++ drivers/net/lguest_net.c | 222 +++++++++++++++++++++++++++++++++++---- include/linux/lguest_bus.h | 5 include/linux/lguest_launcher.h | 60 ++++++++++ 6 files changed, 565 insertions(+), 42 deletions(-) =================================================================== --- a/drivers/block/lguest_blk.c +++ b/drivers/block/lguest_blk.c @@ -1,6 +1,12 @@ -/* A simple block driver for lguest. - * - * Copyright 2006 Rusty Russell <rusty@rustcorp.com.au> IBM Corporation +/*D:400 + * The Guest block driver + * + * This is a simple block driver, which appears as /dev/lgba, lgbb, lgbc etc. + * The mechanism is simple: we place the information about the request in the + * device page, then use SEND_DMA (containing the data for a write, or an empty + * "ping" DMA for a read). + :*/ +/* Copyright 2006 Rusty Russell <rusty@rustcorp.com.au> IBM Corporation * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -25,27 +31,50 @@ static char next_block_index = 'a'; +/*D:420 Here is the structure which holds all the information we need about + * each Guest block device. + * + * I'm sure at this stage, you're wondering "hey, where was the adventure I was + * promised?" and thinking "Rusty sucks, I shall say nasty things about him on + * my blog". I think Real adventures have boring bits, too, and you're in the + * middle of one. But it gets better. Just not quite yet. */ struct blockdev { + /* The block queue infrastructure wants a spinlock: it is held while it + * calls our block request function. We grab it in our interrupt + * handler so the responses don't mess with new requests. */ spinlock_t lock; - /* The disk structure for the kernel. */ + /* The disk structure registered with kernel. */ struct gendisk *disk; - /* The major number for this disk. */ + /* The major device number for this disk, and the interrupt. We only + * really keep them here for completeness; we'd need them if we + * supported device unplugging. */ int major; int irq; + /* The physical address of this device's memory page */ unsigned long phys_addr; - /* The mapped block page. */ + /* The mapped memory page for convenient acces. */ struct lguest_block_page *lb_page; - /* We only have a single request outstanding at a time. */ + /* We only have a single request outstanding at a time: this is it. */ struct lguest_dma dma; struct request *req; }; -/* Jens gave me this nice helper to end all chunks of a request. */ +/*D:495 We originally used end_request() throughout the driver, but it turns + * out that end_request() is deprecated, and doesn't actually end the request + * (which seems like a good reason to deprecate it!). It simply ends the first + * bio. So if we had 3 bios in a "struct request" we would do all 3, + * end_request(), do 2, end_request(), do 1 and end_request(): twice as much + * work as we needed to do. + * + * This reinforced to me that I do not understand the block layer. + * + * Nonetheless, Jens Axboe gave me this nice helper to end all chunks of a + * request. This improved disk speed by 130%. */ static void end_entire_request(struct request *req, int uptodate) { if (end_that_request_first(req, uptodate, req->hard_nr_sectors)) @@ -55,30 +84,62 @@ static void end_entire_request(struct re end_that_request_last(req, uptodate); } +/* I'm told there are only two stories in the world worth telling: love and + * hate. So there used to be a love scene here like this: + * + * Launcher: We could make beautiful I/O together, you and I. + * Guest: My, that's a big disk! + * + * Unfortunately, it was just too raunchy for our otherwise-gentle tale. */ + +/*D:490 This is the interrupt handler, called when a block read or write has + * been completed for us. */ static irqreturn_t lgb_irq(int irq, void *_bd) { + /* We handed our "struct blockdev" as the argument to request_irq(), so + * it is passed through to us here. This tells us which device we're + * dealing with in case we have more than one. */ struct blockdev *bd = _bd; unsigned long flags; + /* We weren't doing anything? Strange, but could happen if we shared + * interrupts (we don't!). */ if (!bd->req) { pr_debug("No work!\n"); return IRQ_NONE; } + /* Not done yet? That's equally strange. */ if (!bd->lb_page->result) { pr_debug("No result!\n"); return IRQ_NONE; } + /* We have to grab the lock before ending the request. */ spin_lock_irqsave(&bd->lock, flags); + /* "result" is 1 for success, 2 for failure: end_entire_request() wants + * to know whether this succeeded or not. */ end_entire_request(bd->req, bd->lb_page->result == 1); + /* Clear out request, it's done. */ bd->req = NULL; + /* Reset incoming DMA for next time. */ bd->dma.used_len = 0; + /* Ready for more reads or writes */ blk_start_queue(bd->disk->queue); spin_unlock_irqrestore(&bd->lock, flags); + + /* The interrupt was for us, we dealt with it. */ return IRQ_HANDLED; } +/*D:480 The block layer's "struct request" contains a number of "struct bio"s, + * each of which contains "struct bio_vec"s, each of which contains a page, an + * offset and a length. + * + * Fortunately there are iterators to help us walk through the "struct + * request". Even more fortunately, there were plenty of places to steal the + * code from. We pack the "struct request" into our "struct lguest_dma" and + * return the total length. */ static unsigned int req_to_dma(struct request *req, struct lguest_dma *dma) { unsigned int i = 0, idx, len = 0; @@ -87,8 +148,13 @@ static unsigned int req_to_dma(struct re rq_for_each_bio(bio, req) { struct bio_vec *bvec; bio_for_each_segment(bvec, bio, idx) { + /* We told the block layer not to give us too many. */ BUG_ON(i == LGUEST_MAX_DMA_SECTIONS); + /* If we had a zero-length segment, it would look like + * the end of the data referred to by the "struct + * lguest_dma", so make sure that doesn't happen. */ BUG_ON(!bvec->bv_len); + /* Convert page & offset to a physical address */ dma->addr[i] = page_to_phys(bvec->bv_page) + bvec->bv_offset; dma->len[i] = bvec->bv_len; @@ -96,26 +162,39 @@ static unsigned int req_to_dma(struct re i++; } } + /* If the array isn't full, we mark the end with a 0 length */ if (i < LGUEST_MAX_DMA_SECTIONS) dma->len[i] = 0; return len; } +/* This creates an empty DMA, useful for prodding the Host without sending data + * (ie. when we want to do a read) */ static void empty_dma(struct lguest_dma *dma) { dma->len[0] = 0; } +/*D:470 Setting up a request is fairly easy: */ static void setup_req(struct blockdev *bd, int type, struct request *req, struct lguest_dma *dma) { + /* The type is 1 (write) or 0 (read). */ bd->lb_page->type = type; + /* The sector on disk where the read or write starts. */ bd->lb_page->sector = req->sector; + /* The result is initialized to 0 (unfinished). */ bd->lb_page->result = 0; + /* The current request (so we can end it in the interrupt handler). */ bd->req = req; + /* The number of bytes: returned as a side-effect of req_to_dma(), + * which packs the block layer's "struct request" into our "struct + * lguest_dma" */ bd->lb_page->bytes = req_to_dma(req, dma); } +/*D:450 Write is pretty straightforward: we pack the request into a "struct + * lguest_dma", then use SEND_DMA to send the request. */ static void do_write(struct blockdev *bd, struct request *req) { struct lguest_dma send; @@ -126,6 +205,9 @@ static void do_write(struct blockdev *bd lguest_send_dma(bd->phys_addr, &send); } +/* Read is similar to write, except we pack the request into our receive + * "struct lguest_dma" and send through an empty DMA just to tell the Host that + * there's a request pending. */ static void do_read(struct blockdev *bd, struct request *req) { struct lguest_dma ping; @@ -137,21 +219,30 @@ static void do_read(struct blockdev *bd, lguest_send_dma(bd->phys_addr, &ping); } +/*D:440 This where requests come in: we get handed the request queue and are + * expected to pull a "struct request" off it until we've finished them or + * we're waiting for a reply: */ static void do_lgb_request(request_queue_t *q) { struct blockdev *bd; struct request *req; again: + /* This sometimes returns NULL even on the very first time around. I + * wonder if it's something to do with letting elves handle the request + * queue... */ req = elv_next_request(q); if (!req) return; + /* We attached the struct blockdev to the disk: get it back */ bd = req->rq_disk->private_data; - /* Sometimes we get repeated requests after blk_stop_queue. */ + /* Sometimes we get repeated requests after blk_stop_queue(), but we + * can only handle one at a time. */ if (bd->req) return; + /* We only do reads and writes: no tricky business! */ if (!blk_fs_request(req)) { pr_debug("Got non-command 0x%08x\n", req->cmd_type); req->errors++; @@ -164,20 +255,31 @@ again: else do_read(bd, req); - /* Wait for interrupt to tell us it's done. */ + /* We've put out the request, so stop any more coming in until we get + * an interrupt, which takes us to lgb_irq() to re-enable the queue. */ blk_stop_queue(q); } +/*D:430 This is the "struct block_device_operations" we attach to the disk at + * the end of lguestblk_probe(). It doesn't seem to want much. */ static struct block_device_operations lguestblk_fops = { .owner = THIS_MODULE, }; +/*D:425 Setting up a disk device seems to involve a lot of code. I'm not sure + * quite why. I do know that the IDE code sent two or three of the maintainers + * insane, perhaps this is the fringe of the same disease? + * + * As in the console code, the probe function gets handed the generic + * lguest_device from lguest_bus.c: */ static int lguestblk_probe(struct lguest_device *lgdev) { struct blockdev *bd; int err; int irqflags = IRQF_SHARED; + /* First we allocate our own "struct blockdev" and initialize the easy + * fields. */ bd = kmalloc(sizeof(*bd), GFP_KERNEL); if (!bd) return -ENOMEM; @@ -187,59 +289,100 @@ static int lguestblk_probe(struct lguest bd->req = NULL; bd->dma.used_len = 0; bd->dma.len[0] = 0; + /* The descriptor in the lguest_devices array provided by the Host + * gives the Guest the physical page number of the device's page. */ bd->phys_addr = (lguest_devices[lgdev->index].pfn << PAGE_SHIFT); + /* We use lguest_map() to get a pointer to the device page */ bd->lb_page = lguest_map(bd->phys_addr, 1); if (!bd->lb_page) { err = -ENOMEM; goto out_free_bd; } + /* We need a major device number: 0 means "assign one dynamically". */ bd->major = register_blkdev(0, "lguestblk"); if (bd->major < 0) { err = bd->major; goto out_unmap; } + /* This allocates a "struct gendisk" where we pack all the information + * about the disk which the rest of Linux sees. We ask for one minor + * number; I do wonder if we should be asking for more. */ bd->disk = alloc_disk(1); if (!bd->disk) { err = -ENOMEM; goto out_unregister_blkdev; } + /* Every disk needs a queue for requests to come in: we set up the + * queue with a callback function (the core of our driver) and the lock + * to use. */ bd->disk->queue = blk_init_queue(do_lgb_request, &bd->lock); if (!bd->disk->queue) { err = -ENOMEM; goto out_put_disk; } - /* We can only handle a certain number of sg entries */ + /* We can only handle a certain number of pointers in our SEND_DMA + * call, so we set that with blk_queue_max_hw_segments(). This is not + * to be confused with blk_queue_max_phys_segments() of course! I + * know, who could possibly confuse the two? + * + * Well, it's simple to tell them apart: this one seems to work and the + * other one didn't. */ blk_queue_max_hw_segments(bd->disk->queue, LGUEST_MAX_DMA_SECTIONS); - /* Buffers must not cross page boundaries */ + + /* Due to technical limitations of our Host (and simple coding) we + * can't have a single buffer which crosses a page boundary. Tell it + * here. This means that our maximum request size is 16 + * (LGUEST_MAX_DMA_SECTIONS) pages. */ blk_queue_segment_boundary(bd->disk->queue, PAGE_SIZE-1); + /* We name our disk: this becomes the device name when udev does its + * magic thing and creates the device node, such as /dev/lgba. + * next_block_index is a global which starts at 'a'. Unfortunately + * this simple increment logic means that the 27th disk will be called + * "/dev/lgb{". In that case, I recommend having at least 29 disks, so + * your /dev directory will be balanced. */ sprintf(bd->disk->disk_name, "lgb%c", next_block_index++); + + /* We look to the device descriptor again to see if this device's + * interrupts are expected to be random. If they are, we tell the irq + * subsystem. At the moment this bit is always set. */ if (lguest_devices[lgdev->index].features & LGUEST_DEVICE_F_RANDOMNESS) irqflags |= IRQF_SAMPLE_RANDOM; + + /* Now we have the name and irqflags, we can request the interrupt; we + * give it the "struct blockdev" we have set up to pass to lgb_irq() + * when there is an interrupt. */ err = request_irq(bd->irq, lgb_irq, irqflags, bd->disk->disk_name, bd); if (err) goto out_cleanup_queue; + /* We bind our one-entry DMA pool to the key for this block device so + * the Host can reply to our requests. The key is equal to the + * physical address of the device's page, which is conveniently + * unique. */ err = lguest_bind_dma(bd->phys_addr, &bd->dma, 1, bd->irq); if (err) goto out_free_irq; + /* We finish our disk initialization and add the disk to the system. */ bd->disk->major = bd->major; bd->disk->first_minor = 0; bd->disk->private_data = bd; bd->disk->fops = &lguestblk_fops; - /* This is initialized to the disk size by the other end. */ + /* This is initialized to the disk size by the Launcher. */ set_capacity(bd->disk, bd->lb_page->num_sectors); add_disk(bd->disk); printk(KERN_INFO "%s: device %i at major %d\n", bd->disk->disk_name, lgdev->index, bd->major); + /* We don't need to keep the "struct blockdev" around, but if we ever + * implemented device removal, we'd need this. */ lgdev->private = bd; return 0; @@ -258,6 +401,8 @@ out_free_bd: return err; } +/*D:410 The boilerplate code for registering the lguest block driver is just + * like the console: */ static struct lguest_driver lguestblk_drv = { .name = "lguestblk", .owner = THIS_MODULE, =================================================================== --- a/drivers/char/hvc_lguest.c +++ b/drivers/char/hvc_lguest.c @@ -1,6 +1,19 @@ -/* Simple console for lguest. +/*D:300 + * The Guest console driver * - * Copyright (C) 2006 Rusty Russell, IBM Corporation + * This is a trivial console driver: we use lguest's DMA mechanism to send + * bytes out, and register a DMA buffer to receive bytes in. It is assumed to + * be present and available from the very beginning of boot. + * + * Writing console drivers is one of the few remaining Dark Arts in Linux. + * Fortunately for us, the path of virtual consoles has been well-trodden by + * the PowerPC folks, who wrote "hvc_console.c" to generically support any + * virtual console. We use that infrastructure which only requires us to write + * the basic put_chars and get_chars functions and call the right register + * functions. + :*/ + +/* Copyright (C) 2006 Rusty Russell, IBM Corporation * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -21,49 +34,81 @@ #include <linux/lguest_bus.h> #include "hvc_console.h" +/*D:340 This is our single console input buffer, with associated "struct + * lguest_dma" referring to it. Note the 0-terminated length array, and the + * use of physical address for the buffer itself. */ static char inbuf[256]; static struct lguest_dma cons_input = { .used_len = 0, .addr[0] = __pa(inbuf), .len[0] = sizeof(inbuf), .len[1] = 0 }; +/*D:310 The put_chars() callback is pretty straightforward. + * + * First we put the pointer and length in a "struct lguest_dma": we only have + * one pointer, so we set the second length to 0. Then we use SEND_DMA to send + * the data to (Host) buffers attached to the console key. Usually a device's + * key is a physical address within the device's memory, but because the + * console device doesn't have any associated physical memory, we use the + * LGUEST_CONSOLE_DMA_KEY constant (aka 0). */ static int put_chars(u32 vtermno, const char *buf, int count) { struct lguest_dma dma; - /* FIXME: what if it's over a page boundary? */ + /* FIXME: DMA buffers in a "struct lguest_dma" are not allowed + * to go over page boundaries. This never seems to happen, + * but if it did we'd need to fix this code. */ dma.len[0] = count; dma.len[1] = 0; dma.addr[0] = __pa(buf); lguest_send_dma(LGUEST_CONSOLE_DMA_KEY, &dma); + /* We're expected to return the amount of data we wrote: all of it. */ return count; } +/*D:350 get_chars() is the callback from the hvc_console infrastructure when + * an interrupt is received. + * + * Firstly we see if our buffer has been filled: if not, we return. The rest + * of the code deals with the fact that the hvc_console() infrastructure only + * asks us for 16 bytes at a time. We keep a "cons_offset" variable for + * partially-read buffers. */ static int get_chars(u32 vtermno, char *buf, int count) { static int cons_offset; + /* Nothing left to see here... */ if (!cons_input.used_len) return 0; + /* You want more than we have to give? Well, try wanting less! */ if (cons_input.used_len - cons_offset < count) count = cons_input.used_len - cons_offset; + /* Copy across to their buffer and increment offset. */ memcpy(buf, inbuf + cons_offset, count); cons_offset += count; + + /* Finished? Zero offset, and reset cons_input so Host will use it + * again. */ if (cons_offset == cons_input.used_len) { cons_offset = 0; cons_input.used_len = 0; } return count; } +/*:*/ static struct hv_ops lguest_cons = { .get_chars = get_chars, .put_chars = put_chars, }; +/*D:320 Console drivers are initialized very early so boot messages can go + * out. At this stage, the console is output-only. Our driver checks we're a + * Guest, and if so hands hvc_instantiate() the console number (0), priority + * (0), and the struct hv_ops containing the put_chars() function. */ static int __init cons_init(void) { if (strcmp(paravirt_ops.name, "lguest") != 0) @@ -73,21 +118,46 @@ static int __init cons_init(void) } console_initcall(cons_init); +/*D:370 To set up and manage our virtual console, we call hvc_alloc() and + * stash the result in the private pointer of the "struct lguest_device". + * Since we never remove the console device we never need this pointer again, + * but using ->private is considered good form, and you never know who's going + * to copy your driver. + * + * Once the console is set up, we bind our input buffer ready for input. */ static int lguestcons_probe(struct lguest_device *lgdev) { int err; + /* The first argument of hvc_alloc() is the virtual console number, so + * we use zero. The second argument is the interrupt number. + * + * The third argument is a "struct hv_ops" containing the put_chars() + * and get_chars() pointers. The final argument is the output buffer + * size: we use 256 and expect the Host to have room for us to send + * that much. */ lgdev->private = hvc_alloc(0, lgdev_irq(lgdev), &lguest_cons, 256); if (IS_ERR(lgdev->private)) return PTR_ERR(lgdev->private); + /* We bind a single DMA buffer at key LGUEST_CONSOLE_DMA_KEY. + * "cons_input" is that statically-initialized global DMA buffer we saw + * above, and we also give the interrupt we want. */ err = lguest_bind_dma(LGUEST_CONSOLE_DMA_KEY, &cons_input, 1, lgdev_irq(lgdev)); if (err) printk("lguest console: failed to bind buffer.\n"); return err; } +/* Note the use of lgdev_irq() for the interrupt number. We tell hvc_alloc() + * to expect input when this interrupt is triggered, and then tell + * lguest_bind_dma() that is the interrupt to send us when input comes in. */ +/*D:360 From now on the console driver follows standard Guest driver form: + * register_lguest_driver() registers the device type and probe function, and + * the probe function sets up the device. + * + * The standard "struct lguest_driver": */ static struct lguest_driver lguestcons_drv = { .name = "lguestcons", .owner = THIS_MODULE, @@ -95,6 +165,7 @@ static struct lguest_driver lguestcons_d .probe = lguestcons_probe, }; +/* The standard init function */ static int __init hvc_lguest_init(void) { return register_lguest_driver(&lguestcons_drv); =================================================================== --- a/drivers/lguest/lguest_bus.c +++ b/drivers/lguest/lguest_bus.c @@ -46,6 +46,10 @@ static struct device_attribute lguest_de __ATTR_NULL }; +/*D:130 The generic bus infrastructure requires a function which says whether a + * device matches a driver. For us, it is simple: "struct lguest_driver" + * contains a "device_type" field which indicates what type of device it can + * handle, so we just cast the args and compare: */ static int lguest_dev_match(struct device *_dev, struct device_driver *_drv) { struct lguest_device *dev = container_of(_dev,struct lguest_device,dev); @@ -53,6 +57,7 @@ static int lguest_dev_match(struct devic return (drv->device_type == lguest_devices[dev->index].type); } +/*:*/ struct lguest_bus { struct bus_type bus; @@ -71,11 +76,24 @@ static struct lguest_bus lguest_bus = { } }; +/*D:140 This is the callback which occurs once the bus infrastructure matches + * up a device and driver, ie. in response to add_lguest_device() calling + * device_register(), or register_lguest_driver() calling driver_register(). + * + * At the moment it's always the latter: the devices are added first, since + * scan_devices() is called from a "core_initcall", and the drivers themselves + * called later as a normal "initcall". But it would work the other way too. + * + * So now we have the happy couple, we add the status bit to indicate that we + * found a driver. If the driver truly loves the device, it will return + * happiness from its probe function (ok, perhaps this wasn't my greatest + * analogy), and we set the final "driver ok" bit so the Host sees it's all + * green. */ static int lguest_dev_probe(struct device *_dev) { int ret; - struct lguest_device *dev = container_of(_dev,struct lguest_device,dev); - struct lguest_driver *drv = container_of(dev->dev.driver, + struct lguest_device*dev = container_of(_dev,struct lguest_device,dev); + struct lguest_driver*drv = container_of(dev->dev.driver, struct lguest_driver, drv); lguest_devices[dev->index].status |= LGUEST_DEVICE_S_DRIVER; @@ -85,6 +103,10 @@ static int lguest_dev_probe(struct devic return ret; } +/* The last part of the bus infrastructure is the function lguest drivers use + * to register themselves. Firstly, we do nothing if there's no lguest bus + * (ie. this is not a Guest), otherwise we fill in the embedded generic "struct + * driver" fields and call the generic driver_register(). */ int register_lguest_driver(struct lguest_driver *drv) { if (!lguest_devices) @@ -97,12 +119,36 @@ int register_lguest_driver(struct lguest return driver_register(&drv->drv); } + +/* At the moment we build all the drivers into the kernel because they're so + * simple: 8144 bytes for all three of them as I type this. And as the console + * really needs to be built in, it's actually only 3527 bytes for the network + * and block drivers. + * + * If they get complex it will make sense for them to be modularized, so we + * need to explicitly export the symbol. + * + * I don't think non-GPL modules make sense, so it's a GPL-only export. + */ EXPORT_SYMBOL_GPL(register_lguest_driver); +/*D:120 This is the core of the lguest bus: actually adding a new device. + * It's a separate function because it's neater that way, and because an + * earlier version of the code supported hotplug and unplug. They were removed + * early on because they were never used. + * + * As Andrew Tridgell says, "Untested code is buggy code". + * + * It's worth reading this carefully: we start with an index into the array of + * "struct lguest_device_desc"s indicating the device which is new: */ static void add_lguest_device(unsigned int index) { struct lguest_device *new; + /* Each "struct lguest_device_desc" has a "status" field, which the + * Guest updates as the device is probed. In the worst case, the Host + * can look at these bits to tell what part of device setup failed, + * even if the console isn't available. */ lguest_devices[index].status |= LGUEST_DEVICE_S_ACKNOWLEDGE; new = kmalloc(sizeof(struct lguest_device), GFP_KERNEL); if (!new) { @@ -111,12 +157,17 @@ static void add_lguest_device(unsigned i return; } + /* The "struct lguest_device" setup is pretty straight-forward example + * code. */ new->index = index; new->private = NULL; memset(&new->dev, 0, sizeof(new->dev)); new->dev.parent = &lguest_bus.dev; new->dev.bus = &lguest_bus.bus; sprintf(new->dev.bus_id, "%u", index); + + /* device_register() causes the bus infrastructure to look for a + * matching driver. */ if (device_register(&new->dev) != 0) { printk(KERN_EMERG "Cannot register lguest device %u\n", index); lguest_devices[index].status |= LGUEST_DEVICE_S_FAILED; @@ -124,6 +175,9 @@ static void add_lguest_device(unsigned i } } +/*D:110 scan_devices() simply iterates through the device array. The type 0 + * is reserved to mean "no device", and anything else means we have found a + * device: add it. */ static void scan_devices(void) { unsigned int i; @@ -133,12 +187,23 @@ static void scan_devices(void) add_lguest_device(i); } +/*D:100 Fairly early in boot, lguest_bus_init() is called to set up the lguest + * bus. We check that we are a Guest by checking paravirt_ops.name: there are + * other ways of checking, but this seems most obvious to me. + * + * So we can access the array of "struct lguest_device_desc"s easily, we map + * that memory and store the pointer in the global "lguest_devices". Then we + * register the bus with the core. Doing two registrations seems clunky to me, + * but it seems to be the correct sysfs incantation. + * + * Finally we call scan_devices() which adds all the devices found in the + * "struct lguest_device_desc" array. */ static int __init lguest_bus_init(void) { if (strcmp(paravirt_ops.name, "lguest") != 0) return 0; - /* Devices are in page above top of "normal" mem. */ + /* Devices are in a single page above top of "normal" mem */ lguest_devices = lguest_map(max_pfn<<PAGE_SHIFT, 1); if (bus_register(&lguest_bus.bus) != 0 @@ -148,4 +213,5 @@ static int __init lguest_bus_init(void) scan_devices(); return 0; } +/* Do this after core stuff, before devices. */ postcore_initcall(lguest_bus_init); =================================================================== --- a/drivers/net/lguest_net.c +++ b/drivers/net/lguest_net.c @@ -1,6 +1,13 @@ -/* A simple network driver for lguest. - * - * Copyright 2006 Rusty Russell <rusty@rustcorp.com.au> IBM Corporation +/*D:500 + * The Guest network driver. + * + * This is very simple a virtual network driver, and our last Guest driver. + * The only trick is that it can talk directly to multiple other recipients + * (ie. other Guests on the same network). It can also be used with only the + * Host on the network. + :*/ + +/* Copyright 2006 Rusty Russell <rusty@rustcorp.com.au> IBM Corporation * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -27,23 +34,28 @@ #define MAX_LANS 4 #define NUM_SKBS 8 +/*D:530 The "struct lguestnet_info" contains all the information we need to + * know about the network device. */ struct lguestnet_info { - /* The shared page(s). */ + /* The mapped device page(s) (an array of "struct lguest_net"). */ struct lguest_net *peer; + /* The physical address of the device page(s) */ unsigned long peer_phys; + /* The size of the device page(s). */ unsigned long mapsize; /* The lguest_device I come from */ struct lguest_device *lgdev; - /* My peerid. */ + /* My peerid (ie. my slot in the array). */ unsigned int me; - /* Receive queue. */ + /* Receive queue: the network packets waiting to be filled. */ struct sk_buff *skb[NUM_SKBS]; struct lguest_dma dma[NUM_SKBS]; }; +/*:*/ /* How many bytes left in this page. */ static unsigned int rest_of_page(void *data) @@ -51,39 +63,82 @@ static unsigned int rest_of_page(void *d return PAGE_SIZE - ((unsigned long)data % PAGE_SIZE); } -/* Simple convention: offset 4 * peernum. */ +/*D:570 Each peer (ie. Guest or Host) on the network binds their receive + * buffers to a different key: we simply use the physical address of the + * device's memory page plus the peer number. The Host insists that all keys + * be a multiple of 4, so we multiply the peer number by 4. */ static unsigned long peer_key(struct lguestnet_info *info, unsigned peernum) { return info->peer_phys + 4 * peernum; } +/* This is the routine which sets up a "struct lguest_dma" to point to a + * network packet, similar to req_to_dma() in lguest_blk.c. The structure of a + * "struct sk_buff" has grown complex over the years: it consists of a "head" + * linear section pointed to by "skb->data", and possibly an array of + * "fragments" in the case of a non-linear packet. + * + * Our receive buffers don't use fragments at all but outgoing skbs might, so + * we handle it. */ static void skb_to_dma(const struct sk_buff *skb, unsigned int headlen, struct lguest_dma *dma) { unsigned int i, seg; + /* First, we put the linear region into the "struct lguest_dma". Each + * entry can't go over a page boundary, so even though all our packets + * are 1514 bytes or less, we might need to use two entries here: */ for (i = seg = 0; i < headlen; seg++, i += rest_of_page(skb->data+i)) { dma->addr[seg] = virt_to_phys(skb->data + i); dma->len[seg] = min((unsigned)(headlen - i), rest_of_page(skb->data + i)); } + + /* Now we handle the fragments: at least they're guaranteed not to go + * over a page. skb_shinfo(skb) returns a pointer to the structure + * which tells us about the number of fragments and the fragment + * array. */ for (i = 0; i < skb_shinfo(skb)->nr_frags; i++, seg++) { const skb_frag_t *f = &skb_shinfo(skb)->frags[i]; /* Should not happen with MTU less than 64k - 2 * PAGE_SIZE. */ if (seg == LGUEST_MAX_DMA_SECTIONS) { + /* We will end up sending a truncated packet should + * this ever happen. Plus, a cool log message! */ printk("Woah dude! Megapacket!\n"); break; } dma->addr[seg] = page_to_phys(f->page) + f->page_offset; dma->len[seg] = f->size; } + + /* If after all that we didn't use the entire "struct lguest_dma" + * array, we terminate it with a 0 length. */ if (seg < LGUEST_MAX_DMA_SECTIONS) dma->len[seg] = 0; } -/* We overload multicast bit to show promiscuous mode. */ +/* + * Packet transmission. + * + * Our packet transmission is a little unusual. A real network card would just + * send out the packet and leave the receivers to decide if they're interested. + * Instead, we look through the network device memory page and see if any of + * the ethernet addresses match the packet destination, and if so we send it to + * that Guest. + * + * This is made a little more complicated in two cases. The first case is + * broadcast packets: for that we send the packet to all Guests on the network, + * one at a time. The second case is "promiscuous" mode, where a Guest wants + * to see all the packets on the network. We need a way for the Guest to tell + * us it wants to see all packets, so it sets the "multicast" bit on its + * published MAC address, which is never valid in a real ethernet address. + */ #define PROMISC_BIT 0x01 +/* This is the callback which is summoned whenever the network device's + * multicast or promiscuous state changes. If the card is in promiscuous mode, + * we advertise that in our ethernet address in the device's memory. We do the + * same if Linux wants any or all multicast traffic. */ static void lguestnet_set_multicast(struct net_device *dev) { struct lguestnet_info *info = netdev_priv(dev); @@ -94,11 +149,14 @@ static void lguestnet_set_multicast(stru info->peer[info->me].mac[0] &= ~PROMISC_BIT; } +/* A simple test function to see if a peer wants to see all packets.*/ static int promisc(struct lguestnet_info *info, unsigned int peer) { return info->peer[peer].mac[0] & PROMISC_BIT; } +/* Another simple function to see if a peer's advertised ethernet address + * matches a packet's destination ethernet address. */ static int mac_eq(const unsigned char mac[ETH_ALEN], struct lguestnet_info *info, unsigned int peer) { @@ -108,6 +166,8 @@ static int mac_eq(const unsigned char ma return memcmp(mac+1, info->peer[peer].mac+1, ETH_ALEN-1) == 0; } +/* This is the function which actually sends a packet once we've decided a + * peer wants it: */ static void transfer_packet(struct net_device *dev, struct sk_buff *skb, unsigned int peernum) @@ -115,76 +175,134 @@ static void transfer_packet(struct net_d struct lguestnet_info *info = netdev_priv(dev); struct lguest_dma dma; + /* We use our handy "struct lguest_dma" packing function to prepare + * the skb for sending. */ skb_to_dma(skb, skb_headlen(skb), &dma); pr_debug("xfer length %04x (%u)\n", htons(skb->len), skb->len); + /* This is the actual send call which copies the packet. */ lguest_send_dma(peer_key(info, peernum), &dma); + + /* Check that the entire packet was transmitted. If not, it could mean + * that the other Guest registered a short receive buffer, but this + * driver should never do that. More likely, the peer is dead. */ if (dma.used_len != skb->len) { dev->stats.tx_carrier_errors++; pr_debug("Bad xfer to peer %i: %i of %i (dma %p/%i)\n", peernum, dma.used_len, skb->len, (void *)dma.addr[0], dma.len[0]); } else { + /* On success we update the stats. */ dev->stats.tx_bytes += skb->len; dev->stats.tx_packets++; } } +/* Another helper function to tell is if a slot in the device memory is unused. + * Since we always set the Local Assignment bit in the ethernet address, the + * first byte can never be 0. */ static int unused_peer(const struct lguest_net peer[], unsigned int num) { return peer[num].mac[0] == 0; } +/* Finally, here is the routine which handles an outgoing packet. It's called + * "start_xmit" for traditional reasons. */ static int lguestnet_start_xmit(struct sk_buff *skb, struct net_device *dev) { unsigned int i; int broadcast; struct lguestnet_info *info = netdev_priv(dev); + /* Extract the destination ethernet address from the packet. */ const unsigned char *dest = ((struct ethhdr *)skb->data)->h_dest; pr_debug("%s: xmit %02x:%02x:%02x:%02x:%02x:%02x\n", dev->name, dest[0],dest[1],dest[2],dest[3],dest[4],dest[5]); + /* If it's a multicast packet, we broadcast to everyone. That's not + * very efficient, but there are very few applications which actually + * use multicast, which is a shame really. + * + * As etherdevice.h points out: "By definition the broadcast address is + * also a multicast address." So we don't have to test for broadcast + * packets separately. */ broadcast = is_multicast_ether_addr(dest); + + /* Look through all the published ethernet addresses to see if we + * should send this packet. */ for (i = 0; i < info->mapsize/sizeof(struct lguest_net); i++) { + /* We don't send to ourselves (we actually can't SEND_DMA to + * ourselves anyway), and don't send to unused slots.*/ if (i == info->me || unused_peer(info->peer, i)) continue; + /* If it's broadcast we send it. If they want every packet we + * send it. If the destination matches their address we send + * it. Otherwise we go to the next peer. */ if (!broadcast && !promisc(info, i) && !mac_eq(dest, info, i)) continue; pr_debug("lguestnet %s: sending from %i to %i\n", dev->name, info->me, i); + /* Our routine which actually does the transfer. */ transfer_packet(dev, skb, i); } + + /* An xmit routine is expected to dispose of the packet, so we do. */ dev_kfree_skb(skb); + + /* As per kernel convention, 0 means success. This is why I love + * networking: even if we never sent to anyone, that's still + * success! */ return 0; } -/* Find a new skb to put in this slot in shared mem. */ +/*D:560 + * Packet receiving. + * + * First, here's a helper routine which fills one of our array of receive + * buffers: */ static int fill_slot(struct net_device *dev, unsigned int slot) { struct lguestnet_info *info = netdev_priv(dev); - /* Try to create and register a new one. */ + + /* We can receive ETH_DATA_LEN (1500) byte packets, plus a standard + * ethernet header of ETH_HLEN (14) bytes. */ info->skb[slot] = netdev_alloc_skb(dev, ETH_HLEN + ETH_DATA_LEN); if (!info->skb[slot]) { printk("%s: could not fill slot %i\n", dev->name, slot); return -ENOMEM; } + /* skb_to_dma() is a helper which sets up the "struct lguest_dma" to + * point to the data in the skb: we also use it for sending out a + * packet. */ skb_to_dma(info->skb[slot], ETH_HLEN + ETH_DATA_LEN, &info->dma[slot]); + + /* This is a Write Memory Barrier: it ensures that the entry in the + * receive buffer array is written *before* we set the "used_len" entry + * to 0. If the Host were looking at the receive buffer array from a + * different CPU, it could potentially see "used_len = 0" and not see + * the updated receive buffer information. This would be a horribly + * nasty bug, so make sure the compiler and CPU know this has to happen + * first. */ wmb(); - /* Now we tell hypervisor it can use the slot. */ + /* Writing 0 to "used_len" tells the Host it can use this receive + * buffer now. */ info->dma[slot].used_len = 0; return 0; } +/* This is the actual receive routine. When we receive an interrupt from the + * Host to tell us a packet has been delivered, we arrive here: */ static irqreturn_t lguestnet_rcv(int irq, void *dev_id) { struct net_device *dev = dev_id; struct lguestnet_info *info = netdev_priv(dev); unsigned int i, done = 0; + /* Look through our entire receive array for an entry which has data + * in it. */ for (i = 0; i < ARRAY_SIZE(info->dma); i++) { unsigned int length; struct sk_buff *skb; @@ -193,10 +311,16 @@ static irqreturn_t lguestnet_rcv(int irq if (length == 0) continue; + /* We've found one! Remember the skb (we grabbed the length + * above), and immediately refill the slot we've taken it + * from. */ done++; skb = info->skb[i]; fill_slot(dev, i); + /* This shouldn't happen: micropackets could be sent by a + * badly-behaved Guest on the network, but the Host will never + * stuff more data in the buffer than the buffer length. */ if (length < ETH_HLEN || length > ETH_HLEN + ETH_DATA_LEN) { pr_debug(KERN_WARNING "%s: unbelievable skb len: %i\n", dev->name, length); @@ -204,36 +328,72 @@ static irqreturn_t lguestnet_rcv(int irq continue; } + /* skb_put(), what a great function! I've ranted about this + * function before (http://lkml.org/lkml/1999/9/26/24). You + * call it after you've added data to the end of an skb (in + * this case, it was the Host which wrote the data). */ skb_put(skb, length); + + /* The ethernet header contains a protocol field: we use the + * standard helper to extract it, and place the result in + * skb->protocol. The helper also sets up skb->pkt_type and + * eats up the ethernet header from the front of the packet. */ skb->protocol = eth_type_trans(skb, dev); - /* This is a reliable transport. */ + + /* If this device doesn't need checksums for sending, we also + * don't need to check the packets when they come in. */ if (dev->features & NETIF_F_NO_CSUM) skb->ip_summed = CHECKSUM_UNNECESSARY; + + /* As a last resort for debugging the driver or the lguest I/O + * subsystem, you can uncomment the "#define DEBUG" at the top + * of this file, which turns all the pr_debug() into printk() + * and floods the logs. */ pr_debug("Receiving skb proto 0x%04x len %i type %i\n", ntohs(skb->protocol), skb->len, skb->pkt_type); + /* Update the packet and byte counts (visible from ifconfig, + * and good for debugging). */ dev->stats.rx_bytes += skb->len; dev->stats.rx_packets++; + + /* Hand our fresh network packet into the stack's "network + * interface receive" routine. That will free the packet + * itself when it's finished. */ netif_rx(skb); } + + /* If we found any packets, we assume the interrupt was for us. */ return done ? IRQ_HANDLED : IRQ_NONE; } +/*D:550 This is where we start: when the device is brought up by dhcpd or + * ifconfig. At this point we advertise our MAC address to the rest of the + * network, and register receive buffers ready for incoming packets. */ static int lguestnet_open(struct net_device *dev) { int i; struct lguestnet_info *info = netdev_priv(dev); - /* Set up our MAC address */ + /* Copy our MAC address into the device page, so others on the network + * can find us. */ memcpy(info->peer[info->me].mac, dev->dev_addr, ETH_ALEN); - /* Turn on promisc mode if needed */ + /* We might already be in promisc mode (dev->flags & IFF_PROMISC). Our + * set_multicast callback handles this already, so we call it now. */ lguestnet_set_multicast(dev); + /* Allocate packets and put them into our "struct lguest_dma" array. + * If we fail to allocate all the packets we could still limp along, + * but it's a sign of real stress so we should probably give up now. */ for (i = 0; i < ARRAY_SIZE(info->dma); i++) { if (fill_slot(dev, i) != 0) goto cleanup; } + + /* Finally we tell the Host where our array of "struct lguest_dma" + * receive buffers is, binding it to the key corresponding to the + * device's physical memory plus our peerid. */ if (lguest_bind_dma(peer_key(info,info->me), info->dma, NUM_SKBS, lgdev_irq(info->lgdev)) != 0) goto cleanup; @@ -244,22 +404,29 @@ cleanup: dev_kfree_skb(info->skb[i]); return -ENOMEM; } - +/*:*/ + +/* The close routine is called when the device is no longer in use: we clean up + * elegantly. */ static int lguestnet_close(struct net_device *dev) { unsigned int i; struct lguestnet_info *info = netdev_priv(dev); - /* Clear all trace: others might deliver packets, we'll ignore it. */ + /* Clear all trace of our existence out of the device memory by setting + * the slot which held our MAC address to 0 (unused). */ memset(&info->peer[info->me], 0, sizeof(info->peer[info->me])); - /* Deregister sg lists. */ + /* Unregister our array of receive buffers */ lguest_unbind_dma(peer_key(info, info->me), info->dma); for (i = 0; i < ARRAY_SIZE(info->dma); i++) dev_kfree_skb(info->skb[i]); return 0; } +/*D:510 The network device probe function is basically a standard ethernet + * device setup. It reads the "struct lguest_device_desc" and sets the "struct + * net_device". Oh, the line-by-line excitement! Let's skip over it. :*/ static int lguestnet_probe(struct lguest_device *lgdev) { int err, irqf = IRQF_SHARED; @@ -289,10 +456,16 @@ static int lguestnet_probe(struct lguest dev->stop = lguestnet_close; dev->hard_start_xmit = lguestnet_start_xmit; - /* Turning on/off promisc will call dev->set_multicast_list. - * We don't actually support multicast yet */ + /* We don't actually support multicast yet, but turning on/off + * promisc also calls dev->set_multicast_list. */ dev->set_multicast_list = lguestnet_set_multicast; SET_NETDEV_DEV(dev, &lgdev->dev); + + /* The network code complains if you have "scatter-gather" capability + * if you don't also handle checksums (it seem that would be + * "illogical"). So we use a lie of omission and don't tell it that we + * can handle scattered packets unless we also don't want checksums, + * even though to us they're completely independent. */ if (desc->features & LGUEST_NET_F_NOCSUM) dev->features = NETIF_F_SG|NETIF_F_NO_CSUM; @@ -324,6 +497,9 @@ static int lguestnet_probe(struct lguest } pr_debug("lguestnet: registered device %s\n", dev->name); + /* Finally, we put the "struct net_device" in the generic "struct + * lguest_device"s private pointer. Again, it's not necessary, but + * makes sure the cool kernel kids don't tease us. */ lgdev->private = dev; return 0; @@ -351,3 +527,11 @@ module_init(lguestnet_init); MODULE_DESCRIPTION("Lguest network driver"); MODULE_LICENSE("GPL"); + +/*D:580 + * This is the last of the Drivers, and with this we have covered the many and + * wonderous and fine (and boring) details of the Guest. + * + * "make Launcher" beckons, where we answer questions like "Where do Guests + * come from?", and "What do you do when someone asks for optimization?" + */ =================================================================== --- a/include/linux/lguest_bus.h +++ b/include/linux/lguest_bus.h @@ -15,11 +15,14 @@ struct lguest_device { void *private; }; -/* By convention, each device can use irq index+1 if it wants to. */ +/*D:380 Since interrupt numbers are arbitrary, we use a convention: each device + * can use the interrupt number corresponding to its index. The +1 is because + * interrupt 0 is not usable (it's actually the timer interrupt). */ static inline int lgdev_irq(const struct lguest_device *dev) { return dev->index + 1; } +/*:*/ /* dma args must not be vmalloced! */ void lguest_send_dma(unsigned long key, struct lguest_dma *dma); =================================================================== --- a/include/linux/lguest_launcher.h +++ b/include/linux/lguest_launcher.h @@ -9,14 +9,45 @@ /* How many devices? Assume each one wants up to two dma arrays per device. */ #define LGUEST_MAX_DEVICES (LGUEST_MAX_DMA/2) +/*D:200 + * Lguest I/O + * + * The lguest I/O mechanism is the only way Guests can talk to devices. There + * are two hypercalls involved: SEND_DMA for output and BIND_DMA for input. In + * each case, "struct lguest_dma" describes the buffer: this contains 16 + * addr/len pairs, and if there are fewer buffer elements the len array is + * terminated with a 0. + * + * I/O is organized by keys: BIND_DMA attaches buffers to a particular key, and + * SEND_DMA transfers to buffers bound to particular key. By convention, keys + * correspond to a physical address within the device's page. This means that + * devices will never accidentally end up with the same keys, and allows the + * Host use The Futex Trick (as we'll see later in our journey). + * + * SEND_DMA simply indicates a key to send to, and the physical address of the + * "struct lguest_dma" to send. The Host will write the number of bytes + * transferred into the "struct lguest_dma"'s used_len member. + * + * BIND_DMA indicates a key to bind to, a pointer to an array of "struct + * lguest_dma"s ready for receiving, the size of that array, and an interrupt + * to trigger when data is received. The Host will only allow transfers into + * buffers with a used_len of zero: it then sets used_len to the number of + * bytes transferred and triggers the interrupt for the Guest to process the + * new input. */ struct lguest_dma { - /* 0 if free to be used, filled by hypervisor. */ + /* 0 if free to be used, filled by the Host. */ u32 used_len; unsigned long addr[LGUEST_MAX_DMA_SECTIONS]; u16 len[LGUEST_MAX_DMA_SECTIONS]; }; +/*:*/ +/*D:460 This is the layout of a block device memory page. The Launcher sets up + * the num_sectors initially to tell the Guest the size of the disk. The Guest + * puts the type, sector and length of the request in the first three fields, + * then DMAs to the Host. The Host processes the request, sets up the result, + * then DMAs back to the Guest. */ struct lguest_block_page { /* 0 is a read, 1 is a write. */ @@ -28,27 +59,47 @@ struct lguest_block_page u32 num_sectors; /* Disk length = num_sectors * 512 */ }; -/* There is a shared page of these. */ +/*D:520 The network device is basically a memory page where all the Guests on + * the network publish their MAC (ethernet) addresses: it's an array of "struct + * lguest_net": */ struct lguest_net { /* Simply the mac address (with multicast bit meaning promisc). */ unsigned char mac[6]; }; +/*:*/ /* Where the Host expects the Guest to SEND_DMA console output to. */ #define LGUEST_CONSOLE_DMA_KEY 0 -/* We have a page of these descriptors in the lguest_device page. */ +/*D:010 + * Drivers + * + * The Guest needs devices to do anything useful. Since we don't let it touch + * real devices (think of the damage it could do!) we provide virtual devices. + * We could emulate a PCI bus with various devices on it, but that is a fairly + * complex burden for the Host and suboptimal for the Guest, so we have our own + * "lguest" bus and simple drivers. + * + * Devices are described by an array of LGUEST_MAX_DEVICES of these structs, + * placed by the Launcher just above the top of physical memory: + */ struct lguest_device_desc { + /* The device type: console, network, disk etc. */ u16 type; #define LGUEST_DEVICE_T_CONSOLE 1 #define LGUEST_DEVICE_T_NET 2 #define LGUEST_DEVICE_T_BLOCK 3 + /* The specific features of this device: these depends on device type + * except for LGUEST_DEVICE_F_RANDOMNESS. */ u16 features; #define LGUEST_NET_F_NOCSUM 0x4000 /* Don't bother checksumming */ #define LGUEST_DEVICE_F_RANDOMNESS 0x8000 /* IRQ is fairly random */ + /* This is how the Guest reports status of the device: the Host can set + * LGUEST_DEVICE_S_REMOVED to indicate removal, but the rest are only + * ever manipulated by the Guest, and only ever set. */ u16 status; /* 256 and above are device specific. */ #define LGUEST_DEVICE_S_ACKNOWLEDGE 1 /* We have seen device. */ @@ -58,9 +109,12 @@ struct lguest_device_desc { #define LGUEST_DEVICE_S_REMOVED_ACK 16 /* Driver has been told. */ #define LGUEST_DEVICE_S_FAILED 128 /* Something actually failed */ + /* Each device exists somewhere in Guest physical memory, over some + * number of pages. */ u16 num_pages; u32 pfn; }; +/*:*/ /* Write command first word is a request. */ enum lguest_req ^ permalink raw reply [flat|nested] 8+ messages in thread
[parent not found: <1181272355.24958.15.camel@localhost.localdomain>]
* [PATCH 4/7] lguest documentation:Chapter IV [not found] ` <1181272355.24958.15.camel@localhost.localdomain> @ 2007-06-08 3:13 ` Rusty Russell 2007-06-08 3:13 ` [PATCH 5/7] lguest documentation:Chapter V Rusty Russell [not found] ` <1181272434.24958.19.camel@localhost.localdomain> 0 siblings, 2 replies; 8+ messages in thread From: Rusty Russell @ 2007-06-08 3:13 UTC (permalink / raw) To: Andrew Morton; +Cc: lkml - Kernel Mailing List, virtualization Documentation: The Launcher Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> --- Documentation/lguest/lguest.c | 611 +++++++++++++++++++++++++++++++++++++---- drivers/lguest/core.c | 24 + drivers/lguest/io.c | 250 +++++++++++++++- drivers/lguest/lg.h | 25 + drivers/lguest/lguest_user.c | 166 ++++++++++- 5 files changed, 1000 insertions(+), 76 deletions(-) =================================================================== --- a/Documentation/lguest/lguest.c +++ b/Documentation/lguest/lguest.c @@ -34,12 +34,20 @@ #include <termios.h> #include <getopt.h> #include <zlib.h> +/*L:110 We can ignore the 28 include files we need for this program, but I do + * want to draw attention to the use of kernel-style types. + * + * As Linus said, "C is a Spartan language, and so should your naming be." I + * like these abbreviations and the header we need uses them, so we define them + * here. + */ typedef unsigned long long u64; typedef uint32_t u32; typedef uint16_t u16; typedef uint8_t u8; #include "../../include/linux/lguest_launcher.h" #include "../../include/asm-i386/e820.h" +/*:*/ #define PAGE_PRESENT 0x7 /* Present, RW, Execute */ #define NET_PEERNUM 1 @@ -48,31 +56,46 @@ typedef uint8_t u8; #define SIOCBRADDIF 0x89a2 /* add interface to bridge */ #endif +/*L:120 verbose is both a global flag and a macro. The C preprocessor allows + * this, and although I wouldn't recommend it, it works quite nicely here. */ static bool verbose; #define verbose(args...) \ do { if (verbose) printf(args); } while(0) +/*:*/ static int waker_fd; + +/*This is our list of devices. */ struct device_list { + /* Summary information about the devices in our list: ready to pass to + * select() to ask which need servicing.*/ fd_set infds; int max_infd; + /* A single linked list of devices. */ struct device *dev; + /* ... And an end pointer so we can easily append new devices */ struct device **lastdev; }; +/* The device structure describes a single device. */ struct device { + /* The linked-list pointer. */ struct device *next; + /* The descriptor for this device, as mapped into the Guest. */ struct lguest_device_desc *desc; + /* The memory page(s) of this device, if any. Also mapped in Guest. */ void *mem; - /* Watch this fd if handle_input non-NULL. */ + /* If handle_input is set, it wants to be called when this file + * descriptor is ready. */ int fd; bool (*handle_input)(int fd, struct device *me); - /* Watch DMA to this key if handle_input non-NULL. */ + /* If handle_output is set, it wants to be called when the Guest sends + * DMA to this key. */ unsigned long watch_key; u32 (*handle_output)(int fd, const struct iovec *iov, unsigned int num, struct device *me); @@ -81,6 +104,11 @@ struct device void *priv; }; +/*L:130 + * Loading the Kernel. + * + * We start with couple of simple helper routines. open_or_die() avoids + * error-checking code cluttering the callers: */ static int open_or_die(const char *name, int flags) { int fd = open(name, flags); @@ -89,26 +117,38 @@ static int open_or_die(const char *name, return fd; } +/* map_zeroed_pages() takes a (page-aligned) address and a number of pages. */ static void *map_zeroed_pages(unsigned long addr, unsigned int num) { + /* We cache the /dev/zero file-descriptor so we only open it once. */ static int fd = -1; if (fd == -1) fd = open_or_die("/dev/zero", O_RDONLY); + /* We use a private mapping (ie. if we write to the page, it will be + * copied), and obviously we insist that it be mapped where we ask. */ if (mmap((void *)addr, getpagesize() * num, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_FIXED|MAP_PRIVATE, fd, 0) != (void *)addr) err(1, "Mmaping %u pages of /dev/zero @%p", num, (void *)addr); + + /* Returning the address is just a courtesy: can simplify callers. */ return (void *)addr; } -/* Find magic string marking entry point, return entry point. */ +/* To find out where to start we look for the magic Guest string, which marks + * the code we see in lguest_asm.S. This is a hack which we are currently + * plotting to replace with the normal Linux entry point. */ static unsigned long entry_point(void *start, void *end, unsigned long page_offset) { void *p; + /* The scan gives us the physical starting address. We want the + * virtual address in this case, and fortunately, we already figured + * out the physical-virtual difference and passed it here in + * "page_offset". */ for (p = start; p < end; p++) if (memcmp(p, "GenuineLguest", strlen("GenuineLguest")) == 0) return (long)p + strlen("GenuineLguest") + page_offset; @@ -116,7 +156,17 @@ static unsigned long entry_point(void *s err(1, "Is this image a genuine lguest?"); } -/* Returns the entry point */ +/* This routine takes an open vmlinux image, which is in ELF, and maps it into + * the Guest memory. ELF = Embedded Linking Format, which is the format used + * by all modern binaries on Linux including the kernel. + * + * The ELF headers give *two* addresses: a physical address, and a virtual + * address. The Guest kernel expects to be placed in memory at the physical + * address, and the page tables set up so it will correspond to that virtual + * address. We return the difference between the virtual and physical + * addresses in the "page_offset" pointer. + * + * We return the starting address. */ static unsigned long map_elf(int elf_fd, const Elf32_Ehdr *ehdr, unsigned long *page_offset) { @@ -125,40 +175,61 @@ static unsigned long map_elf(int elf_fd, unsigned int i; unsigned long start = -1UL, end = 0; - /* Sanity checks. */ + /* Sanity checks on the main ELF header: an x86 executable with a + * reasonable number of correctly-sized program headers. */ if (ehdr->e_type != ET_EXEC || ehdr->e_machine != EM_386 || ehdr->e_phentsize != sizeof(Elf32_Phdr) || ehdr->e_phnum < 1 || ehdr->e_phnum > 65536U/sizeof(Elf32_Phdr)) errx(1, "Malformed elf header"); + /* An ELF executable contains an ELF header and a number of "program" + * headers which indicate which parts ("segments") of the program to + * load where. */ + + /* We read in all the program headers at once: */ if (lseek(elf_fd, ehdr->e_phoff, SEEK_SET) < 0) err(1, "Seeking to program headers"); if (read(elf_fd, phdr, sizeof(phdr)) != sizeof(phdr)) err(1, "Reading program headers"); + /* We don't know page_offset yet. */ *page_offset = 0; - /* We map the loadable segments at virtual addresses corresponding - * to their physical addresses (our virtual == guest physical). */ + + /* Try all the headers: there are usually only three. A read-only one, + * a read-write one, and a "note" section which isn't loadable. */ for (i = 0; i < ehdr->e_phnum; i++) { + /* If this isn't a loadable segment, we ignore it */ if (phdr[i].p_type != PT_LOAD) continue; verbose("Section %i: size %i addr %p\n", i, phdr[i].p_memsz, (void *)phdr[i].p_paddr); - /* We expect linear address space. */ + /* We expect a simple linear address space: every segment must + * have the same difference between virtual (p_vaddr) and + * physical (p_paddr) address. */ if (!*page_offset) *page_offset = phdr[i].p_vaddr - phdr[i].p_paddr; else if (*page_offset != phdr[i].p_vaddr - phdr[i].p_paddr) errx(1, "Page offset of section %i different", i); + /* We track the first and last address we mapped, so we can + * tell entry_point() where to scan. */ if (phdr[i].p_paddr < start) start = phdr[i].p_paddr; if (phdr[i].p_paddr + phdr[i].p_filesz > end) end = phdr[i].p_paddr + phdr[i].p_filesz; - /* We map everything private, writable. */ + /* We map this section of the file at its physical address. We + * map it read & write even if the header says this segment is + * read-only. The kernel really wants to be writable: it + * patches its own instructions which would normally be + * read-only. + * + * MAP_PRIVATE means that the page won't be copied until a + * write is done to it. This allows us to share much of the + * kernel memory between Guests. */ addr = mmap((void *)phdr[i].p_paddr, phdr[i].p_filesz, PROT_READ|PROT_WRITE|PROT_EXEC, @@ -172,7 +243,31 @@ static unsigned long map_elf(int elf_fd, return entry_point((void *)start, (void *)end, *page_offset); } -/* This is amazingly reliable. */ +/*L:170 Prepare to be SHOCKED and AMAZED. And possibly a trifle nauseated. + * + * We know that CONFIG_PAGE_OFFSET sets what virtual address the kernel expects + * to be. We don't know what that option was, but we can figure it out + * approximately by looking at the addresses in the code. I chose the common + * case of reading a memory location into the %eax register: + * + * movl <some-address>, %eax + * + * This gets encoded as five bytes: "0xA1 <4-byte-address>". For example, + * "0xA1 0x18 0x60 0x47 0xC0" reads the address 0xC0476018 into %eax. + * + * In this example can guess that the kernel was compiled with + * CONFIG_PAGE_OFFSET set to 0xC0000000 (it's always a round number). If the + * kernel were larger than 16MB, we might see 0xC1 addresses show up, but our + * kernel isn't that bloated yet. + * + * Unfortunately, x86 has variable-length instructions, so finding this + * particular instruction properly involves writing a disassembler. Instead, + * we rely on statistics. We look for "0xA1" and tally the different bytes + * which occur 4 bytes later (the "0xC0" in our example above). When one of + * those bytes appears three times, we can be reasonably confident that it + * forms the start of CONFIG_PAGE_OFFSET. + * + * This is amazingly reliable. */ static unsigned long intuit_page_offset(unsigned char *img, unsigned long len) { unsigned int i, possibilities[256] = { 0 }; @@ -185,30 +280,52 @@ static unsigned long intuit_page_offset( errx(1, "could not determine page offset"); } +/*L:160 Unfortunately the entire ELF image isn't compressed: the segments + * which need loading are extracted and compressed raw. This denies us the + * information we need to make a fully-general loader. */ static unsigned long unpack_bzimage(int fd, unsigned long *page_offset) { gzFile f; int ret, len = 0; + /* A bzImage always gets loaded at physical address 1M. This is + * actually configurable as CONFIG_PHYSICAL_START, but as the comment + * there says, "Don't change this unless you know what you are doing". + * Indeed. */ void *img = (void *)0x100000; + /* gzdopen takes our file descriptor (carefully placed at the start of + * the GZIP header we found) and returns a gzFile. */ f = gzdopen(fd, "rb"); + /* We read it into memory in 64k chunks until we hit the end. */ while ((ret = gzread(f, img + len, 65536)) > 0) len += ret; if (ret < 0) err(1, "reading image from bzImage"); verbose("Unpacked size %i addr %p\n", len, img); + + /* Without the ELF header, we can't tell virtual-physical gap. This is + * CONFIG_PAGE_OFFSET, and people do actually change it. Fortunately, + * I have a clever way of figuring it out from the code itself. */ *page_offset = intuit_page_offset(img, len); return entry_point(img, img + len, *page_offset); } +/*L:150 A bzImage, unlike an ELF file, is not meant to be loaded. You're + * supposed to jump into it and it will unpack itself. We can't do that + * because the Guest can't run the unpacking code, and adding features to + * lguest kills puppies, so we don't want to. + * + * The bzImage is formed by putting the decompressing code in front of the + * compressed kernel code. So we can simple scan through it looking for the + * first "gzip" header, and start decompressing from there. */ static unsigned long load_bzimage(int fd, unsigned long *page_offset) { unsigned char c; int state = 0; - /* Ugly brute force search for gzip header. */ + /* GZIP header is 0x1F 0x8B <method> <flags>... <compressed-by>. */ while (read(fd, &c, 1) == 1) { switch (state) { case 0: @@ -225,8 +342,10 @@ static unsigned long load_bzimage(int fd state++; break; case 9: + /* Seek back to the start of the gzip header. */ lseek(fd, -10, SEEK_CUR); - if (c != 0x03) /* Compressed under UNIX. */ + /* One final check: "compressed under UNIX". */ + if (c != 0x03) state = -1; else return unpack_bzimage(fd, page_offset); @@ -235,25 +354,43 @@ static unsigned long load_bzimage(int fd errx(1, "Could not find kernel in bzImage"); } +/*L:140 Loading the kernel is easy when it's a "vmlinux", but most kernels + * come wrapped up in the self-decompressing "bzImage" format. With some funky + * coding, we can load those, too. */ static unsigned long load_kernel(int fd, unsigned long *page_offset) { Elf32_Ehdr hdr; + /* Read in the first few bytes. */ if (read(fd, &hdr, sizeof(hdr)) != sizeof(hdr)) err(1, "Reading kernel"); + /* If it's an ELF file, it starts with "\177ELF" */ if (memcmp(hdr.e_ident, ELFMAG, SELFMAG) == 0) return map_elf(fd, &hdr, page_offset); + /* Otherwise we assume it's a bzImage, and try to unpack it */ return load_bzimage(fd, page_offset); } +/* This is a trivial little helper to align pages. Andi Kleen hated it because + * it calls getpagesize() twice: "it's dumb code." + * + * Kernel guys get really het up about optimization, even when it's not + * necessary. I leave this code as a reaction against that. */ static inline unsigned long page_align(unsigned long addr) { + /* Add upwards and truncate downwards. */ return ((addr + getpagesize()-1) & ~(getpagesize()-1)); } -/* initrd gets loaded at top of memory: return length. */ +/*L:180 An "initial ram disk" is a disk image loaded into memory along with + * the kernel which the kernel can use to boot from without needing any + * drivers. Most distributions now use this as standard: the initrd contains + * the code to load the appropriate driver modules for the current machine. + * + * Importantly, James Morris works for RedHat, and Fedora uses initrds for its + * kernels. He sent me this (and tells me when I break it). */ static unsigned long load_initrd(const char *name, unsigned long mem) { int ifd; @@ -262,21 +399,35 @@ static unsigned long load_initrd(const c void *iaddr; ifd = open_or_die(name, O_RDONLY); + /* fstat() is needed to get the file size. */ if (fstat(ifd, &st) < 0) err(1, "fstat() on initrd '%s'", name); + /* The length needs to be rounded up to a page size: mmap needs the + * address to be page aligned. */ len = page_align(st.st_size); + /* We map the initrd at the top of memory. */ iaddr = mmap((void *)mem - len, st.st_size, PROT_READ|PROT_EXEC|PROT_WRITE, MAP_FIXED|MAP_PRIVATE, ifd, 0); if (iaddr != (void *)mem - len) err(1, "Mmaping initrd '%s' returned %p not %p", name, iaddr, (void *)mem - len); + /* Once a file is mapped, you can close the file descriptor. It's a + * little odd, but quite useful. */ close(ifd); verbose("mapped initrd %s size=%lu @ %p\n", name, st.st_size, iaddr); + + /* We return the initrd size. */ return len; } +/* Once we know how much memory we have, and the address the Guest kernel + * expects, we can construct simple linear page tables which will get the Guest + * far enough into the boot to create its own. + * + * We lay them out of the way, just below the initrd (which is why we need to + * know its size). */ static unsigned long setup_pagetables(unsigned long mem, unsigned long initrd_size, unsigned long page_offset) @@ -285,23 +436,32 @@ static unsigned long setup_pagetables(un unsigned int mapped_pages, i, linear_pages; unsigned int ptes_per_page = getpagesize()/sizeof(u32); - /* If we can map all of memory above page_offset, we do so. */ + /* Ideally we map all physical memory starting at page_offset. + * However, if page_offset is 0xC0000000 we can only map 1G of physical + * (0xC0000000 + 1G overflows). */ if (mem > -page_offset) mapped_pages = mem/getpagesize(); else mapped_pages = -page_offset/getpagesize(); - /* Each linear PTE page can map ptes_per_page pages. */ + /* Each PTE page can map ptes_per_page pages: how many do we need? */ linear_pages = (mapped_pages + ptes_per_page-1)/ptes_per_page; - /* We lay out top-level then linear mapping immediately below initrd */ + /* We put the toplevel page directory page at the top of memory. */ pgdir = (void *)mem - initrd_size - getpagesize(); + + /* Now we use the next linear_pages pages as pte pages */ linear = (void *)pgdir - linear_pages*getpagesize(); + /* Linear mapping is easy: put every page's address into the mapping in + * order. PAGE_PRESENT contains the flags Present, Writable and + * Executable. */ for (i = 0; i < mapped_pages; i++) linear[i] = ((i * getpagesize()) | PAGE_PRESENT); - /* Now set up pgd so that this memory is at page_offset */ + /* The top level points to the linear page table pages above. The + * entry representing page_offset points to the first one, and they + * continue from there. */ for (i = 0; i < mapped_pages; i += ptes_per_page) { pgdir[(i + page_offset/getpagesize())/ptes_per_page] = (((u32)linear + i*sizeof(u32)) | PAGE_PRESENT); @@ -310,9 +470,13 @@ static unsigned long setup_pagetables(un verbose("Linear mapping of %u pages in %u pte pages at %p\n", mapped_pages, linear_pages, linear); + /* We return the top level (guest-physical) address: the kernel needs + * to know where it is. */ return (unsigned long)pgdir; } +/* Simple routine to roll all the commandline arguments together with spaces + * between them. */ static void concat(char *dst, char *args[]) { unsigned int i, len = 0; @@ -326,6 +490,10 @@ static void concat(char *dst, char *args dst[len] = '\0'; } +/* This is where we actually tell the kernel to initialize the Guest. We saw + * the arguments it expects when we looked at initialize() in lguest_user.c: + * the top physical page to allow, the top level pagetable, the entry point and + * the page_offset constant for the Guest. */ static int tell_kernel(u32 pgdir, u32 start, u32 page_offset) { u32 args[] = { LHREQ_INITIALIZE, @@ -336,8 +504,11 @@ static int tell_kernel(u32 pgdir, u32 st fd = open_or_die("/dev/lguest", O_RDWR); if (write(fd, args, sizeof(args)) < 0) err(1, "Writing to /dev/lguest"); + + /* We return the /dev/lguest file descriptor to control this Guest */ return fd; } +/*:*/ static void set_fd(int fd, struct device_list *devices) { @@ -346,61 +517,108 @@ static void set_fd(int fd, struct device devices->max_infd = fd; } -/* When input arrives, we tell the kernel to kick lguest out with -EAGAIN. */ +/*L:200 + * The Waker. + * + * With a console and network devices, we can have lots of input which we need + * to process. We could try to tell the kernel what file descriptors to watch, + * but handing a file descriptor mask through to the kernel is fairly icky. + * + * Instead, we fork off a process which watches the file descriptors and writes + * the LHREQ_BREAK command to the /dev/lguest filedescriptor to tell the Host + * loop to stop running the Guest. This causes it to return from the + * /dev/lguest read with -EAGAIN, where it will write to /dev/lguest to reset + * the LHREQ_BREAK and wake us up again. + * + * This, of course, is merely a different *kind* of icky. + */ static void wake_parent(int pipefd, int lguest_fd, struct device_list *devices) { + /* Add the pipe from the Launcher to the fdset in the device_list, so + * we watch it, too. */ set_fd(pipefd, devices); for (;;) { fd_set rfds = devices->infds; u32 args[] = { LHREQ_BREAK, 1 }; + /* Wait until input is ready from one of the devices. */ select(devices->max_infd+1, &rfds, NULL, NULL, NULL); + /* Is it a message from the Launcher? */ if (FD_ISSET(pipefd, &rfds)) { int ignorefd; + /* If read() returns 0, it means the Launcher has + * exited. We silently follow. */ if (read(pipefd, &ignorefd, sizeof(ignorefd)) == 0) exit(0); + /* Otherwise it's telling us there's a problem with one + * of the devices, and we should ignore that file + * descriptor from now on. */ FD_CLR(ignorefd, &devices->infds); - } else + } else /* Send LHREQ_BREAK command. */ write(lguest_fd, args, sizeof(args)); } } +/* This routine just sets up a pipe to the waker process. */ static int setup_waker(int lguest_fd, struct device_list *device_list) { int pipefd[2], child; + /* We create a pipe to talk to the waker, and also so it knows when the + * Launcher dies (and closes pipe). */ pipe(pipefd); child = fork(); if (child == -1) err(1, "forking"); if (child == 0) { + /* Close the "writing" end of our copy of the pipe */ close(pipefd[1]); wake_parent(pipefd[0], lguest_fd, device_list); } + /* Close the reading end of our copy of the pipe. */ close(pipefd[0]); + /* Here is the fd used to talk to the waker. */ return pipefd[1]; } +/*L:210 + * Device Handling. + * + * When the Guest sends DMA to us, it sends us an array of addresses and sizes. + * We need to make sure it's not trying to reach into the Launcher itself, so + * we have a convenient routine which check it and exits with an error message + * if something funny is going on: + */ static void *_check_pointer(unsigned long addr, unsigned int size, unsigned int line) { + /* We have to separately check addr and addr+size, because size could + * be huge and addr + size might wrap around. */ if (addr >= LGUEST_GUEST_TOP || addr + size >= LGUEST_GUEST_TOP) errx(1, "%s:%i: Invalid address %li", __FILE__, line, addr); + /* We return a pointer for the caller's convenience, now we know it's + * safe to use. */ return (void *)addr; } +/* A macro which transparently hands the line number to the real function. */ #define check_pointer(addr,size) _check_pointer(addr, size, __LINE__) -/* Returns pointer to dma->used_len */ +/* The Guest has given us the address of a "struct lguest_dma". We check it's + * OK and convert it to an iovec (which is a simple array of ptr/size + * pairs). */ static u32 *dma2iov(unsigned long dma, struct iovec iov[], unsigned *num) { unsigned int i; struct lguest_dma *udma; + /* First we make sure that the array memory itself is valid. */ udma = check_pointer(dma, sizeof(*udma)); + /* Now we check each element */ for (i = 0; i < LGUEST_MAX_DMA_SECTIONS; i++) { + /* A zero length ends the array. */ if (!udma->len[i]) break; @@ -408,9 +626,15 @@ static u32 *dma2iov(unsigned long dma, s iov[i].iov_len = udma->len[i]; } *num = i; + + /* We return the pointer to where the caller should write the amount of + * the buffer used. */ return &udma->used_len; } +/* This routine gets a DMA buffer from the Guest for a given key, and converts + * it to an iovec array. It returns the interrupt the Guest wants when we're + * finished, and a pointer to the "used_len" field to fill in. */ static u32 *get_dma_buffer(int fd, void *key, struct iovec iov[], unsigned int *num, u32 *irq) { @@ -418,16 +642,21 @@ static u32 *get_dma_buffer(int fd, void unsigned long udma; u32 *res; + /* Ask the kernel for a DMA buffer corresponding to this key. */ udma = write(fd, buf, sizeof(buf)); + /* They haven't registered any, or they're all used? */ if (udma == (unsigned long)-1) return NULL; - /* Kernel stashes irq in ->used_len. */ + /* Convert it into our iovec array */ res = dma2iov(udma, iov, num); + /* The kernel stashes irq in ->used_len to get it out to us. */ *irq = *res; + /* Return a pointer to ((struct lguest_dma *)udma)->used_len. */ return res; } +/* This is a convenient routine to send the Guest an interrupt. */ static void trigger_irq(int fd, u32 irq) { u32 buf[] = { LHREQ_IRQ, irq }; @@ -435,6 +664,10 @@ static void trigger_irq(int fd, u32 irq) err(1, "Triggering irq %i", irq); } +/* This simply sets up an iovec array where we can put data to be discarded. + * This happens when the Guest doesn't want or can't handle the input: we have + * to get rid of it somewhere, and if we bury it in the ceiling space it will + * start to smell after a week. */ static void discard_iovec(struct iovec *iov, unsigned int *num) { static char discard_buf[1024]; @@ -443,19 +676,24 @@ static void discard_iovec(struct iovec * iov->iov_len = sizeof(discard_buf); } +/* Here is the input terminal setting we save, and the routine to restore them + * on exit so the user can see what they type next. */ static struct termios orig_term; static void restore_term(void) { tcsetattr(STDIN_FILENO, TCSANOW, &orig_term); } +/* We associate some data with the console for our exit hack. */ struct console_abort { + /* How many times have they hit ^C? */ int count; + /* When did they start? */ struct timeval start; }; -/* We DMA input to buffer bound at start of console page. */ +/* This is the routine which handles console input (ie. stdin). */ static bool handle_console_input(int fd, struct device *dev) { u32 irq = 0, *lenp; @@ -464,24 +702,38 @@ static bool handle_console_input(int fd, struct iovec iov[LGUEST_MAX_DMA_SECTIONS]; struct console_abort *abort = dev->priv; + /* First we get the console buffer from the Guest. The key is dev->mem + * which was set to 0 in setup_console(). */ lenp = get_dma_buffer(fd, dev->mem, iov, &num, &irq); if (!lenp) { + /* If it's not ready for input, warn and set up to discard. */ warn("console: no dma buffer!"); discard_iovec(iov, &num); } + /* This is why we convert to iovecs: the readv() call uses them, and so + * it reads straight into the Guest's buffer. */ len = readv(dev->fd, iov, num); if (len <= 0) { + /* This implies that the console is closed, is /dev/null, or + * something went terribly wrong. We still go through the rest + * of the logic, though, especially the exit handling below. */ warnx("Failed to get console input, ignoring console."); len = 0; } + /* If we read the data into the Guest, fill in the length and send the + * interrupt. */ if (lenp) { *lenp = len; trigger_irq(fd, irq); } - /* Three ^C within one second? Exit. */ + /* Three ^C within one second? Exit. + * + * This is such a hack, but works surprisingly well. Each ^C has to be + * in a buffer by itself, so they can't be too fast. But we check that + * we get three within about a second, so they can't be too slow. */ if (len == 1 && ((char *)iov[0].iov_base)[0] == 3) { if (!abort->count++) gettimeofday(&abort->start, NULL); @@ -489,43 +741,60 @@ static bool handle_console_input(int fd, struct timeval now; gettimeofday(&now, NULL); if (now.tv_sec <= abort->start.tv_sec+1) { - /* Make sure waker is not blocked in BREAK */ u32 args[] = { LHREQ_BREAK, 0 }; + /* Close the fd so Waker will know it has to + * exit. */ close(waker_fd); + /* Just in case waker is blocked in BREAK, send + * unbreak now. */ write(fd, args, sizeof(args)); exit(2); } abort->count = 0; } } else + /* Any other key resets the abort counter. */ abort->count = 0; + /* Now, if we didn't read anything, put the input terminal back and + * return failure (meaning, don't call us again). */ if (!len) { restore_term(); return false; } + /* Everything went OK! */ return true; } +/* Handling console output is much simpler than input. */ static u32 handle_console_output(int fd, const struct iovec *iov, unsigned num, struct device*dev) { + /* Whatever the Guest sends, write it to standard output. Return the + * number of bytes written. */ return writev(STDOUT_FILENO, iov, num); } +/* Guest->Host network output is also pretty easy. */ static u32 handle_tun_output(int fd, const struct iovec *iov, unsigned num, struct device *dev) { - /* Now we've seen output, we should warn if we can't get buffers. */ + /* We put a flag in the "priv" pointer of the network device, and set + * it as soon as we see output. We'll see why in handle_tun_input() */ *(bool *)dev->priv = true; + /* Whatever packet the Guest sent us, write it out to the tun + * device. */ return writev(dev->fd, iov, num); } +/* This matches the peer_key() in lguest_net.c. The key for any given slot + * is the address of the network device's page plus 4 * the slot number. */ static unsigned long peer_offset(unsigned int peernum) { return 4 * peernum; } +/* This is where we handle a packet coming in from the tun device */ static bool handle_tun_input(int fd, struct device *dev) { u32 irq = 0, *lenp; @@ -533,17 +802,28 @@ static bool handle_tun_input(int fd, str unsigned num; struct iovec iov[LGUEST_MAX_DMA_SECTIONS]; + /* First we get a buffer the Guest has bound to its key. */ lenp = get_dma_buffer(fd, dev->mem+peer_offset(NET_PEERNUM), iov, &num, &irq); if (!lenp) { + /* Now, it's expected that if we try to send a packet too + * early, the Guest won't be ready yet. This is why we set a + * flag when the Guest sends its first packet. If it's sent a + * packet we assume it should be ready to receive them. + * + * Actually, this is what the status bits in the descriptor are + * for: we should *use* them. FIXME! */ if (*(bool *)dev->priv) warn("network: no dma buffer!"); discard_iovec(iov, &num); } + /* Read the packet from the device directly into the Guest's buffer. */ len = readv(dev->fd, iov, num); if (len <= 0) err(1, "reading network"); + + /* Write the used_len, and trigger the interrupt for the Guest */ if (lenp) { *lenp = len; trigger_irq(fd, irq); @@ -551,9 +831,13 @@ static bool handle_tun_input(int fd, str verbose("tun input packet len %i [%02x %02x] (%s)\n", len, ((u8 *)iov[0].iov_base)[0], ((u8 *)iov[0].iov_base)[1], lenp ? "sent" : "discarded"); + /* All good. */ return true; } +/* The last device handling routine is block output: the Guest has sent a DMA + * to the block device. It will have placed the command it wants in the + * "struct lguest_block_page". */ static u32 handle_block_output(int fd, const struct iovec *iov, unsigned num, struct device *dev) { @@ -563,36 +847,64 @@ static u32 handle_block_output(int fd, c struct iovec reply[LGUEST_MAX_DMA_SECTIONS]; off64_t device_len, off = (off64_t)p->sector * 512; + /* First we extract the device length from the dev->priv pointer. */ device_len = *(off64_t *)dev->priv; + /* We first check that the read or write is within the length of the + * block file. */ if (off >= device_len) err(1, "Bad offset %llu vs %llu", off, device_len); + /* Move to the right location in the block file. This shouldn't fail, + * but best to check. */ if (lseek64(dev->fd, off, SEEK_SET) != off) err(1, "Bad seek to sector %i", p->sector); verbose("Block: %s at offset %llu\n", p->type ? "WRITE" : "READ", off); + /* They were supposed to bind a reply buffer at key equal to the start + * of the block device memory. We need this to tell them when the + * request is finished. */ lenp = get_dma_buffer(fd, dev->mem, reply, &reply_num, &irq); if (!lenp) err(1, "Block request didn't give us a dma buffer"); if (p->type) { + /* A write request. The DMA they sent contained the data, so + * write it out. */ len = writev(dev->fd, iov, num); + /* Grr... Now we know how long the "struct lguest_dma" they + * sent was, we make sure they didn't try to write over the end + * of the block file (possibly extending it). */ if (off + len > device_len) { + /* Trim it back to the correct length */ ftruncate(dev->fd, device_len); + /* Die, bad Guest, die. */ errx(1, "Write past end %llu+%u", off, len); } + /* The reply length is 0: we just send back an empty DMA to + * interrupt them and tell them the write is finished. */ *lenp = 0; } else { + /* A read request. They sent an empty DMA to start the + * request, and we put the read contents into the reply + * buffer. */ len = readv(dev->fd, reply, reply_num); *lenp = len; } + /* The result is 1 (done), 2 if there was an error (short read or + * write). */ p->result = 1 + (p->bytes != len); + /* Now tell them we've used their reply buffer. */ trigger_irq(fd, irq); + + /* We're supposed to return the number of bytes of the output buffer we + * used. But the block device uses the "result" field instead, so we + * don't bother. */ return 0; } +/* This is the generic routine we call when the Guest sends some DMA out. */ static void handle_output(int fd, unsigned long dma, unsigned long key, struct device_list *devices) { @@ -601,30 +913,53 @@ static void handle_output(int fd, unsign struct iovec iov[LGUEST_MAX_DMA_SECTIONS]; unsigned num = 0; + /* Convert the "struct lguest_dma" they're sending to a "struct + * iovec". */ lenp = dma2iov(dma, iov, &num); + + /* Check each device: if they expect output to this key, tell them to + * handle it. */ for (i = devices->dev; i; i = i->next) { if (i->handle_output && key == i->watch_key) { + /* We write the result straight into the used_len field + * for them. */ *lenp = i->handle_output(fd, iov, num, i); return; } } + + /* This can happen: the kernel sends any SEND_DMA which doesn't match + * another Guest to us. It could be that another Guest just left a + * network, for example. But it's unusual. */ warnx("Pending dma %p, key %p", (void *)dma, (void *)key); } +/* This is called when the waker wakes us up: check for incoming file + * descriptors. */ static void handle_input(int fd, struct device_list *devices) { + /* select() wants a zeroed timeval to mean "don't wait". */ struct timeval poll = { .tv_sec = 0, .tv_usec = 0 }; for (;;) { struct device *i; fd_set fds = devices->infds; + /* If nothing is ready, we're done. */ if (select(devices->max_infd+1, &fds, NULL, NULL, &poll) == 0) break; + /* Otherwise, call the device(s) which have readable + * file descriptors and a method of handling them. */ for (i = devices->dev; i; i = i->next) { if (i->handle_input && FD_ISSET(i->fd, &fds)) { + /* If handle_input() returns false, it means we + * should no longer service it. + * handle_console_input() does this. */ if (!i->handle_input(fd, i)) { + /* Clear it from the set of input file + * descriptors kept at the head of the + * device list. */ FD_CLR(i->fd, &devices->infds); /* Tell waker to ignore it too... */ write(waker_fd, &i->fd, sizeof(i->fd)); @@ -634,9 +969,22 @@ static void handle_input(int fd, struct } } +/*L:190 + * Device Setup + * + * All devices need a descriptor so the Guest knows it exists, and a "struct + * device" so the Launcher can keep track of it. We have common helper + * routines to allocate them. + * + * This routine allocates a new "struct lguest_device_desc". This descriptor + * ultimitely belongs in the devices array just above the Guest's normal + * memory, but we set up devices while parsing the command line options and we + * don't know how much memory the Guest has yet. So we allocate them here and + * move them later in map_device_descriptors(). */ static struct lguest_device_desc *new_dev_desc(u16 type, u16 features, u16 num_pages) { + /* This is the very top allowed guest-physical address */ static unsigned long top = LGUEST_GUEST_TOP; struct lguest_device_desc *desc; @@ -644,7 +992,11 @@ static struct lguest_device_desc *new_de desc->type = type; desc->num_pages = num_pages; desc->features = features; + /* The Guest sets status bits to indicate its progress. */ desc->status = 0; + + /* If they said the device needs memory, we allocate that now, down + * from the top. */ if (num_pages) { top -= num_pages*getpagesize(); map_zeroed_pages(top, num_pages); @@ -654,6 +1006,9 @@ static struct lguest_device_desc *new_de return desc; } +/* This monster routine does all the creation and setup of a new device, + * including caling new_dev_desc() to allocate the descriptor and device + * memory. */ static struct device *new_device(struct device_list *devices, u16 type, u16 num_pages, u16 features, int fd, @@ -666,12 +1021,18 @@ static struct device *new_device(struct { struct device *dev = malloc(sizeof(*dev)); - /* Append to device list. */ + /* Append to device list. Prepending to a single-linked list is + * easier, but the user expects the devices to be arranged on the bus + * in command-line order. The first network device on the command line + * is eth0, the first block device /dev/lgba, etc. */ *devices->lastdev = dev; dev->next = NULL; devices->lastdev = &dev->next; + /* Now we populate the fields one at a time. */ dev->fd = fd; + /* If we have an input handler for this file descriptor, then we add it + * to the device_list's fdset and maxfd. */ if (handle_input) set_fd(dev->fd, devices); dev->desc = new_dev_desc(type, features, num_pages); @@ -682,27 +1043,37 @@ static struct device *new_device(struct return dev; } +/* Our first setup routine is the console. It's a fairly simple device, but + * UNIX tty handling makes it uglier than it could be. */ static void setup_console(struct device_list *devices) { struct device *dev; + /* If we can save the initial standard input settings... */ if (tcgetattr(STDIN_FILENO, &orig_term) == 0) { struct termios term = orig_term; + /* Then we turn off echo, line buffering and ^C etc. We want a + * raw input stream to the Guest. */ term.c_lflag &= ~(ISIG|ICANON|ECHO); tcsetattr(STDIN_FILENO, TCSANOW, &term); + /* If we exit gracefully, the original settings will be + * restored so the user can see what they're typing. */ atexit(restore_term); } - /* We don't currently require a page for the console. */ + /* We don't currently require any memory for the console, so we ask for + * 0 pages. */ dev = new_device(devices, LGUEST_DEVICE_T_CONSOLE, 0, 0, STDIN_FILENO, handle_console_input, LGUEST_CONSOLE_DMA_KEY, handle_console_output); + /* We store the console state in dev->priv, and initialize it. */ dev->priv = malloc(sizeof(struct console_abort)); ((struct console_abort *)dev->priv)->count = 0; verbose("device %p: console\n", (void *)(dev->desc->pfn * getpagesize())); } +/* Setting up a block file is also fairly straightforward. */ static void setup_block_file(const char *filename, struct device_list *devices) { int fd; @@ -710,20 +1081,47 @@ static void setup_block_file(const char off64_t *device_len; struct lguest_block_page *p; + /* We open with O_LARGEFILE because otherwise we get stuck at 2G. We + * open with O_DIRECT because otherwise our benchmarks go much too + * fast. */ fd = open_or_die(filename, O_RDWR|O_LARGEFILE|O_DIRECT); + + /* We want one page, and have no input handler (the block file never + * has anything interesting to say to us). Our timing will be quite + * random, so it should be a reasonable randomness source. */ dev = new_device(devices, LGUEST_DEVICE_T_BLOCK, 1, LGUEST_DEVICE_F_RANDOMNESS, fd, NULL, 0, handle_block_output); + + /* We store the device size in the private area */ device_len = dev->priv = malloc(sizeof(*device_len)); + /* This is the safe way of establishing the size of our device: it + * might be a normal file or an actual block device like /dev/hdb. */ *device_len = lseek64(fd, 0, SEEK_END); + + /* The device memory is a "struct lguest_block_page". It's zeroed + * already, we just need to put in the device size. Block devices + * think in sectors (ie. 512 byte chunks), so we translate here. */ p = dev->mem; - p->num_sectors = *device_len/512; verbose("device %p: block %i sectors\n", (void *)(dev->desc->pfn * getpagesize()), p->num_sectors); } -/* We use fnctl locks to reserve network slots (autocleanup!) */ +/* + * Network Devices. + * + * Setting up network devices is quite a pain, because we have three types. + * First, we have the inter-Guest network. This is a file which is mapped into + * the address space of the Guests who are on the network. Because it is a + * shared mapping, the same page underlies all the devices, and they can send + * DMA to each other. + * + * Remember from our network driver, the Guest is told what slot in the page it + * is to use. We use exclusive fnctl locks to reserve a slot. If another + * Guest is using a slot, the lock will fail and we try another. Because fnctl + * locks are cleaned up automatically when we die, this cleverly means that our + * reservation on the slot will vanish if we crash. */ static unsigned int find_slot(int netfd, const char *filename) { struct flock fl; @@ -731,26 +1129,33 @@ static unsigned int find_slot(int netfd, fl.l_type = F_WRLCK; fl.l_whence = SEEK_SET; fl.l_len = 1; + /* Try a 1 byte lock in each possible position number */ for (fl.l_start = 0; fl.l_start < getpagesize()/sizeof(struct lguest_net); fl.l_start++) { + /* If we succeed, return the slot number. */ if (fcntl(netfd, F_SETLK, &fl) == 0) return fl.l_start; } errx(1, "No free slots in network file %s", filename); } +/* This function sets up the network file */ static void setup_net_file(const char *filename, struct device_list *devices) { int netfd; struct device *dev; + /* We don't use open_or_die() here: for friendliness we create the file + * if it doesn't already exist. */ netfd = open(filename, O_RDWR, 0); if (netfd < 0) { if (errno == ENOENT) { netfd = open(filename, O_RDWR|O_CREAT, 0600); if (netfd >= 0) { + /* If we succeeded, initialize the file with a + * blank page. */ char page[getpagesize()]; memset(page, 0, sizeof(page)); write(netfd, page, sizeof(page)); @@ -760,11 +1165,15 @@ static void setup_net_file(const char *f err(1, "cannot open net file '%s'", filename); } + /* We need 1 page, and the features indicate the slot to use and that + * no checksum is needed. We never touch this device again; it's + * between the Guests on the network, so we don't register input or + * output handlers. */ dev = new_device(devices, LGUEST_DEVICE_T_NET, 1, find_slot(netfd, filename)|LGUEST_NET_F_NOCSUM, -1, NULL, 0, NULL); - /* We overwrite the /dev/zero mapping with the actual file. */ + /* Map the shared file. */ if (mmap(dev->mem, getpagesize(), PROT_READ|PROT_WRITE, MAP_FIXED|MAP_SHARED, netfd, 0) != dev->mem) err(1, "could not mmap '%s'", filename); @@ -772,6 +1181,7 @@ static void setup_net_file(const char *f (void *)(dev->desc->pfn * getpagesize()), filename, dev->desc->features & ~LGUEST_NET_F_NOCSUM); } +/*:*/ static u32 str2ip(const char *ipaddr) { @@ -781,7 +1191,11 @@ static u32 str2ip(const char *ipaddr) return (byte[0] << 24) | (byte[1] << 16) | (byte[2] << 8) | byte[3]; } -/* adapted from libbridge */ +/* This code is "adapted" from libbridge: it attaches the Host end of the + * network device to the bridge device specified by the command line. + * + * This is yet another James Morris contribution (I'm an IP-level guy, so I + * dislike bridging), and I just try not to break it. */ static void add_to_bridge(int fd, const char *if_name, const char *br_name) { int ifidx; @@ -800,12 +1214,16 @@ static void add_to_bridge(int fd, const err(1, "can't add %s to bridge %s", if_name, br_name); } +/* This sets up the Host end of the network device with an IP address, brings + * it up so packets will flow, the copies the MAC address into the hwaddr + * pointer (in practice, the Host's slot in the network device's memory). */ static void configure_device(int fd, const char *devname, u32 ipaddr, unsigned char hwaddr[6]) { struct ifreq ifr; struct sockaddr_in *sin = (struct sockaddr_in *)&ifr.ifr_addr; + /* Don't read these incantations. Just cut & paste them like I did! */ memset(&ifr, 0, sizeof(ifr)); strcpy(ifr.ifr_name, devname); sin->sin_family = AF_INET; @@ -816,12 +1234,19 @@ static void configure_device(int fd, con if (ioctl(fd, SIOCSIFFLAGS, &ifr) != 0) err(1, "Bringing interface %s up", devname); + /* SIOC stands for Socket I/O Control. G means Get (vs S for Set + * above). IF means Interface, and HWADDR is hardware address. + * Simple! */ if (ioctl(fd, SIOCGIFHWADDR, &ifr) != 0) err(1, "getting hw address for %s", devname); - memcpy(hwaddr, ifr.ifr_hwaddr.sa_data, 6); } +/*L:195 The other kind of network is a Host<->Guest network. This can either + * use briding or routing, but the principle is the same: it uses the "tun" + * device to inject packets into the Host as if they came in from a normal + * network card. We just shunt packets between the Guest and the tun + * device. */ static void setup_tun_net(const char *arg, struct device_list *devices) { struct device *dev; @@ -830,36 +1255,56 @@ static void setup_tun_net(const char *ar u32 ip; const char *br_name = NULL; + /* We open the /dev/net/tun device and tell it we want a tap device. A + * tap device is like a tun device, only somehow different. To tell + * the truth, I completely blundered my way through this code, but it + * works now! */ netfd = open_or_die("/dev/net/tun", O_RDWR); memset(&ifr, 0, sizeof(ifr)); ifr.ifr_flags = IFF_TAP | IFF_NO_PI; strcpy(ifr.ifr_name, "tap%d"); if (ioctl(netfd, TUNSETIFF, &ifr) != 0) err(1, "configuring /dev/net/tun"); + /* We don't need checksums calculated for packets coming in this + * device: trust us! */ ioctl(netfd, TUNSETNOCSUM, 1); - /* You will be peer 1: we should create enough jitter to randomize */ + /* We create the net device with 1 page, using the features field of + * the descriptor to tell the Guest it is in slot 1 (NET_PEERNUM), and + * that the device has fairly random timing. We do *not* specify + * LGUEST_NET_F_NOCSUM: these packets can reach the real world. + * + * We will put our MAC address is slot 0 for the Guest to see, so + * it will send packets to us using the key "peer_offset(0)": */ dev = new_device(devices, LGUEST_DEVICE_T_NET, 1, NET_PEERNUM|LGUEST_DEVICE_F_RANDOMNESS, netfd, handle_tun_input, peer_offset(0), handle_tun_output); + + /* We keep a flag which says whether we've seen packets come out from + * this network device. */ dev->priv = malloc(sizeof(bool)); *(bool *)dev->priv = false; + /* We need a socket to perform the magic network ioctls to bring up the + * tap interface, connect to the bridge etc. Any socket will do! */ ipfd = socket(PF_INET, SOCK_DGRAM, IPPROTO_IP); if (ipfd < 0) err(1, "opening IP socket"); + /* If the command line was --tunnet=bridge:<name> do bridging. */ if (!strncmp(BRIDGE_PFX, arg, strlen(BRIDGE_PFX))) { ip = INADDR_ANY; br_name = arg + strlen(BRIDGE_PFX); add_to_bridge(ipfd, ifr.ifr_name, br_name); - } else + } else /* It is an IP address to set up the device with */ ip = str2ip(arg); - /* We are peer 0, ie. first slot. */ + /* We are peer 0, ie. first slot, so we hand dev->mem to this routine + * to write the MAC address at the start of the device memory. */ configure_device(ipfd, ifr.ifr_name, ip, dev->mem); - /* Set "promisc" bit: we want every single packet. */ + /* Set "promisc" bit: we want every single packet if we're going to + * bridge to other machines (and otherwise it doesn't matter). */ *((u8 *)dev->mem) |= 0x1; close(ipfd); @@ -871,7 +1316,10 @@ static void setup_tun_net(const char *ar verbose("attached to bridge: %s\n", br_name); } -/* Now we know how much memory we have, we copy in device descriptors */ +/* That's the end of device setup. + * + * After we've finished all the command line processing we know how much memory + * we have, so we know where to put the device descriptors: */ static void map_device_descriptors(struct device_list *devs, unsigned long mem) { struct device *i; @@ -889,12 +1337,18 @@ static void map_device_descriptors(struc : i->desc->type == LGUEST_DEVICE_T_CONSOLE ? "console" : i->desc->type == LGUEST_DEVICE_T_BLOCK ? "block" : "unknown"); + /* Copy the device descriptor into the Guest memory and change + * the "struct device"'s pointer to refer to the new copy. + * This means that the device's input and output routines can + * see the status bits. */ descs[num] = *i->desc; free(i->desc); i->desc = &descs[num]; } } +/*L:220 Finally we reach the core of the Launcher, which runs the Guest, + * serves its input and output, and finally, lays it to rest. */ static void __attribute__((noreturn)) run_guest(int lguest_fd, struct device_list *device_list) { @@ -906,20 +1360,37 @@ run_guest(int lguest_fd, struct device_l /* We read from the /dev/lguest device to run the Guest. */ readval = read(lguest_fd, arr, sizeof(arr)); + /* The read can only really return sizeof(arr) (the Guest did a + * SEND_DMA to us), or an error. */ + + /* For a successful read, arr[0] is the address of the "struct + * lguest_dma", and arr[1] is the key the Guest sent to. */ if (readval == sizeof(arr)) { handle_output(lguest_fd, arr[0], arr[1], device_list); continue; + /* ENOENT means the Guest died. Reading tells us why. */ } else if (errno == ENOENT) { char reason[1024] = { 0 }; read(lguest_fd, reason, sizeof(reason)-1); errx(1, "%s", reason); + /* EAGAIN means the waker wanted us to look at some input. + * Anything else means a bug or incompatible change. */ } else if (errno != EAGAIN) err(1, "Running guest failed"); + + /* Service input, then unset the BREAK which releases + * the Waker. */ handle_input(lguest_fd, device_list); if (write(lguest_fd, args, sizeof(args)) < 0) err(1, "Resetting break"); } } +/* + * This is the end of the Launcher. + * + * But wait! We've seen I/O from the Launcher, and we've seen I/O from the + * Drivers. If we were to see the Host kernel I/O code, our understanding + * would be complete... :*/ static struct option opts[] = { { "verbose", 0, NULL, 'v' }, @@ -937,19 +1408,46 @@ static void usage(void) "<mem-in-mb> vmlinux [args...]"); } +/*L:100 The Launcher code itself takes us out into userspace, that scary place + * where pointers run wild and free! Unfortunately, like most userspace + * programs, it's quite boring (which is why everyone like to hack on the + * kernel!). Perhaps if you make up an Lguest Drinking Game at this point, it + * will get you through this section. Or, maybe not. + * + * The Launcher binary sits up high, usually starting at address 0xB8000000. + * Everything below this is the "physical" memory for the Guest. For example, + * if the Guest were to write a "1" at physical address 0, we would see a "1" + * in the Launcher at "(int *)0". Guest physical == Launcher virtual. + * + * This can be tough to get your head around, but usually it just means that we + * don't need to do any conversion when the Guest gives us it's "physical" + * addresses. + */ int main(int argc, char *argv[]) { + /* Memory, top-level pagetable, code startpoint, PAGE_OFFSET and size + * of the (optional) initrd. */ unsigned long mem, pgdir, start, page_offset, initrd_size = 0; + /* A temporary and the /dev/lguest file descriptor. */ int c, lguest_fd; + /* The list of Guest devices, based on command line arguments. */ struct device_list device_list; + /* The boot information for the Guest: at guest-physical address 0. */ void *boot = (void *)0; + /* If they specify an initrd file to load. */ const char *initrd_name = NULL; + /* First we initialize the device list. Since console and network + * device receive input from a file descriptor, we keep an fdset + * (infds) and the maximum fd number (max_infd) with the head of the + * list. We also keep a pointer to the last device, for easy appending + * to the list. */ device_list.max_infd = -1; device_list.dev = NULL; device_list.lastdev = &device_list.dev; FD_ZERO(&device_list.infds); + /* The options are fairly straight-forward */ while ((c = getopt_long(argc, argv, "v", opts, NULL)) != EOF) { switch (c) { case 'v': @@ -972,46 +1470,69 @@ int main(int argc, char *argv[]) usage(); } } + /* After the other arguments we expect memory and kernel image name, + * followed by command line arguments for the kernel. */ if (optind + 2 > argc) usage(); - /* We need a console device */ + /* We always have a console device */ setup_console(&device_list); - /* First we map /dev/zero over all of guest-physical memory. */ + /* First remaining argument is the number of megabytes of memory. */ mem = atoi(argv[optind]) * 1024 * 1024; + + /* We start by mapping anonymous pages over all of guest-physical + * memory range. This fills it with 0, and ensures that the Guest + * won't be killed when it tries to access it. */ map_zeroed_pages(0, mem / getpagesize()); /* Now we load the kernel */ start = load_kernel(open_or_die(argv[optind+1], O_RDONLY), &page_offset); - /* Write the device descriptors into memory. */ + /* Write the device descriptors into memory for the Guest to know what + * devices it has. The descriptors are mapped just above normal + * memory: we couldn't do this while parsing the command-line arguments + * in setup_net_file() etc because we didn't know how much memory the + * Guest was going to have. */ map_device_descriptors(&device_list, mem); - /* Map the initrd image if requested */ + /* Map the initrd image if requested (at top of physical memory) */ if (initrd_name) { initrd_size = load_initrd(initrd_name, mem); + /* These are the location in the Linux boot header where the + * start and size of the initrd are expected to be found. */ *(unsigned long *)(boot+0x218) = mem - initrd_size; *(unsigned long *)(boot+0x21c) = initrd_size; + /* The bootloader type 0xFF means "unknown"; that's OK. */ *(unsigned char *)(boot+0x210) = 0xFF; } - /* Set up the initial linar pagetables. */ + /* Set up the initial linear pagetables, starting below the initrd. */ pgdir = setup_pagetables(mem, initrd_size, page_offset); - /* E820 memory map: ours is a simple, single region. */ + /* The Linux boot header contains an "E820" memory map: ours is a + * simple, single region. */ *(char*)(boot+E820NR) = 1; *((struct e820entry *)(boot+E820MAP)) = ((struct e820entry) { 0, mem, E820_RAM }); - /* Command line pointer and command line (at 4096) */ + /* The boot header contains a command line pointer: we put the command + * line after the boot header (at address 4096) */ *(void **)(boot + 0x228) = boot + 4096; concat(boot + 4096, argv+optind+2); - /* Paravirt type: 1 == lguest */ + + /* The guest type value of "1" tells the Guest it's under lguest. */ *(int *)(boot + 0x23c) = 1; + /* We tell the kernel to initialize the Guest: this returns the open + * /dev/lguest file descriptor. */ lguest_fd = tell_kernel(pgdir, start, page_offset); + + /* We fork off a child process, which wakes the Launcher whenever one + * of the input file descriptors needs attention. Otherwise we would + * run the Guest until it tries to output something. */ waker_fd = setup_waker(lguest_fd, &device_list); + /* Finally, run the Guest. This doesn't return. */ run_guest(lguest_fd, &device_list); } =================================================================== --- a/drivers/lguest/core.c +++ b/drivers/lguest/core.c @@ -209,24 +209,39 @@ static int emulate_insn(struct lguest *l return 1; } +/*L:305 + * Dealing With Guest Memory. + * + * When the Guest gives us (what it thinks is) a physical address, we can use + * the normal copy_from_user() & copy_to_user() on that address: remember, + * Guest physical == Launcher virtual. + * + * But we can't trust the Guest: it might be trying to access the Launcher + * code. We have to check that the range is below the pfn_limit the Launcher + * gave us. We have to make sure that addr + len doesn't give us a false + * positive by overflowing, too. */ int lguest_address_ok(const struct lguest *lg, unsigned long addr, unsigned long len) { return (addr+len) / PAGE_SIZE < lg->pfn_limit && (addr+len >= addr); } -/* Just like get_user, but don't let guest access lguest binary. */ +/* This is a convenient routine to get a 32-bit value from the Guest (a very + * common operation). Here we can see how useful the kill_lguest() routine we + * met in the Launcher can be: we return a random value (0) instead of needing + * to return an error. */ u32 lgread_u32(struct lguest *lg, unsigned long addr) { u32 val = 0; - /* Don't let them access lguest binary */ + /* Don't let them access lguest binary. */ if (!lguest_address_ok(lg, addr, sizeof(val)) || get_user(val, (u32 __user *)addr) != 0) kill_guest(lg, "bad read address %#lx", addr); return val; } +/* Same thing for writing a value. */ void lgwrite_u32(struct lguest *lg, unsigned long addr, u32 val) { if (!lguest_address_ok(lg, addr, sizeof(val)) @@ -234,6 +249,9 @@ void lgwrite_u32(struct lguest *lg, unsi kill_guest(lg, "bad write address %#lx", addr); } +/* This routine is more generic, and copies a range of Guest bytes into a + * buffer. If the copy_from_user() fails, we fill the buffer with zeroes, so + * the caller doesn't end up using uninitialized kernel memory. */ void lgread(struct lguest *lg, void *b, unsigned long addr, unsigned bytes) { if (!lguest_address_ok(lg, addr, bytes) @@ -244,6 +262,7 @@ void lgread(struct lguest *lg, void *b, } } +/* Similarly, our generic routine to copy into a range of Guest bytes. */ void lgwrite(struct lguest *lg, unsigned long addr, const void *b, unsigned bytes) { @@ -251,6 +270,7 @@ void lgwrite(struct lguest *lg, unsigned || copy_to_user((void __user *)addr, b, bytes) != 0) kill_guest(lg, "bad write address %#lx len %u", addr, bytes); } +/* (end of memory access helper routines) :*/ static void set_ts(void) { =================================================================== --- a/drivers/lguest/io.c +++ b/drivers/lguest/io.c @@ -27,8 +27,36 @@ #include <linux/uaccess.h> #include "lg.h" +/*L:300 + * I/O + * + * Getting data in and out of the Guest is quite an art. There are numerous + * ways to do it, and they all suck differently. We try to keep things fairly + * close to "real" hardware so our Guest's drivers don't look like an alien + * visitation in the middle of the Linux code, and yet make sure that Guests + * can talk directly to other Guests, not just the Launcher. + * + * To do this, the Guest gives us a key when it binds or sends DMA buffers. + * The key corresponds to a "physical" address inside the Guest (ie. a virtual + * address inside the Launcher process). We don't, however, use this key + * directly. + * + * We want Guests which share memory to be able to DMA to each other: two + * Launchers can mmap memory the same file, then the Guests can communicate. + * Fortunately, the futex code provides us with a way to get a "union + * futex_key" corresponding to the memory lying at a virtual address: if the + * two processes share memory, the "union futex_key" for that memory will match + * even if the memory is mapped at different addresses in each. So we always + * convert the keys to "union futex_key"s to compare them. + * + * Before we dive into this though, we need to look at another set of helper + * routines used throughout the Host kernel code to access Guest memory. + :*/ static struct list_head dma_hash[61]; +/* An unfortunate side effect of the Linux double-linked list implementation is + * that there's no good way to statically initialize an array of linked + * lists. */ void lguest_io_init(void) { unsigned int i; @@ -60,6 +88,19 @@ kill: return 0; } +/*L:330 This is our hash function, using the wonderful Jenkins hash. + * + * The futex key is a union with three parts: an unsigned long word, a pointer, + * and an int "offset". We could use jhash_2words() which takes three u32s. + * (Ok, the hash functions are great: the naming sucks though). + * + * It's nice to be portable to 64-bit platforms, so we use the more generic + * jhash2(), which takes an array of u32, the number of u32s, and an initial + * u32 to roll in. This is uglier, but breaks down to almost the same code on + * 32-bit platforms like this one. + * + * We want a position in the array, so we modulo ARRAY_SIZE(dma_hash) (ie. 61). + */ static unsigned int hash(const union futex_key *key) { return jhash2((u32*)&key->both.word, @@ -68,6 +109,9 @@ static unsigned int hash(const union fut % ARRAY_SIZE(dma_hash); } +/* This is a convenience routine to compare two keys. It's a much bemoaned C + * weakness that it doesn't allow '==' on structures or unions, so we have to + * open-code it like this. */ static inline int key_eq(const union futex_key *a, const union futex_key *b) { return (a->both.word == b->both.word @@ -75,22 +119,36 @@ static inline int key_eq(const union fut && a->both.offset == b->both.offset); } -/* Must hold read lock on dmainfo owner's current->mm->mmap_sem */ +/*L:360 OK, when we need to actually free up a Guest's DMA array we do several + * things, so we have a convenient function to do it. + * + * The caller must hold a read lock on dmainfo owner's current->mm->mmap_sem + * for the drop_futex_key_refs(). */ static void unlink_dma(struct lguest_dma_info *dmainfo) { + /* You locked this too, right? */ BUG_ON(!mutex_is_locked(&lguest_lock)); + /* This is how we know that the entry is free. */ dmainfo->interrupt = 0; + /* Remove it from the hash table. */ list_del(&dmainfo->list); + /* Drop the references we were holding (to the inode or mm). */ drop_futex_key_refs(&dmainfo->key); } +/*L:350 This is the routine which we call when the Guest asks to unregister a + * DMA array attached to a given key. Returns true if the array was found. */ static int unbind_dma(struct lguest *lg, const union futex_key *key, unsigned long dmas) { int i, ret = 0; + /* We don't bother with the hash table, just look through all this + * Guest's DMA arrays. */ for (i = 0; i < LGUEST_MAX_DMA; i++) { + /* In theory it could have more than one array on the same key, + * or one array on multiple keys, so we check both */ if (key_eq(key, &lg->dma[i].key) && dmas == lg->dma[i].dmas) { unlink_dma(&lg->dma[i]); ret = 1; @@ -100,51 +158,91 @@ static int unbind_dma(struct lguest *lg, return ret; } +/*L:340 BIND_DMA: this is the hypercall which sets up an array of "struct + * lguest_dma" for receiving I/O. + * + * The Guest wants to bind an array of "struct lguest_dma"s to a particular key + * to receive input. This only happens when the Guest is setting up a new + * device, so it doesn't have to be very fast. + * + * It returns 1 on a successful registration (it can fail if we hit the limit + * of registrations for this Guest). + */ int bind_dma(struct lguest *lg, unsigned long ukey, unsigned long dmas, u16 numdmas, u8 interrupt) { unsigned int i; int ret = 0; union futex_key key; + /* Futex code needs the mmap_sem. */ struct rw_semaphore *fshared = ¤t->mm->mmap_sem; + /* Invalid interrupt? (We could kill the guest here). */ if (interrupt >= LGUEST_IRQS) return 0; + /* We need to grab the Big Lguest Lock, because other Guests may be + * trying to look through this Guest's DMAs to send something while + * we're doing this. */ mutex_lock(&lguest_lock); down_read(fshared); if (get_futex_key((u32 __user *)ukey, fshared, &key) != 0) { kill_guest(lg, "bad dma key %#lx", ukey); goto unlock; } + + /* We want to keep this key valid once we drop mmap_sem, so we have to + * hold a reference. */ get_futex_key_refs(&key); + /* If the Guest specified an interrupt of 0, that means they want to + * unregister this array of "struct lguest_dma"s. */ if (interrupt == 0) ret = unbind_dma(lg, &key, dmas); else { + /* Look through this Guest's dma array for an unused entry. */ for (i = 0; i < LGUEST_MAX_DMA; i++) { + /* If the interrupt is non-zero, the entry is already + * used. */ if (lg->dma[i].interrupt) continue; + /* OK, a free one! Fill on our details. */ lg->dma[i].dmas = dmas; lg->dma[i].num_dmas = numdmas; lg->dma[i].next_dma = 0; lg->dma[i].key = key; lg->dma[i].guestid = lg->guestid; lg->dma[i].interrupt = interrupt; + + /* Now we add it to the hash table: the position + * depends on the futex key that we got. */ list_add(&lg->dma[i].list, &dma_hash[hash(&key)]); + /* Success! */ ret = 1; goto unlock; } } + /* If we didn't find a slot to put the key in, drop the reference + * again. */ drop_futex_key_refs(&key); unlock: + /* Unlock and out. */ up_read(fshared); mutex_unlock(&lguest_lock); return ret; } -/* lgread from another guest */ +/*L:385 Note that our routines to access a different Guest's memory are called + * lgread_other() and lgwrite_other(): these names emphasize that they are only + * used when the Guest is *not* the current Guest. + * + * The interface for copying from another process's memory is called + * access_process_vm(), with a final argument of 0 for a read, and 1 for a + * write. + * + * We need lgread_other() to read the destination Guest's "struct lguest_dma" + * array. */ static int lgread_other(struct lguest *lg, void *buf, u32 addr, unsigned bytes) { @@ -157,7 +255,8 @@ static int lgread_other(struct lguest *l return 1; } -/* lgwrite to another guest */ +/* "lgwrite()" to another Guest: used to update the destination "used_len" once + * we've transferred data into the buffer. */ static int lgwrite_other(struct lguest *lg, u32 addr, const void *buf, unsigned bytes) { @@ -170,6 +269,15 @@ static int lgwrite_other(struct lguest * return 1; } +/*L:400 This is the generic engine which copies from a source "struct + * lguest_dma" from this Guest into another Guest's "struct lguest_dma". The + * destination Guest's pages have already been mapped, as contained in the + * pages array. + * + * If you're wondering if there's a nice "copy from one process to another" + * routine, so was I. But Linux isn't really set up to copy between two + * unrelated processes, so we have to write it ourselves. + */ static u32 copy_data(struct lguest *srclg, const struct lguest_dma *src, const struct lguest_dma *dst, @@ -178,33 +286,59 @@ static u32 copy_data(struct lguest *srcl unsigned int totlen, si, di, srcoff, dstoff; void *maddr = NULL; + /* We return the total length transferred. */ totlen = 0; + + /* We keep indexes into the source and destination "struct lguest_dma", + * and an offset within each region. */ si = di = 0; srcoff = dstoff = 0; + + /* We loop until the source or destination is exhausted. */ while (si < LGUEST_MAX_DMA_SECTIONS && src->len[si] && di < LGUEST_MAX_DMA_SECTIONS && dst->len[di]) { + /* We can only transfer the rest of the src buffer, or as much + * as will fit into the destination buffer. */ u32 len = min(src->len[si] - srcoff, dst->len[di] - dstoff); + /* For systems using "highmem" we need to use kmap() to access + * the page we want. We often use the same page over and over, + * so rather than kmap() it on every loop, we set the maddr + * pointer to NULL when we need to move to the next + * destination page. */ if (!maddr) maddr = kmap(pages[di]); - /* FIXME: This is not completely portable, since - archs do different things for copy_to_user_page. */ + /* Copy directly from (this Guest's) source address to the + * destination Guest's kmap()ed buffer. Note that maddr points + * to the start of the page: we need to add the offset of the + * destination address and offset within the buffer. */ + + /* FIXME: This is not completely portable. I looked at + * copy_to_user_page(), and some arch's seem to need special + * flushes. x86 is fine. */ if (copy_from_user(maddr + (dst->addr[di] + dstoff)%PAGE_SIZE, (void *__user)src->addr[si], len) != 0) { + /* If a copy failed, it's the source's fault. */ kill_guest(srclg, "bad address in sending DMA"); totlen = 0; break; } + /* Increment the total and src & dst offsets */ totlen += len; srcoff += len; dstoff += len; + + /* Presumably we reached the end of the src or dest buffers: */ if (srcoff == src->len[si]) { + /* Move to the next buffer at offset 0 */ si++; srcoff = 0; } if (dstoff == dst->len[di]) { + /* We need to unmap that destination page and reset + * maddr ready for the next one. */ kunmap(pages[di]); maddr = NULL; di++; @@ -212,13 +346,15 @@ static u32 copy_data(struct lguest *srcl } } + /* If we still had a page mapped at the end, unmap now. */ if (maddr) kunmap(pages[di]); return totlen; } -/* Src is us, ie. current. */ +/*L:390 This is how we transfer a "struct lguest_dma" from the source Guest + * (the current Guest which called SEND_DMA) to another Guest. */ static u32 do_dma(struct lguest *srclg, const struct lguest_dma *src, struct lguest *dstlg, const struct lguest_dma *dst) { @@ -226,23 +362,31 @@ static u32 do_dma(struct lguest *srclg, u32 ret; struct page *pages[LGUEST_MAX_DMA_SECTIONS]; + /* We check that both source and destination "struct lguest_dma"s are + * within the bounds of the source and destination Guests */ if (!check_dma_list(dstlg, dst) || !check_dma_list(srclg, src)) return 0; - /* First get the destination pages */ + /* We need to map the pages which correspond to each parts of + * destination buffer. */ for (i = 0; i < LGUEST_MAX_DMA_SECTIONS; i++) { if (dst->len[i] == 0) break; + /* get_user_pages() is a complicated function, especially since + * we only want a single page. But it works, and returns the + * number of pages. Note that we're holding the destination's + * mmap_sem, as get_user_pages() requires. */ if (get_user_pages(dstlg->tsk, dstlg->mm, dst->addr[i], 1, 1, 1, pages+i, NULL) != 1) { + /* This means the destination gave us a bogus buffer */ kill_guest(dstlg, "Error mapping DMA pages"); ret = 0; goto drop_pages; } } - /* Now copy until we run out of src or dst. */ + /* Now copy the data until we run out of src or dst. */ ret = copy_data(srclg, src, dst, pages); drop_pages: @@ -251,6 +395,11 @@ drop_pages: return ret; } +/*L:380 Transferring data from one Guest to another is not as simple as I'd + * like. We've found the "struct lguest_dma_info" bound to the same address as + * the send, we need to copy into it. + * + * This function returns true if the destination array was empty. */ static int dma_transfer(struct lguest *srclg, unsigned long udma, struct lguest_dma_info *dst) @@ -259,15 +408,23 @@ static int dma_transfer(struct lguest *s struct lguest *dstlg; u32 i, dma = 0; + /* From the "struct lguest_dma_info" we found in the hash, grab the + * Guest. */ dstlg = &lguests[dst->guestid]; - /* Get our dma list. */ + /* Read in the source "struct lguest_dma" handed to SEND_DMA. */ lgread(srclg, &src_dma, udma, sizeof(src_dma)); - /* We can't deadlock against them dmaing to us, because this - * is all under the lguest_lock. */ + /* We need the destination's mmap_sem, and we already hold the source's + * mmap_sem for the futex key lookup. Normally this would suggest that + * we could deadlock if the destination Guest was trying to send to + * this source Guest at the same time, which is another reason that all + * I/O is done under the big lguest_lock. */ down_read(&dstlg->mm->mmap_sem); + /* Look through the destination DMA array for an available buffer. */ for (i = 0; i < dst->num_dmas; i++) { + /* We keep a "next_dma" pointer which often helps us avoid + * looking at lots of previously-filled entries. */ dma = (dst->next_dma + i) % dst->num_dmas; if (!lgread_other(dstlg, &dst_dma, dst->dmas + dma * sizeof(struct lguest_dma), @@ -277,30 +434,46 @@ static int dma_transfer(struct lguest *s if (!dst_dma.used_len) break; } + + /* If we found a buffer, we do the actual data copy. */ if (i != dst->num_dmas) { unsigned long used_lenp; unsigned int ret; ret = do_dma(srclg, &src_dma, dstlg, &dst_dma); - /* Put used length in src. */ + /* Put used length in the source "struct lguest_dma"'s used_len + * field. It's a little tricky to figure out where that is, + * though. */ lgwrite_u32(srclg, udma+offsetof(struct lguest_dma, used_len), ret); + /* Tranferring 0 bytes is OK if the source buffer was empty. */ if (ret == 0 && src_dma.len[0] != 0) goto fail; - /* Make sure destination sees contents before length. */ + /* The destination Guest might be running on a different CPU: + * we have to make sure that it will see the "used_len" field + * change to non-zero *after* it sees the data we copied into + * the buffer. Hence a write memory barrier. */ wmb(); + /* Figuring out where the destination's used_len field for this + * "struct lguest_dma" in the array is also a little ugly. */ used_lenp = dst->dmas + dma * sizeof(struct lguest_dma) + offsetof(struct lguest_dma, used_len); lgwrite_other(dstlg, used_lenp, &ret, sizeof(ret)); + /* Move the cursor for next time. */ dst->next_dma++; } up_read(&dstlg->mm->mmap_sem); - /* Do this last so dst doesn't simply sleep on lock. */ + /* We trigger the destination interrupt, even if the destination was + * empty and we didn't transfer anything: this gives them a chance to + * wake up and refill. */ set_bit(dst->interrupt, dstlg->irqs_pending); + /* Wake up the destination process. */ set_wakeup_process(srclg, dstlg->tsk); + /* If we passed the last "struct lguest_dma", the receive had no + * buffers left. */ return i == dst->num_dmas; fail: @@ -308,6 +481,8 @@ fail: return 0; } +/*L:370 This is the counter-side to the BIND_DMA hypercall; the SEND_DMA + * hypercall. We find out who's listening, and send to them. */ void send_dma(struct lguest *lg, unsigned long ukey, unsigned long udma) { union futex_key key; @@ -317,32 +492,45 @@ again: again: mutex_lock(&lguest_lock); down_read(fshared); + /* Get the futex key for the key the Guest gave us */ if (get_futex_key((u32 __user *)ukey, fshared, &key) != 0) { kill_guest(lg, "bad sending DMA key"); goto unlock; } - /* Shared mapping? Look for other guests... */ + /* Since the key must be a multiple of 4, the futex key uses the lower + * bit of the "offset" field (which would always be 0) to indicate a + * mapping which is shared with other processes (ie. Guests). */ if (key.shared.offset & 1) { struct lguest_dma_info *i; + /* Look through the hash for other Guests. */ list_for_each_entry(i, &dma_hash[hash(&key)], list) { + /* Don't send to ourselves. */ if (i->guestid == lg->guestid) continue; if (!key_eq(&key, &i->key)) continue; + /* If dma_transfer() tells us the destination has no + * available buffers, we increment "empty". */ empty += dma_transfer(lg, udma, i); break; } + /* If the destination is empty, we release our locks and + * give the destination Guest a brief chance to restock. */ if (empty == 1) { /* Give any recipients one chance to restock. */ up_read(¤t->mm->mmap_sem); mutex_unlock(&lguest_lock); + /* This wakes up the destination. */ set_wakeup_process(lg, NULL); + /* Next time, we won't try again. */ empty++; goto again; } } else { - /* Private mapping: tell our userspace. */ + /* Private mapping: Guest is sending to its Launcher. We set + * the "dma_is_pending" flag so that the main loop will exit + * and the Launcher's read() from /dev/lguest will return. */ lg->dma_is_pending = 1; lg->pending_dma = udma; lg->pending_key = ukey; @@ -351,6 +539,7 @@ unlock: up_read(fshared); mutex_unlock(&lguest_lock); } +/*:*/ void release_all_dma(struct lguest *lg) { @@ -381,7 +570,8 @@ void set_wakeup_process(struct lguest *l get_task_struct(lg->wake); } -/* Userspace wants a dma buffer from this guest. */ +/*L:320 This routine looks for a DMA buffer registered by the Guest on the + * given key (using the BIND_DMA hypercall). */ unsigned long get_dma_buffer(struct lguest *lg, unsigned long ukey, unsigned long *interrupt) { @@ -390,15 +580,29 @@ unsigned long get_dma_buffer(struct lgue struct lguest_dma_info *i; struct rw_semaphore *fshared = ¤t->mm->mmap_sem; + /* Take the Big Lguest Lock to stop other Guests sending this Guest DMA + * at the same time. */ mutex_lock(&lguest_lock); + /* To match between Guests sharing the same underlying memory we steal + * code from the futex infrastructure. This requires that we hold the + * "mmap_sem" for our process (the Launcher), and pass it to the futex + * code. */ down_read(fshared); + + /* This can fail if it's not a valid address, or if the address is not + * divisible by 4 (the futex code needs that, we don't really). */ if (get_futex_key((u32 __user *)ukey, fshared, &key) != 0) { kill_guest(lg, "bad registered DMA buffer"); goto unlock; } + /* Search the hash table for matching entries (the Launcher can only + * send to its own Guest for the moment, so the entry must be for this + * Guest) */ list_for_each_entry(i, &dma_hash[hash(&key)], list) { if (key_eq(&key, &i->key) && i->guestid == lg->guestid) { unsigned int j; + /* Look through the registered DMA array for an + * available buffer. */ for (j = 0; j < i->num_dmas; j++) { struct lguest_dma dma; @@ -407,6 +611,8 @@ unsigned long get_dma_buffer(struct lgue if (dma.used_len == 0) break; } + /* Store the interrupt the Guest wants when the buffer + * is used. */ *interrupt = i->interrupt; break; } @@ -416,4 +622,12 @@ unlock: mutex_unlock(&lguest_lock); return ret; } - +/*:*/ + +/*L:410 This really has completed the Launcher. Not only have we now finished + * the longest chapter in our journey, but this also means we are over halfway + * through! + * + * Enough prevaricating around the bush: it is time for us to dive into the + * core of the Host, in "make Host". + */ =================================================================== --- a/drivers/lguest/lg.h +++ b/drivers/lguest/lg.h @@ -245,6 +245,30 @@ void set_wakeup_process(struct lguest *l /* hypercalls.c: */ void do_hypercalls(struct lguest *lg); +/*L:035 + * Let's step aside for the moment, to study one important routine that's used + * widely in the Host code. + * + * There are many cases where the Guest does something invalid, like pass crap + * to a hypercall. Since only the Guest kernel can make hypercalls, it's quite + * acceptable to simply terminate the Guest and give the Launcher a nicely + * formatted reason. It's also simpler for the Guest itself, which doesn't + * need to check most hypercalls for "success"; if you're still running, it + * succeeded. + * + * Once this is called, the Guest will never run again, so most Host code can + * call this then continue as if nothing had happened. This means many + * functions don't have to explicitly return an error code, which keeps the + * code simple. + * + * It also means that this can be called more than once: only the first one is + * remembered. The only trick is that we still need to kill the Guest even if + * we can't allocate memory to store the reason. Linux has a neat way of + * packing error codes into invalid pointers, so we use that here. + * + * Like any macro which uses an "if", it is safely wrapped in a run-once "do { + * } while(0)". + */ #define kill_guest(lg, fmt...) \ do { \ if (!(lg)->dead) { \ @@ -253,6 +277,7 @@ do { \ (lg)->dead = ERR_PTR(-ENOMEM); \ } \ } while(0) +/* (End of aside) :*/ static inline unsigned long guest_pa(struct lguest *lg, unsigned long vaddr) { =================================================================== --- a/drivers/lguest/lguest_user.c +++ b/drivers/lguest/lguest_user.c @@ -9,33 +9,62 @@ #include <linux/fs.h> #include "lg.h" +/*L:030 setup_regs() doesn't really belong in this file, but it gives us an + * early glimpse deeper into the Host so it's worth having here. + * + * Most of the Guest's registers are left alone: we used get_zeroed_page() to + * allocate the structure, so they will be 0. */ static void setup_regs(struct lguest_regs *regs, unsigned long start) { - /* Write out stack in format lguest expects, so we can switch to it. */ + /* There are four "segment" registers which the Guest needs to boot: + * The "code segment" register (cs) refers to the kernel code segment + * __KERNEL_CS, and the "data", "extra" and "stack" segment registers + * refer to the kernel data segment __KERNEL_DS. + * + * The privilege level is packed into the lower bits. The Guest runs + * at privilege level 1 (GUEST_PL).*/ regs->ds = regs->es = regs->ss = __KERNEL_DS|GUEST_PL; regs->cs = __KERNEL_CS|GUEST_PL; - regs->eflags = 0x202; /* Interrupts enabled. */ + + /* The "eflags" register contains miscellaneous flags. Bit 1 (0x002) + * is supposed to always be "1". Bit 9 (0x200) controls whether + * interrupts are enabled. We always leave interrupts enabled while + * running the Guest. */ + regs->eflags = 0x202; + + /* The "Extended Instruction Pointer" register says where the Guest is + * running. */ regs->eip = start; - /* esi points to our boot information (physical address 0) */ -} - -/* + addr */ + + /* %esi points to our boot information, at physical address 0, so don't + * touch it. */ +} + +/*L:310 To send DMA into the Guest, the Launcher needs to be able to ask for a + * DMA buffer. This is done by writing LHREQ_GETDMA and the key to + * /dev/lguest. */ static long user_get_dma(struct lguest *lg, const u32 __user *input) { unsigned long key, udma, irq; + /* Fetch the key they wrote to us. */ if (get_user(key, input) != 0) return -EFAULT; + /* Look for a free Guest DMA buffer bound to that key. */ udma = get_dma_buffer(lg, key, &irq); if (!udma) return -ENOENT; - /* We put irq number in udma->used_len. */ + /* We need to tell the Launcher what interrupt the Guest expects after + * the buffer is filled. We stash it in udma->used_len. */ lgwrite_u32(lg, udma + offsetof(struct lguest_dma, used_len), irq); + + /* The (guest-physical) address of the DMA buffer is returned from + * the write(). */ return udma; } -/* To force the Guest to stop running and return to the Launcher, the +/*L:315 To force the Guest to stop running and return to the Launcher, the * Waker sets writes LHREQ_BREAK and the value "1" to /dev/lguest. The * Launcher then writes LHREQ_BREAK and "0" to release the Waker. */ static int break_guest_out(struct lguest *lg, const u32 __user *input) @@ -59,7 +88,8 @@ static int break_guest_out(struct lguest } } -/* + irq */ +/*L:050 Sending an interrupt is done by writing LHREQ_IRQ and an interrupt + * number to /dev/lguest. */ static int user_send_irq(struct lguest *lg, const u32 __user *input) { u32 irq; @@ -68,14 +98,19 @@ static int user_send_irq(struct lguest * return -EFAULT; if (irq >= LGUEST_IRQS) return -EINVAL; + /* Next time the Guest runs, the core code will see if it can deliver + * this interrupt. */ set_bit(irq, lg->irqs_pending); return 0; } +/*L:040 Once our Guest is initialized, the Launcher makes it run by reading + * from /dev/lguest. */ static ssize_t read(struct file *file, char __user *user, size_t size,loff_t*o) { struct lguest *lg = file->private_data; + /* You must write LHREQ_INITIALIZE first! */ if (!lg) return -EINVAL; @@ -83,27 +118,52 @@ static ssize_t read(struct file *file, c if (current != lg->tsk) return -EPERM; + /* If the guest is already dead, we indicate why */ if (lg->dead) { size_t len; + /* lg->dead either contains an error code, or a string. */ if (IS_ERR(lg->dead)) return PTR_ERR(lg->dead); + /* We can only return as much as the buffer they read with. */ len = min(size, strlen(lg->dead)+1); if (copy_to_user(user, lg->dead, len) != 0) return -EFAULT; return len; } + /* If we returned from read() last time because the Guest sent DMA, + * clear the flag. */ if (lg->dma_is_pending) lg->dma_is_pending = 0; + /* Run the Guest until something interesting happens. */ return run_guest(lg, (unsigned long __user *)user); } -/* Take: pfnlimit, pgdir, start, pageoffset. */ +/*L:020 The initialization write supplies 4 32-bit values (in addition to the + * 32-bit LHREQ_INITIALIZE value). These are: + * + * pfnlimit: The highest (Guest-physical) page number the Guest should be + * allowed to access. The Launcher has to live in Guest memory, so it sets + * this to ensure the Guest can't reach it. + * + * pgdir: The (Guest-physical) address of the top of the initial Guest + * pagetables (which are set up by the Launcher). + * + * start: The first instruction to execute ("eip" in x86-speak). + * + * page_offset: The PAGE_OFFSET constant in the Guest kernel. We should + * probably wean the code off this, but it's a very useful constant! Any + * address above this is within the Guest kernel, and any kernel address can + * quickly converted from physical to virtual by adding PAGE_OFFSET. It's + * 0xC0000000 (3G) by default, but it's configurable at kernel build time. + */ static int initialize(struct file *file, const u32 __user *input) { + /* "struct lguest" contains everything we (the Host) know about a + * Guest. */ struct lguest *lg; int err, i; u32 args[4]; @@ -111,7 +171,7 @@ static int initialize(struct file *file, /* We grab the Big Lguest lock, which protects the global array * "lguests" and multiple simultaneous initializations. */ mutex_lock(&lguest_lock); - + /* You can't initialize twice! Close the device and start again... */ if (file->private_data) { err = -EBUSY; goto unlock; @@ -122,38 +182,73 @@ static int initialize(struct file *file, goto unlock; } + /* Find an unused guest. */ i = find_free_guest(); if (i < 0) { err = -ENOSPC; goto unlock; } + /* OK, we have an index into the "lguest" array: "lg" is a convenient + * pointer. */ lg = &lguests[i]; + + /* Populate the easy fields of our "struct lguest" */ lg->guestid = i; lg->pfn_limit = args[0]; lg->page_offset = args[3]; + + /* We need a complete page for the Guest registers: they are accessible + * to the Guest and we can only grant it access to whole pages. */ lg->regs_page = get_zeroed_page(GFP_KERNEL); if (!lg->regs_page) { err = -ENOMEM; goto release_guest; } + /* We actually put the registers at the bottom of the page. */ lg->regs = (void *)lg->regs_page + PAGE_SIZE - sizeof(*lg->regs); + /* Initialize the Guest's shadow page tables, using the toplevel + * address the Launcher gave us. This allocates memory, so can + * fail. */ err = init_guest_pagetable(lg, args[1]); if (err) goto free_regs; + /* Now we initialize the Guest's registers, handing it the start + * address. */ setup_regs(lg->regs, args[2]); + + /* There are a couple of GDT entries the Guest expects when first + * booting. */ setup_guest_gdt(lg); + + /* The timer for lguest's clock needs initialization. */ init_clockdev(lg); + + /* We keep a pointer to the Launcher task (ie. current task) for when + * other Guests want to wake this one (inter-Guest I/O). */ lg->tsk = current; + /* We also to hold a reference, so it doesn't vanish while we're + * sending it I/O */ get_task_struct(lg->tsk); + /* We need to keep a pointer to the Launcher's memory map, because if + * the Launcher dies we need to clean it up. If we don't keep a + * reference, it is destroyed before close() is called. */ lg->mm = get_task_mm(lg->tsk); + + /* Initialize the queue for the waker to wait on */ init_waitqueue_head(&lg->break_wq); + + /* We remember which CPU's pages this Guest used last, for optimization + * when the same Guest runs on the same CPU twice. */ lg->last_pages = NULL; + + /* We keep our "struct lguest" in the file's private_data. */ file->private_data = lg; mutex_unlock(&lguest_lock); + /* And because this is a write() call, we return the length used. */ return sizeof(args); free_regs: @@ -165,9 +260,15 @@ unlock: return err; } +/*L:010 The first operation the Launcher does must be a write. All writes + * start with a 32 bit number: for the first write this must be + * LHREQ_INITIALIZE to set up the Guest. After that the Launcher can use + * writes of other values to get DMA buffers and send interrupts. */ static ssize_t write(struct file *file, const char __user *input, size_t size, loff_t *off) { + /* Once the guest is initialized, we hold the "struct lguest" in the + * file private data. */ struct lguest *lg = file->private_data; u32 req; @@ -175,8 +276,11 @@ static ssize_t write(struct file *file, return -EFAULT; input += sizeof(req); + /* If you haven't initialized, you must do that first. */ if (req != LHREQ_INITIALIZE && !lg) return -EINVAL; + + /* Once the Guest is dead, all you can do is read() why it died. */ if (lg && lg->dead) return -ENOENT; @@ -198,34 +302,74 @@ static ssize_t write(struct file *file, } } +/*L:060 The final piece of interface code is the close() routine. It reverses + * everything done in initialize(). This is usually called because the + * Launcher exited. + * + * Note that the close routine returns 0 or a negative error number: it can't + * really fail, but it can whine. I blame Sun for this wart, and K&R C for + * letting them do it. :*/ static int close(struct inode *inode, struct file *file) { struct lguest *lg = file->private_data; + /* If we never successfully initialized, there's nothing to clean up */ if (!lg) return 0; + /* We need the big lock, to protect from inter-guest I/O and other + * Launchers initializing guests. */ mutex_lock(&lguest_lock); /* Cancels the hrtimer set via LHCALL_SET_CLOCKEVENT. */ hrtimer_cancel(&lg->hrt); + /* Free any DMA buffers the Guest had bound. */ release_all_dma(lg); + /* Free up the shadow page tables for the Guest. */ free_guest_pagetable(lg); + /* Release the task reference we took. */ put_task_struct(lg->tsk); + /* Now all the memory cleanups are done, it's safe to release the + * Launcher's memory management structure. */ mmput(lg->mm); + /* If lg->dead doesn't contain an error code it will be NULL or a + * kmalloc()ed string, either of which is ok to hand to kfree(). */ if (!IS_ERR(lg->dead)) kfree(lg->dead); + /* We can free up the register page we allocated. */ free_page(lg->regs_page); + /* We clear the entire structure, which also marks it as free for the + * next user. */ memset(lg, 0, sizeof(*lg)); + /* Release lock and exit. */ mutex_unlock(&lguest_lock); + return 0; } +/*L:000 + * Welcome to our journey through the Launcher! + * + * The Launcher is the Host userspace program which sets up, runs and services + * the Guest. In fact, many comments in the Drivers which refer to "the Host" + * doing things are inaccurate: the Launcher does all the device handling for + * the Guest. The Guest can't tell what's done by the the Launcher and what by + * the Host. + * + * Just to confuse you: to the Host kernel, the Launcher *is* the Guest and we + * shall see more of that later. + * + * We begin our understanding with the Host kernel interface which the Launcher + * uses: reading and writing a character device called /dev/lguest. All the + * work happens in the read(), write() and close() routines: */ static struct file_operations lguest_fops = { .owner = THIS_MODULE, .release = close, .write = write, .read = read, }; + +/* This is a textbook example of a "misc" character device. Populate a "struct + * miscdevice" and register it with misc_register(). */ static struct miscdevice lguest_dev = { .minor = MISC_DYNAMIC_MINOR, .name = "lguest", ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH 5/7] lguest documentation:Chapter V 2007-06-08 3:13 ` [PATCH 4/7] lguest documentation:Chapter IV Rusty Russell @ 2007-06-08 3:13 ` Rusty Russell [not found] ` <1181272434.24958.19.camel@localhost.localdomain> 1 sibling, 0 replies; 8+ messages in thread From: Rusty Russell @ 2007-06-08 3:13 UTC (permalink / raw) To: Andrew Morton; +Cc: lkml - Kernel Mailing List, virtualization Documentation: The Host Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> --- drivers/lguest/core.c | 269 ++++++++++++++++++++++++++- drivers/lguest/hypercalls.c | 115 ++++++++++- drivers/lguest/interrupts_and_traps.c | 176 ++++++++++++++++-- drivers/lguest/lg.h | 19 + drivers/lguest/page_tables.c | 318 +++++++++++++++++++++++++++++---- drivers/lguest/segments.c | 109 ++++++++++- 6 files changed, 918 insertions(+), 88 deletions(-) =================================================================== --- a/drivers/lguest/core.c +++ b/drivers/lguest/core.c @@ -65,11 +65,33 @@ static struct lguest_pages *lguest_pages (SWITCHER_ADDR + SHARED_SWITCHER_PAGES*PAGE_SIZE))[cpu]); } +/*H:010 We need to set up the Switcher at a high virtual address. Remember the + * Switcher is a few hundred bytes of assembler code which actually changes the + * CPU to run the Guest, and then changes back to the Host when a trap or + * interrupt happens. + * + * The Switcher code must be at the same virtual address in the Guest as the + * Host since it will be running as the switchover occurs. + * + * Trying to map memory at a particular address is an unusual thing to do, so + * it's not a simple one-liner. We also set up the per-cpu parts of the + * Switcher here. + */ static __init int map_switcher(void) { int i, err; struct page **pagep; + /* + * Map the Switcher in to high memory. + * + * It turns out that if we choose the address 0xFFC00000 (4MB under the + * top virtual address), it makes setting up the page tables really + * easy. + */ + + /* We allocate an array of "struct page"s. map_vm_area() wants the + * pages in this form, rather than just an array of pointers. */ switcher_page = kmalloc(sizeof(switcher_page[0])*TOTAL_SWITCHER_PAGES, GFP_KERNEL); if (!switcher_page) { @@ -77,6 +99,8 @@ static __init int map_switcher(void) goto out; } + /* Now we actually allocate the pages. The Guest will see these pages, + * so we make sure they're zeroed. */ for (i = 0; i < TOTAL_SWITCHER_PAGES; i++) { unsigned long addr = get_zeroed_page(GFP_KERNEL); if (!addr) { @@ -86,6 +110,9 @@ static __init int map_switcher(void) switcher_page[i] = virt_to_page(addr); } + /* Now we reserve the "virtual memory area" we want: 0xFFC00000 + * (SWITCHER_ADDR). We might not get it in theory, but in practice + * it's worked so far. */ switcher_vma = __get_vm_area(TOTAL_SWITCHER_PAGES * PAGE_SIZE, VM_ALLOC, SWITCHER_ADDR, VMALLOC_END); if (!switcher_vma) { @@ -94,49 +121,105 @@ static __init int map_switcher(void) goto free_pages; } + /* This code actually sets up the pages we've allocated to appear at + * SWITCHER_ADDR. map_vm_area() takes the vma we allocated above, the + * kind of pages we're mapping (kernel pages), and a pointer to our + * array of struct pages. It increments that pointer, but we don't + * care. */ pagep = switcher_page; err = map_vm_area(switcher_vma, PAGE_KERNEL, &pagep); if (err) { printk("lguest: map_vm_area failed: %i\n", err); goto free_vma; } + + /* Now the switcher is mapped at the right address, we can't fail! + * Copy in the compiled-in Switcher code (from switcher.S). */ memcpy(switcher_vma->addr, start_switcher_text, end_switcher_text - start_switcher_text); - /* Fix up IDT entries to point into copied text. */ + /* Most of the switcher.S doesn't care that it's been moved; on Intel, + * jumps are relative, and it doesn't access any references to external + * code or data. + * + * The only exception is the interrupt handlers in switcher.S: their + * addresses are placed in a table (default_idt_entries), so we need to + * update the table with the new addresses. switcher_offset() is a + * convenience function which returns the distance between the builtin + * switcher code and the high-mapped copy we just made. */ for (i = 0; i < IDT_ENTRIES; i++) default_idt_entries[i] += switcher_offset(); + /* + * Set up the Switcher's per-cpu areas. + * + * Each CPU gets two pages of its own within the high-mapped region + * (aka. "struct lguest_pages"). Much of this can be initialized now, + * but some depends on what Guest we are running (which is set up in + * copy_in_guest_info()). + */ for_each_possible_cpu(i) { + /* lguest_pages() returns this CPU's two pages. */ struct lguest_pages *pages = lguest_pages(i); + /* This is a convenience pointer to make the code fit one + * statement to a line. */ struct lguest_ro_state *state = &pages->state; - /* These fields are static: rest done in copy_in_guest_info */ + /* The Global Descriptor Table: the Host has a different one + * for each CPU. We keep a descriptor for the GDT which says + * where it is and how big it is (the size is actually the last + * byte, not the size, hence the "-1"). */ state->host_gdt_desc.size = GDT_SIZE-1; state->host_gdt_desc.address = (long)get_cpu_gdt_table(i); + + /* All CPUs on the Host use the same Interrupt Descriptor + * Table, so we just use store_idt(), which gets this CPU's IDT + * descriptor. */ store_idt(&state->host_idt_desc); + + /* The descriptors for the Guest's GDT and IDT can be filled + * out now, too. We copy the GDT & IDT into ->guest_gdt and + * ->guest_idt before actually running the Guest. */ state->guest_idt_desc.size = sizeof(state->guest_idt)-1; state->guest_idt_desc.address = (long)&state->guest_idt; state->guest_gdt_desc.size = sizeof(state->guest_gdt)-1; state->guest_gdt_desc.address = (long)&state->guest_gdt; + + /* We know where we want the stack to be when the Guest enters + * the switcher: in pages->regs. The stack grows upwards, so + * we start it at the end of that structure. */ state->guest_tss.esp0 = (long)(&pages->regs + 1); + /* And this is the GDT entry to use for the stack: we keep a + * couple of special LGUEST entries. */ state->guest_tss.ss0 = LGUEST_DS; - /* No I/O for you! */ + + /* x86 can have a finegrained bitmap which indicates what I/O + * ports the process can use. We set it to the end of our + * structure, meaning "none". */ state->guest_tss.io_bitmap_base = sizeof(state->guest_tss); + + /* Some GDT entries are the same across all Guests, so we can + * set them up now. */ setup_default_gdt_entries(state); + /* Most IDT entries are the same for all Guests, too.*/ setup_default_idt_entries(state, default_idt_entries); - /* Setup LGUEST segments on all cpus */ + /* The Host needs to be able to use the LGUEST segments on this + * CPU, too, so put them in the Host GDT. */ get_cpu_gdt_table(i)[GDT_ENTRY_LGUEST_CS] = FULL_EXEC_SEGMENT; get_cpu_gdt_table(i)[GDT_ENTRY_LGUEST_DS] = FULL_SEGMENT; } - /* Initialize entry point into switcher. */ + /* In the Switcher, we want the %cs segment register to use the + * LGUEST_CS GDT entry: we've put that in the Host and Guest GDTs, so + * it will be undisturbed when we switch. To change %cs and jump we + * need this structure to feed to Intel's "lcall" instruction. */ lguest_entry.offset = (long)switch_to_guest + switcher_offset(); lguest_entry.segment = LGUEST_CS; printk(KERN_INFO "lguest: mapped switcher at %p\n", switcher_vma->addr); + /* And we succeeded... */ return 0; free_vma: @@ -150,35 +233,58 @@ out: out: return err; } - +/*:*/ + +/* Cleaning up the mapping when the module is unloaded is almost... + * too easy. */ static void unmap_switcher(void) { unsigned int i; + /* vunmap() undoes *both* map_vm_area() and __get_vm_area(). */ vunmap(switcher_vma->addr); + /* Now we just need to free the pages we copied the switcher into */ for (i = 0; i < TOTAL_SWITCHER_PAGES; i++) __free_pages(switcher_page[i], 0); } -/* IN/OUT insns: enough to get us past boot-time probing. */ +/*H:130 Our Guest is usually so well behaved; it never tries to do things it + * isn't allowed to. Unfortunately, "struct paravirt_ops" isn't quite + * complete, because it doesn't contain replacements for the Intel I/O + * instructions. As a result, the Guest sometimes fumbles across one during + * the boot process as it probes for various things which are usually attached + * to a PC. + * + * When the Guest uses one of these instructions, we get trap #13 (General + * Protection Fault) and come here. We see if it's one of those troublesome + * instructions and skip over it. We return true if we did. */ static int emulate_insn(struct lguest *lg) { u8 insn; unsigned int insnlen = 0, in = 0, shift = 0; + /* The eip contains the *virtual* address of the Guest's instruction: + * guest_pa just subtracts the Guest's page_offset. */ unsigned long physaddr = guest_pa(lg, lg->regs->eip); - /* This only works for addresses in linear mapping... */ + /* The guest_pa() function only works for Guest kernel addresses, but + * that's all we're trying to do anyway. */ if (lg->regs->eip < lg->page_offset) return 0; + + /* Decoding x86 instructions is icky. */ lgread(lg, &insn, physaddr, 1); - /* Operand size prefix means it's actually for ax. */ + /* 0x66 is an "operand prefix". It means it's using the upper 16 bits + of the eax register. */ if (insn == 0x66) { shift = 16; + /* The instruction is 1 byte so far, read the next byte. */ insnlen = 1; lgread(lg, &insn, physaddr + insnlen, 1); } + /* We can ignore the lower bit for the moment and decode the 4 opcodes + * we need to emulate. */ switch (insn & 0xFE) { case 0xE4: /* in <next byte>,%al */ insnlen += 2; @@ -195,9 +301,13 @@ static int emulate_insn(struct lguest *l insnlen += 1; break; default: + /* OK, we don't know what this is, can't emulate. */ return 0; } + /* If it was an "IN" instruction, they expect the result to be read + * into %eax, so we change %eax. We always return all-ones, which + * traditionally means "there's nothing there". */ if (in) { /* Lower bit tells is whether it's a 16 or 32 bit access */ if (insn & 0x1) @@ -205,9 +315,12 @@ static int emulate_insn(struct lguest *l else lg->regs->eax |= (0xFFFF << shift); } + /* Finally, we've "done" the instruction, so move past it. */ lg->regs->eip += insnlen; + /* Success! */ return 1; } +/*:*/ /*L:305 * Dealing With Guest Memory. @@ -322,13 +435,24 @@ static void run_guest_once(struct lguest : "memory", "%edx", "%ecx", "%edi", "%esi"); } +/*H:030 Let's jump straight to the the main loop which runs the Guest. + * Remember, this is called by the Launcher reading /dev/lguest, and we keep + * going around and around until something interesting happens. */ int run_guest(struct lguest *lg, unsigned long __user *user) { + /* We stop running once the Guest is dead. */ while (!lg->dead) { + /* We need to initialize this, otherwise gcc complains. It's + * not (yet) clever enough to see that it's initialized when we + * need it. */ unsigned int cr2 = 0; /* Damn gcc */ - /* Hypercalls first: we might have been out to userspace */ + /* First we run any hypercalls the Guest wants done: either in + * the hypercall ring in "struct lguest_data", or directly by + * using int 31 (LGUEST_TRAP_ENTRY). */ do_hypercalls(lg); + /* It's possible the Guest did a SEND_DMA hypercall to the + * Launcher, in which case we return from the read() now. */ if (lg->dma_is_pending) { if (put_user(lg->pending_dma, user) || put_user(lg->pending_key, user+1)) @@ -336,6 +460,7 @@ int run_guest(struct lguest *lg, unsigne return sizeof(unsigned long)*2; } + /* Check for signals */ if (signal_pending(current)) return -ERESTARTSYS; @@ -343,14 +468,24 @@ int run_guest(struct lguest *lg, unsigne if (lg->break_out) return -EAGAIN; + /* Check if there are any interrupts which can be delivered + * now: if so, this sets up the hander to be executed when we + * next run the Guest. */ maybe_do_interrupt(lg); + /* All long-lived kernel loops need to check with this horrible + * thing called the freezer. If the Host is trying to suspend, + * it stops us. */ try_to_freeze(); + /* Just make absolutely sure the Guest is still alive. One of + * those hypercalls could have been fatal, for example. */ if (lg->dead) break; - if (lg->halted) { + /* If the Guest asked to be stopped, we sleep. The Guest's + * clock timer or LHCALL_BREAK from the Waker will wake us. */ + if (lg->halted) { set_current_state(TASK_INTERRUPTIBLE); schedule(); continue; @@ -365,56 +500,117 @@ int run_guest(struct lguest *lg, unsigne return -EFAULT; } + /* OK, now we're ready to jump into the Guest. First we put up + * the "Do Not Disturb" sign: */ local_irq_disable(); - /* Even if *we* don't want FPU trap, guest might... */ + /* Remember the awfully-named TS bit? If the Guest has asked + * to set it we set it now, so we can trap and pass that trap + * to the Guest if it uses the FPU. */ if (lg->ts) set_ts(); + /* Now we actually run the Guest. It will pop back out when + * something interesting happens, and we can examine its + * registers to see what it was doing. */ run_guest_once(lg, lguest_pages(raw_smp_processor_id())); - /* Save cr2 now if we page-faulted. */ + /* The "regs" pointer contains two extra entries which are not + * really registers: a trap number which says what interrupt or + * trap made the switcher code come back, and an error code + * which some traps set. */ + + /* If the Guest page faulted, then the cr2 register will tell + * us the bad virtual address. We have to grab this now, + * because once we re-enable interrupts an interrupt could + * fault and thus overwrite cr2, or we could even move off to a + * different CPU. */ if (lg->regs->trapnum == 14) cr2 = read_cr2(); + /* Similarly, if we took a trap because the Guest used the FPU, + * we have to restore the FPU it expects to see. */ else if (lg->regs->trapnum == 7) math_state_restore(); + + /* Now we're ready to be interrupted or moved to other CPUs */ local_irq_enable(); + /* OK, so what happened? */ switch (lg->regs->trapnum) { case 13: /* We've intercepted a GPF. */ + /* Check if this was one of those annoying IN or OUT + * instructions which we need to emulate. If so, we + * just go back into the Guest after we've done it. */ if (lg->regs->errcode == 0) { if (emulate_insn(lg)) continue; } break; case 14: /* We've intercepted a page fault. */ + /* The Guest accessed a virtual address that wasn't + * mapped. This happens a lot: we don't actually set + * up most of the page tables for the Guest at all when + * we start: as it runs it asks for more and more, and + * we set them up as required. In this case, we don't + * even tell the Guest that the fault happened. + * + * The errcode tells whether this was a read or a + * write, and whether kernel or userspace code. */ if (demand_page(lg, cr2, lg->regs->errcode)) continue; - /* If lguest_data is NULL, this won't hurt. */ + /* OK, it's really not there (or not OK): the Guest + * needs to know. We write out the cr2 value so it + * knows where the fault occurred. + * + * Note that if the Guest were really messed up, this + * could happen before it's done the INITIALIZE + * hypercall, so lg->lguest_data will be NULL, so + * &lg->lguest_data->cr2 will be address 8. Writing + * into that address won't hurt the Host at all, + * though. */ if (put_user(cr2, &lg->lguest_data->cr2)) kill_guest(lg, "Writing cr2"); break; case 7: /* We've intercepted a Device Not Available fault. */ - /* If they don't want to know, just absorb it. */ + /* If the Guest doesn't want to know, we already + * restored the Floating Point Unit, so we just + * continue without telling it. */ if (!lg->ts) continue; break; - case 32 ... 255: /* Real interrupt, fall thru */ + case 32 ... 255: + /* These values mean a real interrupt occurred, in + * which case the Host handler has already been run. + * We just do a friendly check if another process + * should now be run, then fall through to loop + * around: */ cond_resched(); case LGUEST_TRAP_ENTRY: /* Handled at top of loop */ continue; } + /* If we get here, it's a trap the Guest wants to know + * about. */ if (deliver_trap(lg, lg->regs->trapnum)) continue; + /* If the Guest doesn't have a handler (either it hasn't + * registered any yet, or it's one of the faults we don't let + * it handle), it dies with a cryptic error message. */ kill_guest(lg, "unhandled trap %li at %#lx (%#lx)", lg->regs->trapnum, lg->regs->eip, lg->regs->trapnum == 14 ? cr2 : lg->regs->errcode); } + /* The Guest is dead => "No such file or directory" */ return -ENOENT; } + +/* Now we can look at each of the routines this calls, in increasing order of + * complexity: do_hypercalls(), emulate_insn(), maybe_do_interrupt(), + * deliver_trap() and demand_page(). After all those, we'll be ready to + * examine the Switcher, and our philosophical understanding of the Host/Guest + * duality will be complete. :*/ int find_free_guest(void) { @@ -433,55 +629,96 @@ static void adjust_pge(void *on) write_cr4(read_cr4() & ~X86_CR4_PGE); } +/*H:000 + * Welcome to the Host! + * + * By this point your brain has been tickled by the Guest code and numbed by + * the Launcher code; prepare for it to be stretched by the Host code. This is + * the heart. Let's begin at the initialization routine for the Host's lg + * module. + */ static int __init init(void) { int err; + /* Lguest can't run under Xen, VMI or itself. It does Tricky Stuff. */ if (paravirt_enabled()) { printk("lguest is afraid of %s\n", paravirt_ops.name); return -EPERM; } + /* First we put the Switcher up in very high virtual memory. */ err = map_switcher(); if (err) return err; + /* Now we set up the pagetable implementation for the Guests. */ err = init_pagetables(switcher_page, SHARED_SWITCHER_PAGES); if (err) { unmap_switcher(); return err; } + + /* The I/O subsystem needs some things initialized. */ lguest_io_init(); + /* /dev/lguest needs to be registered. */ err = lguest_device_init(); if (err) { free_pagetables(); unmap_switcher(); return err; } + + /* Finally, we need to turn off "Page Global Enable". PGE is an + * optimization where page table entries are specially marked to show + * they never change. The Host kernel marks all the kernel pages this + * way because it's always present, even when userspace is running. + * + * Lguest breaks this: unbeknownst to the rest of the Host kernel, we + * switch to the Guest kernel. If you don't disable this on all CPUs, + * you'll get really weird bugs that you'll chase for two days. + * + * I used to turn PGE off every time we switched to the Guest and back + * on when we return, but that slowed the Switcher down noticibly. */ + + /* We don't need the complexity of CPUs coming and going while we're + * doing this. */ lock_cpu_hotplug(); if (cpu_has_pge) { /* We have a broader idea of "global". */ + /* Remember that this was originally set (for cleanup). */ cpu_had_pge = 1; + /* adjust_pge is a helper function which sets or unsets the PGE + * bit on its CPU, depending on the argument (0 == unset). */ on_each_cpu(adjust_pge, (void *)0, 0, 1); + /* Turn off the feature in the global feature set. */ clear_bit(X86_FEATURE_PGE, boot_cpu_data.x86_capability); } unlock_cpu_hotplug(); + + /* All good! */ return 0; } +/* Cleaning up is just the same code, backwards. With a little French. */ static void __exit fini(void) { lguest_device_remove(); free_pagetables(); unmap_switcher(); + + /* If we had PGE before we started, turn it back on now. */ lock_cpu_hotplug(); if (cpu_had_pge) { set_bit(X86_FEATURE_PGE, boot_cpu_data.x86_capability); + /* adjust_pge's argument "1" means set PGE. */ on_each_cpu(adjust_pge, (void *)1, 0, 1); } unlock_cpu_hotplug(); } +/* The Host side of lguest can be a module. This is a nice way for people to + * play with it. */ module_init(init); module_exit(fini); MODULE_LICENSE("GPL"); =================================================================== --- a/drivers/lguest/hypercalls.c +++ b/drivers/lguest/hypercalls.c @@ -28,37 +28,63 @@ #include <irq_vectors.h> #include "lg.h" +/*H:120 This is the core hypercall routine: where the Guest gets what it + * wants. Or gets killed. Or, in the case of LHCALL_CRASH, both. + * + * Remember from the Guest: %eax == which call to make, and the arguments are + * packed into %edx, %ebx and %ecx if needed. */ static void do_hcall(struct lguest *lg, struct lguest_regs *regs) { switch (regs->eax) { case LHCALL_FLUSH_ASYNC: + /* This call does nothing, except by breaking out of the Guest + * it makes us process all the asynchronous hypercalls. */ break; case LHCALL_LGUEST_INIT: + /* You can't get here unless you're already initialized. Don't + * do that. */ kill_guest(lg, "already have lguest_data"); break; case LHCALL_CRASH: { + /* Crash is such a trivial hypercall that we do it in four + * lines right here. */ char msg[128]; + /* If the lgread fails, it will call kill_guest() itself; the + * kill_guest() with the message will be ignored. */ lgread(lg, msg, regs->edx, sizeof(msg)); msg[sizeof(msg)-1] = '\0'; kill_guest(lg, "CRASH: %s", msg); break; } case LHCALL_FLUSH_TLB: + /* FLUSH_TLB comes in two flavors, depending on the + * argument: */ if (regs->edx) guest_pagetable_clear_all(lg); else guest_pagetable_flush_user(lg); break; case LHCALL_GET_WALLCLOCK: { + /* The Guest wants to know the real time in seconds since 1970, + * in good Unix tradition. */ struct timespec ts; ktime_get_real_ts(&ts); regs->eax = ts.tv_sec; break; } case LHCALL_BIND_DMA: + /* BIND_DMA really wants four arguments, but it's the only call + * which does. So the Guest packs the number of buffers and + * the interrupt number into the final argument, and we decode + * it here. This can legitimately fail, since we currently + * place a limit on the number of DMA pools a Guest can have. + * So we return true or false from this call. */ regs->eax = bind_dma(lg, regs->edx, regs->ebx, regs->ecx >> 8, regs->ecx & 0xFF); break; + + /* All these calls simply pass the arguments through to the right + * routines. */ case LHCALL_SEND_DMA: send_dma(lg, regs->edx, regs->ebx); break; @@ -86,10 +112,13 @@ static void do_hcall(struct lguest *lg, case LHCALL_SET_CLOCKEVENT: guest_set_clockevent(lg, regs->edx); break; + case LHCALL_TS: + /* This sets the TS flag, as we saw used in run_guest(). */ lg->ts = regs->edx; break; case LHCALL_HALT: + /* Similarly, this sets the halted flag for run_guest(). */ lg->halted = 1; break; default: @@ -97,25 +126,42 @@ static void do_hcall(struct lguest *lg, } } -/* We always do queued calls before actual hypercall. */ +/* Asynchronous hypercalls are easy: we just look in the array in the Guest's + * "struct lguest_data" and see if there are any new ones marked "ready". + * + * We are careful to do these in order: obviously we respect the order the + * Guest put them in the ring, but we also promise the Guest that they will + * happen before any normal hypercall (which is why we check this before + * checking for a normal hcall). */ static void do_async_hcalls(struct lguest *lg) { unsigned int i; u8 st[LHCALL_RING_SIZE]; + /* For simplicity, we copy the entire call status array in at once. */ if (copy_from_user(&st, &lg->lguest_data->hcall_status, sizeof(st))) return; + + /* We process "struct lguest_data"s hcalls[] ring once. */ for (i = 0; i < ARRAY_SIZE(st); i++) { struct lguest_regs regs; + /* We remember where we were up to from last time. This makes + * sure that the hypercalls are done in the order the Guest + * places them in the ring. */ unsigned int n = lg->next_hcall; + /* 0xFF means there's no call here (yet). */ if (st[n] == 0xFF) break; + /* OK, we have hypercall. Increment the "next_hcall" cursor, + * and wrap back to 0 if we reach the end. */ if (++lg->next_hcall == LHCALL_RING_SIZE) lg->next_hcall = 0; + /* We copy the hypercall arguments into a fake register + * structure. This makes life simple for do_hcall(). */ if (get_user(regs.eax, &lg->lguest_data->hcalls[n].eax) || get_user(regs.edx, &lg->lguest_data->hcalls[n].edx) || get_user(regs.ecx, &lg->lguest_data->hcalls[n].ecx) @@ -124,66 +170,115 @@ static void do_async_hcalls(struct lgues break; } + /* Do the hypercall, same as a normal one. */ do_hcall(lg, ®s); + + /* Mark the hypercall done. */ if (put_user(0xFF, &lg->lguest_data->hcall_status[n])) { kill_guest(lg, "Writing result for async hypercall"); break; } + /* Stop doing hypercalls if we've just done a DMA to the + * Launcher: it needs to service this first. */ if (lg->dma_is_pending) break; } } +/* Last of all, we look at what happens first of all. The very first time the + * Guest makes a hypercall, we end up here to set things up: */ static void initialize(struct lguest *lg) { + /* You can't do anything until you're initialized. The Guest knows the + * rules, so we're unforgiving here. */ if (lg->regs->eax != LHCALL_LGUEST_INIT) { kill_guest(lg, "hypercall %li before LGUEST_INIT", lg->regs->eax); return; } + /* The pointer to the Guest's "struct lguest_data" is the only + * argument. */ lg->lguest_data = (struct lguest_data __user *)lg->regs->edx; - /* We check here so we can simply copy_to_user/from_user */ + /* If we check the address they gave is OK now, we can simply + * copy_to_user/from_user from now on rather than using lgread/lgwrite. + * I put this in to show that I'm not immune to writing stupid + * optimizations. */ if (!lguest_address_ok(lg, lg->regs->edx, sizeof(*lg->lguest_data))) { kill_guest(lg, "bad guest page %p", lg->lguest_data); return; } + /* The Guest tells us where we're not to deliver interrupts by putting + * the range of addresses into "struct lguest_data". */ if (get_user(lg->noirq_start, &lg->lguest_data->noirq_start) || get_user(lg->noirq_end, &lg->lguest_data->noirq_end) - /* We reserve the top pgd entry. */ + /* We tell the Guest that it can't use the top 4MB of virtual + * addresses used by the Switcher. */ || put_user(4U*1024*1024, &lg->lguest_data->reserve_mem) + /* We also give the Guest a unique id, as used in lguest_net.c. */ || put_user(lg->guestid, &lg->lguest_data->guestid)) kill_guest(lg, "bad guest page %p", lg->lguest_data); - /* This is the one case where the above accesses might have - * been the first write to a Guest page. This may have caused - * a copy-on-write fault, but the Guest might be referring to - * the old (read-only) page. */ + /* This is the one case where the above accesses might have been the + * first write to a Guest page. This may have caused a copy-on-write + * fault, but the Guest might be referring to the old (read-only) + * page. */ guest_pagetable_clear_all(lg); } - -/* Even if we go out to userspace and come back, we don't want to do - * the hypercall again. */ +/* Now we've examined the hypercall code; our Guest can make requests. There + * is one other way we can do things for the Guest, as we see in + * emulate_insn(). */ + +/*H:110 Tricky point: we mark the hypercall as "done" once we've done it. + * Normally we don't need to do this: the Guest will run again and update the + * trap number before we come back around the run_guest() loop to + * do_hypercalls(). + * + * However, if we are signalled or the Guest sends DMA to the Launcher, that + * loop will exit without running the Guest. When it comes back it would try + * to re-run the hypercall. */ static void clear_hcall(struct lguest *lg) { lg->regs->trapnum = 255; } +/*H:100 + * Hypercalls + * + * Remember from the Guest, hypercalls come in two flavors: normal and + * asynchronous. This file handles both of types. + */ void do_hypercalls(struct lguest *lg) { + /* Not initialized yet? */ if (unlikely(!lg->lguest_data)) { + /* Did the Guest make a hypercall? We might have come back for + * some other reason (an interrupt, a different trap). */ if (lg->regs->trapnum == LGUEST_TRAP_ENTRY) { + /* Set up the "struct lguest_data" */ initialize(lg); + /* The hypercall is done. */ clear_hcall(lg); } return; } + /* The Guest has initialized. + * + * Look in the hypercall ring for the async hypercalls: */ do_async_hcalls(lg); + + /* If we stopped reading the hypercall ring because the Guest did a + * SEND_DMA to the Launcher, we want to return now. Otherwise if the + * Guest asked us to do a hypercall, we do it. */ if (!lg->dma_is_pending && lg->regs->trapnum == LGUEST_TRAP_ENTRY) { do_hcall(lg, lg->regs); + /* The hypercall is done. */ clear_hcall(lg); } + + /* This wakes up any other Guests, in case one of the hypercalls was an + * inter-Guest SEND_DMA. */ set_wakeup_process(lg, NULL); } =================================================================== --- a/drivers/lguest/interrupts_and_traps.c +++ b/drivers/lguest/interrupts_and_traps.c @@ -14,100 +14,147 @@ #include <linux/uaccess.h> #include "lg.h" +/* The address of the interrupt handler is split into two bits: */ static unsigned long idt_address(u32 lo, u32 hi) { return (lo & 0x0000FFFF) | (hi & 0xFFFF0000); } +/* The "type" of the interrupt handler is a 4 bit field: we only support a + * couple of types. */ static int idt_type(u32 lo, u32 hi) { return (hi >> 8) & 0xF; } +/* An IDT entry can't be used unless the "present" bit is set. */ static int idt_present(u32 lo, u32 hi) { return (hi & 0x8000); } +/* We need a helper to "push" a value onto the Guest's stack, since that's a + * big part of what delivering an interrupt does. */ static void push_guest_stack(struct lguest *lg, unsigned long *gstack, u32 val) { + /* Stack grows upwards: move stack then write value. */ *gstack -= 4; lgwrite_u32(lg, *gstack, val); } +/*H:210 The set_guest_interrupt() routine actually delivers the interrupt or + * trap. The mechanics of delivering traps and interrupts to the Guest are the + * same, except some traps have an "error code" which gets pushed onto the + * stack as well: the caller tells us if this is one. + * + * "lo" and "hi" are the two parts of the Interrupt Descriptor Table for this + * interrupt or trap. It's split into two parts for traditional reasons: gcc + * on i386 used to be frightened by 64 bit numbers. + * + * We set up the stack just like the CPU does for a real interrupt, so it's + * identical for the Guest (and the standard "iret" instruction will undo + * it). */ static void set_guest_interrupt(struct lguest *lg, u32 lo, u32 hi, int has_err) { unsigned long gstack; u32 eflags, ss, irq_enable; - /* If they want a ring change, we use new stack and push old ss/esp */ + /* There are two cases for interrupts: one where the Guest is already + * in the kernel, and a more complex one where the Guest is in + * userspace. We check the privilege level to find out. */ if ((lg->regs->ss&0x3) != GUEST_PL) { + /* The Guest told us their kernel stack with the SET_STACK + * hypercall: both the virtual address and the segment */ gstack = guest_pa(lg, lg->esp1); ss = lg->ss1; + /* We push the old stack segment and pointer onto the new + * stack: when the Guest does an "iret" back from the interrupt + * handler the CPU will notice they're dropping privilege + * levels and expect these here. */ push_guest_stack(lg, &gstack, lg->regs->ss); push_guest_stack(lg, &gstack, lg->regs->esp); } else { + /* We're staying on the same Guest (kernel) stack. */ gstack = guest_pa(lg, lg->regs->esp); ss = lg->regs->ss; } - /* We use IF bit in eflags to indicate whether irqs were disabled - (it's always 0, since irqs are enabled when guest is running). */ + /* Remember that we never let the Guest actually disable interrupts, so + * the "Interrupt Flag" bit is always zero. We copy that bit from the + * Guest's "irq_enabled" field into the eflags word: the Guest copies + * it back in "lguest_iret". */ eflags = lg->regs->eflags; if (get_user(irq_enable, &lg->lguest_data->irq_enabled)) irq_enable = 0; eflags |= (irq_enable & X86_EFLAGS_IF); + /* An interrupt is expected to push three things on the stack: the old + * "eflags" word, the old code segment, and the old instruction + * pointer. */ push_guest_stack(lg, &gstack, eflags); push_guest_stack(lg, &gstack, lg->regs->cs); push_guest_stack(lg, &gstack, lg->regs->eip); + /* For the six traps which supply an error code, we push that, too. */ if (has_err) push_guest_stack(lg, &gstack, lg->regs->errcode); - /* Change the real stack so switcher returns to trap handler */ + /* Now we've pushed all the old state, we change the stack, the code + * segment and the address to execute. */ lg->regs->ss = ss; lg->regs->esp = gstack + lg->page_offset; lg->regs->cs = (__KERNEL_CS|GUEST_PL); lg->regs->eip = idt_address(lo, hi); - /* Disable interrupts for an interrupt gate. */ + /* There are two kinds of interrupt handlers: 0xE is an "interrupt + * gate" which expects interrupts to be disabled on entry. */ if (idt_type(lo, hi) == 0xE) if (put_user(0, &lg->lguest_data->irq_enabled)) kill_guest(lg, "Disabling interrupts"); } +/*H:200 + * Virtual Interrupts. + * + * maybe_do_interrupt() gets called before every entry to the Guest, to see if + * we should divert the Guest to running an interrupt handler. */ void maybe_do_interrupt(struct lguest *lg) { unsigned int irq; DECLARE_BITMAP(blk, LGUEST_IRQS); struct desc_struct *idt; + /* If the Guest hasn't even initialized yet, we can do nothing. */ if (!lg->lguest_data) return; - /* Mask out any interrupts they have blocked. */ + /* Take our "irqs_pending" array and remove any interrupts the Guest + * wants blocked: the result ends up in "blk". */ if (copy_from_user(&blk, lg->lguest_data->blocked_interrupts, sizeof(blk))) return; bitmap_andnot(blk, lg->irqs_pending, blk, LGUEST_IRQS); + /* Find the first interrupt. */ irq = find_first_bit(blk, LGUEST_IRQS); + /* None? Nothing to do */ if (irq >= LGUEST_IRQS) return; + /* They may be in the middle of an iret, where they asked us never to + * deliver interrupts. */ if (lg->regs->eip >= lg->noirq_start && lg->regs->eip < lg->noirq_end) return; - /* If they're halted, we re-enable interrupts. */ + /* If they're halted, interrupts restart them. */ if (lg->halted) { /* Re-enable interrupts. */ if (put_user(X86_EFLAGS_IF, &lg->lguest_data->irq_enabled)) kill_guest(lg, "Re-enabling interrupts"); lg->halted = 0; } else { - /* Maybe they have interrupts disabled? */ + /* Otherwise we check if they have interrupts disabled. */ u32 irq_enabled; if (get_user(irq_enabled, &lg->lguest_data->irq_enabled)) irq_enabled = 0; @@ -115,112 +162,197 @@ void maybe_do_interrupt(struct lguest *l return; } + /* Look at the IDT entry the Guest gave us for this interrupt. The + * first 32 (FIRST_EXTERNAL_VECTOR) entries are for traps, so we skip + * over them. */ idt = &lg->idt[FIRST_EXTERNAL_VECTOR+irq]; + /* If they don't have a handler (yet?), we just ignore it */ if (idt_present(idt->a, idt->b)) { + /* OK, mark it no longer pending and deliver it. */ clear_bit(irq, lg->irqs_pending); + /* set_guest_interrupt() takes the interrupt descriptor and a + * flag to say whether this interrupt pushes an error code onto + * the stack as well: virtual interrupts never do. */ set_guest_interrupt(lg, idt->a, idt->b, 0); } } +/*H:220 Now we've got the routines to deliver interrupts, delivering traps + * like page fault is easy. The only trick is that Intel decided that some + * traps should have error codes: */ static int has_err(unsigned int trap) { return (trap == 8 || (trap >= 10 && trap <= 14) || trap == 17); } +/* deliver_trap() returns true if it could deliver the trap. */ int deliver_trap(struct lguest *lg, unsigned int num) { u32 lo = lg->idt[num].a, hi = lg->idt[num].b; + /* Early on the Guest hasn't set the IDT entries (or maybe it put a + * bogus one in): if we fail here, the Guest will be killed. */ if (!idt_present(lo, hi)) return 0; set_guest_interrupt(lg, lo, hi, has_err(num)); return 1; } +/*H:250 Here's the hard part: returning to the Host every time a trap happens + * and then calling deliver_trap() and re-entering the Guest is slow. + * Particularly because Guest userspace system calls are traps (trap 128). + * + * So we'd like to set up the IDT to tell the CPU to deliver traps directly + * into the Guest. This is possible, but the complexities cause the size of + * this file to double! However, 150 lines of code is worth writing for taking + * system calls down from 1750ns to 270ns. Plus, if lguest didn't do it, all + * the other hypervisors would tease it. + * + * This routine determines if a trap can be delivered directly. */ static int direct_trap(const struct lguest *lg, const struct desc_struct *trap, unsigned int num) { - /* Hardware interrupts don't go to guest (except syscall). */ + /* Hardware interrupts don't go to the Guest at all (except system + * call). */ if (num >= FIRST_EXTERNAL_VECTOR && num != SYSCALL_VECTOR) return 0; - /* We intercept page fault (demand shadow paging & cr2 saving) - protection fault (in/out emulation) and device not - available (TS handling), and hypercall */ + /* The Host needs to see page faults (for shadow paging and to save the + * fault address), general protection faults (in/out emulation) and + * device not available (TS handling), and of course, the hypercall + * trap. */ if (num == 14 || num == 13 || num == 7 || num == LGUEST_TRAP_ENTRY) return 0; - /* Interrupt gates (0xE) or not present (0x0) can't go direct. */ + /* Only trap gates (type 15) can go direct to the Guest. Interrupt + * gates (type 14) disable interrupts as they are entered, which we + * never let the Guest do. Not present entries (type 0x0) also can't + * go direct, of course 8) */ return idt_type(trap->a, trap->b) == 0xF; } +/*H:260 When we make traps go directly into the Guest, we need to make sure + * the kernel stack is valid (ie. mapped in the page tables). Otherwise, the + * CPU trying to deliver the trap will fault while trying to push the interrupt + * words on the stack: this is called a double fault, and it forces us to kill + * the Guest. + * + * Which is deeply unfair, because (literally!) it wasn't the Guests' fault. */ void pin_stack_pages(struct lguest *lg) { unsigned int i; + /* Depending on the CONFIG_4KSTACKS option, the Guest can have one or + * two pages of stack space. */ for (i = 0; i < lg->stack_pages; i++) + /* The stack grows *upwards*, hence the subtraction */ pin_page(lg, lg->esp1 - i * PAGE_SIZE); } +/* Direct traps also mean that we need to know whenever the Guest wants to use + * a different kernel stack, so we can change the IDT entries to use that + * stack. The IDT entries expect a virtual address, so unlike most addresses + * the Guest gives us, the "esp" (stack pointer) value here is virtual, not + * physical. + * + * In Linux each process has its own kernel stack, so this happens a lot: we + * change stacks on each context switch. */ void guest_set_stack(struct lguest *lg, u32 seg, u32 esp, unsigned int pages) { - /* You cannot have a stack segment with priv level 0. */ + /* You are not allowd have a stack segment with privilege level 0: bad + * Guest! */ if ((seg & 0x3) != GUEST_PL) kill_guest(lg, "bad stack segment %i", seg); + /* We only expect one or two stack pages. */ if (pages > 2) kill_guest(lg, "bad stack pages %u", pages); + /* Save where the stack is, and how many pages */ lg->ss1 = seg; lg->esp1 = esp; lg->stack_pages = pages; + /* Make sure the new stack pages are mapped */ pin_stack_pages(lg); } -/* Set up trap in IDT. */ +/* All this reference to mapping stacks leads us neatly into the other complex + * part of the Host: page table handling. */ + +/*H:235 This is the routine which actually checks the Guest's IDT entry and + * transfers it into our entry in "struct lguest": */ static void set_trap(struct lguest *lg, struct desc_struct *trap, unsigned int num, u32 lo, u32 hi) { u8 type = idt_type(lo, hi); + /* We zero-out a not-present entry */ if (!idt_present(lo, hi)) { trap->a = trap->b = 0; return; } + /* We only support interrupt and trap gates. */ if (type != 0xE && type != 0xF) kill_guest(lg, "bad IDT type %i", type); + /* We only copy the handler address, present bit, privilege level and + * type. The privilege level controls where the trap can be triggered + * manually with an "int" instruction. This is usually GUEST_PL, + * except for system calls which userspace can use. */ trap->a = ((__KERNEL_CS|GUEST_PL)<<16) | (lo&0x0000FFFF); trap->b = (hi&0xFFFFEF00); } +/*H:230 While we're here, dealing with delivering traps and interrupts to the + * Guest, we might as well complete the picture: how the Guest tells us where + * it wants them to go. This would be simple, except making traps fast + * requires some tricks. + * + * We saw the Guest setting Interrupt Descriptor Table (IDT) entries with the + * LHCALL_LOAD_IDT_ENTRY hypercall before: that comes here. */ void load_guest_idt_entry(struct lguest *lg, unsigned int num, u32 lo, u32 hi) { - /* Guest never handles: NMI, doublefault, hypercall, spurious irq. */ + /* Guest never handles: NMI, doublefault, spurious interrupt or + * hypercall. We ignore when it tries to set them. */ if (num == 2 || num == 8 || num == 15 || num == LGUEST_TRAP_ENTRY) return; + /* Mark the IDT as changed: next time the Guest runs we'll know we have + * to copy this again. */ lg->changed |= CHANGED_IDT; + + /* The IDT which we keep in "struct lguest" only contains 32 entries + * for the traps and LGUEST_IRQS (32) entries for interrupts. We + * ignore attempts to set handlers for higher interrupt numbers, except + * for the system call "interrupt" at 128: we have a special IDT entry + * for that. */ if (num < ARRAY_SIZE(lg->idt)) set_trap(lg, &lg->idt[num], num, lo, hi); else if (num == SYSCALL_VECTOR) set_trap(lg, &lg->syscall_idt, num, lo, hi); } +/* The default entry for each interrupt points into the Switcher routines which + * simply return to the Host. The run_guest() loop will then call + * deliver_trap() to bounce it back into the Guest. */ static void default_idt_entry(struct desc_struct *idt, int trap, const unsigned long handler) { + /* A present interrupt gate. */ u32 flags = 0x8e00; - /* They can't "int" into any of them except hypercall. */ + /* Set the privilege level on the entry for the hypercall: this allows + * the Guest to use the "int" instruction to trigger it. */ if (trap == LGUEST_TRAP_ENTRY) flags |= (GUEST_PL << 13); + /* Now pack it into the IDT entry in its weird format. */ idt->a = (LGUEST_CS<<16) | (handler&0x0000FFFF); idt->b = (handler&0xFFFF0000) | flags; } +/* When the Guest first starts, we put default entries into the IDT. */ void setup_default_idt_entries(struct lguest_ro_state *state, const unsigned long *def) { @@ -230,19 +362,25 @@ void setup_default_idt_entries(struct lg default_idt_entry(&state->guest_idt[i], i, def[i]); } +/*H:240 We don't use the IDT entries in the "struct lguest" directly, instead + * we copy them into the IDT which we've set up for Guests on this CPU, just + * before we run the Guest. This routine does that copy. */ void copy_traps(const struct lguest *lg, struct desc_struct *idt, const unsigned long *def) { unsigned int i; - /* All hardware interrupts are same whatever the guest: only the - * traps might be different. */ + /* We can simply copy the direct traps, otherwise we use the default + * ones in the Switcher: they will return to the Host. */ for (i = 0; i < FIRST_EXTERNAL_VECTOR; i++) { if (direct_trap(lg, &lg->idt[i], i)) idt[i] = lg->idt[i]; else default_idt_entry(&idt[i], i, def[i]); } + + /* Don't forget the system call trap! The IDT entries for other + * interupts never change, so no need to copy them. */ i = SYSCALL_VECTOR; if (direct_trap(lg, &lg->syscall_idt, i)) idt[i] = lg->syscall_idt; =================================================================== --- a/drivers/lguest/lg.h +++ b/drivers/lguest/lg.h @@ -58,9 +58,18 @@ struct lguest_dma_info u8 interrupt; /* 0 when not registered */ }; -/* We have separate types for the guest's ptes & pgds and the shadow ptes & - * pgds. Since this host might use three-level pagetables and the guest and - * shadow pagetables don't, we can't use the normal pte_t/pgd_t. */ +/*H:310 The page-table code owes a great debt of gratitude to Andi Kleen. He + * reviewed the original code which used "u32" for all page table entries, and + * insisted that it would be far clearer with explicit typing. I thought it + * was overkill, but he was right: it is much clearer than it was before. + * + * We have separate types for the Guest's ptes & pgds and the shadow ptes & + * pgds. There's already a Linux type for these (pte_t and pgd_t) but they + * change depending on kernel config options (PAE). */ + +/* Each entry is identical: lower 12 bits of flags and upper 20 bits for the + * "page frame number" (0 == first physical page, etc). They are different + * types so the compiler will warn us if we mix them improperly. */ typedef union { struct { unsigned flags:12, pfn:20; }; struct { unsigned long val; } raw; @@ -77,8 +86,12 @@ typedef union { struct { unsigned flags:12, pfn:20; }; struct { unsigned long val; } raw; } gpte_t; + +/* We have two convenient macros to convert a "raw" value as handed to us by + * the Guest into the correct Guest PGD or PTE type. */ #define mkgpte(_val) ((gpte_t){.raw.val = _val}) #define mkgpgd(_val) ((gpgd_t){.raw.val = _val}) +/*:*/ struct pgdir { =================================================================== --- a/drivers/lguest/page_tables.c +++ b/drivers/lguest/page_tables.c @@ -15,38 +15,91 @@ #include <asm/tlbflush.h> #include "lg.h" +/*H:300 + * The Page Table Code + * + * We use two-level page tables for the Guest. If you're not entirely + * comfortable with virtual addresses, physical addresses and page tables then + * I recommend you review lguest.c's "Page Table Handling" (with diagrams!). + * + * The Guest keeps page tables, but we maintain the actual ones here: these are + * called "shadow" page tables. Which is a very Guest-centric name: these are + * the real page tables the CPU uses, although we keep them up to date to + * reflect the Guest's. (See what I mean about weird naming? Since when do + * shadows reflect anything?) + * + * Anyway, this is the most complicated part of the Host code. There are seven + * parts to this: + * (i) Setting up a page table entry for the Guest when it faults, + * (ii) Setting up the page table entry for the Guest stack, + * (iii) Setting up a page table entry when the Guest tells us it has changed, + * (iv) Switching page tables, + * (v) Flushing (thowing away) page tables, + * (vi) Mapping the Switcher when the Guest is about to run, + * (vii) Setting up the page tables initially. + :*/ + +/* Pages a 4k long, and each page table entry is 4 bytes long, giving us 1024 + * (or 2^10) entries per page. */ #define PTES_PER_PAGE_SHIFT 10 #define PTES_PER_PAGE (1 << PTES_PER_PAGE_SHIFT) + +/* 1024 entries in a page table page maps 1024 pages: 4MB. The Switcher is + * conveniently placed at the top 4MB, so it uses a separate, complete PTE + * page. */ #define SWITCHER_PGD_INDEX (PTES_PER_PAGE - 1) +/* We actually need a separate PTE page for each CPU. Remember that after the + * Switcher code itself comes two pages for each CPU, and we don't want this + * CPU's guest to see the pages of any other CPU. */ static DEFINE_PER_CPU(spte_t *, switcher_pte_pages); #define switcher_pte_page(cpu) per_cpu(switcher_pte_pages, cpu) +/*H:320 With our shadow and Guest types established, we need to deal with + * them: the page table code is curly enough to need helper functions to keep + * it clear and clean. + * + * The first helper takes a virtual address, and says which entry in the top + * level page table deals with that address. Since each top level entry deals + * with 4M, this effectively divides by 4M. */ static unsigned vaddr_to_pgd_index(unsigned long vaddr) { return vaddr >> (PAGE_SHIFT + PTES_PER_PAGE_SHIFT); } -/* These access the shadow versions (ie. the ones used by the CPU). */ +/* There are two functions which return pointers to the shadow (aka "real") + * page tables. + * + * spgd_addr() takes the virtual address and returns a pointer to the top-level + * page directory entry for that address. Since we keep track of several page + * tables, the "i" argument tells us which one we're interested in (it's + * usually the current one). */ static spgd_t *spgd_addr(struct lguest *lg, u32 i, unsigned long vaddr) { unsigned int index = vaddr_to_pgd_index(vaddr); + /* We kill any Guest trying to touch the Switcher addresses. */ if (index >= SWITCHER_PGD_INDEX) { kill_guest(lg, "attempt to access switcher pages"); index = 0; } + /* Return a pointer index'th pgd entry for the i'th page table. */ return &lg->pgdirs[i].pgdir[index]; } +/* This routine then takes the PGD entry given above, which contains the + * address of the PTE page. It then returns a pointer to the PTE entry for the + * given address. */ static spte_t *spte_addr(struct lguest *lg, spgd_t spgd, unsigned long vaddr) { spte_t *page = __va(spgd.pfn << PAGE_SHIFT); + /* You should never call this if the PGD entry wasn't valid */ BUG_ON(!(spgd.flags & _PAGE_PRESENT)); return &page[(vaddr >> PAGE_SHIFT) % PTES_PER_PAGE]; } -/* These access the guest versions. */ +/* These two functions just like the above two, except they access the Guest + * page tables. Hence they return a Guest address. */ static unsigned long gpgd_addr(struct lguest *lg, unsigned long vaddr) { unsigned int index = vaddr >> (PAGE_SHIFT + PTES_PER_PAGE_SHIFT); @@ -61,12 +114,24 @@ static unsigned long gpte_addr(struct lg return gpage + ((vaddr>>PAGE_SHIFT) % PTES_PER_PAGE) * sizeof(gpte_t); } -/* Do a virtual -> physical mapping on a user page. */ +/*H:350 This routine takes a page number given by the Guest and converts it to + * an actual, physical page number. It can fail for several reasons: the + * virtual address might not be mapped by the Launcher, the write flag is set + * and the page is read-only, or the write flag was set and the page was + * shared so had to be copied, but we ran out of memory. + * + * This holds a reference to the page, so release_pte() is careful to + * put that back. */ static unsigned long get_pfn(unsigned long virtpfn, int write) { struct page *page; + /* This value indicates failure. */ unsigned long ret = -1UL; + /* get_user_pages() is a complex interface: it gets the "struct + * vm_area_struct" and "struct page" assocated with a range of pages. + * It also needs the task's mmap_sem held, and is not very quick. + * It returns the number of pages it got. */ down_read(¤t->mm->mmap_sem); if (get_user_pages(current, current->mm, virtpfn << PAGE_SHIFT, 1, write, 1, &page, NULL) == 1) @@ -75,28 +140,47 @@ static unsigned long get_pfn(unsigned lo return ret; } +/*H:340 Converting a Guest page table entry to a shadow (ie. real) page table + * entry can be a little tricky. The flags are (almost) the same, but the + * Guest PTE contains a virtual page number: the CPU needs the real page + * number. */ static spte_t gpte_to_spte(struct lguest *lg, gpte_t gpte, int write) { spte_t spte; unsigned long pfn; - /* We ignore the global flag. */ + /* The Guest sets the global flag, because it thinks that it is using + * PGE. We only told it to use PGE so it would tell us whether it was + * flushing a kernel mapping or a userspace mapping. We don't actually + * use the global bit, so throw it away. */ spte.flags = (gpte.flags & ~_PAGE_GLOBAL); + + /* We need a temporary "unsigned long" variable to hold the answer from + * get_pfn(), because it returns 0xFFFFFFFF on failure, which wouldn't + * fit in spte.pfn. get_pfn() finds the real physical number of the + * page, given the virtual number. */ pfn = get_pfn(gpte.pfn, write); if (pfn == -1UL) { kill_guest(lg, "failed to get page %u", gpte.pfn); - /* Must not put_page() bogus page on cleanup. */ + /* When we destroy the Guest, we'll go through the shadow page + * tables and release_pte() them. Make sure we don't think + * this one is valid! */ spte.flags = 0; } + /* Now we assign the page number, and our shadow PTE is complete. */ spte.pfn = pfn; return spte; } +/*H:460 And to complete the chain, release_pte() looks like this: */ static void release_pte(spte_t pte) { + /* Remember that get_user_pages() took a reference to the page, in + * get_pfn()? We have to put it back now. */ if (pte.flags & _PAGE_PRESENT) put_page(pfn_to_page(pte.pfn)); } +/*:*/ static void check_gpte(struct lguest *lg, gpte_t gpte) { @@ -110,11 +194,16 @@ static void check_gpgd(struct lguest *lg kill_guest(lg, "bad page directory entry"); } -/* FIXME: We hold reference to pages, which prevents them from being - swapped. It'd be nice to have a callback when Linux wants to swap out. */ - -/* We fault pages in, which allows us to update accessed/dirty bits. - * Return true if we got page. */ +/*H:330 + * (i) Setting up a page table entry for the Guest when it faults + * + * We saw this call in run_guest(): when we see a page fault in the Guest, we + * come here. That's because we only set up the shadow page tables lazily as + * they're needed, so we get page faults all the time and quietly fix them up + * and return to the Guest without it knowing. + * + * If we fixed up the fault (ie. we mapped the address), this routine returns + * true. */ int demand_page(struct lguest *lg, unsigned long vaddr, int errcode) { gpgd_t gpgd; @@ -123,106 +212,161 @@ int demand_page(struct lguest *lg, unsig gpte_t gpte; spte_t *spte; + /* First step: get the top-level Guest page table entry. */ gpgd = mkgpgd(lgread_u32(lg, gpgd_addr(lg, vaddr))); + /* Toplevel not present? We can't map it in. */ if (!(gpgd.flags & _PAGE_PRESENT)) return 0; + /* Now look at the matching shadow entry. */ spgd = spgd_addr(lg, lg->pgdidx, vaddr); if (!(spgd->flags & _PAGE_PRESENT)) { - /* Get a page of PTEs for them. */ + /* No shadow entry: allocate a new shadow PTE page. */ unsigned long ptepage = get_zeroed_page(GFP_KERNEL); - /* FIXME: Steal from self in this case? */ + /* This is not really the Guest's fault, but killing it is + * simple for this corner case. */ if (!ptepage) { kill_guest(lg, "out of memory allocating pte page"); return 0; } + /* We check that the Guest pgd is OK. */ check_gpgd(lg, gpgd); + /* And we copy the flags to the shadow PGD entry. The page + * number in the shadow PGD is the page we just allocated. */ spgd->raw.val = (__pa(ptepage) | gpgd.flags); } + /* OK, now we look at the lower level in the Guest page table: keep its + * address, because we might update it later. */ gpte_ptr = gpte_addr(lg, gpgd, vaddr); gpte = mkgpte(lgread_u32(lg, gpte_ptr)); - /* No page? */ + /* If this page isn't in the Guest page tables, we can't page it in. */ if (!(gpte.flags & _PAGE_PRESENT)) return 0; - /* Write to read-only page? */ + /* Check they're not trying to write to a page the Guest wants + * read-only (bit 2 of errcode == write). */ if ((errcode & 2) && !(gpte.flags & _PAGE_RW)) return 0; - /* User access to a non-user page? */ + /* User access to a kernel page? (bit 3 == user access) */ if ((errcode & 4) && !(gpte.flags & _PAGE_USER)) return 0; + /* Check that the Guest PTE flags are OK, and the page number is below + * the pfn_limit (ie. not mapping the Launcher binary). */ check_gpte(lg, gpte); + /* Add the _PAGE_ACCESSED and (for a write) _PAGE_DIRTY flag */ gpte.flags |= _PAGE_ACCESSED; if (errcode & 2) gpte.flags |= _PAGE_DIRTY; - /* We're done with the old pte. */ + /* Get the pointer to the shadow PTE entry we're going to set. */ spte = spte_addr(lg, *spgd, vaddr); + /* If there was a valid shadow PTE entry here before, we release it. + * This can happen with a write to a previously read-only entry. */ release_pte(*spte); - /* We don't make it writable if this isn't a write: later - * write will fault so we can set dirty bit in guest. */ + /* If this is a write, we insist that the Guest page is writable (the + * final arg to gpte_to_spte()). */ if (gpte.flags & _PAGE_DIRTY) *spte = gpte_to_spte(lg, gpte, 1); else { + /* If this is a read, don't set the "writable" bit in the page + * table entry, even if the Guest says it's writable. That way + * we come back here when a write does actually ocur, so we can + * update the Guest's _PAGE_DIRTY flag. */ gpte_t ro_gpte = gpte; ro_gpte.flags &= ~_PAGE_RW; *spte = gpte_to_spte(lg, ro_gpte, 0); } - /* Now we update dirty/accessed on guest. */ + /* Finally, we write the Guest PTE entry back: we've set the + * _PAGE_ACCESSED and maybe the _PAGE_DIRTY flags. */ lgwrite_u32(lg, gpte_ptr, gpte.raw.val); + + /* We succeeded in mapping the page! */ return 1; } -/* This is much faster than the full demand_page logic. */ +/*H:360 (ii) Setting up the page table entry for the Guest stack. + * + * Remember pin_stack_pages() which makes sure the stack is mapped? It could + * simply call demand_page(), but as we've seen that logic is quite long, and + * usually the stack pages are already mapped anyway, so it's not required. + * + * This is a quick version which answers the question: is this virtual address + * mapped by the shadow page tables, and is it writable? */ static int page_writable(struct lguest *lg, unsigned long vaddr) { spgd_t *spgd; unsigned long flags; + /* Look at the top level entry: is it present? */ spgd = spgd_addr(lg, lg->pgdidx, vaddr); if (!(spgd->flags & _PAGE_PRESENT)) return 0; + /* Check the flags on the pte entry itself: it must be present and + * writable. */ flags = spte_addr(lg, *spgd, vaddr)->flags; return (flags & (_PAGE_PRESENT|_PAGE_RW)) == (_PAGE_PRESENT|_PAGE_RW); } +/* So, when pin_stack_pages() asks us to pin a page, we check if it's already + * in the page tables, and if not, we call demand_page() with error code 2 + * (meaning "write"). */ void pin_page(struct lguest *lg, unsigned long vaddr) { if (!page_writable(lg, vaddr) && !demand_page(lg, vaddr, 2)) kill_guest(lg, "bad stack page %#lx", vaddr); } +/*H:450 If we chase down the release_pgd() code, it looks like this: */ static void release_pgd(struct lguest *lg, spgd_t *spgd) { + /* If the entry's not present, there's nothing to release. */ if (spgd->flags & _PAGE_PRESENT) { unsigned int i; + /* Converting the pfn to find the actual PTE page is easy: turn + * the page number into a physical address, then convert to a + * virtual address (easy for kernel pages like this one). */ spte_t *ptepage = __va(spgd->pfn << PAGE_SHIFT); + /* For each entry in the page, we might need to release it. */ for (i = 0; i < PTES_PER_PAGE; i++) release_pte(ptepage[i]); + /* Now we can free the page of PTEs */ free_page((long)ptepage); + /* And zero out the PGD entry we we never release it twice. */ spgd->raw.val = 0; } } +/*H:440 (v) Flushing (thowing away) page tables, + * + * We saw flush_user_mappings() called when we re-used a top-level pgdir page. + * It simply releases every PTE page from 0 up to the kernel address. */ static void flush_user_mappings(struct lguest *lg, int idx) { unsigned int i; + /* Release every pgd entry up to the kernel's address. */ for (i = 0; i < vaddr_to_pgd_index(lg->page_offset); i++) release_pgd(lg, lg->pgdirs[idx].pgdir + i); } +/* The Guest also has a hypercall to do this manually: it's used when a large + * number of mappings have been changed. */ void guest_pagetable_flush_user(struct lguest *lg) { + /* Drop the userspace part of the current page table. */ flush_user_mappings(lg, lg->pgdidx); } - +/*:*/ + +/* We keep several page tables. This is a simple routine to find the page + * table (if any) corresponding to this top-level address the Guest has given + * us. */ static unsigned int find_pgdir(struct lguest *lg, unsigned long pgtable) { unsigned int i; @@ -232,21 +376,30 @@ static unsigned int find_pgdir(struct lg return i; } +/*H:435 And this is us, creating the new page directory. If we really do + * allocate a new one (and so the kernel parts are not there), we set + * blank_pgdir. */ static unsigned int new_pgdir(struct lguest *lg, unsigned long cr3, int *blank_pgdir) { unsigned int next; + /* We pick one entry at random to throw out. Choosing the Least + * Recently Used might be better, but this is easy. */ next = random32() % ARRAY_SIZE(lg->pgdirs); + /* If it's never been allocated at all before, try now. */ if (!lg->pgdirs[next].pgdir) { lg->pgdirs[next].pgdir = (spgd_t *)get_zeroed_page(GFP_KERNEL); + /* If the allocation fails, just keep using the one we have */ if (!lg->pgdirs[next].pgdir) next = lg->pgdidx; else - /* There are no mappings: you'll need to re-pin */ + /* This is a blank page, so there are no kernel + * mappings: caller must map the stack! */ *blank_pgdir = 1; } + /* Record which Guest toplevel this shadows. */ lg->pgdirs[next].cr3 = cr3; /* Release all the non-kernel mappings. */ flush_user_mappings(lg, next); @@ -254,82 +407,161 @@ static unsigned int new_pgdir(struct lgu return next; } +/*H:430 (iv) Switching page tables + * + * This is what happens when the Guest changes page tables (ie. changes the + * top-level pgdir). This happens on almost every context switch. */ void guest_new_pagetable(struct lguest *lg, unsigned long pgtable) { int newpgdir, repin = 0; + /* Look to see if we have this one already. */ newpgdir = find_pgdir(lg, pgtable); + /* If not, we allocate or mug an existing one: if it's a fresh one, + * repin gets set to 1. */ if (newpgdir == ARRAY_SIZE(lg->pgdirs)) newpgdir = new_pgdir(lg, pgtable, &repin); + /* Change the current pgd index to the new one. */ lg->pgdidx = newpgdir; + /* If it was completely blank, we map in the Guest kernel stack */ if (repin) pin_stack_pages(lg); } +/*H:470 Finally, a routine which throws away everything: all PGD entries in all + * the shadow page tables. This is used when we destroy the Guest. */ static void release_all_pagetables(struct lguest *lg) { unsigned int i, j; + /* Every shadow pagetable this Guest has */ for (i = 0; i < ARRAY_SIZE(lg->pgdirs); i++) if (lg->pgdirs[i].pgdir) + /* Every PGD entry except the Switcher at the top */ for (j = 0; j < SWITCHER_PGD_INDEX; j++) release_pgd(lg, lg->pgdirs[i].pgdir + j); } +/* We also throw away everything when a Guest tells us it's changed a kernel + * mapping. Since kernel mappings are in every page table, it's easiest to + * throw them all away. This is amazingly slow, but thankfully rare. */ void guest_pagetable_clear_all(struct lguest *lg) { release_all_pagetables(lg); + /* We need the Guest kernel stack mapped again. */ pin_stack_pages(lg); } +/*H:420 This is the routine which actually sets the page table entry for then + * "idx"'th shadow page table. + * + * Normally, we can just throw out the old entry and replace it with 0: if they + * use it demand_page() will put the new entry in. We need to do this anyway: + * The Guest expects _PAGE_ACCESSED to be set on its PTE the first time a page + * is read from, and _PAGE_DIRTY when it's written to. + * + * But Avi Kivity pointed out that most Operating Systems (Linux included) set + * these bits on PTEs immediately anyway. This is done to save the CPU from + * having to update them, but it helps us the same way: if they set + * _PAGE_ACCESSED then we can put a read-only PTE entry in immediately, and if + * they set _PAGE_DIRTY then we can put a writable PTE entry in immediately. + */ static void do_set_pte(struct lguest *lg, int idx, unsigned long vaddr, gpte_t gpte) { + /* Look up the matching shadow page directot entry. */ spgd_t *spgd = spgd_addr(lg, idx, vaddr); + + /* If the top level isn't present, there's no entry to update. */ if (spgd->flags & _PAGE_PRESENT) { + /* Otherwise, we start by releasing the existing entry. */ spte_t *spte = spte_addr(lg, *spgd, vaddr); release_pte(*spte); + + /* If they're setting this entry as dirty or accessed, we might + * as well put that entry they've given us in now. This shaves + * 10% off a copy-on-write micro-benchmark. */ if (gpte.flags & (_PAGE_DIRTY | _PAGE_ACCESSED)) { check_gpte(lg, gpte); *spte = gpte_to_spte(lg, gpte, gpte.flags&_PAGE_DIRTY); } else + /* Otherwise we can demand_page() it in later. */ spte->raw.val = 0; } } +/*H:410 Updating a PTE entry is a little trickier. + * + * We keep track of several different page tables (the Guest uses one for each + * process, so it makes sense to cache at least a few). Each of these have + * identical kernel parts: ie. every mapping above PAGE_OFFSET is the same for + * all processes. So when the page table above that address changes, we update + * all the page tables, not just the current one. This is rare. + * + * The benefit is that when we have to track a new page table, we can copy keep + * all the kernel mappings. This speeds up context switch immensely. */ void guest_set_pte(struct lguest *lg, unsigned long cr3, unsigned long vaddr, gpte_t gpte) { - /* Kernel mappings must be changed on all top levels. */ + /* Kernel mappings must be changed on all top levels. Slow, but + * doesn't happen often. */ if (vaddr >= lg->page_offset) { unsigned int i; for (i = 0; i < ARRAY_SIZE(lg->pgdirs); i++) if (lg->pgdirs[i].pgdir) do_set_pte(lg, i, vaddr, gpte); } else { + /* Is this page table one we have a shadow for? */ int pgdir = find_pgdir(lg, cr3); if (pgdir != ARRAY_SIZE(lg->pgdirs)) + /* If so, do the update. */ do_set_pte(lg, pgdir, vaddr, gpte); } } +/*H:400 + * (iii) Setting up a page table entry when the Guest tells us it has changed. + * + * Just like we did in interrupts_and_traps.c, it makes sense for us to deal + * with the other side of page tables while we're here: what happens when the + * Guest asks for a page table to be updated? + * + * We already saw that demand_page() will fill in the shadow page tables when + * needed, so we can simply remove shadow page table entries whenever the Guest + * tells us they've changed. When the Guest tries to use the new entry it will + * fault and demand_page() will fix it up. + * + * So with that in mind here's our code to to update a (top-level) PGD entry: + */ void guest_set_pmd(struct lguest *lg, unsigned long cr3, u32 idx) { int pgdir; + /* The kernel seems to try to initialize this early on: we ignore its + * attempts to map over the Switcher. */ if (idx >= SWITCHER_PGD_INDEX) return; + /* If they're talking about a page table we have a shadow for... */ pgdir = find_pgdir(lg, cr3); if (pgdir < ARRAY_SIZE(lg->pgdirs)) + /* ... throw it away. */ release_pgd(lg, lg->pgdirs[pgdir].pgdir + idx); } +/*H:500 (vii) Setting up the page tables initially. + * + * When a Guest is first created, the Launcher tells us where the toplevel of + * its first page table is. We set some things up here: */ int init_guest_pagetable(struct lguest *lg, unsigned long pgtable) { - /* We assume this in flush_user_mappings, so check now */ + /* In flush_user_mappings() we loop from 0 to + * "vaddr_to_pgd_index(lg->page_offset)". This assumes it won't hit + * the Switcher mappings, so check that now. */ if (vaddr_to_pgd_index(lg->page_offset) >= SWITCHER_PGD_INDEX) return -EINVAL; + /* We start on the first shadow page table, and give it a blank PGD + * page. */ lg->pgdidx = 0; lg->pgdirs[lg->pgdidx].cr3 = pgtable; lg->pgdirs[lg->pgdidx].pgdir = (spgd_t*)get_zeroed_page(GFP_KERNEL); @@ -338,33 +570,48 @@ int init_guest_pagetable(struct lguest * return 0; } +/* When a Guest dies, our cleanup is fairly simple. */ void free_guest_pagetable(struct lguest *lg) { unsigned int i; + /* Throw away all page table pages. */ release_all_pagetables(lg); + /* Now free the top levels: free_page() can handle 0 just fine. */ for (i = 0; i < ARRAY_SIZE(lg->pgdirs); i++) free_page((long)lg->pgdirs[i].pgdir); } -/* Caller must be preempt-safe */ +/*H:480 (vi) Mapping the Switcher when the Guest is about to run. + * + * The Switcher and the two pages for this CPU need to be available to the + * Guest (and not the pages for other CPUs). We have the appropriate PTE pages + * for each CPU already set up, we just need to hook them in. */ void map_switcher_in_guest(struct lguest *lg, struct lguest_pages *pages) { spte_t *switcher_pte_page = __get_cpu_var(switcher_pte_pages); spgd_t switcher_pgd; spte_t regs_pte; - /* Since switcher less that 4MB, we simply mug top pte page. */ + /* Make the last PGD entry for this Guest point to the Switcher's PTE + * page for this CPU (with appropriate flags). */ switcher_pgd.pfn = __pa(switcher_pte_page) >> PAGE_SHIFT; switcher_pgd.flags = _PAGE_KERNEL; lg->pgdirs[lg->pgdidx].pgdir[SWITCHER_PGD_INDEX] = switcher_pgd; - /* Map our regs page over stack page. */ + /* We also change the Switcher PTE page. When we're running the Guest, + * we want the Guest's "regs" page to appear where the first Switcher + * page for this CPU is. This is an optimization: when the Switcher + * saves the Guest registers, it saves them into the first page of this + * CPU's "struct lguest_pages": if we make sure the Guest's register + * page is already mapped there, we don't have to copy them out + * again. */ regs_pte.pfn = __pa(lg->regs_page) >> PAGE_SHIFT; regs_pte.flags = _PAGE_KERNEL; switcher_pte_page[(unsigned long)pages/PAGE_SIZE%PTES_PER_PAGE] = regs_pte; } +/*:*/ static void free_switcher_pte_pages(void) { @@ -374,6 +621,10 @@ static void free_switcher_pte_pages(void free_page((long)switcher_pte_page(i)); } +/*H:520 Setting up the Switcher PTE page for given CPU is fairly easy, given + * the CPU number and the "struct page"s for the Switcher code itself. + * + * Currently the Switcher is less than a page long, so "pages" is always 1. */ static __init void populate_switcher_pte_page(unsigned int cpu, struct page *switcher_page[], unsigned int pages) @@ -381,21 +632,26 @@ static __init void populate_switcher_pte unsigned int i; spte_t *pte = switcher_pte_page(cpu); + /* The first entries are easy: they map the Switcher code. */ for (i = 0; i < pages; i++) { pte[i].pfn = page_to_pfn(switcher_page[i]); pte[i].flags = _PAGE_PRESENT|_PAGE_ACCESSED; } - /* We only map this CPU's pages, so guest can't see others. */ + /* The only other thing we map is this CPU's pair of pages. */ i = pages + cpu*2; - /* First page (regs) is rw, second (state) is ro. */ + /* First page (Guest registers) is writable from the Guest */ pte[i].pfn = page_to_pfn(switcher_page[i]); pte[i].flags = _PAGE_PRESENT|_PAGE_ACCESSED|_PAGE_RW; + /* The second page contains the "struct lguest_ro_state", and is + * read-only. */ pte[i+1].pfn = page_to_pfn(switcher_page[i+1]); pte[i+1].flags = _PAGE_PRESENT|_PAGE_ACCESSED; } +/*H:510 At boot or module load time, init_pagetables() allocates and populates + * the Switcher PTE page for each CPU. */ __init int init_pagetables(struct page **switcher_page, unsigned int pages) { unsigned int i; @@ -410,7 +666,9 @@ __init int init_pagetables(struct page * } return 0; } - +/*:*/ + +/* Cleaning up simply involves freeing the PTE page for each CPU. */ void free_pagetables(void) { free_switcher_pte_pages(); =================================================================== --- a/drivers/lguest/segments.c +++ b/drivers/lguest/segments.c @@ -11,17 +11,58 @@ * from frolicking through its parklike serenity. :*/ #include "lg.h" +/*H:600 + * We've almost completed the Host; there's just one file to go! + * + * Segments & The Global Descriptor Table + * + * (That title sounds like a bad Nerdcore group. Not to suggest that there are + * any good Nerdcore groups, but in high school a friend of mine had a band + * called Joe Fish and the Chips, so there are definitely worse band names). + * + * To refresh: the GDT is a table of 8-byte values describing segments. Once + * set up, these segments can be loaded into one of the 6 "segment registers". + * + * GDT entries are passed around as "struct desc_struct"s, which like IDT + * entries are split into two 32-bit members, "a" and "b". One day, someone + * will clean that up, and be declared a Hero. (No pressure, I'm just saying). + * + * Anyway, the GDT entry contains a base (the start address of the segment), a + * limit (the size of the segment - 1), and some flags. Sounds simple, and it + * would be, except those zany Intel engineers decided that it was too boring + * to put the base at one end, the limit at the other, and the flags in + * between. They decided to shotgun the bits at random throughout the 8 bytes, + * like so: + * + * 0 16 40 48 52 56 63 + * [ limit part 1 ][ base part 1 ][ flags ][li][fl][base ] + * mit ags part 2 + * part 2 + * + * As a result, this file contains a certain amount of magic numeracy. Let's + * begin. + */ + +/* Is the descriptor the Guest wants us to put in OK? + * + * The flag which Intel says must be zero: must be zero. The descriptor must + * be present, (this is actually checked earlier but is here for thorougness), + * and the descriptor type must be 1 (a memory segment). */ static int desc_ok(const struct desc_struct *gdt) { - /* MBZ=0, P=1, DT=1 */ return ((gdt->b & 0x00209000) == 0x00009000); } +/* Is the segment present? (Otherwise it can't be used by the Guest). */ static int segment_present(const struct desc_struct *gdt) { return gdt->b & 0x8000; } +/* There are several entries we don't let the Guest set. The TSS entry is the + * "Task State Segment" which controls all kinds of delicate things. The + * LGUEST_CS and LGUEST_DS entries are reserved for the Switcher, and the + * the Guest can't be trusted to deal with double faults. */ static int ignored_gdt(unsigned int num) { return (num == GDT_ENTRY_TSS @@ -30,9 +71,18 @@ static int ignored_gdt(unsigned int num) || num == GDT_ENTRY_DOUBLEFAULT_TSS); } -/* We don't allow removal of CS, DS or SS; it doesn't make sense. */ +/* If the Guest asks us to remove an entry from the GDT, we have to be careful. + * If one of the segment registers is pointing at that entry the Switcher will + * crash when it tries to reload the segment registers for the Guest. + * + * It doesn't make much sense for the Guest to try to remove its own code, data + * or stack segments while they're in use: assume that's a Guest bug. If it's + * one of the lesser segment registers using the removed entry, we simply set + * that register to 0 (unusable). */ static void check_segment_use(struct lguest *lg, unsigned int desc) { + /* GDT entries are 8 bytes long, so we divide to get the index and + * ignore the bottom bits. */ if (lg->regs->gs / 8 == desc) lg->regs->gs = 0; if (lg->regs->fs / 8 == desc) @@ -45,12 +95,16 @@ static void check_segment_use(struct lgu kill_guest(lg, "Removed live GDT entry %u", desc); } +/*H:610 Once the GDT has been changed, we look through the changed entries and + * see if they're OK. If not, we'll call kill_guest() and the Guest will never + * get to use the invalid entries. */ static void fixup_gdt_table(struct lguest *lg, unsigned start, unsigned end) { unsigned int i; for (i = start; i < end; i++) { - /* We never copy these ones to real gdt */ + /* We never copy these ones to real GDT, so we don't care what + * they say */ if (ignored_gdt(i)) continue; @@ -64,41 +118,57 @@ static void fixup_gdt_table(struct lgues if (!desc_ok(&lg->gdt[i])) kill_guest(lg, "Bad GDT descriptor %i", i); - /* DPL 0 presumably means "for use by guest". */ + /* Segment descriptors contain a privilege level: the Guest is + * sometimes careless and leaves this as 0, even though it's + * running at privilege level 1. If so, we fix it here. */ if ((lg->gdt[i].b & 0x00006000) == 0) lg->gdt[i].b |= (GUEST_PL << 13); - /* Set accessed bit, since gdt isn't writable. */ + /* Each descriptor has an "accessed" bit. If we don't set it + * now, the CPU will try to set it when the Guest first loads + * that entry into a segment register. But the GDT isn't + * writable by the Guest, so bad things can happen. */ lg->gdt[i].b |= 0x00000100; } } +/* This routine is called at boot or modprobe time for each CPU to set up the + * "constant" GDT entries for Guests running on that CPU. */ void setup_default_gdt_entries(struct lguest_ro_state *state) { struct desc_struct *gdt = state->guest_gdt; unsigned long tss = (unsigned long)&state->guest_tss; - /* Hypervisor segments. */ + /* The hypervisor segments are full 0-4G segments, privilege level 0 */ gdt[GDT_ENTRY_LGUEST_CS] = FULL_EXEC_SEGMENT; gdt[GDT_ENTRY_LGUEST_DS] = FULL_SEGMENT; - /* This is the one which we *cannot* copy from guest, since tss - is depended on this lguest_ro_state, ie. this cpu. */ + /* The TSS segment refers to the TSS entry for this CPU, so we cannot + * copy it from the Guest. Forgive the magic flags */ gdt[GDT_ENTRY_TSS].a = 0x00000067 | (tss << 16); gdt[GDT_ENTRY_TSS].b = 0x00008900 | (tss & 0xFF000000) | ((tss >> 16) & 0x000000FF); } +/* This routine is called before the Guest is run for the first time. */ void setup_guest_gdt(struct lguest *lg) { + /* Start with full 0-4G segments... */ lg->gdt[GDT_ENTRY_KERNEL_CS] = FULL_EXEC_SEGMENT; lg->gdt[GDT_ENTRY_KERNEL_DS] = FULL_SEGMENT; + /* ...except the Guest is allowed to use them, so set the privilege + * level appropriately in the flags. */ lg->gdt[GDT_ENTRY_KERNEL_CS].b |= (GUEST_PL << 13); lg->gdt[GDT_ENTRY_KERNEL_DS].b |= (GUEST_PL << 13); } -/* This is a fast version for the common case where only the three TLS entries - * have changed. */ +/* Like the IDT, we never simply use the GDT the Guest gives us. We set up the + * GDTs for each CPU, then we copy across the entries each time we want to run + * a different Guest on that CPU. */ + +/* A partial GDT load, for the three "thead-local storage" entries. Otherwise + * it's just like load_guest_gdt(). So much, in fact, it would probably be + * neater to have a single hypercall to cover both. */ void copy_gdt_tls(const struct lguest *lg, struct desc_struct *gdt) { unsigned int i; @@ -107,22 +177,31 @@ void copy_gdt_tls(const struct lguest *l gdt[i] = lg->gdt[i]; } +/* This is the full version */ void copy_gdt(const struct lguest *lg, struct desc_struct *gdt) { unsigned int i; + /* The default entries from setup_default_gdt_entries() are not + * replaced. See ignored_gdt() above. */ for (i = 0; i < GDT_ENTRIES; i++) if (!ignored_gdt(i)) gdt[i] = lg->gdt[i]; } +/* This is where the Guest asks us to load a new GDT (LHCALL_LOAD_GDT). */ void load_guest_gdt(struct lguest *lg, unsigned long table, u32 num) { + /* We assume the Guest has the same number of GDT entries as the + * Host, otherwise we'd have to dynamically allocate the Guest GDT. */ if (num > ARRAY_SIZE(lg->gdt)) kill_guest(lg, "too many gdt entries %i", num); + /* We read the whole thing in, then fix it up. */ lgread(lg, lg->gdt, table, num * sizeof(lg->gdt[0])); fixup_gdt_table(lg, 0, ARRAY_SIZE(lg->gdt)); + /* Mark that the GDT changed so the core knows it has to copy it again, + * even if the Guest is run on the same CPU. */ lg->changed |= CHANGED_GDT; } @@ -134,3 +213,13 @@ void guest_load_tls(struct lguest *lg, u fixup_gdt_table(lg, GDT_ENTRY_TLS_MIN, GDT_ENTRY_TLS_MAX+1); lg->changed |= CHANGED_GDT_TLS; } + +/* + * With this, we have finished the Host. + * + * Five of the seven parts of our task are complete. You have made it through + * the Bit of Despair (I think that's somewhere in the page table code, + * myself). + * + * Next, we examine "make Switcher". It's short, but intense. + */ ^ permalink raw reply [flat|nested] 8+ messages in thread
[parent not found: <1181272434.24958.19.camel@localhost.localdomain>]
* [PATCH 6/7] lguest documentation:Chapter VI [not found] ` <1181272434.24958.19.camel@localhost.localdomain> @ 2007-06-08 3:14 ` Rusty Russell [not found] ` <1181272472.24958.21.camel@localhost.localdomain> 1 sibling, 0 replies; 8+ messages in thread From: Rusty Russell @ 2007-06-08 3:14 UTC (permalink / raw) To: Andrew Morton; +Cc: lkml - Kernel Mailing List, virtualization Documentation: The Switcher Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> --- drivers/lguest/core.c | 51 +++++++- drivers/lguest/switcher.S | 271 ++++++++++++++++++++++++++++++++++++++------- 2 files changed, 276 insertions(+), 46 deletions(-) =================================================================== --- a/drivers/lguest/core.c +++ b/drivers/lguest/core.c @@ -394,46 +394,89 @@ static void set_ts(void) write_cr0(cr0|8); } +/*S:010 + * We are getting close to the Switcher. + * + * Remember that each CPU has two pages which are visible to the Guest when it + * runs on that CPU. This has to contain the state for that Guest: we copy the + * state in just before we run the Guest. + * + * Each Guest has "changed" flags which indicate what has changed in the Guest + * since it last ran. We saw this set in interrupts_and_traps.c and + * segments.c. + */ static void copy_in_guest_info(struct lguest *lg, struct lguest_pages *pages) { + /* Copying all this data can be quite expensive. We usually run the + * same Guest we ran last time (and that Guest hasn't run anywhere else + * meanwhile). If that's not the case, we pretend everything in the + * Guest has changed. */ if (__get_cpu_var(last_guest) != lg || lg->last_pages != pages) { __get_cpu_var(last_guest) = lg; lg->last_pages = pages; lg->changed = CHANGED_ALL; } - /* These are pretty cheap, so we do them unconditionally. */ + /* These copies are pretty cheap, so we do them unconditionally: */ + /* Save the current Host top-level page directory. */ pages->state.host_cr3 = __pa(current->mm->pgd); + /* Set up the Guest's page tables to see this CPU's pages (and no + * other CPU's pages). */ map_switcher_in_guest(lg, pages); + /* Set up the two "TSS" members which tell the CPU what stack to use + * for traps which do directly into the Guest (ie. traps at privilege + * level 1). */ pages->state.guest_tss.esp1 = lg->esp1; pages->state.guest_tss.ss1 = lg->ss1; - /* Copy direct trap entries. */ + /* Copy direct-to-Guest trap entries. */ if (lg->changed & CHANGED_IDT) copy_traps(lg, pages->state.guest_idt, default_idt_entries); - /* Copy all GDT entries but the TSS. */ + /* Copy all GDT entries which the Guest can change. */ if (lg->changed & CHANGED_GDT) copy_gdt(lg, pages->state.guest_gdt); /* If only the TLS entries have changed, copy them. */ else if (lg->changed & CHANGED_GDT_TLS) copy_gdt_tls(lg, pages->state.guest_gdt); + /* Mark the Guest as unchanged for next time. */ lg->changed = 0; } +/* Finally: the code to actually call into the Switcher to run the Guest. */ static void run_guest_once(struct lguest *lg, struct lguest_pages *pages) { + /* This is a dummy value we need for GCC's sake. */ unsigned int clobber; + /* Copy the guest-specific information into this CPU's "struct + * lguest_pages". */ copy_in_guest_info(lg, pages); - /* Put eflags on stack, lcall does rest: suitable for iret return. */ + /* Now: we push the "eflags" register on the stack, then do an "lcall". + * This is how we change from using the kernel code segment to using + * the dedicated lguest code segment, as well as jumping into the + * Switcher. + * + * The lcall also pushes the old code segment (KERNEL_CS) onto the + * stack, then the address of this call. This stack layout happens to + * exactly match the stack of an interrupt... */ asm volatile("pushf; lcall *lguest_entry" + /* This is how we tell GCC that %eax ("a") and %ebx ("b") + * are changed by this routine. The "=" means output. */ : "=a"(clobber), "=b"(clobber) + /* %eax contains the pages pointer. ("0" refers to the + * 0-th argument above, ie "a"). %ebx contains the + * physical address of the Guest's top-level page + * directory. */ : "0"(pages), "1"(__pa(lg->pgdirs[lg->pgdidx].pgdir)) + /* We tell gcc that all these registers could change, + * which means we don't have to save and restore them in + * the Switcher. */ : "memory", "%edx", "%ecx", "%edi", "%esi"); } +/*:*/ /*H:030 Let's jump straight to the the main loop which runs the Guest. * Remember, this is called by the Launcher reading /dev/lguest, and we keep =================================================================== --- a/drivers/lguest/switcher.S +++ b/drivers/lguest/switcher.S @@ -6,41 +6,131 @@ * are feeling invigorated and refreshed then the next, more challenging stage * can be found in "make Guest". :*/ +/*S:100 + * Welcome to the Switcher itself! + * + * This file contains the low-level code which changes the CPU to run the Guest + * code, and returns to the Host when something happens. Understand this, and + * you understand the heart of our journey. + * + * Because this is in assembler rather than C, our tale switches from prose to + * verse. First I tried limericks: + * + * There once was an eax reg, + * To which our pointer was fed, + * It needed an add, + * Which asm-offsets.h had + * But this limerick is hurting my head. + * + * Next I tried haikus, but fitting the required reference to the seasons in + * every stanza was quickly becoming tiresome: + * + * The %eax reg + * Holds "struct lguest_pages" now: + * Cherry blossoms fall. + * + * Then I started with Heroic Verse, but the rhyming requirement leeched away + * the content density and led to some uniquely awful oblique rhymes: + * + * These constants are coming from struct offsets + * For use within the asm switcher text. + * + * Finally, I settled for something between heroic hexameter, and normal prose + * with inappropriate linebreaks. Anyway, it aint no Shakespeare. + */ + +// Not all kernel headers work from assembler +// But these ones are needed: the ENTRY() define +// And constants extracted from struct offsets +// To avoid magic numbers and breakage: +// Should they change the compiler can't save us +// Down here in the depths of assembler code. #include <linux/linkage.h> #include <asm/asm-offsets.h> #include "lg.h" +// We mark the start of the code to copy +// It's placed in .text tho it's never run here +// You'll see the trick macro at the end +// Which interleaves data and text to effect. .text ENTRY(start_switcher_text) -/* %eax points to lguest pages for this CPU. %ebx contains cr3 value. - All normal registers can be clobbered! */ +// When we reach switch_to_guest we have just left +// The safe and comforting shores of C code +// %eax has the "struct lguest_pages" to use +// Where we save state and still see it from the Guest +// And %ebx holds the Guest shadow pagetable: +// Once set we have truly left Host behind. ENTRY(switch_to_guest) - /* Save host segments on host stack. */ + // We told gcc all its regs could fade, + // Clobbered by our journey into the Guest + // We could have saved them, if we tried + // But time is our master and cycles count. + + // Segment registers must be saved for the Host + // We push them on the Host stack for later pushl %es pushl %ds pushl %gs pushl %fs - /* With CONFIG_FRAME_POINTER, gcc doesn't let us clobber this! */ + // But the compiler is fickle, and heeds + // No warning of %ebp clobbers + // When frame pointers are used. That register + // Must be saved and restored or chaos strikes. pushl %ebp - /* Save host stack. */ + // The Host's stack is done, now save it away + // In our "struct lguest_pages" at offset + // Distilled into asm-offsets.h movl %esp, LGUEST_PAGES_host_sp(%eax) - /* Switch to guest stack: if we get NMI we expect to be there. */ + + // All saved and there's now five steps before us: + // Stack, GDT, IDT, TSS + // And last of all the page tables are flipped. + + // Yet beware that our stack pointer must be + // Always valid lest an NMI hits + // %edx does the duty here as we juggle + // %eax is lguest_pages: our stack lies within. movl %eax, %edx addl $LGUEST_PAGES_regs, %edx movl %edx, %esp - /* Switch to guest's GDT, IDT. */ + + // The Guest's GDT we so carefully + // Placed in the "struct lguest_pages" before lgdt LGUEST_PAGES_guest_gdt_desc(%eax) + + // The Guest's IDT we did partially + // Move to the "struct lguest_pages" as well. lidt LGUEST_PAGES_guest_idt_desc(%eax) - /* Switch to guest's TSS while GDT still writable. */ + + // The TSS entry which controls traps + // Must be loaded up with "ltr" now: + // For after we switch over our page tables + // It (as the rest) will be writable no more. + // (The GDT entry TSS needs + // Changes type when we load it: damn Intel!) movl $(GDT_ENTRY_TSS*8), %edx ltr %dx - /* Set host's TSS GDT entry to available (clear byte 5 bit 2). */ + + // Look back now, before we take this last step! + // The Host's TSS entry was also marked used; + // Let's clear it again, ere we return. + // The GDT descriptor of the Host + // Points to the table after two "size" bytes movl (LGUEST_PAGES_host_gdt_desc+2)(%eax), %edx + // Clear the type field of "used" (byte 5, bit 2) andb $0xFD, (GDT_ENTRY_TSS*8 + 5)(%edx) - /* Switch to guest page tables: lguest_pages->state now read-only. */ + + // Once our page table's switched, the Guest is live! + // The Host fades as we run this final step. + // Our "struct lguest_pages" is now read-only. movl %ebx, %cr3 - /* Restore guest regs */ + + // The page table change did one tricky thing: + // The Guest's register page has been mapped + // Writable onto our %esp (stack) -- + // We can simply pop off all Guest regs. popl %ebx popl %ecx popl %edx @@ -52,12 +142,27 @@ ENTRY(switch_to_guest) popl %fs popl %ds popl %es - /* Skip error code and trap number */ + + // Near the base of the stack lurk two strange fields + // Which we fill as we exit the Guest + // These are the trap number and its error + // We can simply step past them on our way. addl $8, %esp + + // The last five stack slots hold return address + // And everything needed to change privilege + // Into the Guest privilege level of 1, + // And the stack where the Guest had last left it. + // Interrupts are turned back on: we are Guest. iret +// There are two paths where we switch to the Host +// So we put the routine in a macro. +// We are on our way home, back to the Host +// Interrupted out of the Guest, we come here. #define SWITCH_TO_HOST \ - /* Save guest state */ \ + /* We save the Guest state: all registers first \ + * Laid out just as "struct lguest_regs" defines */ \ pushl %es; \ pushl %ds; \ pushl %fs; \ @@ -69,58 +174,119 @@ ENTRY(switch_to_guest) pushl %edx; \ pushl %ecx; \ pushl %ebx; \ - /* Load lguest ds segment for convenience. */ \ + /* Our stack and our code are using segments \ + * Set in the TSS and IDT \ + * Yet if we were to touch data we'd use \ + * Whatever data segment the Guest had. \ + * Load the lguest ds segment for now. */ \ movl $(LGUEST_DS), %eax; \ movl %eax, %ds; \ - /* Figure out where we are, based on stack (at top of regs). */ \ + /* So where are we? Which CPU, which struct? \ + * The stack is our clue: our TSS sets \ + * It at the end of "struct lguest_pages" \ + * And we then pushed and pushed and pushed Guest regs: \ + * Now stack points atop the "struct lguest_regs". \ + * Subtract that offset, and we find our struct. */ \ movl %esp, %eax; \ subl $LGUEST_PAGES_regs, %eax; \ - /* Put trap number in %ebx before we switch cr3 and lose it. */ \ + /* Save our trap number: the switch will obscure it \ + * (The Guest regs are not mapped here in the Host) \ + * %ebx holds it safe for deliver_to_host */ \ movl LGUEST_PAGES_regs_trapnum(%eax), %ebx; \ - /* Switch to host page tables (host GDT, IDT and stack are in host \ - mem, so need this first) */ \ + /* The Host GDT, IDT and stack! \ + * All these lie safely hidden from the Guest: \ + * We must return to the Host page tables \ + * (Hence that was saved in struct lguest_pages) */ \ movl LGUEST_PAGES_host_cr3(%eax), %edx; \ movl %edx, %cr3; \ - /* Set guest's TSS to available (clear byte 5 bit 2). */ \ + /* As before, when we looked back at the Host \ + * As we left and marked TSS unused \ + * So must we now for the Guest left behind. */ \ andb $0xFD, (LGUEST_PAGES_guest_gdt+GDT_ENTRY_TSS*8+5)(%eax); \ - /* Switch to host's GDT & IDT. */ \ + /* Switch to Host's GDT, IDT. */ \ lgdt LGUEST_PAGES_host_gdt_desc(%eax); \ lidt LGUEST_PAGES_host_idt_desc(%eax); \ - /* Switch to host's stack. */ \ + /* Restore the Host's stack where it's saved regs lie */ \ movl LGUEST_PAGES_host_sp(%eax), %esp; \ - /* Switch to host's TSS */ \ + /* Last the TSS: our Host is complete */ \ movl $(GDT_ENTRY_TSS*8), %edx; \ ltr %dx; \ + /* Restore now the regs saved right at the first. */ \ popl %ebp; \ popl %fs; \ popl %gs; \ popl %ds; \ popl %es -/* Return to run_guest_once. */ +// Here's where we come when the Guest has just trapped: +// (Which trap we'll see has been pushed on the stack). +// We need only switch back, and the Host will decode +// Why we came home, and what needs to be done. return_to_host: SWITCH_TO_HOST iret +// An interrupt, with some cause external +// Has ajerked us rudely from the Guest's code +// Again we must return home to the Host deliver_to_host: SWITCH_TO_HOST - /* Decode IDT and jump to hosts' irq handler. When that does iret, it - * will return to run_guest_once. This is a feature. */ + // But now we must go home via that place + // Where that interrupt was supposed to go + // Had we not been ensconced, running the Guest. + // Here we see the cleverness of our stack: + // The Host stack is formed like an interrupt + // With EIP, CS and EFLAGS layered. + // Interrupt handlers end with "iret" + // And that will take us home at long long last. + + // But first we must find the handler to call! + // The IDT descriptor for the Host + // Has two bytes for size, and four for address: + // %edx will hold it for us for now. movl (LGUEST_PAGES_host_idt_desc+2)(%eax), %edx + // We now know the table address we need, + // And saved the trap's number inside %ebx. + // Yet the pointer to the handler is smeared + // Across the bits of the table entry. + // What oracle can tell us how to extract + // From such a convoluted encoding? + // I consulted gcc, and it gave + // These instructions, which I gladly credit: leal (%edx,%ebx,8), %eax movzwl (%eax),%edx movl 4(%eax), %eax xorw %ax, %ax orl %eax, %edx + // Now the address of the handler's in %edx + // We call it now: its "iret" takes us home. jmp *%edx -/* Real hardware interrupts are delivered straight to the host. Others - cause us to return to run_guest_once so it can decide what to do. Note - that some of these are overridden by the guest to deliver directly, and - never enter here (see load_guest_idt_entry). */ +// Every interrupt can come to us here +// But we must truly tell each apart. +// They number two hundred and fifty six +// And each must land in a different spot, +// Push its number on stack, and join the stream. + +// And worse, a mere six of the traps stand apart +// And push on their stack an addition: +// An error number, thirty two bits long +// So we punish the other two fifty +// And make them push a zero so they match. + +// Yet two fifty six entries is long +// And all will look most the same as the last +// So we create a macro which can make +// As many entries as we need to fill. + +// Note the change to .data then .text: +// We plant the address of each entry +// Into a (data) table for the Host +// To know where each Guest interrupt should go. .macro IRQ_STUB N TARGET .data; .long 1f; .text; 1: - /* Make an error number for most traps, which don't have one. */ + // Trap eight, ten through fourteen and seventeen + // Supply an error number. Else zero. .if (\N <> 8) && (\N < 10 || \N > 14) && (\N <> 17) pushl $0 .endif @@ -129,6 +295,8 @@ deliver_to_host: ALIGN .endm +// This macro creates numerous entries +// Using GAS macros which out-power C's. .macro IRQ_STUBS FIRST LAST TARGET irq=\FIRST .rept \LAST-\FIRST+1 @@ -137,24 +305,43 @@ deliver_to_host: .endr .endm -/* We intercept every interrupt, because we may need to switch back to - * host. Unfortunately we can't tell them apart except by entry - * point, so we need 256 entry points. - */ +// Here's the marker for our pointer table +// Laid in the data section just before +// Each macro places the address of code +// Forming an array: each one points to text +// Which handles interrupt in its turn. .data .global default_idt_entries default_idt_entries: .text - IRQ_STUBS 0 1 return_to_host /* First two traps */ - IRQ_STUB 2 handle_nmi /* NMI */ - IRQ_STUBS 3 31 return_to_host /* Rest of traps */ - IRQ_STUBS 32 127 deliver_to_host /* Real interrupts */ - IRQ_STUB 128 return_to_host /* System call (overridden) */ - IRQ_STUBS 129 255 deliver_to_host /* Other real interrupts */ - -/* We ignore NMI and return. */ + // The first two traps go straight back to the Host + IRQ_STUBS 0 1 return_to_host + // We'll say nothing, yet, about NMI + IRQ_STUB 2 handle_nmi + // Other traps also return to the Host + IRQ_STUBS 3 31 return_to_host + // All interrupts go via their handlers + IRQ_STUBS 32 127 deliver_to_host + // 'Cept system calls coming from userspace + // Are to go to the Guest, never the Host. + IRQ_STUB 128 return_to_host + IRQ_STUBS 129 255 deliver_to_host + +// The NMI, what a fabulous beast +// Which swoops in and stops us no matter that +// We're suspended between heaven and hell, +// (Or more likely between the Host and Guest) +// When in it comes! We are dazed and confused +// So we do the simplest thing which one can. +// Though we've pushed the trap number and zero +// We discard them, return, and hope we live. handle_nmi: addl $8, %esp iret +// We are done; all that's left is Mastery +// And "make Mastery" is a journey long +// Designed to make your fingers itch to code. + +// Here ends the text, the file and poem. ENTRY(end_switcher_text) ^ permalink raw reply [flat|nested] 8+ messages in thread
[parent not found: <1181272472.24958.21.camel@localhost.localdomain>]
* [PATCH 7/7] lguest documentation:Chapter VII [not found] ` <1181272472.24958.21.camel@localhost.localdomain> @ 2007-06-08 3:15 ` Rusty Russell 0 siblings, 0 replies; 8+ messages in thread From: Rusty Russell @ 2007-06-08 3:15 UTC (permalink / raw) To: Andrew Morton; +Cc: lkml - Kernel Mailing List, virtualization Documentation: The FIXMEs Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> --- Documentation/lguest/lguest.c | 12 ++++++++++++ drivers/char/hvc_lguest.c | 3 +++ drivers/lguest/interrupts_and_traps.c | 14 ++++++++++++++ drivers/lguest/io.c | 10 ++++++++++ drivers/lguest/lguest.c | 8 ++++++++ drivers/lguest/lguest_asm.S | 14 ++++++++++++++ drivers/lguest/page_tables.c | 5 +++++ drivers/lguest/segments.c | 4 ++++ drivers/net/lguest_net.c | 19 +++++++++++++++++++ 9 files changed, 89 insertions(+) =================================================================== --- a/Documentation/lguest/lguest.c +++ b/Documentation/lguest/lguest.c @@ -1536,3 +1536,15 @@ int main(int argc, char *argv[]) /* Finally, run the Guest. This doesn't return. */ run_guest(lguest_fd, &device_list); } +/*:*/ + +/*M:999 + * Mastery is done: you now know everything I do. + * + * But surely you have seen code, features and bugs in your wanderings which + * you now yearn to attack? That is the real game, and I look forward to you + * patching and forking lguest into the Your-Name-Here-visor. + * + * Farewell, and good coding! + * Rusty Russell. + */ =================================================================== --- a/drivers/char/hvc_lguest.c +++ b/drivers/char/hvc_lguest.c @@ -13,6 +13,9 @@ * functions. :*/ +/*M:002 The console can be flooded: while the Guest is processing input the + * Host can send more. Buffering in the Host could alleviate this, but it is a + * difficult problem in general. :*/ /* Copyright (C) 2006 Rusty Russell, IBM Corporation * * This program is free software; you can redistribute it and/or modify =================================================================== --- a/drivers/lguest/interrupts_and_traps.c +++ b/drivers/lguest/interrupts_and_traps.c @@ -231,6 +231,20 @@ static int direct_trap(const struct lgue * go direct, of course 8) */ return idt_type(trap->a, trap->b) == 0xF; } +/*:*/ + +/*M:005 The Guest has the ability to turn its interrupt gates into trap gates, + * if it is careful. The Host will let trap gates can go directly to the + * Guest, but the Guest needs the interrupts atomically disabled for an + * interrupt gate. It can do this by pointing the trap gate at instructions + * within noirq_start and noirq_end, where it can safely disable interrupts. */ + +/*M:006 The Guests do not use the sysenter (fast system call) instruction, + * because it's hardcoded to enter privilege level 0 and so can't go direct. + * It's about twice as fast as the older "int 0x80" system call, so it might + * still be worthwhile to handle it in the Switcher and lcall down to the + * Guest. The sysenter semantics are hairy tho: search for that keyword in + * entry.S :*/ /*H:260 When we make traps go directly into the Guest, we need to make sure * the kernel stack is valid (ie. mapped in the page tables). Otherwise, the =================================================================== --- a/drivers/lguest/io.c +++ b/drivers/lguest/io.c @@ -570,6 +570,16 @@ void set_wakeup_process(struct lguest *l get_task_struct(lg->wake); } +/*M:007 We only return a single DMA buffer to the Launcher, but it would be + * more efficient to return a pointer to the entire array of DMA buffers, which + * it can cache and choose one whenever it wants. + * + * Currently the Launcher uses a write to /dev/lguest, and the return value is + * the address of the DMA structure with the interrupt number placed in + * dma->used_len. If we wanted to return the entire array, we need to return + * the address, array size and interrupt number: this seems to require an + * ioctl(). :*/ + /*L:320 This routine looks for a DMA buffer registered by the Guest on the * given key (using the BIND_DMA hypercall). */ unsigned long get_dma_buffer(struct lguest *lg, =================================================================== --- a/drivers/lguest/lguest.c +++ b/drivers/lguest/lguest.c @@ -251,6 +251,14 @@ static void irq_enable(void) { lguest_data.irq_enabled = X86_EFLAGS_IF; } +/*:*/ +/*M:003 Note that we don't check for outstanding interrupts when we re-enable + * them (or when we unmask an interrupt). This seems to work for the moment, + * since interrupts are rare and we'll just get the interrupt on the next timer + * tick, but when we turn on CONFIG_NO_HZ, we should revisit this. One way + * would be to put the "irq_enabled" field in a page by itself, and have the + * Host write-protect it when an interrupt comes in when irqs are disabled. + * There will then be a page fault as soon as interrupts are re-enabled. :*/ /*G:034 * The Interrupt Descriptor Table (IDT). =================================================================== --- a/drivers/lguest/lguest_asm.S +++ b/drivers/lguest/lguest_asm.S @@ -41,6 +41,20 @@ LGUEST_PATCH(pushf, movl lguest_data+LGU .global lguest_noirq_start .global lguest_noirq_end +/*M:004 When the Host reflects a trap or injects an interrupt into the Guest, + * it sets the eflags interrupt bit on the stack based on + * lguest_data.irq_enabled, so the Guest iret logic does the right thing when + * restoring it. However, when the Host sets the Guest up for direct traps, + * such as system calls, the processor is the one to push eflags onto the + * stack, and the interrupt bit will be 1 (in reality, interrupts are always + * enabled in the Guest). + * + * This turns out to be harmless: the only trap which should happen under Linux + * with interrupts disabled is Page Fault (due to our lazy mapping of vmalloc + * regions), which has to be reflected through the Host anyway. If another + * trap *does* go off when interrupts are disabled, the Guest will panic, and + * we'll never get to this iret! :*/ + /*G:045 There is one final paravirt_op that the Guest implements, and glancing * at it you can see why I left it to last. It's *cool*! It's in *assembler*! * =================================================================== --- a/drivers/lguest/page_tables.c +++ b/drivers/lguest/page_tables.c @@ -14,6 +14,11 @@ #include <linux/percpu.h> #include <asm/tlbflush.h> #include "lg.h" + +/*M:008 We hold reference to pages, which prevents them from being swapped. + * It'd be nice to have a callback in the "struct mm_struct" when Linux wants + * to swap out. If we had this, and a shrinker callback to trim PTE pages, we + * could probably consider launching Guests as non-root. :*/ /*H:300 * The Page Table Code =================================================================== --- a/drivers/lguest/segments.c +++ b/drivers/lguest/segments.c @@ -94,6 +94,10 @@ static void check_segment_use(struct lgu || lg->regs->ss / 8 == desc) kill_guest(lg, "Removed live GDT entry %u", desc); } +/*:*/ +/*M:009 We wouldn't need to check for removal of in-use segments if we handled + * faults in the Switcher. However, it's probably not a worthwhile + * optimization. :*/ /*H:610 Once the GDT has been changed, we look through the changed entries and * see if they're OK. If not, we'll call kill_guest() and the Guest will never =================================================================== --- a/drivers/net/lguest_net.c +++ b/drivers/net/lguest_net.c @@ -33,6 +33,25 @@ #define SHARED_SIZE PAGE_SIZE #define MAX_LANS 4 #define NUM_SKBS 8 + +/*M:011 Network code master Jeff Garzik points out numerous shortcomings in + * this driver if it aspires to greatness. + * + * Firstly, it doesn't use "NAPI": the networking's New API, and is poorer for + * it. As he says "NAPI means system-wide load leveling, across multiple + * network interfaces. Lack of NAPI can mean competition at higher loads." + * + * He also points out that we don't implement set_mac_address, so users cannot + * change the devices hardware address. When I asked why one would want to: + * "Bonding, and situations where you /do/ want the MAC address to "leak" out + * of the host onto the wider net." + * + * Finally, he would like module unloading: "It is not unrealistic to think of + * [un|re|]loading the net support module in an lguest guest. And, adding + * module support makes the programmer more responsible, because they now have + * to learn to clean up after themselves. Any driver that cannot clean up + * after itself is an incomplete driver in my book." + :*/ /*D:530 The "struct lguestnet_info" contains all the information we need to * know about the network device. */ ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 1/7] lguest documentation: infrastructure and Chapter I [not found] <1181272257.24958.12.camel@localhost.localdomain> 2007-06-08 3:11 ` [PATCH 2/7] lguest documentation:Chapter II Rusty Russell [not found] ` <1181272316.24958.13.camel@localhost.localdomain> @ 2007-06-08 6:23 ` Andrew Morton [not found] ` <20070607232349.b4673058.akpm@linux-foundation.org> 3 siblings, 0 replies; 8+ messages in thread From: Andrew Morton @ 2007-06-08 6:23 UTC (permalink / raw) To: Rusty Russell; +Cc: lkml - Kernel Mailing List, virtualization On Fri, 08 Jun 2007 13:10:57 +1000 Rusty Russell <rusty@rustcorp.com.au> wrote: > The netfilter code had very good documentation: the Netfilter Hacking > HOWTO. Noone ever read it. > > So this time I'm trying something different, using a bit of > Knuthiness. > > Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> > --- > Documentation/lguest/extract | 58 +++++++++++++++++++++++++++++++++ > Documentation/lguest/lguest.c | 9 +++-- > drivers/lguest/Makefile | 12 ++++++ > drivers/lguest/README | 47 ++++++++++++++++++++++++++ > drivers/lguest/core.c | 7 ++- > drivers/lguest/hypercalls.c | 9 +++-- > drivers/lguest/interrupts_and_traps.c | 13 +++++++ > drivers/lguest/io.c | 8 +++- > drivers/lguest/lguest.c | 31 ++++++++++++++++- > drivers/lguest/lguest_bus.c | 3 + > drivers/lguest/lguest_user.c | 7 +++ > drivers/lguest/page_tables.c | 10 ++++- > drivers/lguest/segments.c | 11 ++++++ > drivers/lguest/switcher.S | 13 +++---- > 14 files changed, 219 insertions(+), 19 deletions(-) This patch series craps all over the existing one: lguest-export-symbols-for-lguest-as-a-module.patch lguest-the-guest-code.patch lguest-the-host-code.patch lguest-the-asm-offsets.patch lguest-the-makefile-and-kconfig.patch lguest-the-console-driver.patch lguest-the-net-driver.patch lguest-the-block-driver.patch lguest-the-documentation-example-launcher.patch obviously we could append this new series to the existing one, but that limits our ability to make further changes over the next few weeks. Or we just give up trying to present the series nicely and it hits mainline with a random stream of uninteresting and irrelevant afterthoughts at the end, like we're some lameo git users or something. (I could split all this up, but I'm getting kinda bored of preventing you from mucking up your own patch series) So... what course dost thou wish to take? ^ permalink raw reply [flat|nested] 8+ messages in thread
[parent not found: <20070607232349.b4673058.akpm@linux-foundation.org>]
* Re: [PATCH 1/7] lguest documentation: infrastructure and Chapter I [not found] ` <20070607232349.b4673058.akpm@linux-foundation.org> @ 2007-06-08 7:05 ` Rusty Russell 0 siblings, 0 replies; 8+ messages in thread From: Rusty Russell @ 2007-06-08 7:05 UTC (permalink / raw) To: Andrew Morton; +Cc: lkml - Kernel Mailing List, virtualization On Thu, 2007-06-07 at 23:23 -0700, Andrew Morton wrote: > On Fri, 08 Jun 2007 13:10:57 +1000 Rusty Russell <rusty@rustcorp.com.au> wrote: > > > The netfilter code had very good documentation: the Netfilter Hacking > > HOWTO. Noone ever read it. > > > > So this time I'm trying something different, using a bit of > > Knuthiness. > > > > Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> > > --- > > Documentation/lguest/extract | 58 +++++++++++++++++++++++++++++++++ > > Documentation/lguest/lguest.c | 9 +++-- > > drivers/lguest/Makefile | 12 ++++++ > > drivers/lguest/README | 47 ++++++++++++++++++++++++++ > > drivers/lguest/core.c | 7 ++- > > drivers/lguest/hypercalls.c | 9 +++-- > > drivers/lguest/interrupts_and_traps.c | 13 +++++++ > > drivers/lguest/io.c | 8 +++- > > drivers/lguest/lguest.c | 31 ++++++++++++++++- > > drivers/lguest/lguest_bus.c | 3 + > > drivers/lguest/lguest_user.c | 7 +++ > > drivers/lguest/page_tables.c | 10 ++++- > > drivers/lguest/segments.c | 11 ++++++ > > drivers/lguest/switcher.S | 13 +++---- > > 14 files changed, 219 insertions(+), 19 deletions(-) > > This patch series craps all over the existing one: > > lguest-export-symbols-for-lguest-as-a-module.patch > lguest-the-guest-code.patch > lguest-the-host-code.patch > lguest-the-asm-offsets.patch > lguest-the-makefile-and-kconfig.patch > lguest-the-console-driver.patch > lguest-the-net-driver.patch > lguest-the-block-driver.patch > lguest-the-documentation-example-launcher.patch > > obviously we could append this new series to the existing one, but that > limits our ability to make further changes over the next few weeks. Or we > just give up trying to present the series nicely and it hits mainline with > a random stream of uninteresting and irrelevant afterthoughts at the end, > like we're some lameo git users or something. Well, I've been painfully maintaining that documentation patch for months (it touches *everything*). I *had* intended to send it straight to Linus one he took lguest, but that may never happen. Don't take it. I'll push it to the end of my own series, send it direct to Linus if and when. > (I could split all this up, but I'm getting kinda bored of preventing you > from mucking up your own patch series) Little sympathy. Perhaps there's a moral about timely merging here. Rusty. ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2007-06-08 7:05 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <1181272257.24958.12.camel@localhost.localdomain>
2007-06-08 3:11 ` [PATCH 2/7] lguest documentation:Chapter II Rusty Russell
[not found] ` <1181272316.24958.13.camel@localhost.localdomain>
2007-06-08 3:12 ` [PATCH 3/7] lguest documentation:Chapter III Rusty Russell
[not found] ` <1181272355.24958.15.camel@localhost.localdomain>
2007-06-08 3:13 ` [PATCH 4/7] lguest documentation:Chapter IV Rusty Russell
2007-06-08 3:13 ` [PATCH 5/7] lguest documentation:Chapter V Rusty Russell
[not found] ` <1181272434.24958.19.camel@localhost.localdomain>
2007-06-08 3:14 ` [PATCH 6/7] lguest documentation:Chapter VI Rusty Russell
[not found] ` <1181272472.24958.21.camel@localhost.localdomain>
2007-06-08 3:15 ` [PATCH 7/7] lguest documentation:Chapter VII Rusty Russell
2007-06-08 6:23 ` [PATCH 1/7] lguest documentation: infrastructure and Chapter I Andrew Morton
[not found] ` <20070607232349.b4673058.akpm@linux-foundation.org>
2007-06-08 7:05 ` Rusty Russell
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox