From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Chen, Tiejun" Subject: Re: [v3][PATCH 07/16] hvmloader/pci: skip reserved ranges Date: Wed, 17 Jun 2015 16:47:58 +0800 Message-ID: <5581343E.5050100@intel.com> References: <1433985325-16676-1-git-send-email-tiejun.chen@intel.com> <1433985325-16676-8-git-send-email-tiejun.chen@intel.com> <557A9000.600@intel.com> <557FEC6A.1080003@intel.com> <55800B21020000780008561E@mail.emea.novell.com> <55811D50.2070002@intel.com> <55813BA60200007800085D93@mail.emea.novell.com> <558127B7.5060800@intel.com> <5581466C0200007800085DEC@mail.emea.novell.com> <55812F2D.2030504@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <55812F2D.2030504@intel.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Jan Beulich Cc: Kevin Tian , "wei.liu2@citrix.com" , "ian.campbell@citrix.com" , "tim@xen.org" , "Ian.Jackson@eu.citrix.com" , "xen-devel@lists.xen.org" , "stefano.stabellini@citrix.com" , "andrew.cooper3@citrix.com" , Yang Z Zhang List-Id: xen-devel@lists.xenproject.org On 2015/6/17 16:26, Chen, Tiejun wrote: > On 2015/6/17 16:05, Jan Beulich wrote: >>>>> On 17.06.15 at 09:54, wrote: >>> On 2015/6/17 15:19, Jan Beulich wrote: >>>>>>> On 17.06.15 at 09:10, wrote: >>>>> Yeah, this may waste some spaces in this worst case but I this think >>>>> this can guarantee our change don't impact on the original >>>>> expectation, >>>>> right? >>>> >>>> "Some space" may be multiple Gb (e.g. the frame buffer of a graphics >>> >>> Sure. >>> >>>> card), which is totally unacceptable. >>>> >>> >>> But then I don't understand what's your way. How can we fit all pci >>> devices just with "the smallest power-of-2 region enclosing the reserved >>> device memory"? >>> >>> For example, the whole pci memory is sitting at >>> [0xa0000000, 0xa2000000]. And there are two PCI devices, A and B. Note >>> each device needs to be allocated with 0x1000000. So if without >>> concerning RMRR, >>> >>> A. [0xa0000000,0xa1000000] >>> B. [0xa1000000,0xa2000000] >>> >>> But if one RMRR resides at [0xa0f00000, 0xa1f00000] which obviously >>> generate its own alignment with 0x1000000. So the pci memory is expended >>> as [0xa0000000, 0xa3000000], right? >>> >>> Then actually the whole pci memory can be separated three segments like, >>> >>> #1. [0xa0000000, 0xa0f00000] >>> #2. [0xa0f00000, 0xa1f00000] -> RMRR would occupy >>> #3. [0xa1f00000, 0xa3000000] >>> >>> So just #3 can suffice to allocate but just for one device, right? >> >> Right, i.e. this isn't even sufficient - you need [a0000000,a3ffffff] >> to fit everything (but of course you can put smaller BARs into the >> unused ranges [a0000000,a0efffff] and [a1f00000,a1ffffff]). > > Yes, I knew there's this sort of hole that we should use efficiently as > you said. And I also thought about this way previously but current pci > allocation framework isn't friend to implement this easily, > > /* Assign iomem and ioport resources in descending order of size. */ > for ( i = 0; i < nr_bars; i++ ) > { > > I mean it isn't easy to calculate what's the most sufficient size in > advance, and its also difficult to find to fit a appropriate pci device > into those "holes", so see below, > >> That's why I said it's not going to be tricky to get all corner cases >> right _and_ not use up more space than needed. >> >>>>>> ought to work out the smallest power-of-2 region enclosing the >>>>> >>>>> Okay. I remember the smallest size of a given PCI I/O space is 8 >>>>> bytes, >>>>> and the smallest size of a PCI memory space is 16 bytes. So >>>>> >>>>> /* At least 16 bytes to align a PCI BAR size. */ >>>>> uint64_t align = 16; >>>>> >>>>> reserved_start = memory_map.map[j].addr; >>>>> reserved_size = memory_map.map[j].size; >>>>> >>>>> reserved_start = (reserved_star + align) & ~(align - 1); >>>>> reserved_size = (reserved_size + align) & ~(align - 1); >>>>> >>>>> Is this correct? >>>> >>>> Simply aligning the region doesn't help afaict. You need to fit it >>>> with the other MMIO allocations. >>> >>> I guess you're saying just those mmio allocations conflicting with RMRR? >>> But we don't know these exact addresses until we finalize to allocate >>> them, right? >> >> That's the point - you need to allocate them _around_ the reserved >> regions. >> > > Something hits me to generate another idea, > > #1. Still allocate all devices as before. > #2. Lookup all actual bars to check if they're conflicting RMRR > > We can skip these bars to keep zero. Then later it would make lookup > easily. Sorry this may bring a confusion when I reread these lines. I mean, Or we can skip allocating these bars to keep zero during #1. But further consider this way, I think this lead that it can't make sure the remaining resources don't conflict RMRR. So please ignore this option and just do #1 purely. Thanks Tiejun > > #3. Need to reallocate these conflicting bars. > #3.1 Trying to reallocate them with the remaining resources > #3.2 If the remaining resources aren't enough, we need to allocate them > from high_mem_resource. > > I just feel this way may be easy and better. And even, this way also can > help terminate the preexisting allocation failures, right? > > Thanks > Tiejun > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel > >