ccio_clear_io_tlb() and __raw_write() more question?

ccio_clear_io_tlb() and __raw_write() more question?

[rubisher]

Hello all,

I would try to explain my worry about this function.

It is:
static CCIO_INLINE void
ccio_clear_io_tlb(struct ioc *ioc, dma_addr_t iovp, size_t byte_cnt)
{
        u32 chain_size = 1 << ioc->chainid_shift;

        iovp &= IOVP_MASK;      /* clear offset bits, just want pagenum */
        byte_cnt += chain_size;

        while(byte_cnt > chain_size) {
                __asm__ __volatile__ ("# ccio_clear_io_tlb-in");
                WRITE_U32(CMD_TLB_PURGE | iovp, &ioc->ioc_regs->io_command);
                __asm__ __volatile__ ("# ccio_clear_io_tlb-out");
                iovp += chain_size;
                byte_cnt -= chain_size;
        }
}

Without doc I guess that this call to WRITE_U32() is supposed to do what it
name says: i.e purge a TLB entry?

In details:
#define WRITE_U32(value, addr)  __raw_writel((value), (addr))

and finaly:
static inline void __raw_writel(unsigned int b, volatile void __iomem *addr)
{
        __asm__ __volatile__ ("# __raw_writel-in");
        *(volatile unsigned int __force *) addr = b;
        __asm__ __volatile__ ("# __raw_writel-out");
}

I just added __asm__ to see excatly how __raw_write() is assembled:

        # __raw_writel-in
#NO_APP
        stw %r19,48(%r28)
#APP
        # __raw_writel-out

Otoh, ioc->ioc_regs is set in:
static int __init ccio_probe(struct parisc_device *dev)
{
[snip]      
        ioc->ioc_regs = ioremap_nocache(dev->hpa.start, 4096);
[snip]
};

and according to __ioremap comment:
/*
 * Remap an arbitrary physical address space into the kernel virtual
 * address space.
 *
[snip]
 */

ioc_regs contains well a virtual address (what seems to me coherent with paper
on iommu design previously mentioned)?

puting all those stuff togther: for CPU, WRITE_U32 (aka __raw_writel), just
write something at some virtual address, right?

So my worry is that this data write could be cached and btw the actual
CMD_TLB_PURGE operation could be delay untill this entry would be purged?

What's your opinion?
What would be the best way to actualy do this write (back) to the io ctrlr?

Tx again,
  r.
---
Scarlet One, ADSL 6 Mbps + Telephone, from EUR 29,95...
http://www.scarlet.be/

Re: ccio_clear_io_tlb() and __raw_write() more question?

[Matthew Wilcox]

On Wed, Jan 16, 2008 at 01:53:25PM +0100, rubisher wrote:
> So my worry is that this data write could be cached and btw the actual
> CMD_TLB_PURGE operation could be delay untill this entry would be purged?

f-space is uncached.

-- 
Intel are signing my paycheques ... these opinions are still mine
"Bill, look, we understand that you're interested in selling us this
operating system, but compare it to ours.  We can't possibly take such
a retrograde step."

Re: ccio_clear_io_tlb() and __raw_write() more question?

[Grant Grundler]

On Wed, Jan 16, 2008 at 01:53:25PM +0100, rubisher wrote:
> Without doc I guess that this call to WRITE_U32() is supposed to do what it
> name says: i.e purge a TLB entry?
...
> puting all those stuff togther: for CPU, WRITE_U32 (aka __raw_writel), just
> write something at some virtual address, right?
> 
> So my worry is that this data write could be cached and btw the actual
> CMD_TLB_PURGE operation could be delay untill this entry would be purged?

Willy is correct - writes to this IO space (0xf0000000) address is uncached.

While the write is "posted", it's a fairly short path to the IOMMU (1 hop).
Since this is the "unmap path" I don't expect any race conditions since
we would have to call map using the same mapping right away in order to
pickup the stale IOTLB entry. We avoid this race by allocating IO Pdir
entries in a "circular" fashion (start at bottom and work through
the entire resource bit map).

hth,
grant

Re: ccio_clear_io_tlb() and __raw_write() more question?

[rubisher]

Grant Grundler wrote:
> On Wed, Jan 16, 2008 at 01:53:25PM +0100, rubisher wrote:
>> Without doc I guess that this call to WRITE_U32() is supposed to do what it
>> name says: i.e purge a TLB entry?
> ...
>> puting all those stuff togther: for CPU, WRITE_U32 (aka __raw_writel), just
>> write something at some virtual address, right?
>>
>> So my worry is that this data write could be cached and btw the actual
>> CMD_TLB_PURGE operation could be delay untill this entry would be purged?
> 
> Willy is correct - writes to this IO space (0xf0000000) address is uncached.
> 
mmm, is it the reason why you (I mean hp engineers) call it "f-space": IO space addresses starting by 0xf?

> While the write is "posted", it's a fairly short path to the IOMMU (1 hop).
> Since this is the "unmap path" I don't expect any race conditions since
> we would have to call map using the same mapping right away in order to
> pickup the stale IOTLB entry. We avoid this race by allocating IO Pdir
> entries in a "circular" fashion (start at bottom and work through
> the entire resource bit map).
> 
> hth,
> grant
> 
> 
Tx,
	r.

Re: ccio_clear_io_tlb() and __raw_write() more question?

[Grant Grundler]

On Sun, Jan 20, 2008 at 10:06:30AM +0000, rubisher wrote:
...
>> Willy is correct - writes to this IO space (0xf0000000) address is 
>> uncached.
> mmm, is it the reason why you (I mean hp engineers) call it "f-space": IO 
> space addresses starting by 0xf?

Yes.

grant