* [PATCH] powerpc/pseries: Fix CONFIG_NUMA=n build
From: Michael Ellerman @ 2018-05-09 14:58 UTC (permalink / raw)
To: linuxppc-dev; +Cc: mwb
The build is failing with CONFIG_NUMA=n and some compiler versions:
arch/powerpc/platforms/pseries/hotplug-cpu.o: In function `dlpar_online_cpu':
hotplug-cpu.c:(.text+0x12c): undefined reference to `timed_topology_update'
arch/powerpc/platforms/pseries/hotplug-cpu.o: In function `dlpar_cpu_remove':
hotplug-cpu.c:(.text+0x400): undefined reference to `timed_topology_update'
Fix it by moving the empty version of timed_topology_update() into the
existing #ifdef block, which has the right guard of SPLPAR && NUMA.
Fixes: cee5405da402 ("powerpc/hotplug: Improve responsiveness of hotplug change")
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/include/asm/topology.h | 13 +++++--------
1 file changed, 5 insertions(+), 8 deletions(-)
diff --git a/arch/powerpc/include/asm/topology.h b/arch/powerpc/include/asm/topology.h
index 9f421641a35c..16b077801a5f 100644
--- a/arch/powerpc/include/asm/topology.h
+++ b/arch/powerpc/include/asm/topology.h
@@ -91,6 +91,7 @@ extern int start_topology_update(void);
extern int stop_topology_update(void);
extern int prrn_is_enabled(void);
extern int find_and_online_cpu_nid(int cpu);
+extern int timed_topology_update(int nsecs);
#else
static inline int start_topology_update(void)
{
@@ -108,16 +109,12 @@ static inline int find_and_online_cpu_nid(int cpu)
{
return 0;
}
+static inline int timed_topology_update(int nsecs)
+{
+ return 0;
+}
#endif /* CONFIG_NUMA && CONFIG_PPC_SPLPAR */
-#if defined(CONFIG_HOTPLUG_CPU) || defined(CONFIG_NEED_MULTIPLE_NODES)
-#if defined(CONFIG_PPC_SPLPAR)
-extern int timed_topology_update(int nsecs);
-#else
-#define timed_topology_update(nsecs)
-#endif /* CONFIG_PPC_SPLPAR */
-#endif /* CONFIG_HOTPLUG_CPU || CONFIG_NEED_MULTIPLE_NODES */
-
#include <asm-generic/topology.h>
#ifdef CONFIG_SMP
--
2.14.1
^ permalink raw reply related
* Re: [PATCH 13/13] powerpc/eeh: Refactor report functions
From: Michael Ellerman @ 2018-05-09 14:51 UTC (permalink / raw)
To: Sam Bobroff; +Cc: linuxppc-dev
In-Reply-To: <20180507052354.GB18123@tungsten.ozlabs.ibm.com>
Sam Bobroff <sbobroff@linux.ibm.com> writes:
> On Thu, May 03, 2018 at 11:27:12PM +1000, Michael Ellerman wrote:
>> Sam Bobroff <sbobroff@linux.ibm.com> writes:
>> > diff --git a/arch/powerpc/kernel/eeh_driver.c b/arch/powerpc/kernel/eeh_driver.c
>> > index eb4feee81ff4..1c4336dcf9f5 100644
>> > --- a/arch/powerpc/kernel/eeh_driver.c
>> > +++ b/arch/powerpc/kernel/eeh_driver.c
>> > @@ -54,6 +54,25 @@ static int eeh_result_priority(enum pci_ers_result result)
>> > }
>> > };
>> >
>> > +const char *pci_ers_result_name(enum pci_ers_result r)
>> > +{
>> > + switch (r) {
>> > + case PCI_ERS_RESULT_NONE: return "none";
>> > + case PCI_ERS_RESULT_CAN_RECOVER: return "can recover";
>> > + case PCI_ERS_RESULT_NEED_RESET: return "need reset";
>> > + case PCI_ERS_RESULT_DISCONNECT: return "disconnect";
>> > + case PCI_ERS_RESULT_RECOVERED: return "recovered";
>> > + case PCI_ERS_RESULT_NO_AER_DRIVER: return "no AER driver";
>> > + default:
>> > + WARN_ONCE(1, "Unknown result type");
>> > + return "unknown";
>> > + }
>> > +};
>> > +
>> > +#define eeh_infoline(EDEV, FMT, ...) \
>> > +pr_info("EEH: PE#%x (PCI %s): " pr_fmt(FMT) "\n", EDEV->pe_config_addr, \
>> > +((EDEV->pdev) ? dev_name(&EDEV->pdev->dev) : "NONE"), ##__VA_ARGS__)
>>
>> Ooof, I guess.
>>
>> It would be nicer as a function.
>
> OK (I'd prefer to avoid macros too), but I'm not sure what kind of
> function you mean. I initially tried to use a function, but the existing
> pr_*() type macros seemed to be macros all the way down to printk, so there
> didn't seem to be anywhere to hook into, and I ended up having to
> open-code the varargs and allocate a buffer to print into. Then it can
> overflow etc. and it ended up seeming worse. Is there a better way I'm
> missing here?
Something like (not tested):
void eeh_edev_info(const struct eeh_info *info, const char *fmt, ...)
{
struct va_format vaf;
va_list args;
va_start(args, fmt);
vaf.fmt = fmt;
vaf.va = &args;
printk(KERN_INFO "EEH: PE#%x (PCI %s): %pV\n",
info->pe_config_addr,
info->pdev ? dev_name(&info->pdev->dev) : "none",
&vaf);
va_end(args);
}
>
>> "infoline" annoys me for some reason, "eeh_info" ?
>
> OK. How about eeh_edev_info(), to indicate that it acts on an 'edev'?
Sounds good.
cheers
^ permalink raw reply
* Re: [PATCH] pkeys: Introduce PKEY_ALLOC_SIGNALINHERIT and change signal semantics
From: Andy Lutomirski @ 2018-05-09 14:41 UTC (permalink / raw)
To: Florian Weimer
Cc: Andrew Lutomirski, linuxram, Dave Hansen, Linux-MM, Linux API,
linux-x86_64, linux-arch, X86 ML, linuxppc-dev
In-Reply-To: <314e1a48-db94-9b37-8793-a95a2082c9e2@redhat.com>
On Tue, May 8, 2018 at 5:40 AM Florian Weimer <fweimer@redhat.com> wrote:
> On 05/08/2018 04:49 AM, Andy Lutomirski wrote:
> > On Mon, May 7, 2018 at 2:48 AM Florian Weimer <fweimer@redhat.com>
wrote:
> >
> >> On 05/03/2018 06:05 AM, Andy Lutomirski wrote:
> >>> On Wed, May 2, 2018 at 7:11 PM Ram Pai <linuxram@us.ibm.com> wrote:
> >>>
> >>>> On Wed, May 02, 2018 at 09:23:49PM +0000, Andy Lutomirski wrote:
> >>>>>
> >>>>>> If I recall correctly, the POWER maintainer did express a strong
> >>> desire
> >>>>>> back then for (what is, I believe) their current semantics, which
my
> >>>>>> PKEY_ALLOC_SIGNALINHERIT patch implements for x86, too.
> >>>>>
> >>>>> Ram, I really really don't like the POWER semantics. Can you give
> > some
> >>>>> justification for them? Does POWER at least have an atomic way for
> >>>>> userspace to modify just the key it wants to modify or, even better,
> >>>>> special load and store instructions to use alternate keys?
> >>>
> >>>> I wouldn't call it POWER semantics. The way I implemented it on power
> >>>> lead to the semantics, given that nothing was explicitly stated
> >>>> about how the semantics should work within a signal handler.
> >>>
> >>> I think that this is further evidence that we should introduce a new
> >>> pkey_alloc() mode and deprecate the old. To the extent possible, this
> >>> thing should work the same way on x86 and POWER.
> >
> >> Do you propose to change POWER or to change x86?
> >
> > Sorry for being slow to reply. I propose to introduce a new
> > PKEY_ALLOC_something variant on x86 and POWER and to make the behavior
> > match on both.
> So basically implement PKEY_ALLOC_SETSIGNAL for POWER, and keep the
> existing (different) behavior without the flag?
> Ram, would you be okay with that? Could you give me a hand if
> necessary? (I assume we have silicon in-house because it's a
> long-standing feature of the POWER platform which was simply dormant on
> Linux until now.)
> > It should at least update the values loaded when a signal
> > is delivered and it should probably also update it for new threads.
> I think we should keep inheritance for new threads and fork. pkey_alloc
> only has a single access rights argument, which makes it hard to reuse
> this interface if there are two (three) separate sets of access rights.
Hmm. I can get on board with the idea that fork() / clone() /
pthread_create() are all just special cases of the idea that the thread
that *calls* them should have the right pkey values, and the latter is
already busted given our inability to asynchronously propagate the new mode
in pkey_alloc(). So let's so PKEY_ALLOC_SETSIGNAL as a starting point.
One thing we could do, though: the current initual state on process
creation is all access blocked on all keys. We could change it so that
half the keys are fully blocked and half are read-only. Then we could add
a PKEY_ALLOC_STRICT or similar that allocates a key with the correct
initial state *and* does the setsignal thing. If there are no keys left
with the correct initial state, then it fails.
^ permalink raw reply
* make a few block drivers highmem safe
From: Christoph Hellwig @ 2018-05-09 13:59 UTC (permalink / raw)
To: Jens Axboe
Cc: Ed L. Cashin, Geoff Levand, Maxim Levitsky, linux-block,
linuxppc-dev, linux-mtd, sparclinux
Hi all,
this series converts a few random block drivers to be highmem safe,
in preparation of eventually getting rid of the block layer bounce
buffering support.
^ permalink raw reply
* [PATCH 7/7] ps3disk: handle highmem pages
From: Christoph Hellwig @ 2018-05-09 13:59 UTC (permalink / raw)
To: Jens Axboe
Cc: Ed L. Cashin, Geoff Levand, Maxim Levitsky, linux-block,
linuxppc-dev, linux-mtd, sparclinux
In-Reply-To: <20180509135948.20329-1-hch@lst.de>
The ps3disk driver already kmaps all pages when copying from/to the
internal bounce buffer, so it can accept highmem pages just fine.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
drivers/block/ps3disk.c | 2 --
1 file changed, 2 deletions(-)
diff --git a/drivers/block/ps3disk.c b/drivers/block/ps3disk.c
index 075662f2cf46..afe1508d82c6 100644
--- a/drivers/block/ps3disk.c
+++ b/drivers/block/ps3disk.c
@@ -465,8 +465,6 @@ static int ps3disk_probe(struct ps3_system_bus_device *_dev)
priv->queue = queue;
queue->queuedata = dev;
- blk_queue_bounce_limit(queue, BLK_BOUNCE_HIGH);
-
blk_queue_max_hw_sectors(queue, dev->bounce_size >> 9);
blk_queue_segment_boundary(queue, -1UL);
blk_queue_dma_alignment(queue, dev->blk_size-1);
--
2.17.0
^ permalink raw reply related
* [PATCH 6/7] jsflash: handle highmem pages
From: Christoph Hellwig @ 2018-05-09 13:59 UTC (permalink / raw)
To: Jens Axboe
Cc: Ed L. Cashin, Geoff Levand, Maxim Levitsky, linux-block,
linuxppc-dev, linux-mtd, sparclinux
In-Reply-To: <20180509135948.20329-1-hch@lst.de>
Just kmap the bio single page payload before processing it.
(and yes, now highmem on sparc32 anyway, but kmap_(un)map atomic are nops,
so this gives the right example)
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
drivers/sbus/char/jsflash.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/drivers/sbus/char/jsflash.c b/drivers/sbus/char/jsflash.c
index 14f377ac1280..8520587b8d09 100644
--- a/drivers/sbus/char/jsflash.c
+++ b/drivers/sbus/char/jsflash.c
@@ -215,6 +215,7 @@ static void jsfd_request(void)
unsigned long offset = blk_rq_pos(req) << 9;
size_t len = blk_rq_cur_bytes(req);
blk_status_t err = BLK_STS_IOERR;
+ void *p;
if ((offset + len) > jdp->dsize)
goto end;
@@ -229,7 +230,9 @@ static void jsfd_request(void)
goto end;
}
- jsfd_read(bio_data(req->bio), jdp->dbase + offset, len);
+ p = kmap_atomic(bio_page(bio)) + bio_offset(bio);
+ jsfd_read(p, jdp->dbase + offset, len);
+ kunmap_atomic(p);
err = BLK_STS_OK;
end:
if (!__blk_end_request_cur(req, err))
@@ -592,7 +595,6 @@ static int jsfd_init(void)
put_disk(disk);
goto out;
}
- blk_queue_bounce_limit(disk->queue, BLK_BOUNCE_HIGH);
jsfd_disk[i] = disk;
}
--
2.17.0
^ permalink raw reply related
* [PATCH 5/7] aoe: handle highmem pages
From: Christoph Hellwig @ 2018-05-09 13:59 UTC (permalink / raw)
To: Jens Axboe
Cc: Ed L. Cashin, Geoff Levand, Maxim Levitsky, linux-block,
linuxppc-dev, linux-mtd, sparclinux
In-Reply-To: <20180509135948.20329-1-hch@lst.de>
Use kmap_atomic when copying out of a bio_vec.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
drivers/block/aoe/aoeblk.c | 1 -
drivers/block/aoe/aoecmd.c | 3 ++-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/block/aoe/aoeblk.c b/drivers/block/aoe/aoeblk.c
index 6797e6c23c8a..027b876370bc 100644
--- a/drivers/block/aoe/aoeblk.c
+++ b/drivers/block/aoe/aoeblk.c
@@ -388,7 +388,6 @@ aoeblk_gdalloc(void *vp)
d->aoemajor, d->aoeminor);
goto err_mempool;
}
- blk_queue_bounce_limit(q, BLK_BOUNCE_HIGH);
spin_lock_irqsave(&d->lock, flags);
WARN_ON(!(d->flags & DEVFL_GD_NOW));
diff --git a/drivers/block/aoe/aoecmd.c b/drivers/block/aoe/aoecmd.c
index 540bb60cd071..096882e54095 100644
--- a/drivers/block/aoe/aoecmd.c
+++ b/drivers/block/aoe/aoecmd.c
@@ -1032,8 +1032,9 @@ bvcpy(struct sk_buff *skb, struct bio *bio, struct bvec_iter iter, long cnt)
iter.bi_size = cnt;
__bio_for_each_segment(bv, bio, iter, iter) {
- char *p = page_address(bv.bv_page) + bv.bv_offset;
+ char *p = kmap_atomic(bv.bv_page) + bv.bv_offset;
skb_copy_bits(skb, soff, p, bv.bv_len);
+ kunmap_atomic(p);
soff += bv.bv_len;
}
}
--
2.17.0
^ permalink raw reply related
* [PATCH 4/7] mtd_blkdevs: handle highmem pages
From: Christoph Hellwig @ 2018-05-09 13:59 UTC (permalink / raw)
To: Jens Axboe
Cc: Ed L. Cashin, Geoff Levand, Maxim Levitsky, linux-block,
linuxppc-dev, linux-mtd, sparclinux
In-Reply-To: <20180509135948.20329-1-hch@lst.de>
Just kmap the single payload page before passing it on to the FTL.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
drivers/mtd/mtd_blkdevs.c | 20 ++++++++++++++------
1 file changed, 14 insertions(+), 6 deletions(-)
diff --git a/drivers/mtd/mtd_blkdevs.c b/drivers/mtd/mtd_blkdevs.c
index 16ae4ae8e8f9..29c0bfd74e8a 100644
--- a/drivers/mtd/mtd_blkdevs.c
+++ b/drivers/mtd/mtd_blkdevs.c
@@ -82,7 +82,6 @@ static blk_status_t do_blktrans_request(struct mtd_blktrans_ops *tr,
block = blk_rq_pos(req) << 9 >> tr->blkshift;
nsect = blk_rq_cur_bytes(req) >> tr->blkshift;
- buf = bio_data(req->bio);
if (req_op(req) == REQ_OP_FLUSH) {
if (tr->flush(dev))
@@ -100,9 +99,14 @@ static blk_status_t do_blktrans_request(struct mtd_blktrans_ops *tr,
return BLK_STS_IOERR;
return BLK_STS_OK;
case REQ_OP_READ:
- for (; nsect > 0; nsect--, block++, buf += tr->blksize)
- if (tr->readsect(dev, block, buf))
+ buf = kmap(bio_page(req->bio)) + bio_offset(req->bio);
+ for (; nsect > 0; nsect--, block++, buf += tr->blksize) {
+ if (tr->readsect(dev, block, buf)) {
+ kunmap(bio_page(req->bio));
return BLK_STS_IOERR;
+ }
+ }
+ kunmap(bio_page(req->bio));
rq_flush_dcache_pages(req);
return BLK_STS_OK;
case REQ_OP_WRITE:
@@ -110,9 +114,14 @@ static blk_status_t do_blktrans_request(struct mtd_blktrans_ops *tr,
return BLK_STS_IOERR;
rq_flush_dcache_pages(req);
- for (; nsect > 0; nsect--, block++, buf += tr->blksize)
- if (tr->writesect(dev, block, buf))
+ buf = kmap(bio_page(req->bio)) + bio_offset(req->bio);
+ for (; nsect > 0; nsect--, block++, buf += tr->blksize) {
+ if (tr->writesect(dev, block, buf)) {
+ kunmap(bio_page(req->bio));
return BLK_STS_IOERR;
+ }
+ }
+ kunmap(bio_page(req->bio));
return BLK_STS_OK;
default:
return BLK_STS_IOERR;
@@ -418,7 +427,6 @@ int add_mtd_blktrans_dev(struct mtd_blktrans_dev *new)
new->rq->queuedata = new;
blk_queue_logical_block_size(new->rq, tr->blksize);
- blk_queue_bounce_limit(new->rq, BLK_BOUNCE_HIGH);
blk_queue_flag_set(QUEUE_FLAG_NONROT, new->rq);
blk_queue_flag_clear(QUEUE_FLAG_ADD_RANDOM, new->rq);
--
2.17.0
^ permalink raw reply related
* [PATCH 3/7] memstick: don't call blk_queue_bounce_limit
From: Christoph Hellwig @ 2018-05-09 13:59 UTC (permalink / raw)
To: Jens Axboe
Cc: Ed L. Cashin, Geoff Levand, Maxim Levitsky, linux-block,
linuxppc-dev, linux-mtd, sparclinux
In-Reply-To: <20180509135948.20329-1-hch@lst.de>
All in-tree host drivers set up a proper dma mask and use the dma-mapping
helpers. This means they will be able to deal with any address that we
are throwing at them.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
drivers/memstick/core/ms_block.c | 5 -----
drivers/memstick/core/mspro_block.c | 5 -----
2 files changed, 10 deletions(-)
diff --git a/drivers/memstick/core/ms_block.c b/drivers/memstick/core/ms_block.c
index 57b13dfbd21e..b2d025f42d14 100644
--- a/drivers/memstick/core/ms_block.c
+++ b/drivers/memstick/core/ms_block.c
@@ -2096,12 +2096,8 @@ static int msb_init_disk(struct memstick_dev *card)
struct msb_data *msb = memstick_get_drvdata(card);
struct memstick_host *host = card->host;
int rc;
- u64 limit = BLK_BOUNCE_HIGH;
unsigned long capacity;
- if (host->dev.dma_mask && *(host->dev.dma_mask))
- limit = *(host->dev.dma_mask);
-
mutex_lock(&msb_disk_lock);
msb->disk_id = idr_alloc(&msb_disk_idr, card, 0, 256, GFP_KERNEL);
mutex_unlock(&msb_disk_lock);
@@ -2123,7 +2119,6 @@ static int msb_init_disk(struct memstick_dev *card)
msb->queue->queuedata = card;
- blk_queue_bounce_limit(msb->queue, limit);
blk_queue_max_hw_sectors(msb->queue, MS_BLOCK_MAX_PAGES);
blk_queue_max_segments(msb->queue, MS_BLOCK_MAX_SEGS);
blk_queue_max_segment_size(msb->queue,
diff --git a/drivers/memstick/core/mspro_block.c b/drivers/memstick/core/mspro_block.c
index 8897962781bb..a2fadc605750 100644
--- a/drivers/memstick/core/mspro_block.c
+++ b/drivers/memstick/core/mspro_block.c
@@ -1175,12 +1175,8 @@ static int mspro_block_init_disk(struct memstick_dev *card)
struct mspro_sys_info *sys_info = NULL;
struct mspro_sys_attr *s_attr = NULL;
int rc, disk_id;
- u64 limit = BLK_BOUNCE_HIGH;
unsigned long capacity;
- if (host->dev.dma_mask && *(host->dev.dma_mask))
- limit = *(host->dev.dma_mask);
-
for (rc = 0; msb->attr_group.attrs[rc]; ++rc) {
s_attr = mspro_from_sysfs_attr(msb->attr_group.attrs[rc]);
@@ -1219,7 +1215,6 @@ static int mspro_block_init_disk(struct memstick_dev *card)
msb->queue->queuedata = card;
- blk_queue_bounce_limit(msb->queue, limit);
blk_queue_max_hw_sectors(msb->queue, MSPRO_BLOCK_MAX_PAGES);
blk_queue_max_segments(msb->queue, MSPRO_BLOCK_MAX_SEGS);
blk_queue_max_segment_size(msb->queue,
--
2.17.0
^ permalink raw reply related
* [PATCH 2/7] DAC960: don't use block layer bounce buffers
From: Christoph Hellwig @ 2018-05-09 13:59 UTC (permalink / raw)
To: Jens Axboe
Cc: Ed L. Cashin, Geoff Levand, Maxim Levitsky, linux-block,
linuxppc-dev, linux-mtd, sparclinux
In-Reply-To: <20180509135948.20329-1-hch@lst.de>
DAC960 just sets the block bounce limit to the dma mask, which means
that the iommu or swiotlb already take care of the bounce buffering,
and the block bouncing can be removed.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
drivers/block/DAC960.c | 9 ++-------
drivers/block/DAC960.h | 1 -
2 files changed, 2 insertions(+), 8 deletions(-)
diff --git a/drivers/block/DAC960.c b/drivers/block/DAC960.c
index f781eff7d23e..c9ba48519d0f 100644
--- a/drivers/block/DAC960.c
+++ b/drivers/block/DAC960.c
@@ -1179,7 +1179,6 @@ static bool DAC960_V1_EnableMemoryMailboxInterface(DAC960_Controller_T
if (pci_set_dma_mask(Controller->PCIDevice, DMA_BIT_MASK(32)))
return DAC960_Failure(Controller, "DMA mask out of range");
- Controller->BounceBufferLimit = DMA_BIT_MASK(32);
if ((hw_type == DAC960_PD_Controller) || (hw_type == DAC960_P_Controller)) {
CommandMailboxesSize = 0;
@@ -1380,11 +1379,8 @@ static bool DAC960_V2_EnableMemoryMailboxInterface(DAC960_Controller_T
dma_addr_t CommandMailboxDMA;
DAC960_V2_CommandStatus_T CommandStatus;
- if (!pci_set_dma_mask(Controller->PCIDevice, DMA_BIT_MASK(64)))
- Controller->BounceBufferLimit = DMA_BIT_MASK(64);
- else if (!pci_set_dma_mask(Controller->PCIDevice, DMA_BIT_MASK(32)))
- Controller->BounceBufferLimit = DMA_BIT_MASK(32);
- else
+ if (pci_set_dma_mask(Controller->PCIDevice, DMA_BIT_MASK(64)) &&
+ pci_set_dma_mask(Controller->PCIDevice, DMA_BIT_MASK(32)))
return DAC960_Failure(Controller, "DMA mask out of range");
/* This is a temporary dma mapping, used only in the scope of this function */
@@ -2540,7 +2536,6 @@ static bool DAC960_RegisterBlockDevice(DAC960_Controller_T *Controller)
continue;
}
Controller->RequestQueue[n] = RequestQueue;
- blk_queue_bounce_limit(RequestQueue, Controller->BounceBufferLimit);
RequestQueue->queuedata = Controller;
blk_queue_max_segments(RequestQueue, Controller->DriverScatterGatherLimit);
blk_queue_max_hw_sectors(RequestQueue, Controller->MaxBlocksPerCommand);
diff --git a/drivers/block/DAC960.h b/drivers/block/DAC960.h
index 21aff470d268..1439e651928b 100644
--- a/drivers/block/DAC960.h
+++ b/drivers/block/DAC960.h
@@ -2295,7 +2295,6 @@ typedef struct DAC960_Controller
unsigned short MaxBlocksPerCommand;
unsigned short ControllerScatterGatherLimit;
unsigned short DriverScatterGatherLimit;
- u64 BounceBufferLimit;
unsigned int CombinedStatusBufferLength;
unsigned int InitialStatusLength;
unsigned int CurrentStatusLength;
--
2.17.0
^ permalink raw reply related
* [PATCH 1/7] mtip32xx: don't use block layer bounce buffers
From: Christoph Hellwig @ 2018-05-09 13:59 UTC (permalink / raw)
To: Jens Axboe
Cc: Ed L. Cashin, Geoff Levand, Maxim Levitsky, linux-block,
linuxppc-dev, linux-mtd, sparclinux
In-Reply-To: <20180509135948.20329-1-hch@lst.de>
mtip32xx just sets the block bounce limit to the dma mask, which means
that the iommu or swiotlb already take care of the bounce buffering,
and the block bouncing can be removed.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
drivers/block/mtip32xx/mtip32xx.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/drivers/block/mtip32xx/mtip32xx.c b/drivers/block/mtip32xx/mtip32xx.c
index 769c551e3d71..b03bb27dcc58 100644
--- a/drivers/block/mtip32xx/mtip32xx.c
+++ b/drivers/block/mtip32xx/mtip32xx.c
@@ -3862,7 +3862,6 @@ static int mtip_block_initialize(struct driver_data *dd)
blk_queue_max_hw_sectors(dd->queue, 0xffff);
blk_queue_max_segment_size(dd->queue, 0x400000);
blk_queue_io_min(dd->queue, 4096);
- blk_queue_bounce_limit(dd->queue, dd->pdev->dma_mask);
/* Signal trim support */
if (dd->trim_supp == true) {
--
2.17.0
^ permalink raw reply related
* [PATCH 3/3] powerpc/prom: Clean up local variables in early_init_dt_scan_cpus()
From: Michael Ellerman @ 2018-05-09 13:42 UTC (permalink / raw)
To: linuxppc-dev
In-Reply-To: <20180509134229.26278-1-mpe@ellerman.id.au>
This is cosmetic, but the current arrangement is hell ugly, so clean
it up.
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/kernel/prom.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/arch/powerpc/kernel/prom.c b/arch/powerpc/kernel/prom.c
index 9edb0d0af986..0e47fb85f47d 100644
--- a/arch/powerpc/kernel/prom.c
+++ b/arch/powerpc/kernel/prom.c
@@ -308,12 +308,11 @@ static int __init early_init_dt_scan_cpus(unsigned long node,
const char *uname, int depth,
void *data)
{
- const char *type = of_get_flat_dt_prop(node, "device_type", NULL);
- const __be32 *prop;
- const __be32 *intserv;
- int i, nthreads;
- int len;
- int found = -1, found_hwid = -1;
+ int i, len, nthreads, found_hwid, found;
+ const __be32 *prop, *intserv;
+ const char *type;
+
+ type = of_get_flat_dt_prop(node, "device_type", NULL);
/* We are scanning "cpu" nodes only */
if (type == NULL || strcmp(type, "cpu") != 0)
@@ -330,6 +329,7 @@ static int __init early_init_dt_scan_cpus(unsigned long node,
* Now see if any of these threads match our boot cpu.
* NOTE: This must match the parsing done in smp_setup_cpu_maps.
*/
+ found = found_hwid = -1;
for (i = 0; i < nthreads; i++) {
found_hwid = be32_to_cpu(intserv[i]);
if (found_hwid == fdt_boot_cpuid_phys(initial_boot_params))
--
2.14.1
^ permalink raw reply related
* [PATCH 2/3] powerpc/prom: Save the boot CPU hardware id in a local
From: Michael Ellerman @ 2018-05-09 13:42 UTC (permalink / raw)
To: linuxppc-dev
In-Reply-To: <20180509134229.26278-1-mpe@ellerman.id.au>
In early_init_dt_scan_cpus() we find the boot CPU, and get it's
hardware CPU number. Currently we do that by using the thread number
and doing be32_to_cpu(instserv[i]) three times.
The endian flips are not very efficient, but the main concern is just
that it's ugly to do the same flip in three places. Instead just put
the endian flipped value in a local and use it.
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/kernel/prom.c | 14 +++++---------
1 file changed, 5 insertions(+), 9 deletions(-)
diff --git a/arch/powerpc/kernel/prom.c b/arch/powerpc/kernel/prom.c
index 05e7fb47a7a4..9edb0d0af986 100644
--- a/arch/powerpc/kernel/prom.c
+++ b/arch/powerpc/kernel/prom.c
@@ -313,8 +313,7 @@ static int __init early_init_dt_scan_cpus(unsigned long node,
const __be32 *intserv;
int i, nthreads;
int len;
- int found = -1;
- int found_thread = 0;
+ int found = -1, found_hwid = -1;
/* We are scanning "cpu" nodes only */
if (type == NULL || strcmp(type, "cpu") != 0)
@@ -332,11 +331,9 @@ static int __init early_init_dt_scan_cpus(unsigned long node,
* NOTE: This must match the parsing done in smp_setup_cpu_maps.
*/
for (i = 0; i < nthreads; i++) {
- if (be32_to_cpu(intserv[i]) ==
- fdt_boot_cpuid_phys(initial_boot_params)) {
+ found_hwid = be32_to_cpu(intserv[i]);
+ if (found_hwid == fdt_boot_cpuid_phys(initial_boot_params))
found = boot_cpu_count;
- found_thread = i;
- }
#ifdef CONFIG_SMP
/* logical cpu id is always 0 on UP kernels */
boot_cpu_count++;
@@ -347,8 +344,7 @@ static int __init early_init_dt_scan_cpus(unsigned long node,
if (found < 0)
return 0;
- DBG("boot cpu: logical %d physical %d\n", found,
- be32_to_cpu(intserv[found_thread]));
+ DBG("boot cpu: logical %d physical %d\n", found, found_hwid);
boot_cpuid = found;
/*
@@ -389,7 +385,7 @@ static int __init early_init_dt_scan_cpus(unsigned long node,
cur_cpu_spec->cpu_features |= CPU_FTR_SMT;
allocate_paca(boot_cpuid);
#endif
- set_hard_smp_processor_id(found, be32_to_cpu(intserv[found_thread]));
+ set_hard_smp_processor_id(found, found_hwid);
return 0;
}
--
2.14.1
^ permalink raw reply related
* [PATCH 1/3] powerpc/prom: Drop support for old FDT versions
From: Michael Ellerman @ 2018-05-09 13:42 UTC (permalink / raw)
To: linuxppc-dev
In commit e6a6928c3ea1 ("of/fdt: Convert FDT functions to use
libfdt") (Apr 2014), the generic flat device tree code dropped support
for flat device tree's older than version 0x10 (16).
We still have code in our CPU scanning to cope with flat device tree
versions earlier than 2, which can now never trigger, so drop it.
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/kernel/prom.c | 23 ++++-------------------
1 file changed, 4 insertions(+), 19 deletions(-)
diff --git a/arch/powerpc/kernel/prom.c b/arch/powerpc/kernel/prom.c
index 9dbed488aba1..05e7fb47a7a4 100644
--- a/arch/powerpc/kernel/prom.c
+++ b/arch/powerpc/kernel/prom.c
@@ -332,25 +332,10 @@ static int __init early_init_dt_scan_cpus(unsigned long node,
* NOTE: This must match the parsing done in smp_setup_cpu_maps.
*/
for (i = 0; i < nthreads; i++) {
- /*
- * version 2 of the kexec param format adds the phys cpuid of
- * booted proc.
- */
- if (fdt_version(initial_boot_params) >= 2) {
- if (be32_to_cpu(intserv[i]) ==
- fdt_boot_cpuid_phys(initial_boot_params)) {
- found = boot_cpu_count;
- found_thread = i;
- }
- } else {
- /*
- * Check if it's the boot-cpu, set it's hw index now,
- * unfortunately this format did not support booting
- * off secondary threads.
- */
- if (of_get_flat_dt_prop(node,
- "linux,boot-cpu", NULL) != NULL)
- found = boot_cpu_count;
+ if (be32_to_cpu(intserv[i]) ==
+ fdt_boot_cpuid_phys(initial_boot_params)) {
+ found = boot_cpu_count;
+ found_thread = i;
}
#ifdef CONFIG_SMP
/* logical cpu id is always 0 on UP kernels */
--
2.14.1
^ permalink raw reply related
* Re: [PATCH kernel v2] powerpc/ioda: Use ibm, supported-tce-sizes for IOMMU page size mask
From: Michael Ellerman @ 2018-05-09 13:38 UTC (permalink / raw)
To: Alexey Kardashevskiy, linuxppc-dev
Cc: Alexey Kardashevskiy, David Gibson, Russell Currey
In-Reply-To: <20180503042924.27308-1-aik@ozlabs.ru>
Alexey Kardashevskiy <aik@ozlabs.ru> writes:
> At the moment we assume that IODA2 and newer PHBs can always do 4K/64K/16M
> IOMMU pages, however this is not the case for POWER9 and now skiboot
> advertises the supported sizes via the device so we use that instead
> of hard coding the mask.
>
> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> ---
> Changes:
> v2:
> * added quirk for POWER8 to advertise 16M if skiboot has not provided info
> ---
> arch/powerpc/platforms/powernv/pci-ioda.c | 26 +++++++++++++++++++++++++-
> 1 file changed, 25 insertions(+), 1 deletion(-)
>
> diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
> index 3f9c69d..891b4b6 100644
> --- a/arch/powerpc/platforms/powernv/pci-ioda.c
> +++ b/arch/powerpc/platforms/powernv/pci-ioda.c
> @@ -2910,6 +2910,30 @@ static void pnv_pci_ioda2_table_free_pages(struct iommu_table *tbl)
> tbl->it_indirect_levels);
> }
>
> +static unsigned long pnv_ioda_parse_tce_sizes(struct pnv_phb *phb)
> +{
> + struct pci_controller *hose = phb->hose;
> + struct device_node *dn = hose->dn;
> + int i, len = 0;
> + const __be32 *r;
> + unsigned long mask = 0;
> +
> + r = of_get_property(dn, "ibm,supported-tce-sizes", &len);
> + if (!r || !len) {
> + mask = SZ_4K | SZ_64K;
> + /* Add 16M for POWER8 by default */
> + if (cpu_has_feature(CPU_FTR_ARCH_207S) &&
> + !cpu_has_feature(CPU_FTR_ARCH_300))
> + mask |= SZ_16M;
> + return mask;
> + }
> +
> + for (i = 0; i < len / sizeof(*r); ++i)
> + mask |= 1ULL << be32_to_cpu(r[i]);
> +
> + return mask;
> +}
Can we use the modern device tree accessors?
eg:
static unsigned long pnv_ioda_parse_tce_sizes(struct pnv_phb *phb)
{
struct pci_controller *hose = phb->hose;
struct device_node *dn = hose->dn;
unsigned long mask = 0;
int i, rc, count;
u32 val;
count = of_property_count_u32_elems(dn, "ibm,supported-tce-sizes");
if (count <= 0) {
mask = SZ_4K | SZ_64K;
/* Add 16M for POWER8 by default */
if (cpu_has_feature(CPU_FTR_ARCH_207S) &&
!cpu_has_feature(CPU_FTR_ARCH_300))
mask |= SZ_16M;
return mask;
}
for (i = 0; i < count; i++) {
rc = of_property_read_u32_index(dn, "ibm,supported-tce-sizes",
i, &val);
if (rc == 0)
mask |= 1ULL << val;
}
return mask;
}
cheers
^ permalink raw reply
* [PATCH v2 4/4] powerpc: Allow LD_DEAD_CODE_DATA_ELIMINATION to be selected
From: Nicholas Piggin @ 2018-05-09 13:00 UTC (permalink / raw)
To: Masahiro Yamada; +Cc: Nicholas Piggin, linux-arch, linux-kbuild, linuxppc-dev
In-Reply-To: <20180509130001.24276-1-npiggin@gmail.com>
This requires further changes to linker script to KEEP some tables
and wildcard compiler generated sections into the right place. This
includes pp32 modifications from Christophe Leroy.
When compiling powernv_defconfig with this option, the resulting
kernel is almost 400kB smaller (and still boots):
text data bss dec filename
11827621 4810490 1341080 17979191 vmlinux
11752437 4598858 1338776 17690071 vmlinux.dcde
Mathieu's numbers for custom Mac Mini G4 config has almost 200kB
saving. It also had some increase in vmlinux size for as-yet
unknown reasons.
text data bss dec filename
7461457 2475122 1428064 11364643 vmlinux
7386425 2364370 1425432 11176227 vmlinux.dcde
Tested-by: Christophe Leroy <christophe.leroy@c-s.fr> [8xx]
Tested-by: Mathieu Malaterre <malat@debian.org> [32-bit powermac]
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
arch/powerpc/Kconfig | 1 +
arch/powerpc/kernel/vmlinux.lds.S | 22 +++++++++++-----------
2 files changed, 12 insertions(+), 11 deletions(-)
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index c32a181a7cbb..ee6dbe2efc8b 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -205,6 +205,7 @@ config PPC
select HAVE_KPROBES
select HAVE_KPROBES_ON_FTRACE
select HAVE_KRETPROBES
+ select HAVE_LD_DEAD_CODE_DATA_ELIMINATION
select HAVE_LIVEPATCH if HAVE_DYNAMIC_FTRACE_WITH_REGS
select HAVE_MEMBLOCK
select HAVE_MEMBLOCK_NODE_MAP
diff --git a/arch/powerpc/kernel/vmlinux.lds.S b/arch/powerpc/kernel/vmlinux.lds.S
index c8af90ff49f0..52a93cdd04bc 100644
--- a/arch/powerpc/kernel/vmlinux.lds.S
+++ b/arch/powerpc/kernel/vmlinux.lds.S
@@ -89,7 +89,7 @@ SECTIONS
*/
.text BLOCK(0) : AT(ADDR(.text) - LOAD_OFFSET) {
#ifdef CONFIG_LD_HEAD_STUB_CATCH
- *(.linker_stub_catch);
+ KEEP(*(.linker_stub_catch));
. = . ;
#endif
@@ -98,7 +98,7 @@ SECTIONS
ALIGN_FUNCTION();
#endif
/* careful! __ftr_alt_* sections need to be close to .text */
- *(.text.hot .text .text.fixup .text.unlikely .fixup __ftr_alt_* .ref.text);
+ *(.text.hot TEXT_MAIN .text.fixup .text.unlikely .fixup __ftr_alt_* .ref.text);
SCHED_TEXT
CPUIDLE_TEXT
LOCK_TEXT
@@ -170,10 +170,10 @@ SECTIONS
.init.data : AT(ADDR(.init.data) - LOAD_OFFSET) {
INIT_DATA
__vtop_table_begin = .;
- *(.vtop_fixup);
+ KEEP(*(.vtop_fixup));
__vtop_table_end = .;
__ptov_table_begin = .;
- *(.ptov_fixup);
+ KEEP(*(.ptov_fixup));
__ptov_table_end = .;
}
@@ -194,26 +194,26 @@ SECTIONS
. = ALIGN(8);
__ftr_fixup : AT(ADDR(__ftr_fixup) - LOAD_OFFSET) {
__start___ftr_fixup = .;
- *(__ftr_fixup)
+ KEEP(*(__ftr_fixup))
__stop___ftr_fixup = .;
}
. = ALIGN(8);
__mmu_ftr_fixup : AT(ADDR(__mmu_ftr_fixup) - LOAD_OFFSET) {
__start___mmu_ftr_fixup = .;
- *(__mmu_ftr_fixup)
+ KEEP(*(__mmu_ftr_fixup))
__stop___mmu_ftr_fixup = .;
}
. = ALIGN(8);
__lwsync_fixup : AT(ADDR(__lwsync_fixup) - LOAD_OFFSET) {
__start___lwsync_fixup = .;
- *(__lwsync_fixup)
+ KEEP(*(__lwsync_fixup))
__stop___lwsync_fixup = .;
}
#ifdef CONFIG_PPC64
. = ALIGN(8);
__fw_ftr_fixup : AT(ADDR(__fw_ftr_fixup) - LOAD_OFFSET) {
__start___fw_ftr_fixup = .;
- *(__fw_ftr_fixup)
+ KEEP(*(__fw_ftr_fixup))
__stop___fw_ftr_fixup = .;
}
#endif
@@ -226,7 +226,7 @@ SECTIONS
. = ALIGN(8);
.machine.desc : AT(ADDR(.machine.desc) - LOAD_OFFSET) {
__machine_desc_start = . ;
- *(.machine.desc)
+ KEEP(*(.machine.desc))
__machine_desc_end = . ;
}
#ifdef CONFIG_RELOCATABLE
@@ -274,7 +274,7 @@ SECTIONS
.data : AT(ADDR(.data) - LOAD_OFFSET) {
DATA_DATA
*(.data.rel*)
- *(.sdata)
+ *(SDATA_MAIN)
*(.sdata2)
*(.got.plt) *(.got)
*(.plt)
@@ -289,7 +289,7 @@ SECTIONS
.opd : AT(ADDR(.opd) - LOAD_OFFSET) {
__start_opd = .;
- *(.opd)
+ KEEP(*(.opd))
__end_opd = .;
}
--
2.17.0
^ permalink raw reply related
* [PATCH v2 3/4] kbuild: Allow LD_DEAD_CODE_DATA_ELIMINATION to be selectable if enabled
From: Nicholas Piggin @ 2018-05-09 13:00 UTC (permalink / raw)
To: Masahiro Yamada; +Cc: Nicholas Piggin, linux-arch, linux-kbuild, linuxppc-dev
In-Reply-To: <20180509130001.24276-1-npiggin@gmail.com>
Architectures that are capable can select
HAVE_LD_DEAD_CODE_DATA_ELIMINATION to enable selection of that
option (as an EXPERT kernel option).
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
arch/Kconfig | 15 ---------------
init/Kconfig | 27 +++++++++++++++++++++++++++
2 files changed, 27 insertions(+), 15 deletions(-)
diff --git a/arch/Kconfig b/arch/Kconfig
index 8e0d665c8d53..3234b73d4626 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -596,21 +596,6 @@ config CC_STACKPROTECTOR_AUTO
endchoice
-config LD_DEAD_CODE_DATA_ELIMINATION
- bool
- help
- Select this if the architecture wants to do dead code and
- data elimination with the linker by compiling with
- -ffunction-sections -fdata-sections and linking with
- --gc-sections.
-
- This requires that the arch annotates or otherwise protects
- its external entry points from being discarded. Linker scripts
- must also merge .text.*, .data.*, and .bss.* correctly into
- output sections. Care must be taken not to pull in unrelated
- sections (e.g., '.text.init'). Typically '.' in section names
- is used to distinguish them from label names / C identifiers.
-
config HAVE_ARCH_WITHIN_STACK_FRAMES
bool
help
diff --git a/init/Kconfig b/init/Kconfig
index f013afc74b11..1706d963766b 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -1038,6 +1038,33 @@ config CC_OPTIMIZE_FOR_SIZE
endchoice
+config HAVE_LD_DEAD_CODE_DATA_ELIMINATION
+ bool
+ help
+ This requires that the arch annotates or otherwise protects
+ its external entry points from being discarded. Linker scripts
+ must also merge .text.*, .data.*, and .bss.* correctly into
+ output sections. Care must be taken not to pull in unrelated
+ sections (e.g., '.text.init'). Typically '.' in section names
+ is used to distinguish them from label names / C identifiers.
+
+config LD_DEAD_CODE_DATA_ELIMINATION
+ bool "Dead code and data elimination (EXPERIMENTAL)"
+ depends on HAVE_LD_DEAD_CODE_DATA_ELIMINATION
+ depends on EXPERT
+ help
+ Select this if the architecture wants to do dead code and
+ data elimination with the linker by compiling with
+ -ffunction-sections -fdata-sections, and linking with
+ --gc-sections.
+
+ This can reduce on disk and in-memory size of the kernel
+ code and static data, particularly for small configs and
+ on small systems. This has the possibility of introducing
+ silently broken kernel if the required annotations are not
+ present. This option is not well tested yet, so use at your
+ own risk.
+
config SYSCTL
bool
--
2.17.0
^ permalink raw reply related
* [PATCH v2 2/4] kbuild: LD_DEAD_CODE_DATA_ELIMINATION no -ffunction-sections/-fdata-sections for module build
From: Nicholas Piggin @ 2018-05-09 12:59 UTC (permalink / raw)
To: Masahiro Yamada; +Cc: Nicholas Piggin, linux-arch, linux-kbuild, linuxppc-dev
In-Reply-To: <20180509130001.24276-1-npiggin@gmail.com>
Modules do not tend to cope with -ffunction-sections, even though they
do not link with -gc-sections. It may be possible for unused symbols to
be trimmed from modules, but in general that would take much more work
in architecture module linker scripts.
For now, enable these only for kernel build.
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
Makefile | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/Makefile b/Makefile
index 619a85ad716b..9dcbfb353f81 100644
--- a/Makefile
+++ b/Makefile
@@ -799,8 +799,8 @@ KBUILD_CFLAGS += $(call cc-option, -fno-inline-functions-called-once)
endif
ifdef CONFIG_LD_DEAD_CODE_DATA_ELIMINATION
-KBUILD_CFLAGS += $(call cc-option,-ffunction-sections,)
-KBUILD_CFLAGS += $(call cc-option,-fdata-sections,)
+KBUILD_CFLAGS_KERNEL += $(call cc-option,-ffunction-sections,)
+KBUILD_CFLAGS_KERNEL += $(call cc-option,-fdata-sections,)
endif
# arch Makefile may override CC so keep this after arch Makefile is included
--
2.17.0
^ permalink raw reply related
* [PATCH v2 1/4] kbuild: Fix asm-generic/vmlinux.lds.h for LD_DEAD_CODE_DATA_ELIMINATION
From: Nicholas Piggin @ 2018-05-09 12:59 UTC (permalink / raw)
To: Masahiro Yamada; +Cc: Nicholas Piggin, linux-arch, linux-kbuild, linuxppc-dev
In-Reply-To: <20180509130001.24276-1-npiggin@gmail.com>
KEEP more tables, and add the function/data section wildcard to more
section selections.
This is a little ad-hoc at the moment, but kernel code should be moved
to consistently use .text..x (note: double dots) for explicit sections
and all references to it in the linker script can be made with
TEXT_MAIN, and similarly for other sections.
For now, let's see if major architectures move to enabling this option
then we can do some refactoring passes. Otherwise if it remains unused
or superseded by LTO, this may not be required.
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
include/asm-generic/vmlinux.lds.h | 49 ++++++++++++++++++-------------
1 file changed, 29 insertions(+), 20 deletions(-)
diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index af240573e482..9efb82a9a528 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -64,15 +64,24 @@
* generates .data.identifier sections, which need to be pulled in with
* .data. We don't want to pull in .data..other sections, which Linux
* has defined. Same for text and bss.
+ *
+ * RODATA_MAIN is not used because existing code already defines .rodata.x
+ * sections to be brought in with rodata.
*/
#ifdef CONFIG_LD_DEAD_CODE_DATA_ELIMINATION
#define TEXT_MAIN .text .text.[0-9a-zA-Z_]*
#define DATA_MAIN .data .data.[0-9a-zA-Z_]*
+#define SDATA_MAIN .sdata .sdata.[0-9a-zA-Z_]*
+#define RODATA_MAIN .rodata .rodata.[0-9a-zA-Z_]*
#define BSS_MAIN .bss .bss.[0-9a-zA-Z_]*
+#define SBSS_MAIN .sbss .sbss.[0-9a-zA-Z_]*
#else
#define TEXT_MAIN .text
#define DATA_MAIN .data
+#define SDATA_MAIN .sdata
+#define RODATA_MAIN .rodata
#define BSS_MAIN .bss
+#define SBSS_MAIN .sbss
#endif
/*
@@ -105,7 +114,7 @@
#ifdef CONFIG_FTRACE_MCOUNT_RECORD
#define MCOUNT_REC() . = ALIGN(8); \
VMLINUX_SYMBOL(__start_mcount_loc) = .; \
- *(__mcount_loc) \
+ KEEP(*(__mcount_loc)) \
VMLINUX_SYMBOL(__stop_mcount_loc) = .;
#else
#define MCOUNT_REC()
@@ -113,7 +122,7 @@
#ifdef CONFIG_TRACE_BRANCH_PROFILING
#define LIKELY_PROFILE() VMLINUX_SYMBOL(__start_annotated_branch_profile) = .; \
- *(_ftrace_annotated_branch) \
+ KEEP(*(_ftrace_annotated_branch)) \
VMLINUX_SYMBOL(__stop_annotated_branch_profile) = .;
#else
#define LIKELY_PROFILE()
@@ -121,7 +130,7 @@
#ifdef CONFIG_PROFILE_ALL_BRANCHES
#define BRANCH_PROFILE() VMLINUX_SYMBOL(__start_branch_profile) = .; \
- *(_ftrace_branch) \
+ KEEP(*(_ftrace_branch)) \
VMLINUX_SYMBOL(__stop_branch_profile) = .;
#else
#define BRANCH_PROFILE()
@@ -238,8 +247,8 @@
*(DATA_MAIN) \
*(.ref.data) \
*(.data..shared_aligned) /* percpu related */ \
- MEM_KEEP(init.data) \
- MEM_KEEP(exit.data) \
+ MEM_KEEP(init.data*) \
+ MEM_KEEP(exit.data*) \
*(.data.unlikely) \
VMLINUX_SYMBOL(__start_once) = .; \
*(.data.once) \
@@ -289,8 +298,8 @@
VMLINUX_SYMBOL(__start_init_task) = .; \
VMLINUX_SYMBOL(init_thread_union) = .; \
VMLINUX_SYMBOL(init_stack) = .; \
- *(.data..init_task) \
- *(.data..init_thread_info) \
+ KEEP(*(.data..init_task)) \
+ KEEP(*(.data..init_thread_info)) \
. = VMLINUX_SYMBOL(__start_init_task) + THREAD_SIZE; \
VMLINUX_SYMBOL(__end_init_task) = .;
@@ -487,8 +496,8 @@
*(.text.hot TEXT_MAIN .text.fixup .text.unlikely) \
*(.text..refcount) \
*(.ref.text) \
- MEM_KEEP(init.text) \
- MEM_KEEP(exit.text) \
+ MEM_KEEP(init.text*) \
+ MEM_KEEP(exit.text*) \
/* sched.text is aling to function alignment to secure we have same
@@ -538,7 +547,7 @@
VMLINUX_SYMBOL(__softirqentry_text_end) = .;
/* Section used for early init (in .S files) */
-#define HEAD_TEXT *(.head.text)
+#define HEAD_TEXT KEEP(*(.head.text))
#define HEAD_TEXT_SECTION \
.head.text : AT(ADDR(.head.text) - LOAD_OFFSET) { \
@@ -579,11 +588,11 @@
/* init and exit section handling */
#define INIT_DATA \
KEEP(*(SORT(___kentry+*))) \
- *(.init.data) \
- MEM_DISCARD(init.data) \
+ *(.init.data init.data.*) \
+ MEM_DISCARD(init.data*) \
KERNEL_CTORS() \
MCOUNT_REC() \
- *(.init.rodata) \
+ *(.init.rodata .init.rodata.*) \
FTRACE_EVENTS() \
TRACE_SYSCALLS() \
KPROBE_BLACKLIST() \
@@ -602,16 +611,16 @@
EARLYCON_TABLE()
#define INIT_TEXT \
- *(.init.text) \
+ *(.init.text .init.text.*) \
*(.text.startup) \
- MEM_DISCARD(init.text)
+ MEM_DISCARD(init.text*)
#define EXIT_DATA \
- *(.exit.data) \
+ *(.exit.data .exit.data.*) \
*(.fini_array) \
*(.dtors) \
- MEM_DISCARD(exit.data) \
- MEM_DISCARD(exit.rodata)
+ MEM_DISCARD(exit.data*) \
+ MEM_DISCARD(exit.rodata*)
#define EXIT_TEXT \
*(.exit.text) \
@@ -629,7 +638,7 @@
. = ALIGN(sbss_align); \
.sbss : AT(ADDR(.sbss) - LOAD_OFFSET) { \
*(.dynsbss) \
- *(.sbss) \
+ *(SBSS_MAIN) \
*(.scommon) \
}
@@ -754,7 +763,7 @@
#define NOTES \
.notes : AT(ADDR(.notes) - LOAD_OFFSET) { \
VMLINUX_SYMBOL(__start_notes) = .; \
- *(.note.*) \
+ KEEP(*(.note.*)) \
VMLINUX_SYMBOL(__stop_notes) = .; \
}
--
2.17.0
^ permalink raw reply related
* [PATCH v2 0/4] LD_DEAD_CODE_DATA_ELIMINATION fixes and enabling for powerpc
From: Nicholas Piggin @ 2018-05-09 12:59 UTC (permalink / raw)
To: Masahiro Yamada; +Cc: Nicholas Piggin, linux-arch, linux-kbuild, linuxppc-dev
v2 changes only patch 4, updates the changelog with ppc32 numbers and
Tested-by tags, and also use TEXT_MAIN in the linker script as
Christophe suggested.
Thanks,
Nick
Nicholas Piggin (4):
kbuild: Fix asm-generic/vmlinux.lds.h for
LD_DEAD_CODE_DATA_ELIMINATION
kbuild: LD_DEAD_CODE_DATA_ELIMINATION no
-ffunction-sections/-fdata-sections for module build
kbuild: Allow LD_DEAD_CODE_DATA_ELIMINATION to be selectable if
enabled
powerpc: Allow LD_DEAD_CODE_DATA_ELIMINATION to be selected
Makefile | 4 +--
arch/Kconfig | 15 ----------
arch/powerpc/Kconfig | 1 +
arch/powerpc/kernel/vmlinux.lds.S | 22 +++++++-------
include/asm-generic/vmlinux.lds.h | 49 ++++++++++++++++++-------------
init/Kconfig | 27 +++++++++++++++++
6 files changed, 70 insertions(+), 48 deletions(-)
--
2.17.0
^ permalink raw reply
* Re: [PATCH] powerpc/64s/radix: reset mm_cpumask for single thread process when possible
From: Nicholas Piggin @ 2018-05-09 11:53 UTC (permalink / raw)
To: Benjamin Herrenschmidt; +Cc: linuxppc-dev
In-Reply-To: <0c3d18301bc3eb86da197ecefb22aec5c6f70a57.camel@au1.ibm.com>
On Wed, 09 May 2018 18:31:55 +1000
Benjamin Herrenschmidt <benh@au1.ibm.com> wrote:
> On Wed, 2018-05-09 at 16:56 +1000, Nicholas Piggin wrote:
> > When a single-threaded process has a non-local mm_cpumask and requires
> > a full PID tlbie invalidation, use that as an opportunity to reset the
> > cpumask back to the current CPU we're running on.
> >
> > No other thread can concurrently switch to this mm, because it must
> > have had a reference on mm_users before it could use_mm. mm_users can
> > be asynchronously incremented e.g., by mmget_not_zero, but those users
> > must not be doing use_mm.
>
> What do you mean ? I don't fully understand how this isn't racy with
> another thread being created, switching to that mm, and then having its
> bit cleared by us ?
We are the single thread, so nothing else can call clone on our mm.
> Also why not use_mm ? what prevents it ?
I'm not sure if it has an assertion (it probably should), but existing
convention. Such things would be broken there are already other
architectures doing similar things with TLB flushing for example, alpha,
ia64, mips, sh... sparc does something very similar resetting its cpumask
by the look.
I grepped mmget_not_zero callers too, couldn't see any obvious use_mm
cases.
Thanks,
Nick
^ permalink raw reply
* Re: [PATCH 1/2] powerpc/64s/radix: do not flush TLB when relaxing access
From: Nicholas Piggin @ 2018-05-09 11:39 UTC (permalink / raw)
To: Balbir Singh
Cc: open list:LINUX FOR POWERPC (32-BIT AND 64-BIT), Alistair Popple
In-Reply-To: <CAKTCnznPYt-8SdP81vnpev5c5Cu9a95RoSCP7WaF+LxnrZv2Ng@mail.gmail.com>
On Wed, 9 May 2018 18:27:07 +1000
Balbir Singh <bsingharora@gmail.com> wrote:
> On Wed, May 9, 2018 at 5:43 PM, Nicholas Piggin <npiggin@gmail.com> wrote:
> > On Wed, 9 May 2018 17:07:47 +1000
> > Balbir Singh <bsingharora@gmail.com> wrote:
> >
> >> On Wed, May 9, 2018 at 4:51 PM, Nicholas Piggin <npiggin@gmail.com> wrote:
> >> > Radix flushes the TLB when updating ptes to increase permissiveness
> >> > of protection (increase access authority). Book3S does not require
> >> > TLB flushing in this case, and it is not done on hash. This patch
> >> > avoids the flush for radix.
> >> >
> >> > From Power ISA v3.0B, p.1090:
> >> >
> >> > Setting a Reference or Change Bit or Upgrading Access Authority
> >> > (PTE Subject to Atomic Hardware Updates)
> >> >
> >> > If the only change being made to a valid PTE that is subject to
> >> > atomic hardware updates is to set the Reference or Change bit to 1
> >> > or to add access authorities, a simpler sequence suffices because
> >> > the translation hardware will refetch the PTE if an access is
> >> > attempted for which the only problems were reference and/or change
> >> > bits needing to be set or insufficient access authority.
> >> >
> >> > Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> >> > ---
> >> > arch/powerpc/mm/pgtable-book3s64.c | 1 -
> >> > arch/powerpc/mm/pgtable.c | 3 ++-
> >> > 2 files changed, 2 insertions(+), 2 deletions(-)
> >> >
> >> > diff --git a/arch/powerpc/mm/pgtable-book3s64.c b/arch/powerpc/mm/pgtable-book3s64.c
> >> > index 518518fb7c45..6e991eaccab4 100644
> >> > --- a/arch/powerpc/mm/pgtable-book3s64.c
> >> > +++ b/arch/powerpc/mm/pgtable-book3s64.c
> >> > @@ -40,7 +40,6 @@ int pmdp_set_access_flags(struct vm_area_struct *vma, unsigned long address,
> >> > if (changed) {
> >> > __ptep_set_access_flags(vma->vm_mm, pmdp_ptep(pmdp),
> >> > pmd_pte(entry), address);
> >> > - flush_pmd_tlb_range(vma, address, address + HPAGE_PMD_SIZE);
> >>
> >> The comment states that this can be used for missing execution
> >> permissions as well. I am not convinced we can skip a flush in those
> >> cases
> >
> > Why not? Execute is part of the access authority. And they're already no
> > ops on hash. What am I missing?
>
> I have not reviewed the hash code, but if relaxing access means
> allowing the code to provide execute permission,
Yes, adding RWX or RC bits would qualify at least.
> won't this result in
> spurious faults?
Well it gets called as part of page faults, which means the MMU will
already reload the pte when the access is retried, as per the ISA.
A few paths actually don't call it from page fault paths, but the
cost of a superfluous fault occasionally versus always doing a tlbie
makes this the wrong thing to do. x86 and hash are the same, they
don't flush here. Hash mode should behave basically the same way in
terms of what it stores in the TLB and whether it would take more
faults. Radix is the newcomer so it should match hash without some
reason not to.
> A simple test might be to run a JIT workload and see
> if the number of faults go up with and without the patch?
What pattern of accesses are you worried about? Why is execute
different from write, for example?
Thanks,
Nick
^ permalink raw reply
* Re: [PATCH] powerpc/64s/radix: reset mm_cpumask for single thread process when possible
From: Nicholas Piggin @ 2018-05-09 11:22 UTC (permalink / raw)
To: Balbir Singh; +Cc: open list:LINUX FOR POWERPC (32-BIT AND 64-BIT)
In-Reply-To: <CAKTCnznBdZV5fBz4kQHuiiAYREB-CNKNobe2+R347r4=xOGO9Q@mail.gmail.com>
On Wed, 9 May 2018 18:23:48 +1000
Balbir Singh <bsingharora@gmail.com> wrote:
> On Wed, May 9, 2018 at 4:56 PM, Nicholas Piggin <npiggin@gmail.com> wrote:
> > When a single-threaded process has a non-local mm_cpumask and requires
> > a full PID tlbie invalidation, use that as an opportunity to reset the
> > cpumask back to the current CPU we're running on.
> >
> > No other thread can concurrently switch to this mm, because it must
> > have had a reference on mm_users before it could use_mm. mm_users can
> > be asynchronously incremented e.g., by mmget_not_zero, but those users
> > must not be doing use_mm.
> >
> > Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> > ---
> > arch/powerpc/include/asm/mmu_context.h | 19 +++++++++
> > arch/powerpc/include/asm/tlb.h | 8 ++++
> > arch/powerpc/mm/tlb-radix.c | 57 +++++++++++++++++++-------
> > 3 files changed, 70 insertions(+), 14 deletions(-)
> >
> > diff --git a/arch/powerpc/include/asm/mmu_context.h b/arch/powerpc/include/asm/mmu_context.h
> > index 1835ca1505d6..df12a994529f 100644
> > --- a/arch/powerpc/include/asm/mmu_context.h
> > +++ b/arch/powerpc/include/asm/mmu_context.h
> > @@ -6,6 +6,7 @@
> > #include <linux/kernel.h>
> > #include <linux/mm.h>
> > #include <linux/sched.h>
> > +#include <linux/sched/mm.h>
> > #include <linux/spinlock.h>
> > #include <asm/mmu.h>
> > #include <asm/cputable.h>
> > @@ -201,6 +202,24 @@ static inline void activate_mm(struct mm_struct *prev, struct mm_struct *next)
> > static inline void enter_lazy_tlb(struct mm_struct *mm,
> > struct task_struct *tsk)
> > {
> > +#ifdef CONFIG_PPC_BOOK3S_64
> > + /*
> > + * Under radix, we do not want to keep lazy PIDs around because
> > + * even if the CPU does not access userspace, it can still bring
> > + * in translations through speculation and prefetching.
> > + *
> > + * Switching away here allows us to trim back the mm_cpumask in
> > + * cases where we know the process is not running on some CPUs
> > + * (see mm/tlb-radix.c).
> > + */
> > + if (radix_enabled() && mm != &init_mm) {
> > + mmgrab(&init_mm);
>
> This is called when a kernel thread decides to unuse a mm, I agree switching
> to init_mm as active_mm is reasonable thing to do.
We lose lazy PIDR switching. Should probably ifdef CONFIG_SMP it, and
possibly one day we could look into having an arch notification for
scheduler migrating tasks so we could restore the optimisation.
For now I could not measure a cost, but it will be a few cycles.
>
> > + tsk->active_mm = &init_mm;
>
> Are we called with irqs disabled? Don't we need it below?
Hmm, yeah you might be right.
> > + switch_mm_irqs_off(mm, tsk->active_mm, tsk);
> > + mmdrop(mm);
> > + }
> > +#endif
> > +
> > /* 64-bit Book3E keeps track of current PGD in the PACA */
> > #ifdef CONFIG_PPC_BOOK3E_64
> > get_paca()->pgd = NULL;
> > diff --git a/arch/powerpc/include/asm/tlb.h b/arch/powerpc/include/asm/tlb.h
> > index a7eabff27a0f..006fce98c403 100644
> > --- a/arch/powerpc/include/asm/tlb.h
> > +++ b/arch/powerpc/include/asm/tlb.h
> > @@ -76,6 +76,14 @@ static inline int mm_is_thread_local(struct mm_struct *mm)
> > return false;
> > return cpumask_test_cpu(smp_processor_id(), mm_cpumask(mm));
> > }
> > +static inline void mm_reset_thread_local(struct mm_struct *mm)
> reset_thread_local --> reset_to_thread_local?
>
> > +{
> > + WARN_ON(atomic_read(&mm->context.copros) > 0);
>
> Can we put this under DEBUG_VM, VM_WARN_ON?
Yeah we could, I worry nobody tests with them on...
>
> > + WARN_ON(!(atomic_read(&mm->mm_users) == 1 && current->mm == mm));
>
>
>
> > + atomic_set(&mm->context.active_cpus, 1);
> > + cpumask_clear(mm_cpumask(mm));
> > + cpumask_set_cpu(smp_processor_id(), mm_cpumask(mm));
> > +}
> > #else /* CONFIG_PPC_BOOK3S_64 */
> > static inline int mm_is_thread_local(struct mm_struct *mm)
> > {
> > diff --git a/arch/powerpc/mm/tlb-radix.c b/arch/powerpc/mm/tlb-radix.c
> > index 5ac3206c51cc..d5593a78702a 100644
> > --- a/arch/powerpc/mm/tlb-radix.c
> > +++ b/arch/powerpc/mm/tlb-radix.c
> > @@ -504,6 +504,15 @@ void radix__local_flush_tlb_page(struct vm_area_struct *vma, unsigned long vmadd
> > }
> > EXPORT_SYMBOL(radix__local_flush_tlb_page);
> >
> > +static bool mm_is_singlethreaded(struct mm_struct *mm)
> > +{
>
> mm_tlb_context_is_local?
It's not the same thing as _local. _local means only local TLBs.
_singlethreaded means only this thread can access the mappings.
> We should also skip init_mm from these checks
I don't think we should see init_mm here ever -- it has no user
mappings so won't get this user tlb flushing.
>
> > + if (atomic_read(&mm->context.copros) > 0)
> > + return false;
> > + if (atomic_read(&mm->mm_users) == 1 && current->mm == mm)
> > + return true;
> > + return false;
> > +}
> > +
> > static bool mm_needs_flush_escalation(struct mm_struct *mm)
> > {
> > /*
> > @@ -511,7 +520,9 @@ static bool mm_needs_flush_escalation(struct mm_struct *mm)
> > * caching PTEs and not flushing them properly when
> > * RIC = 0 for a PID/LPID invalidate
> > */
> > - return atomic_read(&mm->context.copros) != 0;
> > + if (atomic_read(&mm->context.copros) > 0)
> > + return true;
> > + return false;
> > }
> >
> > #ifdef CONFIG_SMP
> > @@ -525,12 +536,17 @@ void radix__flush_tlb_mm(struct mm_struct *mm)
> >
> > preempt_disable();
> > if (!mm_is_thread_local(mm)) {
> > - if (mm_needs_flush_escalation(mm))
> > + if (mm_is_singlethreaded(mm)) {
> > _tlbie_pid(pid, RIC_FLUSH_ALL);
> > - else
> > + mm_reset_thread_local(mm);
> > + } else if (mm_needs_flush_escalation(mm)) {
> > + _tlbie_pid(pid, RIC_FLUSH_ALL);
> > + } else {
> > _tlbie_pid(pid, RIC_FLUSH_TLB);
> > - } else
> > + }
> > + } else {
> > _tlbiel_pid(pid, RIC_FLUSH_TLB);
> > + }
> > preempt_enable();
> > }
> > EXPORT_SYMBOL(radix__flush_tlb_mm);
> > @@ -544,10 +560,13 @@ void radix__flush_all_mm(struct mm_struct *mm)
> > return;
> >
> > preempt_disable();
> > - if (!mm_is_thread_local(mm))
> > + if (!mm_is_thread_local(mm)) {
> > _tlbie_pid(pid, RIC_FLUSH_ALL);
> > - else
> > + if (mm_is_singlethreaded(mm))
> > + mm_reset_thread_local(mm);
> > + } else {
> > _tlbiel_pid(pid, RIC_FLUSH_ALL);
> > + }
> > preempt_enable();
> > }
> > EXPORT_SYMBOL(radix__flush_all_mm);
> > @@ -644,10 +663,14 @@ void radix__flush_tlb_range(struct vm_area_struct *vma, unsigned long start,
> > if (local) {
> > _tlbiel_pid(pid, RIC_FLUSH_TLB);
> > } else {
> > - if (mm_needs_flush_escalation(mm))
> > + if (mm_is_singlethreaded(mm)) {
> > + _tlbie_pid(pid, RIC_FLUSH_ALL);
> > + mm_reset_thread_local(mm);
> > + } else if (mm_needs_flush_escalation(mm)) {
> > _tlbie_pid(pid, RIC_FLUSH_ALL);
> > - else
> > + } else {
> > _tlbie_pid(pid, RIC_FLUSH_TLB);
> > + }
> > }
> > } else {
> > bool hflush = false;
> > @@ -802,13 +825,19 @@ static inline void __radix__flush_tlb_range_psize(struct mm_struct *mm,
> > }
> >
> > if (full) {
> > - if (!local && mm_needs_flush_escalation(mm))
> > - also_pwc = true;
> > -
> > - if (local)
> > + if (local) {
> > _tlbiel_pid(pid, also_pwc ? RIC_FLUSH_ALL : RIC_FLUSH_TLB);
> > - else
> > - _tlbie_pid(pid, also_pwc ? RIC_FLUSH_ALL: RIC_FLUSH_TLB);
> > + } else {
> > + if (mm_is_singlethreaded(mm)) {
> > + _tlbie_pid(pid, RIC_FLUSH_ALL);
> > + mm_reset_thread_local(mm);
> > + } else {
> > + if (mm_needs_flush_escalation(mm))
> > + also_pwc = true;
> > +
> > + _tlbie_pid(pid, also_pwc ? RIC_FLUSH_ALL : RIC_FLUSH_TLB);
> > + }
> > + }
> > } else {
> > if (local)
> > _tlbiel_va_range(start, end, pid, page_size, psize, also_pwc);
>
>
> Looks good otherwise
Thanks,
Nick
^ permalink raw reply
* Re: [PATCH v3] ppc64le livepatch: implement reliable stacktrace for newer consistency models
From: Torsten Duwe @ 2018-05-09 10:35 UTC (permalink / raw)
To: Michael Ellerman
Cc: Josh Poimboeuf, Jiri Kosina, linuxppc-dev, linux-kernel,
Nicholas Piggin, live-patching
In-Reply-To: <87sh71zjuy.fsf@concordia.ellerman.id.au>
On Wed, May 09, 2018 at 11:41:09AM +1000, Michael Ellerman wrote:
> Josh Poimboeuf <jpoimboe@redhat.com> writes:
>
> > Generally we refer to it as "the consistency model".
[...]
>
> We use "powerpc" as the prefix.
>
> So I've used:
>
> powerpc/livepatch: Implement reliable stack tracing for the consistency model
Perfect. Thanks!
Torsten
^ permalink raw reply
* Re: [PATCH] powerpc/pseries: hcall_exit tracepoint retval should be signed
From: Ravi Bangoria @ 2018-05-09 9:46 UTC (permalink / raw)
To: Michael Ellerman; +Cc: linuxppc-dev, anton
In-Reply-To: <20180507130355.25115-1-mpe@ellerman.id.au>
On 05/07/2018 06:33 PM, Michael Ellerman wrote:
> The hcall_exit() tracepoint has retval defined as unsigned long. That
> leads to humours results like:
>
> bash-3686 [009] d..2 854.134094: hcall_entry: opcode=24
> bash-3686 [009] d..2 854.134095: hcall_exit: opcode=24 retval=18446744073709551609
>
> It's normal for some hcalls to return negative values, displaying them
> as unsigned isn't very helpful. So change it to signed.
>
> bash-3711 [001] d..2 471.691008: hcall_entry: opcode=24
> bash-3711 [001] d..2 471.691008: hcall_exit: opcode=24 retval=-7
>
> Which can be more easily compared to H_NOT_FOUND in hvcall.h
Tested-by: Ravi Bangoria <ravi.bangoria@linux.ibm.com>
Thank,
Ravi
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox