[PATCH] sparc64: Handle extremely large kernel TSB range flushes sanely.

[PATCH] sparc64: Handle extremely large kernel TSB range flushes sanely.

[David Miller]

If the number of pages we are flushing is more than twice the number
of entries in the TSB, just scan the TSB table for matches rather
than probing each and every page in the range.

Based upon a patch and report by James Clarke.

Signed-off-by: David S. Miller <davem@davemloft.net>
---

James this is the final version I pushed into the tree.

 arch/sparc/mm/tsb.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/arch/sparc/mm/tsb.c b/arch/sparc/mm/tsb.c
index f2b7711..e20fbba 100644
--- a/arch/sparc/mm/tsb.c
+++ b/arch/sparc/mm/tsb.c
@@ -27,6 +27,20 @@ static inline int tag_compare(unsigned long tag, unsigned long vaddr)
 	return (tag = (vaddr >> 22));
 }
 
+static void flush_tsb_kernel_range_scan(unsigned long start, unsigned long end)
+{
+	unsigned long idx;
+
+	for (idx = 0; idx < KERNEL_TSB_NENTRIES; idx++) {
+		struct tsb *ent = &swapper_tsb[idx];
+		unsigned long match = idx << 13;
+
+		match |= (ent->tag << 22);
+		if (match >= start && match < end)
+			ent->tag = (1UL << TSB_TAG_INVALID_BIT);
+	}
+}
+
 /* TSB flushes need only occur on the processor initiating the address
  * space modification, not on each cpu the address space has run on.
  * Only the TLB flush needs that treatment.
@@ -36,6 +50,9 @@ void flush_tsb_kernel_range(unsigned long start, unsigned long end)
 {
 	unsigned long v;
 
+	if ((end - start) >> PAGE_SHIFT >= 2 * KERNEL_TSB_NENTRIES)
+		return flush_tsb_kernel_range_scan(start, end);
+
 	for (v = start; v < end; v += PAGE_SIZE) {
 		unsigned long hash = tsb_hash(v, PAGE_SHIFT,
 					      KERNEL_TSB_NENTRIES);
-- 
2.1.2.532.g19b5d50

Re: [PATCH] sparc64: Handle extremely large kernel TSB range flushes sanely.

[James Clarke]

> On 26 Oct 2016, at 03:44, David Miller <davem@davemloft.net> wrote:
> 
> 
> If the number of pages we are flushing is more than twice the number
> of entries in the TSB, just scan the TSB table for matches rather
> than probing each and every page in the range.
> 
> Based upon a patch and report by James Clarke.
> 
> Signed-off-by: David S. Miller <davem@davemloft.net>
> ---
> 
> James this is the final version I pushed into the tree.

Great, thanks. Any progress on TLB flushing?

James

Re: [PATCH] sparc64: Handle extremely large kernel TSB range flushes sanely.

[David Miller]

From: James Clarke <jrtc27@jrtc27.com>
Date: Wed, 26 Oct 2016 09:28:05 +0100

> Any progress on TLB flushing?

I was half-way through an implementation when I noticed that
hypervisor TLB flush handler relative branch bug I posted the
fix for last night.

I'll keep plugging away at it today.

Re: [PATCH] sparc64: Handle extremely large kernel TSB range flushes sanely.

[James Clarke]

> On 26 Oct 2016, at 16:54, David Miller <davem@davemloft.net> wrote:
> 
> From: James Clarke <jrtc27@jrtc27.com>
> Date: Wed, 26 Oct 2016 09:28:05 +0100
> 
>> Any progress on TLB flushing?
> 
> I was half-way through an implementation when I noticed that
> hypervisor TLB flush handler relative branch bug I posted the
> fix for last night.

Yep, I saw that. Looks like you forgot to update the comment on
__hypervisor_flush_tlb_pending; it still says 16 insns rather than 27.

> I'll keep plugging away at it today.

Great; let me know if you need a guinea pig, as it’s pretty easy for me to
reproduce.

Thanks,
James

Re: [PATCH] sparc64: Handle extremely large kernel TSB range flushes sanely.

[David Miller]

RnJvbTogSmFtZXMgQ2xhcmtlIDxqcnRjMjdAanJ0YzI3LmNvbT4NCkRhdGU6IFdlZCwgMjYgT2N0
IDIwMTYgMTc6NTg6MTYgKzAxMDANCg0KPj4gT24gMjYgT2N0IDIwMTYsIGF0IDE2OjU0LCBEYXZp
ZCBNaWxsZXIgPGRhdmVtQGRhdmVtbG9mdC5uZXQ+IHdyb3RlOg0KPj4gDQo+PiBGcm9tOiBKYW1l
cyBDbGFya2UgPGpydGMyN0BqcnRjMjcuY29tPg0KPj4gRGF0ZTogV2VkLCAyNiBPY3QgMjAxNiAw
OToyODowNSArMDEwMA0KPj4gDQo+Pj4gQW55IHByb2dyZXNzIG9uIFRMQiBmbHVzaGluZz8NCj4+
IA0KPj4gSSB3YXMgaGFsZi13YXkgdGhyb3VnaCBhbiBpbXBsZW1lbnRhdGlvbiB3aGVuIEkgbm90
aWNlZCB0aGF0DQo+PiBoeXBlcnZpc29yIFRMQiBmbHVzaCBoYW5kbGVyIHJlbGF0aXZlIGJyYW5j
aCBidWcgSSBwb3N0ZWQgdGhlDQo+PiBmaXggZm9yIGxhc3QgbmlnaHQuDQo+IA0KPiBZZXAsIEkg
c2F3IHRoYXQuIExvb2tzIGxpa2UgeW91IGZvcmdvdCB0byB1cGRhdGUgdGhlIGNvbW1lbnQgb24N
Cj4gX19oeXBlcnZpc29yX2ZsdXNoX3RsYl9wZW5kaW5nOyBpdCBzdGlsbCBzYXlzIDE2IGluc25z
IHJhdGhlciB0aGFuIDI3Lg0KDQpGaXhlZCwgdGhhbmtzLg0KDQpBbmQgbm93IEkgbm90aWNlZCB0
aGF0IHRoZSBjcm9zcy1jYWxsIGh5cGVydmlzb3IgdGxiIGZsdXNoIGFzc2VtYmxlcg0KaGFzIHRo
ZSBidWcgYW5kIG5lZWRzIHRvIGJlIGZpeGVkIHRvby4uLg0KDQo+PiBJJ2xsIGtlZXAgcGx1Z2dp
bmcgYXdheSBhdCBpdCB0b2RheS4NCj4gDQo+IEdyZWF0OyBsZXQgbWUga25vdyBpZiB5b3UgbmVl
ZCBhIGd1aW5lYSBwaWcsIGFzIGl0onMgcHJldHR5IGVhc3kgZm9yIG1lIHRvDQo+IHJlcHJvZHVj
ZS4NCg0KV2lsbCBkbywgd2hhdCBraW5kIG9mIGNwdXMgZG8geW91IGhhdmU/DQo

Re: [PATCH] sparc64: Handle extremely large kernel TSB range flushes sanely.

[James Clarke]

> On 26 Oct 2016, at 18:09, David Miller <davem@davemloft.net> wrote:
> 
> From: James Clarke <jrtc27@jrtc27.com>
> Date: Wed, 26 Oct 2016 17:58:16 +0100
> 
>>> On 26 Oct 2016, at 16:54, David Miller <davem@davemloft.net> wrote:
>>> 
>>> From: James Clarke <jrtc27@jrtc27.com>
>>> Date: Wed, 26 Oct 2016 09:28:05 +0100
>>> 
>>>> Any progress on TLB flushing?
>>> 
>>> I'll keep plugging away at it today.
>> 
>> Great; let me know if you need a guinea pig, as it’s pretty easy for me to
>> reproduce.
> 
> Will do, what kind of cpus do you have?

* UltraSparc T5 (Niagara5)
* UltraSparc T1 (Niagara)
* UltraSPARC IIIi

The IIIi seems to be down at the moment though.

James

Re: [PATCH] sparc64: Handle extremely large kernel TSB range flushes sanely.

[David Miller]

RnJvbTogSmFtZXMgQ2xhcmtlIDxqcnRjMjdAanJ0YzI3LmNvbT4NCkRhdGU6IFdlZCwgMjYgT2N0
IDIwMTYgMTg6MjE6MDYgKzAxMDANCg0KPj4gT24gMjYgT2N0IDIwMTYsIGF0IDE4OjA5LCBEYXZp
ZCBNaWxsZXIgPGRhdmVtQGRhdmVtbG9mdC5uZXQ+IHdyb3RlOg0KPj4gDQo+PiBGcm9tOiBKYW1l
cyBDbGFya2UgPGpydGMyN0BqcnRjMjcuY29tPg0KPj4gRGF0ZTogV2VkLCAyNiBPY3QgMjAxNiAx
Nzo1ODoxNiArMDEwMA0KPj4gDQo+Pj4+IE9uIDI2IE9jdCAyMDE2LCBhdCAxNjo1NCwgRGF2aWQg
TWlsbGVyIDxkYXZlbUBkYXZlbWxvZnQubmV0PiB3cm90ZToNCj4+Pj4gDQo+Pj4+IEZyb206IEph
bWVzIENsYXJrZSA8anJ0YzI3QGpydGMyNy5jb20+DQo+Pj4+IERhdGU6IFdlZCwgMjYgT2N0IDIw
MTYgMDk6Mjg6MDUgKzAxMDANCj4+Pj4gDQo+Pj4+PiBBbnkgcHJvZ3Jlc3Mgb24gVExCIGZsdXNo
aW5nPw0KPj4+PiANCj4+Pj4gSSdsbCBrZWVwIHBsdWdnaW5nIGF3YXkgYXQgaXQgdG9kYXkuDQo+
Pj4gDQo+Pj4gR3JlYXQ7IGxldCBtZSBrbm93IGlmIHlvdSBuZWVkIGEgZ3VpbmVhIHBpZywgYXMg
aXSicyBwcmV0dHkgZWFzeSBmb3IgbWUgdG8NCj4+PiByZXByb2R1Y2UuDQo+PiANCj4+IFdpbGwg
ZG8sIHdoYXQga2luZCBvZiBjcHVzIGRvIHlvdSBoYXZlPw0KPiANCj4gKiBVbHRyYVNwYXJjIFQ1
IChOaWFnYXJhNSkNCj4gKiBVbHRyYVNwYXJjIFQxIChOaWFnYXJhKQ0KPiAqIFVsdHJhU1BBUkMg
SUlJaQ0KPiANCj4gVGhlIElJSWkgc2VlbXMgdG8gYmUgZG93biBhdCB0aGUgbW9tZW50IHRob3Vn
aC4NCg0KSmFtZXMsIGhlcmUgaXMgd2hhdCBJIGhhdmUgc28gZmFyLiAgSSBvbmx5IGdhdmUgaXQg
YSBicmllZiB0ZXN0aW5nIG9uDQpzdW40diwgc28gbm8gZ3VhcmFudGVlcyBmb3IgdGhlIHN1bjR1
IGNhc2VzLiAgVGhpcyBpcyBhZ2FpbnN0IHRoZQ0KY3VycmVudCBzcGFyYyBHSVQgdHJlZS4NCg0K
VGhlIGN1dC1vZmYgaXMgMzIgcGFnZXMsIHdlIGNhbiBkaXNjdXNzIHdoZXRoZXIgdGhhdCdzIGEg
Z29vZCB2YWx1ZQ0KdG8gdXNlIG9yIG5vdC4gIEZXSVcsIHg4Nl82NCBoYXMgc2ltaWxhciBjb2Rl
IGZvciB0aGlzIHNpdHVhdGlvbiBhbmQNCnVzZXMgYSBjdXQtb2ZmIG9mIDMzLiAgUGVyaGFwcyA2
NCBpcyBhIGJldHRlciB2YWx1ZSwgd2hvIGtub3dzLg0KDQpJdCBtaWdodCBldmVuIG1ha2Ugc2Vu
c2UgdG8gdXNlIGEgZGlmZmVyZW50IGN1dC1vZmYgZm9yIHRoZSBoeXBlcnZpc29yDQpjYXNlIHNp
bmNlIHRoZSBoeXBlcnZpc29yIHRyYXAgd2UgaGF2ZSB0byB1c2UgdG8gZG8gdGhlIFRMQiBvcGVy
YXRpb24NCmFkZHMgZXZlbiBtb3JlIGV4cGVuc2UgdG8gZWFjaCBpdGVyYXRpb24gb2YgdGhlIHJh
bmdlIGxvb3AuDQoNClRoZSBwb2xpY3kgaW1wbGVtZW50ZWQgZm9yIGh1Z2UgcmFuZ2UgZmx1c2hl
cyBiZWxvdyBpczoNCg0KMSkgU3BpdGZpcmUgLSBGbHVzaCBhbGwgbm9uLWxvY2tlZCBlbnRyaWVz
LCBieSBoYW5kIHVzaW5nIGRpYWdub3N0aWMNCiAgIFRMQiBhY2Nlc3Nlcy4NCg0KMikgQ2hlZXRh
aCAtIEZsdXNoIGFsbCBub24tbG9ja2VkIGVudHJpZXMgdXNpbmcgImZsdXNoIGFsbCIgb3BlcmF0
aW9uLg0KDQozKSBzdW40di9oeXBlcnZpc29yIC0gRmx1c2ggZW50aXJlIGtlcm5lbCBjb250ZXh0
LCB3aGljaCBkb2VzIG5vdA0KICAgcmVtb3ZlIGxvY2tlZCBvciAicGVybWFuZW50IiBlbnRyaWVz
Lg0KDQpBbnl3YXlzLCBsZXQgbWUga25vdyBob3cgaXQgZ29lcy4NCg0KZGlmZiAtLWdpdCBhL2Fy
Y2gvc3BhcmMvbW0vdWx0cmEuUyBiL2FyY2gvc3BhcmMvbW0vdWx0cmEuUw0KaW5kZXggMGZhMmU2
Mi4uNWQyZmQ2YyAxMDA2NDQNCi0tLSBhL2FyY2gvc3BhcmMvbW0vdWx0cmEuUw0KKysrIGIvYXJj
aC9zcGFyYy9tbS91bHRyYS5TDQpAQCAtMTEzLDEyICsxMTMsMTQgQEAgX19mbHVzaF90bGJfcGVu
ZGluZzoJLyogMjcgaW5zbnMgKi8NCiANCiAJLmFsaWduCQkzMg0KIAkuZ2xvYmwJCV9fZmx1c2hf
dGxiX2tlcm5lbF9yYW5nZQ0KLV9fZmx1c2hfdGxiX2tlcm5lbF9yYW5nZToJLyogMTkgaW5zbnMg
Ki8NCitfX2ZsdXNoX3RsYl9rZXJuZWxfcmFuZ2U6CS8qIDMxIGluc25zICovDQogCS8qICVvMD1z
dGFydCwgJW8xPWVuZCAqLw0KIAljbXAJCSVvMCwgJW8xDQogCWJlLHBuCQkleGNjLCAyZg0KKwkg
c3ViCQklbzEsICVvMCwgJW8zDQorCXNybHgJCSVvMywgMTgsICVvNA0KKwlicm56LHBuCQklbzQs
IF9fc3BpdGZpcmVfZmx1c2hfdGxiX2tlcm5lbF9yYW5nZV9zbG93DQogCSBzZXRoaQkJJWhpKFBB
R0VfU0laRSksICVvNA0KLQlzdWIJCSVvMSwgJW8wLCAlbzMNCiAJc3ViCQklbzMsICVvNCwgJW8z
DQogCW9yCQklbzAsIDB4MjAsICVvMAkJISBOdWNsZXVzDQogMToJc3R4YQkJJWcwLCBbJW8wICsg
JW8zXSBBU0lfRE1NVV9ERU1BUA0KQEAgLTEzNCw2ICsxMzYsMzggQEAgX19mbHVzaF90bGJfa2Vy
bmVsX3JhbmdlOgkvKiAxOSBpbnNucyAqLw0KIAlub3ANCiAJbm9wDQogCW5vcA0KKwlub3ANCisJ
bm9wDQorCW5vcA0KKwlub3ANCisJbm9wDQorCW5vcA0KKwlub3ANCisJbm9wDQorCW5vcA0KKwlu
b3ANCisNCitfX3NwaXRmaXJlX2ZsdXNoX3RsYl9rZXJuZWxfcmFuZ2Vfc2xvdzoNCisJbW92CQk2
MyAqIDgsICVvNA0KKzE6CWxkeGEJCVslbzRdIEFTSV9JVExCX0RBVEFfQUNDRVNTLCAlbzMNCisJ
YW5kY2MJCSVvMywgMHg0MCwgJWcwCQkJLyogX1BBR0VfTF80VSAqLw0KKwlibmUscG4JCSV4Y2Ms
IDJmDQorCSBtb3YJCVRMQl9UQUdfQUNDRVNTLCAlbzMNCisJc3R4YQkJJWcwLCBbJW8zXSBBU0lf
SU1NVQ0KKwlzdHhhCQklZzAsIFslbzRdIEFTSV9JVExCX0RBVEFfQUNDRVNTDQorCW1lbWJhcgkJ
I1N5bmMNCisyOglsZHhhCQlbJW80XSBBU0lfRFRMQl9EQVRBX0FDQ0VTUywgJW8zDQorCWFuZGNj
CQklbzMsIDB4NDAsICVnMA0KKwlibmUscG4JCSV4Y2MsIDJmDQorCSBtb3YJCVRMQl9UQUdfQUND
RVNTLCAlbzMNCisJc3R4YQkJJWcwLCBbJW8zXSBBU0lfRE1NVQ0KKwlzdHhhCQklZzAsIFslbzRd
IEFTSV9EVExCX0RBVEFfQUNDRVNTDQorCW1lbWJhcgkJI1N5bmMNCisyOglzdWIJCSVvNCwgOCwg
JW80DQorCWJyZ2V6LHB0CSVvNCwgMWINCisJIG5vcA0KKwlyZXRsDQorCSBub3ANCiANCiBfX3Nw
aXRmaXJlX2ZsdXNoX3RsYl9tbV9zbG93Og0KIAlyZHByCQklcHN0YXRlLCAlZzENCkBAIC0yODgs
NiArMzIyLDQwIEBAIF9fY2hlZXRhaF9mbHVzaF90bGJfcGVuZGluZzoJLyogMjcgaW5zbnMgKi8N
CiAJcmV0bA0KIAkgd3JwcgkJJWc3LCAweDAsICVwc3RhdGUNCiANCitfX2NoZWV0YWhfZmx1c2hf
dGxiX2tlcm5lbF9yYW5nZToJLyogMzEgaW5zbnMgKi8NCisJLyogJW8wPXN0YXJ0LCAlbzE9ZW5k
ICovDQorCWNtcAkJJW8wLCAlbzENCisJYmUscG4JCSV4Y2MsIDJmDQorCSBzdWIJCSVvMSwgJW8w
LCAlbzMNCisJc3JseAkJJW8zLCAxOCwgJW80DQorCWJybnoscG4JCSVvNCwgM2YNCisJIHNldGhp
CQklaGkoUEFHRV9TSVpFKSwgJW80DQorCXN1YgkJJW8zLCAlbzQsICVvMw0KKwlvcgkJJW8wLCAw
eDIwLCAlbzAJCSEgTnVjbGV1cw0KKzE6CXN0eGEJCSVnMCwgWyVvMCArICVvM10gQVNJX0RNTVVf
REVNQVANCisJc3R4YQkJJWcwLCBbJW8wICsgJW8zXSBBU0lfSU1NVV9ERU1BUA0KKwltZW1iYXIJ
CSNTeW5jDQorCWJybnoscHQJCSVvMywgMWINCisJIHN1YgkJJW8zLCAlbzQsICVvMw0KKzI6CXNl
dGhpCQklaGkoS0VSTkJBU0UpLCAlbzMNCisJZmx1c2gJCSVvMw0KKwlyZXRsDQorCSBub3ANCisz
Ogltb3YJCTB4ODAsICVvNA0KKwlzdHhhCQklZzAsIFslbzRdIEFTSV9ETU1VX0RFTUFQDQorCW1l
bWJhcgkJI1N5bmMNCisJc3R4YQkJJWcwLCBbJW80XSBBU0lfSU1NVV9ERU1BUA0KKwltZW1iYXIJ
CSNTeW5jDQorCXJldGwNCisJIG5vcA0KKwlub3ANCisJbm9wDQorCW5vcA0KKwlub3ANCisJbm9w
DQorCW5vcA0KKwlub3ANCisNCiAjaWZkZWYgRENBQ0hFX0FMSUFTSU5HX1BPU1NJQkxFDQogX19j
aGVldGFoX2ZsdXNoX2RjYWNoZV9wYWdlOiAvKiAxMSBpbnNucyAqLw0KIAlzZXRoaQkJJWhpKFBB
R0VfT0ZGU0VUKSwgJWcxDQpAQCAtMzg4LDEzICs0NTYsMTUgQEAgX19oeXBlcnZpc29yX2ZsdXNo
X3RsYl9wZW5kaW5nOiAvKiAyNyBpbnNucyAqLw0KIAlub3ANCiAJbm9wDQogDQotX19oeXBlcnZp
c29yX2ZsdXNoX3RsYl9rZXJuZWxfcmFuZ2U6IC8qIDE5IGluc25zICovDQorX19oeXBlcnZpc29y
X2ZsdXNoX3RsYl9rZXJuZWxfcmFuZ2U6IC8qIDMxIGluc25zICovDQogCS8qICVvMD1zdGFydCwg
JW8xPWVuZCAqLw0KIAljbXAJCSVvMCwgJW8xDQogCWJlLHBuCQkleGNjLCAyZg0KLQkgc2V0aGkJ
CSVoaShQQUdFX1NJWkUpLCAlZzMNCi0JbW92CQklbzAsICVnMQ0KLQlzdWIJCSVvMSwgJWcxLCAl
ZzINCisJIHN1YgkJJW8xLCAlbzAsICVnMg0KKwlzcmx4CQklZzIsIDE4LCAlZzMNCisJYnJueixw
bgkJJWczLCA0Zg0KKwkgbW92CQklbzAsICVnMQ0KKwlzZXRoaQkJJWhpKFBBR0VfU0laRSksICVn
Mw0KIAlzdWIJCSVnMiwgJWczLCAlZzINCiAxOglhZGQJCSVnMSwgJWcyLCAlbzAJLyogQVJHMDog
dmlydHVhbCBhZGRyZXNzICovDQogCW1vdgkJMCwgJW8xCQkvKiBBUkcxOiBtbXUgY29udGV4dCAq
Lw0KQEAgLTQwOSw2ICs0NzksMTYgQEAgX19oeXBlcnZpc29yX2ZsdXNoX3RsYl9rZXJuZWxfcmFu
Z2U6IC8qIDE5IGluc25zICovDQogMzoJc2V0aGkJCSVoaShfX2h5cGVydmlzb3JfdGxiX3RsMF9l
cnJvciksICVvMg0KIAlqbXBsCQklbzIgKyAlbG8oX19oeXBlcnZpc29yX3RsYl90bDBfZXJyb3Ip
LCAlZzANCiAJIG5vcA0KKzQ6CW1vdgkJMCwgJW8wCQkvKiBBUkcwOiBDUFUgbGlzdHMgdW5pbXBs
ZW1lbnRlZCAqLw0KKwltb3YJCTAsICVvMQkJLyogQVJHMTogQ1BVIGxpc3RzIHVuaW1wbGVtZW50
ZWQgKi8NCisJbW92CQkwLCAlbzIJCS8qIEFSRzI6IG1tdSBjb250ZXh0ID09IG51Y2xldXMgKi8N
CisJbW92CQlIVl9NTVVfQUxMLCAlbzMJLyogQVJHMzogZmxhZ3MgKi8NCisJbW92CQlIVl9GQVNU
X01NVV9ERU1BUF9DVFgsICVvNQ0KKwl0YQkJSFZfRkFTVF9UUkFQDQorCWJybnoscG4JCSVvMCwg
M2INCisJIG1vdgkJSFZfRkFTVF9NTVVfREVNQVBfQ1RYLCAlbzENCisJcmV0bA0KKwkgbm9wDQog
DQogI2lmZGVmIERDQUNIRV9BTElBU0lOR19QT1NTSUJMRQ0KIAkvKiBYWFggTmlhZ2FyYSBhbmQg
ZnJpZW5kcyBoYXZlIGFuIDhLIGNhY2hlLCBzbyBubyBhbGlhc2luZyBpcw0KQEAgLTQzMSw0MyAr
NTExLDYgQEAgdGxiX3BhdGNoX29uZToNCiAJcmV0bA0KIAkgbm9wDQogDQotCS5nbG9ibAkJY2hl
ZXRhaF9wYXRjaF9jYWNoZXRsYm9wcw0KLWNoZWV0YWhfcGF0Y2hfY2FjaGV0bGJvcHM6DQotCXNh
dmUJCSVzcCwgLTEyOCwgJXNwDQotDQotCXNldGhpCQklaGkoX19mbHVzaF90bGJfbW0pLCAlbzAN
Ci0Jb3IJCSVvMCwgJWxvKF9fZmx1c2hfdGxiX21tKSwgJW8wDQotCXNldGhpCQklaGkoX19jaGVl
dGFoX2ZsdXNoX3RsYl9tbSksICVvMQ0KLQlvcgkJJW8xLCAlbG8oX19jaGVldGFoX2ZsdXNoX3Rs
Yl9tbSksICVvMQ0KLQljYWxsCQl0bGJfcGF0Y2hfb25lDQotCSBtb3YJCTE5LCAlbzINCi0NCi0J
c2V0aGkJCSVoaShfX2ZsdXNoX3RsYl9wYWdlKSwgJW8wDQotCW9yCQklbzAsICVsbyhfX2ZsdXNo
X3RsYl9wYWdlKSwgJW8wDQotCXNldGhpCQklaGkoX19jaGVldGFoX2ZsdXNoX3RsYl9wYWdlKSwg
JW8xDQotCW9yCQklbzEsICVsbyhfX2NoZWV0YWhfZmx1c2hfdGxiX3BhZ2UpLCAlbzENCi0JY2Fs
bAkJdGxiX3BhdGNoX29uZQ0KLQkgbW92CQkyMiwgJW8yDQotDQotCXNldGhpCQklaGkoX19mbHVz
aF90bGJfcGVuZGluZyksICVvMA0KLQlvcgkJJW8wLCAlbG8oX19mbHVzaF90bGJfcGVuZGluZyks
ICVvMA0KLQlzZXRoaQkJJWhpKF9fY2hlZXRhaF9mbHVzaF90bGJfcGVuZGluZyksICVvMQ0KLQlv
cgkJJW8xLCAlbG8oX19jaGVldGFoX2ZsdXNoX3RsYl9wZW5kaW5nKSwgJW8xDQotCWNhbGwJCXRs
Yl9wYXRjaF9vbmUNCi0JIG1vdgkJMjcsICVvMg0KLQ0KLSNpZmRlZiBEQ0FDSEVfQUxJQVNJTkdf
UE9TU0lCTEUNCi0Jc2V0aGkJCSVoaShfX2ZsdXNoX2RjYWNoZV9wYWdlKSwgJW8wDQotCW9yCQkl
bzAsICVsbyhfX2ZsdXNoX2RjYWNoZV9wYWdlKSwgJW8wDQotCXNldGhpCQklaGkoX19jaGVldGFo
X2ZsdXNoX2RjYWNoZV9wYWdlKSwgJW8xDQotCW9yCQklbzEsICVsbyhfX2NoZWV0YWhfZmx1c2hf
ZGNhY2hlX3BhZ2UpLCAlbzENCi0JY2FsbAkJdGxiX3BhdGNoX29uZQ0KLQkgbW92CQkxMSwgJW8y
DQotI2VuZGlmIC8qIERDQUNIRV9BTElBU0lOR19QT1NTSUJMRSAqLw0KLQ0KLQlyZXQNCi0JIHJl
c3RvcmUNCi0NCiAjaWZkZWYgQ09ORklHX1NNUA0KIAkvKiBUaGVzZSBhcmUgYWxsIGNhbGxlZCBi
eSB0aGUgc2xhdmVzIG9mIGEgY3Jvc3MgY2FsbCwgYXQNCiAJICogdHJhcCBsZXZlbCAxLCB3aXRo
IGludGVycnVwdHMgZnVsbHkgZGlzYWJsZWQuDQpAQCAtNTM1LDEzICs1NzgsMTUgQEAgeGNhbGxf
Zmx1c2hfdGxiX3BhZ2U6CS8qIDIwIGluc25zICovDQogCW5vcA0KIA0KIAkuZ2xvYmwJCXhjYWxs
X2ZsdXNoX3RsYl9rZXJuZWxfcmFuZ2UNCi14Y2FsbF9mbHVzaF90bGJfa2VybmVsX3JhbmdlOgkv
KiAyOCBpbnNucyAqLw0KK3hjYWxsX2ZsdXNoX3RsYl9rZXJuZWxfcmFuZ2U6CS8qIDQ0IGluc25z
ICovDQogCXNldGhpCQklaGkoUEFHRV9TSVpFIC0gMSksICVnMg0KIAlvcgkJJWcyLCAlbG8oUEFH
RV9TSVpFIC0gMSksICVnMg0KIAlhbmRuCQklZzEsICVnMiwgJWcxDQogCWFuZG4JCSVnNywgJWcy
LCAlZzcNCiAJc3ViCQklZzcsICVnMSwgJWczDQotCWFkZAkJJWcyLCAxLCAlZzINCisJc3JseAkJ
JWczLCAxOCwgJWcyDQorCWJybnoscG4JCSVnMiwgMmYNCisJIGFkZAkJJWcyLCAxLCAlZzINCiAJ
c3ViCQklZzMsICVnMiwgJWczDQogCW9yCQklZzEsIDB4MjAsICVnMQkJISBOdWNsZXVzDQogMToJ
c3R4YQkJJWcwLCBbJWcxICsgJWczXSBBU0lfRE1NVV9ERU1BUA0KQEAgLTU1MCwxMSArNTk1LDI1
IEBAIHhjYWxsX2ZsdXNoX3RsYl9rZXJuZWxfcmFuZ2U6CS8qIDI4IGluc25zICovDQogCWJybnos
cHQJCSVnMywgMWINCiAJIHN1YgkJJWczLCAlZzIsICVnMw0KIAlyZXRyeQ0KLQlub3ANCi0Jbm9w
DQotCW5vcA0KLQlub3ANCi0Jbm9wDQorMjoJbW92CQk2MyAqIDgsICVnMQ0KKzE6CWxkeGEJCVsl
ZzFdIEFTSV9JVExCX0RBVEFfQUNDRVNTLCAlZzINCisJYW5kY2MJCSVnMiwgMHg0MCwgJWcwCQkJ
LyogX1BBR0VfTF80VSAqLw0KKwlibmUscG4JCSV4Y2MsIDJmDQorCSBtb3YJCVRMQl9UQUdfQUND
RVNTLCAlZzINCisJc3R4YQkJJWcwLCBbJWcyXSBBU0lfSU1NVQ0KKwlzdHhhCQklZzAsIFslZzFd
IEFTSV9JVExCX0RBVEFfQUNDRVNTDQorCW1lbWJhcgkJI1N5bmMNCisyOglsZHhhCQlbJWcxXSBB
U0lfRFRMQl9EQVRBX0FDQ0VTUywgJWcyDQorCWFuZGNjCQklZzIsIDB4NDAsICVnMA0KKwlibmUs
cG4JCSV4Y2MsIDJmDQorCSBtb3YJCVRMQl9UQUdfQUNDRVNTLCAlZzINCisJc3R4YQkJJWcwLCBb
JWcyXSBBU0lfRE1NVQ0KKwlzdHhhCQklZzAsIFslZzFdIEFTSV9EVExCX0RBVEFfQUNDRVNTDQor
CW1lbWJhcgkJI1N5bmMNCisyOglzdWIJCSVnMSwgOCwgJWcxDQorCWJyZ2V6LHB0CSVnMSwgMWIN
CisJIG5vcA0KKwlyZXRyeQ0KIAlub3ANCiAJbm9wDQogCW5vcA0KQEAgLTY4Myw2ICs3NDIsNTIg
QEAgeGNhbGxfZmV0Y2hfZ2xvYl9wbXVfbjQ6DQogDQogCXJldHJ5DQogDQorX19jaGVldGFoX3hj
YWxsX2ZsdXNoX3RsYl9rZXJuZWxfcmFuZ2U6CS8qIDQ0IGluc25zICovDQorCXNldGhpCQklaGko
UEFHRV9TSVpFIC0gMSksICVnMg0KKwlvcgkJJWcyLCAlbG8oUEFHRV9TSVpFIC0gMSksICVnMg0K
KwlhbmRuCQklZzEsICVnMiwgJWcxDQorCWFuZG4JCSVnNywgJWcyLCAlZzcNCisJc3ViCQklZzcs
ICVnMSwgJWczDQorCXNybHgJCSVnMywgMTgsICVnMg0KKwlicm56LHBuCQklZzIsIDJmDQorCSBh
ZGQJCSVnMiwgMSwgJWcyDQorCXN1YgkJJWczLCAlZzIsICVnMw0KKwlvcgkJJWcxLCAweDIwLCAl
ZzEJCSEgTnVjbGV1cw0KKzE6CXN0eGEJCSVnMCwgWyVnMSArICVnM10gQVNJX0RNTVVfREVNQVAN
CisJc3R4YQkJJWcwLCBbJWcxICsgJWczXSBBU0lfSU1NVV9ERU1BUA0KKwltZW1iYXIJCSNTeW5j
DQorCWJybnoscHQJCSVnMywgMWINCisJIHN1YgkJJWczLCAlZzIsICVnMw0KKwlyZXRyeQ0KKzI6
CW1vdgkJMHg4MCwgJWcyDQorCXN0eGEJCSVnMCwgWyVnMl0gQVNJX0RNTVVfREVNQVANCisJbWVt
YmFyCQkjU3luYw0KKwlzdHhhCQklZzAsIFslZzJdIEFTSV9JTU1VX0RFTUFQDQorCW1lbWJhcgkJ
I1N5bmMNCisJcmV0cnkNCisJbm9wDQorCW5vcA0KKwlub3ANCisJbm9wDQorCW5vcA0KKwlub3AN
CisJbm9wDQorCW5vcA0KKwlub3ANCisJbm9wDQorCW5vcA0KKwlub3ANCisJbm9wDQorCW5vcA0K
Kwlub3ANCisJbm9wDQorCW5vcA0KKwlub3ANCisJbm9wDQorCW5vcA0KKwlub3ANCisJbm9wDQor
DQogI2lmZGVmIERDQUNIRV9BTElBU0lOR19QT1NTSUJMRQ0KIAkuYWxpZ24JCTMyDQogCS5nbG9i
bAkJeGNhbGxfZmx1c2hfZGNhY2hlX3BhZ2VfY2hlZXRhaA0KQEAgLTc5OCwxOCArOTAzLDIwIEBA
IF9faHlwZXJ2aXNvcl94Y2FsbF9mbHVzaF90bGJfcGFnZTogLyogMjAgaW5zbnMgKi8NCiAJIG5v
cA0KIA0KIAkuZ2xvYmwJCV9faHlwZXJ2aXNvcl94Y2FsbF9mbHVzaF90bGJfa2VybmVsX3Jhbmdl
DQotX19oeXBlcnZpc29yX3hjYWxsX2ZsdXNoX3RsYl9rZXJuZWxfcmFuZ2U6IC8qIDI4IGluc25z
ICovDQorX19oeXBlcnZpc29yX3hjYWxsX2ZsdXNoX3RsYl9rZXJuZWxfcmFuZ2U6IC8qIDQ0IGlu
c25zICovDQogCS8qICVnMT1zdGFydCwgJWc3PWVuZCwgZzIsZzMsZzQsZzUsZzY9c2NyYXRjaCAq
Lw0KIAlzZXRoaQkJJWhpKFBBR0VfU0laRSAtIDEpLCAlZzINCiAJb3IJCSVnMiwgJWxvKFBBR0Vf
U0laRSAtIDEpLCAlZzINCiAJYW5kbgkJJWcxLCAlZzIsICVnMQ0KIAlhbmRuCQklZzcsICVnMiwg
JWc3DQogCXN1YgkJJWc3LCAlZzEsICVnMw0KKwlzcmx4CQklZzMsIDE4LCAlZzcNCiAJYWRkCQkl
ZzIsIDEsICVnMg0KIAlzdWIJCSVnMywgJWcyLCAlZzMNCiAJbW92CQklbzAsICVnMg0KIAltb3YJ
CSVvMSwgJWc0DQotCW1vdgkJJW8yLCAlZzcNCisJYnJueixwbgkJJWc3LCAyZg0KKwkgbW92CQkl
bzIsICVnNw0KIDE6CWFkZAkJJWcxLCAlZzMsICVvMAkvKiBBUkcwOiB2aXJ0dWFsIGFkZHJlc3Mg
Ki8NCiAJbW92CQkwLCAlbzEJCS8qIEFSRzE6IG1tdSBjb250ZXh0ICovDQogCW1vdgkJSFZfTU1V
X0FMTCwgJW8yCS8qIEFSRzI6IGZsYWdzICovDQpAQCAtODIwLDcgKzkyNyw3IEBAIF9faHlwZXJ2
aXNvcl94Y2FsbF9mbHVzaF90bGJfa2VybmVsX3JhbmdlOiAvKiAyOCBpbnNucyAqLw0KIAlzZXRo
aQkJJWhpKFBBR0VfU0laRSksICVvMg0KIAlicm56LHB0CQklZzMsIDFiDQogCSBzdWIJCSVnMywg
JW8yLCAlZzMNCi0JbW92CQklZzIsICVvMA0KKzU6CW1vdgkJJWcyLCAlbzANCiAJbW92CQklZzQs
ICVvMQ0KIAltb3YJCSVnNywgJW8yDQogCW1lbWJhcgkJI1N5bmMNCkBAIC04MjgsNiArOTM1LDIw
IEBAIF9faHlwZXJ2aXNvcl94Y2FsbF9mbHVzaF90bGJfa2VybmVsX3JhbmdlOiAvKiAyOCBpbnNu
cyAqLw0KIDE6CXNldGhpCQklaGkoX19oeXBlcnZpc29yX3RsYl94Y2FsbF9lcnJvciksICVnNA0K
IAlqbXBsCQklZzQgKyAlbG8oX19oeXBlcnZpc29yX3RsYl94Y2FsbF9lcnJvciksICVnMA0KIAkg
bm9wDQorMjoJbW92CQklbzMsICVnMQ0KKwltb3YJCSVvNSwgJWczDQorCW1vdgkJMCwgJW8wCQkv
KiBBUkcwOiBDUFUgbGlzdHMgdW5pbXBsZW1lbnRlZCAqLw0KKwltb3YJCTAsICVvMQkJLyogQVJH
MTogQ1BVIGxpc3RzIHVuaW1wbGVtZW50ZWQgKi8NCisJbW92CQkwLCAlbzIJCS8qIEFSRzI6IG1t
dSBjb250ZXh0ID09IG51Y2xldXMgKi8NCisJbW92CQlIVl9NTVVfQUxMLCAlbzMJLyogQVJHMzog
ZmxhZ3MgKi8NCisJbW92CQlIVl9GQVNUX01NVV9ERU1BUF9DVFgsICVvNQ0KKwl0YQkJSFZfRkFT
VF9UUkFQDQorCW1vdgkJJWcxLCAlbzMNCisJYnJ6LHB0CQklbzAsIDViDQorCSBtb3YJCSVnMywg
JW81DQorCW1vdgkJSFZfRkFTVF9NTVVfREVNQVBfQ1RYLCAlZzYNCisJYmEscHQJCSV4Y2MsIDFi
DQorCSBjbHIJCSVnNQ0KIA0KIAkvKiBUaGVzZSBqdXN0IGdldCByZXNjaGVkdWxlZCB0byBQSUwg
dmVjdG9ycy4gKi8NCiAJLmdsb2JsCQl4Y2FsbF9jYWxsX2Z1bmN0aW9uDQpAQCAtODY0LDYgKzk4
NSw1OCBAQCB4Y2FsbF9rZ2RiX2NhcHR1cmU6DQogDQogI2VuZGlmIC8qIENPTkZJR19TTVAgKi8N
CiANCisJLmdsb2JsCQljaGVldGFoX3BhdGNoX2NhY2hldGxib3BzDQorY2hlZXRhaF9wYXRjaF9j
YWNoZXRsYm9wczoNCisJc2F2ZQkJJXNwLCAtMTI4LCAlc3ANCisNCisJc2V0aGkJCSVoaShfX2Zs
dXNoX3RsYl9tbSksICVvMA0KKwlvcgkJJW8wLCAlbG8oX19mbHVzaF90bGJfbW0pLCAlbzANCisJ
c2V0aGkJCSVoaShfX2NoZWV0YWhfZmx1c2hfdGxiX21tKSwgJW8xDQorCW9yCQklbzEsICVsbyhf
X2NoZWV0YWhfZmx1c2hfdGxiX21tKSwgJW8xDQorCWNhbGwJCXRsYl9wYXRjaF9vbmUNCisJIG1v
dgkJMTksICVvMg0KKw0KKwlzZXRoaQkJJWhpKF9fZmx1c2hfdGxiX3BhZ2UpLCAlbzANCisJb3IJ
CSVvMCwgJWxvKF9fZmx1c2hfdGxiX3BhZ2UpLCAlbzANCisJc2V0aGkJCSVoaShfX2NoZWV0YWhf
Zmx1c2hfdGxiX3BhZ2UpLCAlbzENCisJb3IJCSVvMSwgJWxvKF9fY2hlZXRhaF9mbHVzaF90bGJf
cGFnZSksICVvMQ0KKwljYWxsCQl0bGJfcGF0Y2hfb25lDQorCSBtb3YJCTIyLCAlbzINCisNCisJ
c2V0aGkJCSVoaShfX2ZsdXNoX3RsYl9wZW5kaW5nKSwgJW8wDQorCW9yCQklbzAsICVsbyhfX2Zs
dXNoX3RsYl9wZW5kaW5nKSwgJW8wDQorCXNldGhpCQklaGkoX19jaGVldGFoX2ZsdXNoX3RsYl9w
ZW5kaW5nKSwgJW8xDQorCW9yCQklbzEsICVsbyhfX2NoZWV0YWhfZmx1c2hfdGxiX3BlbmRpbmcp
LCAlbzENCisJY2FsbAkJdGxiX3BhdGNoX29uZQ0KKwkgbW92CQkyNywgJW8yDQorDQorCXNldGhp
CQklaGkoX19mbHVzaF90bGJfa2VybmVsX3JhbmdlKSwgJW8wDQorCW9yCQklbzAsICVsbyhfX2Zs
dXNoX3RsYl9rZXJuZWxfcmFuZ2UpLCAlbzANCisJc2V0aGkJCSVoaShfX2NoZWV0YWhfZmx1c2hf
dGxiX2tlcm5lbF9yYW5nZSksICVvMQ0KKwlvcgkJJW8xLCAlbG8oX19jaGVldGFoX2ZsdXNoX3Rs
Yl9rZXJuZWxfcmFuZ2UpLCAlbzENCisJY2FsbAkJdGxiX3BhdGNoX29uZQ0KKwkgbW92CQkzMSwg
JW8yDQorDQorI2lmZGVmIERDQUNIRV9BTElBU0lOR19QT1NTSUJMRQ0KKwlzZXRoaQkJJWhpKF9f
Zmx1c2hfZGNhY2hlX3BhZ2UpLCAlbzANCisJb3IJCSVvMCwgJWxvKF9fZmx1c2hfZGNhY2hlX3Bh
Z2UpLCAlbzANCisJc2V0aGkJCSVoaShfX2NoZWV0YWhfZmx1c2hfZGNhY2hlX3BhZ2UpLCAlbzEN
CisJb3IJCSVvMSwgJWxvKF9fY2hlZXRhaF9mbHVzaF9kY2FjaGVfcGFnZSksICVvMQ0KKwljYWxs
CQl0bGJfcGF0Y2hfb25lDQorCSBtb3YJCTExLCAlbzINCisjZW5kaWYgLyogRENBQ0hFX0FMSUFT
SU5HX1BPU1NJQkxFICovDQorDQorI2lmZGVmIENPTkZJR19TTVANCisJc2V0aGkJCSVoaSh4Y2Fs
bF9mbHVzaF90bGJfa2VybmVsX3JhbmdlKSwgJW8wDQorCW9yCQklbzAsICVsbyh4Y2FsbF9mbHVz
aF90bGJfa2VybmVsX3JhbmdlKSwgJW8wDQorCXNldGhpCQklaGkoX19jaGVldGFoX3hjYWxsX2Zs
dXNoX3RsYl9rZXJuZWxfcmFuZ2UpLCAlbzENCisJb3IJCSVvMSwgJWxvKF9fY2hlZXRhaF94Y2Fs
bF9mbHVzaF90bGJfa2VybmVsX3JhbmdlKSwgJW8xDQorCWNhbGwJCXRsYl9wYXRjaF9vbmUNCisJ
IG1vdgkJNDQsICVvMg0KKyNlbmRpZiAvKiBDT05GSUdfU01QICovDQorDQorCXJldA0KKwkgcmVz
dG9yZQ0KIA0KIAkuZ2xvYmwJCWh5cGVydmlzb3JfcGF0Y2hfY2FjaGV0bGJvcHMNCiBoeXBlcnZp
c29yX3BhdGNoX2NhY2hldGxib3BzOg0KQEAgLTg5NSw3ICsxMDY4LDcgQEAgaHlwZXJ2aXNvcl9w
YXRjaF9jYWNoZXRsYm9wczoNCiAJc2V0aGkJCSVoaShfX2h5cGVydmlzb3JfZmx1c2hfdGxiX2tl
cm5lbF9yYW5nZSksICVvMQ0KIAlvcgkJJW8xLCAlbG8oX19oeXBlcnZpc29yX2ZsdXNoX3RsYl9r
ZXJuZWxfcmFuZ2UpLCAlbzENCiAJY2FsbAkJdGxiX3BhdGNoX29uZQ0KLQkgbW92CQkxOSwgJW8y
DQorCSBtb3YJCTMxLCAlbzINCiANCiAjaWZkZWYgRENBQ0hFX0FMSUFTSU5HX1BPU1NJQkxFDQog
CXNldGhpCQklaGkoX19mbHVzaF9kY2FjaGVfcGFnZSksICVvMA0KQEAgLTkyNiw3ICsxMDk5LDcg
QEAgaHlwZXJ2aXNvcl9wYXRjaF9jYWNoZXRsYm9wczoNCiAJc2V0aGkJCSVoaShfX2h5cGVydmlz
b3JfeGNhbGxfZmx1c2hfdGxiX2tlcm5lbF9yYW5nZSksICVvMQ0KIAlvcgkJJW8xLCAlbG8oX19o
eXBlcnZpc29yX3hjYWxsX2ZsdXNoX3RsYl9rZXJuZWxfcmFuZ2UpLCAlbzENCiAJY2FsbAkJdGxi
X3BhdGNoX29uZQ0KLQkgbW92CQkyOCwgJW8yDQorCSBtb3YJCTQ0LCAlbzINCiAjZW5kaWYgLyog
Q09ORklHX1NNUCAqLw0KIA0KIAlyZXQNCg=

Re: [PATCH] sparc64: Handle extremely large kernel TSB range flushes sanely.

[James Clarke]

On Wed, Oct 26, 2016 at 03:04:59PM -0400, David Miller wrote:
> From: James Clarke <jrtc27@jrtc27.com>
> Date: Wed, 26 Oct 2016 18:21:06 +0100
>
> >> On 26 Oct 2016, at 18:09, David Miller <davem@davemloft.net> wrote:
> >>
> >> From: James Clarke <jrtc27@jrtc27.com>
> >> Date: Wed, 26 Oct 2016 17:58:16 +0100
> >>
> >>>> On 26 Oct 2016, at 16:54, David Miller <davem@davemloft.net> wrote:
> >>>>
> >>>> From: James Clarke <jrtc27@jrtc27.com>
> >>>> Date: Wed, 26 Oct 2016 09:28:05 +0100
> >>>>
> >>>>> Any progress on TLB flushing?
> >>>>
> >>>> I'll keep plugging away at it today.
> >>>
> >>> Great; let me know if you need a guinea pig, as it’s pretty easy for me to
> >>> reproduce.
> >>
> >> Will do, what kind of cpus do you have?
> >
> > * UltraSparc T5 (Niagara5)
> > * UltraSparc T1 (Niagara)
> > * UltraSPARC IIIi
> >
> > The IIIi seems to be down at the moment though.
>
> James, here is what I have so far.  I only gave it a brief testing on
> sun4v, so no guarantees for the sun4u cases.  This is against the
> current sparc GIT tree.
>
> The cut-off is 32 pages, we can discuss whether that's a good value
> to use or not.  FWIW, x86_64 has similar code for this situation and
> uses a cut-off of 33.  Perhaps 64 is a better value, who knows.
>
> It might even make sense to use a different cut-off for the hypervisor
> case since the hypervisor trap we have to use to do the TLB operation
> adds even more expense to each iteration of the range loop.
>
> The policy implemented for huge range flushes below is:
>
> 1) Spitfire - Flush all non-locked entries, by hand using diagnostic
>    TLB accesses.
>
> 2) Cheetah - Flush all non-locked entries using "flush all" operation.
>
> 3) sun4v/hypervisor - Flush entire kernel context, which does not
>    remove locked or "permanent" entries.
>
> Anyways, let me know how it goes.

Thanks for this, it's now compiling. I'll let you know if it works
within the next 24 hours.

Before I forget, what do you think about the following patch? I know
Debian used to use the 64-bit kernel for a 32-bit sparc userland, and so
"Architecture: sparc" was correct, but obviously sparc64 also exists. It
seems more sane to make sparc64 default to "Architecture: sparc64", with
sparc users needing to override this with KBUILD_DEBARCH if they want
to, rather than providing a setup that's broken out of the box for
sparc64 users.

From: James Clarke <jrtc27@jrtc27.com>
Date: Wed, 26 Oct 2016 20:17:10 +0100
Subject: [PATCH] builddeb: Add support for sparc64

Signed-off-by: James Clarke <jrtc27@jrtc27.com>
---
 scripts/package/builddeb | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/scripts/package/builddeb b/scripts/package/builddeb
index 8ea9fd2..63b3112 100755
--- a/scripts/package/builddeb
+++ b/scripts/package/builddeb
@@ -41,6 +41,8 @@ set_debarch() {
 		debarch="$UTS_MACHINE" ;;
 	x86_64)
 		debarch=amd64 ;;
+	sparc64)
+		debarch=sparc64 ;;
 	sparc*)
 		debarch=sparc ;;
 	s390*)
--
2.9.3

Re: [PATCH] sparc64: Handle extremely large kernel TSB range flushes sanely.

[David Miller]

From: James Clarke <jrtc27@jrtc27.com>
Date: Wed, 26 Oct 2016 21:05:36 +0100

> Thanks for this, it's now compiling. I'll let you know if it works
> within the next 24 hours.

Thanks.

> Before I forget, what do you think about the following patch? I know
> Debian used to use the 64-bit kernel for a 32-bit sparc userland, and so
> "Architecture: sparc" was correct, but obviously sparc64 also exists. It
> seems more sane to make sparc64 default to "Architecture: sparc64", with
> sparc users needing to override this with KBUILD_DEBARCH if they want
> to, rather than providing a setup that's broken out of the box for
> sparc64 users.
> 
> From: James Clarke <jrtc27@jrtc27.com>
> Date: Wed, 26 Oct 2016 20:17:10 +0100
> Subject: [PATCH] builddeb: Add support for sparc64
> 
> Signed-off-by: James Clarke <jrtc27@jrtc27.com>

I don't know.

I still personally use a 32-bit userland on my sparc64 systems because
that is what performs the best and is what I will be using for as long
as I possibly can.

I've actually never used this target, is this for build the kernel or
userland components?

Re: [PATCH] sparc64: Handle extremely large kernel TSB range flushes sanely.

[James Clarke]

> On 26 Oct 2016, at 22:02, David Miller <davem@davemloft.net> wrote:
> 
> From: James Clarke <jrtc27@jrtc27.com>
> Date: Wed, 26 Oct 2016 21:05:36 +0100
> 
>> Thanks for this, it's now compiling. I'll let you know if it works
>> within the next 24 hours.
> 
> Thanks.
> 
>> Before I forget, what do you think about the following patch? I know
>> Debian used to use the 64-bit kernel for a 32-bit sparc userland, and so
>> "Architecture: sparc" was correct, but obviously sparc64 also exists. It
>> seems more sane to make sparc64 default to "Architecture: sparc64", with
>> sparc users needing to override this with KBUILD_DEBARCH if they want
>> to, rather than providing a setup that's broken out of the box for
>> sparc64 users.
>> 
>> From: James Clarke <jrtc27@jrtc27.com>
>> Date: Wed, 26 Oct 2016 20:17:10 +0100
>> Subject: [PATCH] builddeb: Add support for sparc64
>> 
>> Signed-off-by: James Clarke <jrtc27@jrtc27.com>
> 
> I don't know.
> 
> I still personally use a 32-bit userland on my sparc64 systems because
> that is what performs the best and is what I will be using for as long
> as I possibly can.
> 
> I've actually never used this target, is this for build the kernel or
> userland components?

Yes, make pkg-deb builds kernel, firmware, headers and linux-libc packages.
By the way, the first build I made of 4.9 (using Debian’s 4.8 config as old
config) wouldn’t boot, since:

* sunvdc module needs _mcount
* sunvnet module needs _mcount and count_bits
* crc32c_sparc64 module needs _mcount and VISenter
[* raid6_pq module needs memcpy, though that’s just for a data partition]

The workaround is not to use CONFIG_MODVERSIONS, but this wasn’t at all clear
at first. This is because of d3867f0483, which moved these to being exported in
their .S.

Anyway, the new kernel is running now and being stress-tested.

James

Re: [PATCH] sparc64: Handle extremely large kernel TSB range flushes sanely.

[James Clarke]

> On 27 Oct 2016, at 02:27, James Clarke <jrtc27@jrtc27.com> wrote:
> 
>> On 26 Oct 2016, at 22:02, David Miller <davem@davemloft.net> wrote:
>> 
>> From: James Clarke <jrtc27@jrtc27.com>
>> Date: Wed, 26 Oct 2016 21:05:36 +0100
>> 
>>> Thanks for this, it's now compiling. I'll let you know if it works
>>> within the next 24 hours.
>> 
>> Thanks.
>> 
>>> Before I forget, what do you think about the following patch? I know
>>> Debian used to use the 64-bit kernel for a 32-bit sparc userland, and so
>>> "Architecture: sparc" was correct, but obviously sparc64 also exists. It
>>> seems more sane to make sparc64 default to "Architecture: sparc64", with
>>> sparc users needing to override this with KBUILD_DEBARCH if they want
>>> to, rather than providing a setup that's broken out of the box for
>>> sparc64 users.
>>> 
>>> From: James Clarke <jrtc27@jrtc27.com>
>>> Date: Wed, 26 Oct 2016 20:17:10 +0100
>>> Subject: [PATCH] builddeb: Add support for sparc64
>>> 
>>> Signed-off-by: James Clarke <jrtc27@jrtc27.com>
>> 
>> I don't know.
>> 
>> I still personally use a 32-bit userland on my sparc64 systems because
>> that is what performs the best and is what I will be using for as long
>> as I possibly can.
>> 
>> I've actually never used this target, is this for build the kernel or
>> userland components?
> 
> Yes, make pkg-deb builds kernel, firmware, headers and linux-libc packages.
> By the way, the first build I made of 4.9 (using Debian’s 4.8 config as old
> config) wouldn’t boot, since:
> 
> * sunvdc module needs _mcount
> * sunvnet module needs _mcount and count_bits
> * crc32c_sparc64 module needs _mcount and VISenter
> [* raid6_pq module needs memcpy, though that’s just for a data partition]
> 
> The workaround is not to use CONFIG_MODVERSIONS, but this wasn’t at all clear
> at first. This is because of d3867f0483, which moved these to being exported in
> their .S.
> 
> Anyway, the new kernel is running now and being stress-tested.

Hi David,
I’ve run it on the T5 and it seems to work without lockups:

[5948090.988821] vln_init: *vmap_lazy_nr is 32754
[5948090.989943] vln_init: lazy_max_pages() is 32768
[5948091.157381] TSB[insmod:261876]: DEBUG flush_tsb_kernel_range start=0000000010006000 end=00000000f0000000 PAGE_SIZE=2000
[5948091.157530] TSB[insmod:261876]: DEBUG flush_tsb_kernel_range start=0000000100000000 end=0005ffff8c000000 PAGE_SIZE=2000
[5948091.158240] vln_init: vmap_lazy_nr is caeb1c
[5948091.158252] vln_init: *vmap_lazy_nr is 0
[5948091.159311] vln_init: lazy_max_pages() is 32768
... continues on as normal ...

(again, that’s my debugging module to see how close the system is to a flush)

I can't (yet) vouch for the IIIi, but when it comes back up I’ll give it a go[1].
I'll also put it on the T1 at some point today, but it *should* also work since
it's using the same sun4v/hypervisor implementation as the T5.

Thanks,
James

[1] Not sure how long that will take...

Re: [PATCH] sparc64: Handle extremely large kernel TSB range flushes sanely.

[David Miller]

RnJvbTogSmFtZXMgQ2xhcmtlIDxqcnRjMjdAanJ0YzI3LmNvbT4NCkRhdGU6IFRodSwgMjcgT2N0
IDIwMTYgMDk6MjU6MzYgKzAxMDANCg0KPiBJonZlIHJ1biBpdCBvbiB0aGUgVDUgYW5kIGl0IHNl
ZW1zIHRvIHdvcmsgd2l0aG91dCBsb2NrdXBzOg0KPiANCj4gWzU5NDgwOTAuOTg4ODIxXSB2bG5f
aW5pdDogKnZtYXBfbGF6eV9uciBpcyAzMjc1NA0KPiBbNTk0ODA5MC45ODk5NDNdIHZsbl9pbml0
OiBsYXp5X21heF9wYWdlcygpIGlzIDMyNzY4DQo+IFs1OTQ4MDkxLjE1NzM4MV0gVFNCW2luc21v
ZDoyNjE4NzZdOiBERUJVRyBmbHVzaF90c2Jfa2VybmVsX3JhbmdlIHN0YXJ0PTAwMDAwMDAwMTAw
MDYwMDAgZW5kPTAwMDAwMDAwZjAwMDAwMDAgUEFHRV9TSVpFPTIwMDANCj4gWzU5NDgwOTEuMTU3
NTMwXSBUU0JbaW5zbW9kOjI2MTg3Nl06IERFQlVHIGZsdXNoX3RzYl9rZXJuZWxfcmFuZ2Ugc3Rh
cnQ9MDAwMDAwMDEwMDAwMDAwMCBlbmQ9MDAwNWZmZmY4YzAwMDAwMCBQQUdFX1NJWkU9MjAwMA0K
PiBbNTk0ODA5MS4xNTgyNDBdIHZsbl9pbml0OiB2bWFwX2xhenlfbnIgaXMgY2FlYjFjDQo+IFs1
OTQ4MDkxLjE1ODI1Ml0gdmxuX2luaXQ6ICp2bWFwX2xhenlfbnIgaXMgMA0KPiBbNTk0ODA5MS4x
NTkzMTFdIHZsbl9pbml0OiBsYXp5X21heF9wYWdlcygpIGlzIDMyNzY4DQo+IC4uLiBjb250aW51
ZXMgb24gYXMgbm9ybWFsIC4uLg0KPiANCj4gKGFnYWluLCB0aGF0onMgbXkgZGVidWdnaW5nIG1v
ZHVsZSB0byBzZWUgaG93IGNsb3NlIHRoZSBzeXN0ZW0gaXMgdG8gYSBmbHVzaCkNCj4gDQo+IEkg
Y2FuJ3QgKHlldCkgdm91Y2ggZm9yIHRoZSBJSUlpLCBidXQgd2hlbiBpdCBjb21lcyBiYWNrIHVw
IEmibGwgZ2l2ZSBpdCBhIGdvWzFdLg0KPiBJJ2xsIGFsc28gcHV0IGl0IG9uIHRoZSBUMSBhdCBz
b21lIHBvaW50IHRvZGF5LCBidXQgaXQgKnNob3VsZCogYWxzbyB3b3JrIHNpbmNlDQo+IGl0J3Mg
dXNpbmcgdGhlIHNhbWUgc3VuNHYvaHlwZXJ2aXNvciBpbXBsZW1lbnRhdGlvbiBhcyB0aGUgVDUu
DQoNCkknbSBhYm91dCB0byB0ZXN0IGl0IG9uIG15IElJSWkgYW5kIHdpbGwgY29tbWl0IHRoaXMg
aWYgaXQgc2VlbXMgdG8gd29yayBwcm9wZXJseS4NCg0KSSBndWVzcyB5b3UgaGF2ZSBubyBvcGlu
aW9uIGFib3V0IHRoZSBjdXQtb2ZmIGNob29zZW4/IDotKQ0KDQpBbnl3YXlzLCB3ZSBjYW4gZmlu
ZSB0dW5lIGl0IGxhdGVyLg0K

Re: [PATCH] sparc64: Handle extremely large kernel TSB range flushes sanely.

[James Clarke]

> On 27 Oct 2016, at 16:51, David Miller <davem@davemloft.net> wrote:
> 
> From: James Clarke <jrtc27@jrtc27.com>
> Date: Thu, 27 Oct 2016 09:25:36 +0100
> 
>> I’ve run it on the T5 and it seems to work without lockups:
>> 
>> [5948090.988821] vln_init: *vmap_lazy_nr is 32754
>> [5948090.989943] vln_init: lazy_max_pages() is 32768
>> [5948091.157381] TSB[insmod:261876]: DEBUG flush_tsb_kernel_range start=0000000010006000 end=00000000f0000000 PAGE_SIZE=2000
>> [5948091.157530] TSB[insmod:261876]: DEBUG flush_tsb_kernel_range start=0000000100000000 end=0005ffff8c000000 PAGE_SIZE=2000
>> [5948091.158240] vln_init: vmap_lazy_nr is caeb1c
>> [5948091.158252] vln_init: *vmap_lazy_nr is 0
>> [5948091.159311] vln_init: lazy_max_pages() is 32768
>> ... continues on as normal ...
>> 
>> (again, that’s my debugging module to see how close the system is to a flush)
>> 
>> I can't (yet) vouch for the IIIi, but when it comes back up I’ll give it a go[1].
>> I'll also put it on the T1 at some point today, but it *should* also work since
>> it's using the same sun4v/hypervisor implementation as the T5.
> 
> I'm about to test it on my IIIi and will commit this if it seems to work properly.
> 
> I guess you have no opinion about the cut-off choosen? :-)
> 
> Anyways, we can fine tune it later.

I was just testing it on the IIIi when I got this. Anyway, it seems to work fine.
It hasn’t (yet) had one of the stupidly high allocations, but it did flush a block
of 3658 pages just fine (assuming the flush actually worked). Similarly for the T1.

The cut-off seems pretty arbitrary, and the only way to determine it properly would
be benchmarking (or finding out what the relevant delays are). Given x86 uses 33,
32 or 64 seem perfectly fine, but going into the hundreds doesn’t sound stupid
either... For such small numbers it’s probably hardly going to matter.

Tested-by: James Clarke <jrtc27@jrtc27.com>

James

Re: [PATCH] sparc64: Handle extremely large kernel TSB range flushes sanely.

[David Miller]

RnJvbTogSmFtZXMgQ2xhcmtlIDxqcnRjMjdAanJ0YzI3LmNvbT4NCkRhdGU6IFRodSwgMjcgT2N0
IDIwMTYgMTc6MDI6MzIgKzAxMDANCg0KPiBJIHdhcyBqdXN0IHRlc3RpbmcgaXQgb24gdGhlIElJ
SWkgd2hlbiBJIGdvdCB0aGlzLiBBbnl3YXksIGl0IHNlZW1zIHRvIHdvcmsgZmluZS4NCj4gSXQg
aGFzbqJ0ICh5ZXQpIGhhZCBvbmUgb2YgdGhlIHN0dXBpZGx5IGhpZ2ggYWxsb2NhdGlvbnMsIGJ1
dCBpdCBkaWQgZmx1c2ggYSBibG9jaw0KPiBvZiAzNjU4IHBhZ2VzIGp1c3QgZmluZSAoYXNzdW1p
bmcgdGhlIGZsdXNoIGFjdHVhbGx5IHdvcmtlZCkuIFNpbWlsYXJseSBmb3IgdGhlIFQxLg0KDQpU
aGFua3MgZm9yIHRlc3RpbmcuICBJJ2xsIHBvc3QgdGhlIGZpbmFsIHBhdGNoIEkgY29tbWl0dGVk
Lg0KDQo+IFRoZSBjdXQtb2ZmIHNlZW1zIHByZXR0eSBhcmJpdHJhcnksIGFuZCB0aGUgb25seSB3
YXkgdG8gZGV0ZXJtaW5lIGl0IHByb3Blcmx5IHdvdWxkDQo+IGJlIGJlbmNobWFya2luZyAob3Ig
ZmluZGluZyBvdXQgd2hhdCB0aGUgcmVsZXZhbnQgZGVsYXlzIGFyZSkuIEdpdmVuIHg4NiB1c2Vz
IDMzLA0KPiAzMiBvciA2NCBzZWVtIHBlcmZlY3RseSBmaW5lLCBidXQgZ29pbmcgaW50byB0aGUg
aHVuZHJlZHMgZG9lc26idCBzb3VuZCBzdHVwaWQNCj4gZWl0aGVyLi4uIEZvciBzdWNoIHNtYWxs
IG51bWJlcnMgaXSicyBwcm9iYWJseSBoYXJkbHkgZ29pbmcgdG8gbWF0dGVyLg0KDQpJdCdzIG5v
dCB0b28gaGFyZCB0byB3cml0ZSBhIGtlcm5lbCBtb2R1bGUgd2hpY2gganVzdCBkb2VzIGR1bW15
IFRMQiBmbHVzaGVzIGluDQp0aGUgbG9vcCBhbmQgY291bnQgdGhlIGN5Y2xlcyB1c2luZyB0aGUg
JXRpY2sgcmVnaXN0ZXIuICBBbmQgSSBwbGFuIHRvIGhhY2sgb24NCnNvbWV0aGluZyBsaWtlIHRo
YXQgc29vbidpc2guDQoNCkFub3RoZXIgcGFydCBvZiB0aGUgZXF1YXRpb24gaXMgdGhhdCBpdCBi
bG93cyBhd2F5LCBhdCBhIG1pbmltdW0sIGFsbCBrZXJuZWwNClRMQiBlbnRyaWVzLiAgQW5kIHRo
YXQgaGFzIGEgY2VydGFpbiBjb3N0IHRvby4NCg=

[PATCH] sparc64: Handle extremely large kernel TLB range flushes more gracefully.

[David Miller]

When the vmalloc area gets fragmented, and because the firmware
mapping area sits between where modules live and the vmalloc area, we
can sometimes receive requests for enormous kernel TLB range flushes.

When this happens the cpu just spins flushing billions of pages and
this triggers the NMI watchdog and other problems.

We took care of this on the TSB side by doing a linear scan of the
table once we pass a certain threshold.

Do something similar for the TLB flush, however we are limited by
the TLB flush facilities provided by the different chip variants.

First of all we use an (mostly arbitrary) cut-off of 256K which is
about 32 pages.  This can be tuned in the future.

The huge range code path for each chip works as follows:

1) On spitfire we flush all non-locked TLB entries using diagnostic
   acceses.

2) On cheetah we use the "flush all" TLB flush.

3) On sun4v/hypervisor we do a TLB context flush on context 0, which
   unlike previous chips does not remove "permanent" or locked
   entries.

We could probably do something better on spitfire, such as limiting
the flush to kernel TLB entries or even doing range comparisons.
However that probably isn't worth it since those chips are old and
the TLB only had 64 entries.

Reported-by: James Clarke <jrtc27@jrtc27.com>
Tested-by: James Clarke <jrtc27@jrtc27.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
---
 arch/sparc/mm/ultra.S | 283 ++++++++++++++++++++++++++++++++++++++++----------
 1 file changed, 228 insertions(+), 55 deletions(-)

diff --git a/arch/sparc/mm/ultra.S b/arch/sparc/mm/ultra.S
index 0fa2e62..5d2fd6c 100644
--- a/arch/sparc/mm/ultra.S
+++ b/arch/sparc/mm/ultra.S
@@ -113,12 +113,14 @@ __flush_tlb_pending:	/* 27 insns */
 
 	.align		32
 	.globl		__flush_tlb_kernel_range
-__flush_tlb_kernel_range:	/* 19 insns */
+__flush_tlb_kernel_range:	/* 31 insns */
 	/* %o0=start, %o1=end */
 	cmp		%o0, %o1
 	be,pn		%xcc, 2f
+	 sub		%o1, %o0, %o3
+	srlx		%o3, 18, %o4
+	brnz,pn		%o4, __spitfire_flush_tlb_kernel_range_slow
 	 sethi		%hi(PAGE_SIZE), %o4
-	sub		%o1, %o0, %o3
 	sub		%o3, %o4, %o3
 	or		%o0, 0x20, %o0		! Nucleus
 1:	stxa		%g0, [%o0 + %o3] ASI_DMMU_DEMAP
@@ -134,6 +136,38 @@ __flush_tlb_kernel_range:	/* 19 insns */
 	nop
 	nop
 	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+
+__spitfire_flush_tlb_kernel_range_slow:
+	mov		63 * 8, %o4
+1:	ldxa		[%o4] ASI_ITLB_DATA_ACCESS, %o3
+	andcc		%o3, 0x40, %g0			/* _PAGE_L_4U */
+	bne,pn		%xcc, 2f
+	 mov		TLB_TAG_ACCESS, %o3
+	stxa		%g0, [%o3] ASI_IMMU
+	stxa		%g0, [%o4] ASI_ITLB_DATA_ACCESS
+	membar		#Sync
+2:	ldxa		[%o4] ASI_DTLB_DATA_ACCESS, %o3
+	andcc		%o3, 0x40, %g0
+	bne,pn		%xcc, 2f
+	 mov		TLB_TAG_ACCESS, %o3
+	stxa		%g0, [%o3] ASI_DMMU
+	stxa		%g0, [%o4] ASI_DTLB_DATA_ACCESS
+	membar		#Sync
+2:	sub		%o4, 8, %o4
+	brgez,pt	%o4, 1b
+	 nop
+	retl
+	 nop
 
 __spitfire_flush_tlb_mm_slow:
 	rdpr		%pstate, %g1
@@ -288,6 +322,40 @@ __cheetah_flush_tlb_pending:	/* 27 insns */
 	retl
 	 wrpr		%g7, 0x0, %pstate
 
+__cheetah_flush_tlb_kernel_range:	/* 31 insns */
+	/* %o0=start, %o1=end */
+	cmp		%o0, %o1
+	be,pn		%xcc, 2f
+	 sub		%o1, %o0, %o3
+	srlx		%o3, 18, %o4
+	brnz,pn		%o4, 3f
+	 sethi		%hi(PAGE_SIZE), %o4
+	sub		%o3, %o4, %o3
+	or		%o0, 0x20, %o0		! Nucleus
+1:	stxa		%g0, [%o0 + %o3] ASI_DMMU_DEMAP
+	stxa		%g0, [%o0 + %o3] ASI_IMMU_DEMAP
+	membar		#Sync
+	brnz,pt		%o3, 1b
+	 sub		%o3, %o4, %o3
+2:	sethi		%hi(KERNBASE), %o3
+	flush		%o3
+	retl
+	 nop
+3:	mov		0x80, %o4
+	stxa		%g0, [%o4] ASI_DMMU_DEMAP
+	membar		#Sync
+	stxa		%g0, [%o4] ASI_IMMU_DEMAP
+	membar		#Sync
+	retl
+	 nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+
 #ifdef DCACHE_ALIASING_POSSIBLE
 __cheetah_flush_dcache_page: /* 11 insns */
 	sethi		%hi(PAGE_OFFSET), %g1
@@ -388,13 +456,15 @@ __hypervisor_flush_tlb_pending: /* 27 insns */
 	nop
 	nop
 
-__hypervisor_flush_tlb_kernel_range: /* 19 insns */
+__hypervisor_flush_tlb_kernel_range: /* 31 insns */
 	/* %o0=start, %o1=end */
 	cmp		%o0, %o1
 	be,pn		%xcc, 2f
-	 sethi		%hi(PAGE_SIZE), %g3
-	mov		%o0, %g1
-	sub		%o1, %g1, %g2
+	 sub		%o1, %o0, %g2
+	srlx		%g2, 18, %g3
+	brnz,pn		%g3, 4f
+	 mov		%o0, %g1
+	sethi		%hi(PAGE_SIZE), %g3
 	sub		%g2, %g3, %g2
 1:	add		%g1, %g2, %o0	/* ARG0: virtual address */
 	mov		0, %o1		/* ARG1: mmu context */
@@ -409,6 +479,16 @@ __hypervisor_flush_tlb_kernel_range: /* 19 insns */
 3:	sethi		%hi(__hypervisor_tlb_tl0_error), %o2
 	jmpl		%o2 + %lo(__hypervisor_tlb_tl0_error), %g0
 	 nop
+4:	mov		0, %o0		/* ARG0: CPU lists unimplemented */
+	mov		0, %o1		/* ARG1: CPU lists unimplemented */
+	mov		0, %o2		/* ARG2: mmu context = nucleus */
+	mov		HV_MMU_ALL, %o3	/* ARG3: flags */
+	mov		HV_FAST_MMU_DEMAP_CTX, %o5
+	ta		HV_FAST_TRAP
+	brnz,pn		%o0, 3b
+	 mov		HV_FAST_MMU_DEMAP_CTX, %o1
+	retl
+	 nop
 
 #ifdef DCACHE_ALIASING_POSSIBLE
 	/* XXX Niagara and friends have an 8K cache, so no aliasing is
@@ -431,43 +511,6 @@ tlb_patch_one:
 	retl
 	 nop
 
-	.globl		cheetah_patch_cachetlbops
-cheetah_patch_cachetlbops:
-	save		%sp, -128, %sp
-
-	sethi		%hi(__flush_tlb_mm), %o0
-	or		%o0, %lo(__flush_tlb_mm), %o0
-	sethi		%hi(__cheetah_flush_tlb_mm), %o1
-	or		%o1, %lo(__cheetah_flush_tlb_mm), %o1
-	call		tlb_patch_one
-	 mov		19, %o2
-
-	sethi		%hi(__flush_tlb_page), %o0
-	or		%o0, %lo(__flush_tlb_page), %o0
-	sethi		%hi(__cheetah_flush_tlb_page), %o1
-	or		%o1, %lo(__cheetah_flush_tlb_page), %o1
-	call		tlb_patch_one
-	 mov		22, %o2
-
-	sethi		%hi(__flush_tlb_pending), %o0
-	or		%o0, %lo(__flush_tlb_pending), %o0
-	sethi		%hi(__cheetah_flush_tlb_pending), %o1
-	or		%o1, %lo(__cheetah_flush_tlb_pending), %o1
-	call		tlb_patch_one
-	 mov		27, %o2
-
-#ifdef DCACHE_ALIASING_POSSIBLE
-	sethi		%hi(__flush_dcache_page), %o0
-	or		%o0, %lo(__flush_dcache_page), %o0
-	sethi		%hi(__cheetah_flush_dcache_page), %o1
-	or		%o1, %lo(__cheetah_flush_dcache_page), %o1
-	call		tlb_patch_one
-	 mov		11, %o2
-#endif /* DCACHE_ALIASING_POSSIBLE */
-
-	ret
-	 restore
-
 #ifdef CONFIG_SMP
 	/* These are all called by the slaves of a cross call, at
 	 * trap level 1, with interrupts fully disabled.
@@ -535,13 +578,15 @@ xcall_flush_tlb_page:	/* 20 insns */
 	nop
 
 	.globl		xcall_flush_tlb_kernel_range
-xcall_flush_tlb_kernel_range:	/* 28 insns */
+xcall_flush_tlb_kernel_range:	/* 44 insns */
 	sethi		%hi(PAGE_SIZE - 1), %g2
 	or		%g2, %lo(PAGE_SIZE - 1), %g2
 	andn		%g1, %g2, %g1
 	andn		%g7, %g2, %g7
 	sub		%g7, %g1, %g3
-	add		%g2, 1, %g2
+	srlx		%g3, 18, %g2
+	brnz,pn		%g2, 2f
+	 add		%g2, 1, %g2
 	sub		%g3, %g2, %g3
 	or		%g1, 0x20, %g1		! Nucleus
 1:	stxa		%g0, [%g1 + %g3] ASI_DMMU_DEMAP
@@ -550,11 +595,25 @@ xcall_flush_tlb_kernel_range:	/* 28 insns */
 	brnz,pt		%g3, 1b
 	 sub		%g3, %g2, %g3
 	retry
-	nop
-	nop
-	nop
-	nop
-	nop
+2:	mov		63 * 8, %g1
+1:	ldxa		[%g1] ASI_ITLB_DATA_ACCESS, %g2
+	andcc		%g2, 0x40, %g0			/* _PAGE_L_4U */
+	bne,pn		%xcc, 2f
+	 mov		TLB_TAG_ACCESS, %g2
+	stxa		%g0, [%g2] ASI_IMMU
+	stxa		%g0, [%g1] ASI_ITLB_DATA_ACCESS
+	membar		#Sync
+2:	ldxa		[%g1] ASI_DTLB_DATA_ACCESS, %g2
+	andcc		%g2, 0x40, %g0
+	bne,pn		%xcc, 2f
+	 mov		TLB_TAG_ACCESS, %g2
+	stxa		%g0, [%g2] ASI_DMMU
+	stxa		%g0, [%g1] ASI_DTLB_DATA_ACCESS
+	membar		#Sync
+2:	sub		%g1, 8, %g1
+	brgez,pt	%g1, 1b
+	 nop
+	retry
 	nop
 	nop
 	nop
@@ -683,6 +742,52 @@ xcall_fetch_glob_pmu_n4:
 
 	retry
 
+__cheetah_xcall_flush_tlb_kernel_range:	/* 44 insns */
+	sethi		%hi(PAGE_SIZE - 1), %g2
+	or		%g2, %lo(PAGE_SIZE - 1), %g2
+	andn		%g1, %g2, %g1
+	andn		%g7, %g2, %g7
+	sub		%g7, %g1, %g3
+	srlx		%g3, 18, %g2
+	brnz,pn		%g2, 2f
+	 add		%g2, 1, %g2
+	sub		%g3, %g2, %g3
+	or		%g1, 0x20, %g1		! Nucleus
+1:	stxa		%g0, [%g1 + %g3] ASI_DMMU_DEMAP
+	stxa		%g0, [%g1 + %g3] ASI_IMMU_DEMAP
+	membar		#Sync
+	brnz,pt		%g3, 1b
+	 sub		%g3, %g2, %g3
+	retry
+2:	mov		0x80, %g2
+	stxa		%g0, [%g2] ASI_DMMU_DEMAP
+	membar		#Sync
+	stxa		%g0, [%g2] ASI_IMMU_DEMAP
+	membar		#Sync
+	retry
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+
 #ifdef DCACHE_ALIASING_POSSIBLE
 	.align		32
 	.globl		xcall_flush_dcache_page_cheetah
@@ -798,18 +903,20 @@ __hypervisor_xcall_flush_tlb_page: /* 20 insns */
 	 nop
 
 	.globl		__hypervisor_xcall_flush_tlb_kernel_range
-__hypervisor_xcall_flush_tlb_kernel_range: /* 28 insns */
+__hypervisor_xcall_flush_tlb_kernel_range: /* 44 insns */
 	/* %g1=start, %g7=end, g2,g3,g4,g5,g6=scratch */
 	sethi		%hi(PAGE_SIZE - 1), %g2
 	or		%g2, %lo(PAGE_SIZE - 1), %g2
 	andn		%g1, %g2, %g1
 	andn		%g7, %g2, %g7
 	sub		%g7, %g1, %g3
+	srlx		%g3, 18, %g7
 	add		%g2, 1, %g2
 	sub		%g3, %g2, %g3
 	mov		%o0, %g2
 	mov		%o1, %g4
-	mov		%o2, %g7
+	brnz,pn		%g7, 2f
+	 mov		%o2, %g7
 1:	add		%g1, %g3, %o0	/* ARG0: virtual address */
 	mov		0, %o1		/* ARG1: mmu context */
 	mov		HV_MMU_ALL, %o2	/* ARG2: flags */
@@ -820,7 +927,7 @@ __hypervisor_xcall_flush_tlb_kernel_range: /* 28 insns */
 	sethi		%hi(PAGE_SIZE), %o2
 	brnz,pt		%g3, 1b
 	 sub		%g3, %o2, %g3
-	mov		%g2, %o0
+5:	mov		%g2, %o0
 	mov		%g4, %o1
 	mov		%g7, %o2
 	membar		#Sync
@@ -828,6 +935,20 @@ __hypervisor_xcall_flush_tlb_kernel_range: /* 28 insns */
 1:	sethi		%hi(__hypervisor_tlb_xcall_error), %g4
 	jmpl		%g4 + %lo(__hypervisor_tlb_xcall_error), %g0
 	 nop
+2:	mov		%o3, %g1
+	mov		%o5, %g3
+	mov		0, %o0		/* ARG0: CPU lists unimplemented */
+	mov		0, %o1		/* ARG1: CPU lists unimplemented */
+	mov		0, %o2		/* ARG2: mmu context = nucleus */
+	mov		HV_MMU_ALL, %o3	/* ARG3: flags */
+	mov		HV_FAST_MMU_DEMAP_CTX, %o5
+	ta		HV_FAST_TRAP
+	mov		%g1, %o3
+	brz,pt		%o0, 5b
+	 mov		%g3, %o5
+	mov		HV_FAST_MMU_DEMAP_CTX, %g6
+	ba,pt		%xcc, 1b
+	 clr		%g5
 
 	/* These just get rescheduled to PIL vectors. */
 	.globl		xcall_call_function
@@ -864,6 +985,58 @@ xcall_kgdb_capture:
 
 #endif /* CONFIG_SMP */
 
+	.globl		cheetah_patch_cachetlbops
+cheetah_patch_cachetlbops:
+	save		%sp, -128, %sp
+
+	sethi		%hi(__flush_tlb_mm), %o0
+	or		%o0, %lo(__flush_tlb_mm), %o0
+	sethi		%hi(__cheetah_flush_tlb_mm), %o1
+	or		%o1, %lo(__cheetah_flush_tlb_mm), %o1
+	call		tlb_patch_one
+	 mov		19, %o2
+
+	sethi		%hi(__flush_tlb_page), %o0
+	or		%o0, %lo(__flush_tlb_page), %o0
+	sethi		%hi(__cheetah_flush_tlb_page), %o1
+	or		%o1, %lo(__cheetah_flush_tlb_page), %o1
+	call		tlb_patch_one
+	 mov		22, %o2
+
+	sethi		%hi(__flush_tlb_pending), %o0
+	or		%o0, %lo(__flush_tlb_pending), %o0
+	sethi		%hi(__cheetah_flush_tlb_pending), %o1
+	or		%o1, %lo(__cheetah_flush_tlb_pending), %o1
+	call		tlb_patch_one
+	 mov		27, %o2
+
+	sethi		%hi(__flush_tlb_kernel_range), %o0
+	or		%o0, %lo(__flush_tlb_kernel_range), %o0
+	sethi		%hi(__cheetah_flush_tlb_kernel_range), %o1
+	or		%o1, %lo(__cheetah_flush_tlb_kernel_range), %o1
+	call		tlb_patch_one
+	 mov		31, %o2
+
+#ifdef DCACHE_ALIASING_POSSIBLE
+	sethi		%hi(__flush_dcache_page), %o0
+	or		%o0, %lo(__flush_dcache_page), %o0
+	sethi		%hi(__cheetah_flush_dcache_page), %o1
+	or		%o1, %lo(__cheetah_flush_dcache_page), %o1
+	call		tlb_patch_one
+	 mov		11, %o2
+#endif /* DCACHE_ALIASING_POSSIBLE */
+
+#ifdef CONFIG_SMP
+	sethi		%hi(xcall_flush_tlb_kernel_range), %o0
+	or		%o0, %lo(xcall_flush_tlb_kernel_range), %o0
+	sethi		%hi(__cheetah_xcall_flush_tlb_kernel_range), %o1
+	or		%o1, %lo(__cheetah_xcall_flush_tlb_kernel_range), %o1
+	call		tlb_patch_one
+	 mov		44, %o2
+#endif /* CONFIG_SMP */
+
+	ret
+	 restore
 
 	.globl		hypervisor_patch_cachetlbops
 hypervisor_patch_cachetlbops:
@@ -895,7 +1068,7 @@ hypervisor_patch_cachetlbops:
 	sethi		%hi(__hypervisor_flush_tlb_kernel_range), %o1
 	or		%o1, %lo(__hypervisor_flush_tlb_kernel_range), %o1
 	call		tlb_patch_one
-	 mov		19, %o2
+	 mov		31, %o2
 
 #ifdef DCACHE_ALIASING_POSSIBLE
 	sethi		%hi(__flush_dcache_page), %o0
@@ -926,7 +1099,7 @@ hypervisor_patch_cachetlbops:
 	sethi		%hi(__hypervisor_xcall_flush_tlb_kernel_range), %o1
 	or		%o1, %lo(__hypervisor_xcall_flush_tlb_kernel_range), %o1
 	call		tlb_patch_one
-	 mov		28, %o2
+	 mov		44, %o2
 #endif /* CONFIG_SMP */
 
 	ret
-- 
2.1.2.532.g19b5d50