From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id oBD1LrIn147865 for ; Sun, 12 Dec 2010 19:21:53 -0600 Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id A078D1E29A4 for ; Sun, 12 Dec 2010 17:23:43 -0800 (PST) Received: from mail.internode.on.net (bld-mail15.adl6.internode.on.net [150.101.137.100]) by cuda.sgi.com with ESMTP id oZPiz5LLQfgT719x for ; Sun, 12 Dec 2010 17:23:43 -0800 (PST) Received: from dastard (unverified [121.44.88.148]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 38234065-1927428 for ; Mon, 13 Dec 2010 11:53:42 +1030 (CDT) From: Dave Chinner Subject: =?UTF-8?q?=5BPATCH=201/3=5D=20lib=3A=20percpu=20counter=20add=20unless=20less=20than=20functionality?= Date: Mon, 13 Dec 2010 12:21:51 +1100 Message-Id: <1292203313-15570-2-git-send-email-david@fromorbit.com> In-Reply-To: <1292203313-15570-1-git-send-email-david@fromorbit.com> References: <1292203313-15570-1-git-send-email-david@fromorbit.com> MIME-Version: 1.0 List-Id: XFS Filesystem from SGI List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: xfs-bounces@oss.sgi.com Errors-To: xfs-bounces@oss.sgi.com To: xfs@oss.sgi.com Cc: linux-kernel@vger.kernel.org, a.p.zijlstra@chello.nl RnJvbTogRGF2ZSBDaGlubmVyIDxkY2hpbm5lckByZWRoYXQuY29tPgoKVG8gdXNlIHRoZSBnZW5l cmljIHBlcmNwdSBjb3VudGVyIGluZnJhc3RydWN0dXJlIGZvciBjb3VudGVycyB0aGF0CnJlcXVp cmUgY29uZGl0aW9uYWwgYWRkaXRpb24gYmFzZWQgb24gYSB0aHJlc2hvbGQgdmFsdWUgd2UgbmVl ZApzcGVjaWFsIGhhbmRsaW5nIG9mIHRoZSBjb3VudGVyLiBGdXJ0aGVyLCB0aGUgY2FsbGVyIG5l ZWRzIHRvIGtub3cKdGhlIHN0YXR1cyBvZiB0aGUgY29uZGl0aW9uYWwgYWRkaXRpb24gdG8gZGV0 ZXJtaW5lIHdoYXQgYWN0aW9uIHRvCnRha2UgZGVwZW5kaW5nIG9uIHdoZXRoZXIgdGhlIGFkZGl0 aW9uIG9jY3VycmVkIG9yIG5vdC4gIEV4YW1wbGVzIG9mCnRoaXMgc29ydCBvZiB1c2FnZSBhcmUg cmVzb3VyY2UgY291bnRlcnMgdGhhdCBjYW5ub3QgZ28gYmVsb3cgemVybwooZS5nLiBmaWxlc3lz dGVtIGZyZWUgYmxvY2tzKS4KClRvIGFsbG93IFhGUyB0byByZXBsYWNlIGl0J3MgY29tcGxleCBy b2xsLXlvdXItb3duIHBlci1jcHUKc3VwZXJibG9jayBjb3VudGVycywgYSBzaW5nbGUgZ2VuZXJp YyBjb25kaXRpb25hbCBmdW5jdGlvbiBpcwpyZXF1aXJlZDogcGVyY3B1X2NvdW50ZXJfYWRkX3Vu bGVzc19sdCgpLiBUaGlzIHdpbGwgYWRkIHRoZSBhbW91bnQKdG8gdGhlIGNvdW50ZXIgdW5sZXNz IHRoZSByZXN1bHQgd291bGQgYmUgbGVzcyB0aGFuIHRoZSBnaXZlbgp0aHJlc2hvbGQuIEEgY2Fs bGVyIHN1cHBsaWVkIHRocmVzaG9sZCBpcyByZXF1aXJlZCBiZWNhdXNlIFhGUyBkb2VzCm5vdCBu ZWNlc3NhcmlseSB1c2UgdGhlIHNhbWUgdGhyZXNob2xkIGZvciBldmVyeSBjb3VudGVyLgoKcGVy Y3B1X2NvdW50ZXJfYWRkX3VubGVzc19sdCgpIGF0dGVtcHRzIHRvIG1pbmltaXNlIGNvdW50ZXIg bG9jawp0cmF2ZXJzYWxzIGJ5IG9ubHkgdGFraW5nIHRoZSBjb3VudGVyIGxvY2sgd2hlbiB0aGUg dGhyZXNob2xkIGlzCndpdGhpbiB0aGUgZXJyb3IgcmFuZ2Ugb2YgdGhlIGN1cnJlbnQgY291bnRl ciB2YWx1ZS4gSGVuY2Ugd2hlbiB0aGUKdGhyZXNob2xkIGlzIG5vdCB3aXRoaW4gdGhlIGNvdW50 ZXIgZXJyb3IgcmFuZ2UsIHRoZSBjb3VudGVyIHdpbGwKc3RpbGwgaGF2ZSB0aGUgc2FtZSBzY2Fs YWJpbGl0eSBjaGFyYWN0ZXJpc3RpY3MgYXMgdGhlIG5vcm1hbApwZXJjcHVfY291bnRlcl9hZGQo KSBmdW5jdGlvbi4KCkFkZGluZyB0aGlzIGZ1bmN0aW9uYWxpdHkgdG8gdGhlIGdlbmVyaWMgcGVy Y3B1IGNvdW50ZXJzIGFsbG93cyB1cwp0byByZW1vdmUgdGhlIG11Y2ggbW9yZSBjb21wbGV4IGFu ZCBsZXNzIGVmZmljaWVudCBYRlMgcGVyY3B1CmNvdW50ZXIgY29kZSAofjcwMCBsaW5lcyBvZiBj b2RlKSBhbmQgcmVwbGFjZSBpdCB3aXRoIGdlbmVyaWMKcGVyY3B1IGNvdW50ZXJzLgoKU2lnbmVk LW9mZi1ieTogRGF2ZSBDaGlubmVyIDxkY2hpbm5lckByZWRoYXQuY29tPgotLS0KIGluY2x1ZGUv bGludXgvcGVyY3B1X2NvdW50ZXIuaCB8ICAgMjcgKysrKysrKysrKysrKysKIGxpYi9wZXJjcHVf Y291bnRlci5jICAgICAgICAgICB8ICAgNzkgKysrKysrKysrKysrKysrKysrKysrKysrKysrKysr KysrKysrKysrKwogMiBmaWxlcyBjaGFuZ2VkLCAxMDYgaW5zZXJ0aW9ucygrKSwgMCBkZWxldGlv bnMoLSkKCmRpZmYgLS1naXQgYS9pbmNsdWRlL2xpbnV4L3BlcmNwdV9jb3VudGVyLmggYi9pbmNs dWRlL2xpbnV4L3BlcmNwdV9jb3VudGVyLmgKaW5kZXggNDZmNmJhNS4uYWQxODc3OSAxMDA2NDQK LS0tIGEvaW5jbHVkZS9saW51eC9wZXJjcHVfY291bnRlci5oCisrKyBiL2luY2x1ZGUvbGludXgv cGVyY3B1X2NvdW50ZXIuaApAQCAtNDEsMTIgKzQxLDIxIEBAIHZvaWQgcGVyY3B1X2NvdW50ZXJf c2V0KHN0cnVjdCBwZXJjcHVfY291bnRlciAqZmJjLCBzNjQgYW1vdW50KTsKIHZvaWQgX19wZXJj cHVfY291bnRlcl9hZGQoc3RydWN0IHBlcmNwdV9jb3VudGVyICpmYmMsIHM2NCBhbW91bnQsIHMz MiBiYXRjaCk7CiBzNjQgX19wZXJjcHVfY291bnRlcl9zdW0oc3RydWN0IHBlcmNwdV9jb3VudGVy ICpmYmMpOwogaW50IHBlcmNwdV9jb3VudGVyX2NvbXBhcmUoc3RydWN0IHBlcmNwdV9jb3VudGVy ICpmYmMsIHM2NCByaHMpOworaW50IF9fcGVyY3B1X2NvdW50ZXJfYWRkX3VubGVzc19sdChzdHJ1 Y3QgcGVyY3B1X2NvdW50ZXIgKmZiYywgczY0IGFtb3VudCwKKwkJCQkJCXM2NCB0aHJlc2hvbGQs IHMzMiBiYXRjaCk7CiAKIHN0YXRpYyBpbmxpbmUgdm9pZCBwZXJjcHVfY291bnRlcl9hZGQoc3Ry dWN0IHBlcmNwdV9jb3VudGVyICpmYmMsIHM2NCBhbW91bnQpCiB7CiAJX19wZXJjcHVfY291bnRl cl9hZGQoZmJjLCBhbW91bnQsIHBlcmNwdV9jb3VudGVyX2JhdGNoKTsKIH0KIAorc3RhdGljIGlu bGluZSBpbnQgcGVyY3B1X2NvdW50ZXJfYWRkX3VubGVzc19sdChzdHJ1Y3QgcGVyY3B1X2NvdW50 ZXIgKmZiYywKKwkJCQkJczY0IGFtb3VudCwgczY0IHRocmVzaG9sZCkKK3sKKwlyZXR1cm4gX19w ZXJjcHVfY291bnRlcl9hZGRfdW5sZXNzX2x0KGZiYywgYW1vdW50LCB0aHJlc2hvbGQsCisJCQkJ CXBlcmNwdV9jb3VudGVyX2JhdGNoKTsKK30KKwogc3RhdGljIGlubGluZSBzNjQgcGVyY3B1X2Nv dW50ZXJfc3VtX3Bvc2l0aXZlKHN0cnVjdCBwZXJjcHVfY291bnRlciAqZmJjKQogewogCXM2NCBy ZXQgPSBfX3BlcmNwdV9jb3VudGVyX3N1bShmYmMpOwpAQCAtMTUzLDYgKzE2MiwyNCBAQCBzdGF0 aWMgaW5saW5lIGludCBwZXJjcHVfY291bnRlcl9pbml0aWFsaXplZChzdHJ1Y3QgcGVyY3B1X2Nv dW50ZXIgKmZiYykKIAlyZXR1cm4gMTsKIH0KIAorc3RhdGljIGlubGluZSBpbnQgcGVyY3B1X2Nv dW50ZXJfYWRkX3VubGVzc19sdChzdHJ1Y3QgcGVyY3B1X2NvdW50ZXIgKmZiYywgczY0IGFtb3Vu dCwKKwkJCQkJCQlzNjQgdGhyZXNob2xkKQoreworCXM2NCBjb3VudDsKKwlpbnQgcmV0ID0g4oCQ MTsKKworCXByZWVtcHRfZGlzYWJsZSgpOworCWNvdW50ID0gZmJjLT5jb3VudCArIGFtb3VudDsK KwlpZiAoY291bnQgPCB0aHJlc2hvbGQpCisJCWdvdG8gb3V0OworCWZiYy0+Y291bnQgPSBjb3Vu dDsKKwlyZXQgPSBjb3VudCA9PSB0aHJlc2hvbGQgPyAwIDogMTsKK291dDoKKwlwcmVlbXB0X2Vu YWJsZSgpOworCXJldHVybiByZXQ7Cit9CisKKwogI2VuZGlmCS8qIENPTkZJR19TTVAgKi8KIAog c3RhdGljIGlubGluZSB2b2lkIHBlcmNwdV9jb3VudGVyX2luYyhzdHJ1Y3QgcGVyY3B1X2NvdW50 ZXIgKmZiYykKZGlmZiAtLWdpdCBhL2xpYi9wZXJjcHVfY291bnRlci5jIGIvbGliL3BlcmNwdV9j b3VudGVyLmMKaW5kZXggNjA0Njc4ZC4uZWFjY2NiNyAxMDA2NDQKLS0tIGEvbGliL3BlcmNwdV9j b3VudGVyLmMKKysrIGIvbGliL3BlcmNwdV9jb3VudGVyLmMKQEAgLTIxMyw2ICsyMTMsODUgQEAg aW50IHBlcmNwdV9jb3VudGVyX2NvbXBhcmUoc3RydWN0IHBlcmNwdV9jb3VudGVyICpmYmMsIHM2 NCByaHMpCiB9CiBFWFBPUlRfU1lNQk9MKHBlcmNwdV9jb3VudGVyX2NvbXBhcmUpOwogCisvKioK KyAqIF9fcGVyY3B1X2NvdW50ZXJfYWRkX3VubGVzc19sdCAtIGFkZCB0byBhIGNvdW50ZXIgYXZv aWRpbmcgdW5kZXJydW5zCisgKiBAZmJjOgljb3VudGVyCisgKiBAYW1vdW50OglhbW91bnQgdG8g YWRkCisgKiBAdGhyZXNob2xkOgl1bmRlcnJ1biB0aHJlc2hvbGQKKyAqIEBiYXRjaDoJcGVyY3B1 IGNvdW50ZXIgYmF0Y2ggc2l6ZS4KKyAqCisgKiBBZGQgQGFtb3VudCB0byBAZmRjIGlmIGFuZCBv bmx5IGlmIHJlc3VsdCBvZiBhZGRpdGlvbiBpcyBncmVhdGVyIHRoYW4gb3IKKyAqIGVxdWFsIHRv IEB0aHJlc2hvbGQgIFJldHVybiAxIGlmIGdyZWF0ZXIgYW5kIGFkZGVkLCAwIGlmIGVxdWFsIGFu ZCBhZGRlZAorICogYW5kIC0xIGlmIGFuZCB1bmRlcnJ1biB3b3VsZCBoYXZlIG9jY3VyZWQuCisg KgorICogVGhpcyBpcyB1c2VmdWwgZm9yIG9wZXJhdGlvbnMgdGhhdCBtdXN0IGFjY3VyYXRlbHkg YW5kIGF0b21pY2FsbHkgb25seSBhZGQgYQorICogZGVsdGEgdG8gYSBjb3VudGVyIGlmIHRoZSBy ZXN1bHQgaXMgZ3JlYXRlciB0aGFuIGEgZ2l2ZW4gKGUuZy4gZm9yIGZyZWVzcGFjZQorICogYWNj b3VudGluZyB3aXRoIEVOT1NQQyBjaGVja2luZyBpbiBmaWxlc3lzdGVtcykuCisgKi8KK2ludCBf X3BlcmNwdV9jb3VudGVyX2FkZF91bmxlc3NfbHQoc3RydWN0IHBlcmNwdV9jb3VudGVyICpmYmMs IHM2NCBhbW91bnQsCisJCQkJCQlzNjQgdGhyZXNob2xkLCBzMzIgYmF0Y2gpCit7CisJczY0CWNv dW50OworCXM2NAllcnJvciA9IDIgKiBiYXRjaCAqIG51bV9vbmxpbmVfY3B1cygpOworCWludAlj cHU7CisJaW50CXJldCA9IC0xOworCisJcHJlZW1wdF9kaXNhYmxlKCk7CisKKwkvKiBDaGVjayB0 byBzZWUgaWYgcm91Z2ggY291bnQgd2lsbCBiZSBzdWZmaWNpZW50IGZvciBjb21wYXJpc29uICov CisJY291bnQgPSBwZXJjcHVfY291bnRlcl9yZWFkKGZiYyk7CisJaWYgKGNvdW50ICsgYW1vdW50 IDwgdGhyZXNob2xkIC0gZXJyb3IpCisJCWdvdG8gb3V0OworCisJLyoKKwkgKiBJZiB0aGUgY291 bnRlciBpcyBvdmVyIHRoZSB0aHJlc2hvbGQgYW5kIHRoZSBjaGFuZ2UgaXMgbGVzcyB0aGFuIHRo ZQorCSAqIGJhdGNoIHNpemUsIHdlIG1pZ2h0IGJlIGFibGUgdG8gYXZvaWQgbG9ja2luZy4KKwkg Ki8KKwlpZiAoY291bnQgPiB0aHJlc2hvbGQgKyBlcnJvciAmJiBhYnMoYW1vdW50KSA8IGJhdGNo KSB7CisJCV9fcGVyY3B1X2NvdW50ZXJfYWRkKGZiYywgYW1vdW50LCBiYXRjaCk7CisJCXJldCA9 IDE7CisJCWdvdG8gb3V0OworCX0KKworCS8qCisJICogSWYgdGhlIHJlc3VsdCBpcyBvdmVyIHRo ZSBlcnJvciB0aHJlc2hvbGQsIHdlIGNhbiBqdXN0IGFkZCBpdAorCSAqIGludG8gdGhlIGdsb2Jh bCBjb3VudGVyIGlnbm9yaW5nIHdoYXQgaXMgaW4gdGhlIHBlci1jcHUgY291bnRlcnMKKwkgKiBh cyB0aGV5IHdpbGwgbm90IGNoYW5nZSB0aGUgcmVzdWx0IG9mIHRoZSBjYWxjdWxhdGlvbi4KKwkg Ki8KKwlzcGluX2xvY2soJmZiYy0+bG9jayk7CisJaWYgKGZiYy0+Y291bnQgKyBhbW91bnQgPiB0 aHJlc2hvbGQgKyBlcnJvcikgeworCQlmYmMtPmNvdW50ICs9IGFtb3VudDsKKwkJcmV0ID0gMTsK KwkJZ290byBvdXRfdW5sb2NrOworCX0KKworCS8qCisJICogUmVzdWx0IGlzIHdpdGhpbmcgdGhl IGVycm9yIG1hcmdpbi4gUnVuIGFuIG9wZW4tY29kZWQgc3VtIG9mIHRoZQorCSAqIHBlci1jcHUg Y291bnRlcnMgdG8gZ2V0IHRoZSBleGFjdCB2YWx1ZSBhdCB0aGlzIHBvaW50IGluIHRpbWUsCisJ ICogYW5kIGlmIHRoZSByZXN1bHQgZ3JlYXRlciB0aGFuIHRoZSB0aHJlc2hvbGQsIGFkZCB0aGUg YW1vdW50IHRvCisJICogdGhlIGdsb2JhbCBjb3VudGVyLgorCSAqLworCWNvdW50ID0gZmJjLT5j b3VudDsKKwlmb3JfZWFjaF9vbmxpbmVfY3B1KGNwdSkgeworCQlzMzIgKnBjb3VudCA9IHBlcl9j cHVfcHRyKGZiYy0+Y291bnRlcnMsIGNwdSk7CisJCWNvdW50ICs9ICpwY291bnQ7CisJfQorCVdB Uk5fT04oY291bnQgPCB0aHJlc2hvbGQpOworCisJaWYgKGNvdW50ICsgYW1vdW50ID49IHRocmVz aG9sZCkgeworCQlyZXQgPSAwOworCQlpZiAoY291bnQgKyBhbW91bnQgPiB0aHJlc2hvbGQpCisJ CQlyZXQgPSAxOworCQlmYmMtPmNvdW50ICs9IGFtb3VudDsKKwl9CitvdXRfdW5sb2NrOgorCXNw aW5fdW5sb2NrKCZmYmMtPmxvY2spOworb3V0OgorCXByZWVtcHRfZW5hYmxlKCk7CisJcmV0dXJu IHJldDsKK30KK0VYUE9SVF9TWU1CT0wocGVyY3B1X2NvdW50ZXJfYWRkX3VubGVzc19sdCk7CisK IHN0YXRpYyBpbnQgX19pbml0IHBlcmNwdV9jb3VudGVyX3N0YXJ0dXAodm9pZCkKIHsKIAljb21w dXRlX2JhdGNoX3ZhbHVlKCk7Ci0tIAoxLjcuMi4zCgpfX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fXwp4ZnMgbWFpbGluZyBsaXN0Cnhmc0Bvc3Muc2dpLmNvbQpo dHRwOi8vb3NzLnNnaS5jb20vbWFpbG1hbi9saXN0aW5mby94ZnMK From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756138Ab0LMBXw (ORCPT ); Sun, 12 Dec 2010 20:23:52 -0500 Received: from bld-mail17.adl2.internode.on.net ([150.101.137.102]:36395 "EHLO mail.internode.on.net" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1755716Ab0LMBXu (ORCPT ); Sun, 12 Dec 2010 20:23:50 -0500 From: Dave Chinner To: xfs@oss.sgi.com Cc: linux-kernel@vger.kernel.org, a.p.zijlstra@chello.nl Subject: =?UTF-8?q?=5BPATCH=201/3=5D=20lib=3A=20percpu=20counter=20add=20unless=20less=20than=20functionality?= Date: Mon, 13 Dec 2010 12:21:51 +1100 Message-Id: <1292203313-15570-2-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 1.7.2.3 In-Reply-To: <1292203313-15570-1-git-send-email-david@fromorbit.com> References: <1292203313-15570-1-git-send-email-david@fromorbit.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Dave Chinner To use the generic percpu counter infrastructure for counters that require conditional addition based on a threshold value we need special handling of the counter. Further, the caller needs to know the status of the conditional addition to determine what action to take depending on whether the addition occurred or not. Examples of this sort of usage are resource counters that cannot go below zero (e.g. filesystem free blocks). To allow XFS to replace it's complex roll-your-own per-cpu superblock counters, a single generic conditional function is required: percpu_counter_add_unless_lt(). This will add the amount to the counter unless the result would be less than the given threshold. A caller supplied threshold is required because XFS does not necessarily use the same threshold for every counter. percpu_counter_add_unless_lt() attempts to minimise counter lock traversals by only taking the counter lock when the threshold is within the error range of the current counter value. Hence when the threshold is not within the counter error range, the counter will still have the same scalability characteristics as the normal percpu_counter_add() function. Adding this functionality to the generic percpu counters allows us to remove the much more complex and less efficient XFS percpu counter code (~700 lines of code) and replace it with generic percpu counters. Signed-off-by: Dave Chinner --- include/linux/percpu_counter.h | 27 ++++++++++++++ lib/percpu_counter.c | 79 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 106 insertions(+), 0 deletions(-) diff --git a/include/linux/percpu_counter.h b/include/linux/percpu_counter.h index 46f6ba5..ad18779 100644 --- a/include/linux/percpu_counter.h +++ b/include/linux/percpu_counter.h @@ -41,12 +41,21 @@ void percpu_counter_set(struct percpu_counter *fbc, s64 amount); void __percpu_counter_add(struct percpu_counter *fbc, s64 amount, s32 batch); s64 __percpu_counter_sum(struct percpu_counter *fbc); int percpu_counter_compare(struct percpu_counter *fbc, s64 rhs); +int __percpu_counter_add_unless_lt(struct percpu_counter *fbc, s64 amount, + s64 threshold, s32 batch); static inline void percpu_counter_add(struct percpu_counter *fbc, s64 amount) { __percpu_counter_add(fbc, amount, percpu_counter_batch); } +static inline int percpu_counter_add_unless_lt(struct percpu_counter *fbc, + s64 amount, s64 threshold) +{ + return __percpu_counter_add_unless_lt(fbc, amount, threshold, + percpu_counter_batch); +} + static inline s64 percpu_counter_sum_positive(struct percpu_counter *fbc) { s64 ret = __percpu_counter_sum(fbc); @@ -153,6 +162,24 @@ static inline int percpu_counter_initialized(struct percpu_counter *fbc) return 1; } +static inline int percpu_counter_add_unless_lt(struct percpu_counter *fbc, s64 amount, + s64 threshold) +{ + s64 count; + int ret = ‐1; + + preempt_disable(); + count = fbc->count + amount; + if (count < threshold) + goto out; + fbc->count = count; + ret = count == threshold ? 0 : 1; +out: + preempt_enable(); + return ret; +} + + #endif /* CONFIG_SMP */ static inline void percpu_counter_inc(struct percpu_counter *fbc) diff --git a/lib/percpu_counter.c b/lib/percpu_counter.c index 604678d..eacccb7 100644 --- a/lib/percpu_counter.c +++ b/lib/percpu_counter.c @@ -213,6 +213,85 @@ int percpu_counter_compare(struct percpu_counter *fbc, s64 rhs) } EXPORT_SYMBOL(percpu_counter_compare); +/** + * __percpu_counter_add_unless_lt - add to a counter avoiding underruns + * @fbc: counter + * @amount: amount to add + * @threshold: underrun threshold + * @batch: percpu counter batch size. + * + * Add @amount to @fdc if and only if result of addition is greater than or + * equal to @threshold Return 1 if greater and added, 0 if equal and added + * and -1 if and underrun would have occured. + * + * This is useful for operations that must accurately and atomically only add a + * delta to a counter if the result is greater than a given (e.g. for freespace + * accounting with ENOSPC checking in filesystems). + */ +int __percpu_counter_add_unless_lt(struct percpu_counter *fbc, s64 amount, + s64 threshold, s32 batch) +{ + s64 count; + s64 error = 2 * batch * num_online_cpus(); + int cpu; + int ret = -1; + + preempt_disable(); + + /* Check to see if rough count will be sufficient for comparison */ + count = percpu_counter_read(fbc); + if (count + amount < threshold - error) + goto out; + + /* + * If the counter is over the threshold and the change is less than the + * batch size, we might be able to avoid locking. + */ + if (count > threshold + error && abs(amount) < batch) { + __percpu_counter_add(fbc, amount, batch); + ret = 1; + goto out; + } + + /* + * If the result is over the error threshold, we can just add it + * into the global counter ignoring what is in the per-cpu counters + * as they will not change the result of the calculation. + */ + spin_lock(&fbc->lock); + if (fbc->count + amount > threshold + error) { + fbc->count += amount; + ret = 1; + goto out_unlock; + } + + /* + * Result is withing the error margin. Run an open-coded sum of the + * per-cpu counters to get the exact value at this point in time, + * and if the result greater than the threshold, add the amount to + * the global counter. + */ + count = fbc->count; + for_each_online_cpu(cpu) { + s32 *pcount = per_cpu_ptr(fbc->counters, cpu); + count += *pcount; + } + WARN_ON(count < threshold); + + if (count + amount >= threshold) { + ret = 0; + if (count + amount > threshold) + ret = 1; + fbc->count += amount; + } +out_unlock: + spin_unlock(&fbc->lock); +out: + preempt_enable(); + return ret; +} +EXPORT_SYMBOL(percpu_counter_add_unless_lt); + static int __init percpu_counter_startup(void) { compute_batch_value(); -- 1.7.2.3