From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jike Song Subject: Re: [ANNOUNCE][RFC] KVMGT - the implementation of Intel GVT-g(full GPU virtualization) for KVM Date: Tue, 27 Oct 2015 17:36:52 +0800 Message-ID: <562F45B4.70004@intel.com> References: <547FC5DE.4010701@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8"; Format="flowed" Content-Transfer-Encoding: base64 Return-path: Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by gabe.freedesktop.org (Postfix) with ESMTP id 558316EA6B for ; Tue, 27 Oct 2015 02:36:31 -0700 (PDT) In-Reply-To: <547FC5DE.4010701@intel.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" To: kvm@vger.kernel.org, igvt-g@lists.01.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org Cc: "Reddy, Raghuveer" , "Cowperthwaite, David J" , "Li, Susie" , "Dong, Eddie" , "Downs, Mike" List-Id: intel-gfx@lists.freedesktop.org SGkgYWxsLAoKV2UgYXJlIHBsZWFzZWQgdG8gYW5ub3VuY2UgYW5vdGhlciB1cGRhdGUgb2YgSW50 ZWwgR1ZULWcgZm9yIEtWTS4KCkludGVsIEdWVC1nIGlzIGEgZnVsbCBHUFUgdmlydHVhbGl6YXRp b24gc29sdXRpb24gd2l0aCBtZWRpYXRlZCBwYXNzLXRocm91Z2gsIHN0YXJ0aW5nIGZyb20gNHRo IGdlbmVyYXRpb24gSW50ZWwgQ29yZShUTSkgcHJvY2Vzc29ycyB3aXRoIEludGVsIEdyYXBoaWNz IHByb2Nlc3NvcnMuIEEgdmlydHVhbCBHUFUgaW5zdGFuY2UgaXMgbWFpbnRhaW5lZCBmb3IgZWFj aCBWTSwgd2l0aCBwYXJ0IG9mIHBlcmZvcm1hbmNlIGNyaXRpY2FsIHJlc291cmNlcyBkaXJlY3Rs eSBhc3NpZ25lZC4gVGhlIGNhcGFiaWxpdHkgb2YgcnVubmluZyBuYXRpdmUgZ3JhcGhpY3MgZHJp dmVyIGluc2lkZSBhIFZNLCB3aXRob3V0IGh5cGVydmlzb3IgaW50ZXJ2ZW50aW9uIGluIHBlcmZv cm1hbmNlIGNyaXRpY2FsIHBhdGhzLCBhY2hpZXZlcyBhIGdvb2QgYmFsYW5jZSBhbW9uZyBwZXJm b3JtYW5jZSwgZmVhdHVyZSwgYW5kIHNoYXJpbmcgY2FwYWJpbGl0eS4gS1ZNIGlzIHN1cHBvcnRl ZCBieSBJbnRlbCBHVlQtZyhhLmsuYS4gS1ZNR1QpLgoKClJlcG9zaXRvcmllcwoKICAgICBLZXJu ZWw6IGh0dHBzOi8vZ2l0aHViLmNvbS8wMW9yZy9pZ3Z0Zy1rZXJuZWwgKDIwMTVxMy0zLjE4LjAg YnJhbmNoKQogICAgIFFlbXU6IGh0dHBzOi8vZ2l0aHViLmNvbS8wMW9yZy9pZ3Z0Zy1xZW11IChr dm1ndF9wdWJsaWMyMDE1cTMgYnJhbmNoKQoKClRoaXMgdXBkYXRlIGNvbnNpc3RzIG9mOgoKICAg ICAtIEtWTUdUIGlzIG5vdyBtZXJnZWQgd2l0aCBYZW5HVCBpbiB1bmlmaWVkIHJlcG9zaXRvcmll cyhrZXJuZWwgYW5kIHFlbXUpLCBidXQgY3VycmVudGx5CiAgICAgICBkaWZmZXJlbnQgYnJhbmNo ZXMgZm9yIHFlbXUuICBLVk1HVCBhbmQgWGVuR1Qgc2hhcmUgc2FtZSBpR1ZULWcgY29yZSBsb2dp Yy4KICAgICAtIFBQR1RUIHN1cHBvcnRlZCwgaGVuY2UgdGhlIFdpbmRvd3MgZ3Vlc3Qgc3VwcG9y dAogICAgIC0gS1ZNR1Qgbm93IHN1cHBvcnRzIGJvdGggNHRoIGdlbmVyYXRpb24gKEhhc3dlbGwp IGFuZCA1dGggZ2VuZXJhdGlvbiAoQnJvYWR3ZWxsKSBJbnRlbCBDb3JlKFRNKSBwcm9jZXNzb3Jz CiAgICAgLSAyRC8zRC9NZWRpYSBkZWNvZGluZyBoYXZlIGJlZW4gdmFsaWRhdGVkIG9uIFVidW50 dSAxNC4wNCBhbmQgV2luZG93czcvV2luZG93cyA4LjEKCk5leHQgdXBkYXRlIHdpbGwgYmUgYXJv dW5kIGVhcmx5IEphbiwgMjAxNi4KCktub3duIGlzc3VlczoKCiAgICAgLSBBdCBsZWFzdCAyR0Ig bWVtb3J5IGlzIHN1Z2dlc3RlZCBmb3IgVk0gdG8gcnVuIG1vc3QgM0Qgd29ya2xvYWRzLgogICAg IC0gM0RtYXJrMDYgcnVubmluZyBpbiBXaW5kb3dzIFZNIG1heSBoYXZlIHNvbWUgc3RhYmlsaXR5 IGlzc3VlLgogICAgIC0gVXNpbmcgVkxDIHRvIHBsYXkgLm9nZyBmaWxlIG1heSBjYXVzZSBtb3Nh aWMgb3Igc2xvdyByZXNwb25zZS4KCgpQbGVhc2Ugc3Vic2NyaWJlIHRoZSBtYWlsaW5nIGxpc3Qg dG8gcmVwb3J0IEJVR3MsIGRpc2N1c3MsIGFuZC9vciBjb250cmlidXRlOgoKICAgICBodHRwczov L2xpc3RzLjAxLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2lndnQtZwoKTW9yZSBpbmZvcm1hdGlvbiBh Ym91dCBJbnRlbCBHVlQtZyBiYWNrZ3JvdW5kLCBhcmNoaXRlY3R1cmUsIGV0YyBjYW4gYmUgZm91 bmQgYXQobWF5IG5vdCBiZSB1cC10by1kYXRlKToKCiAgICAgaHR0cHM6Ly8wMS5vcmcvaWd2dC1n CiAgICAgaHR0cDovL3d3dy5saW51eC1rdm0ub3JnL2ltYWdlcy9mL2YzLzAxeDA4Yi1LVk1HVC1h LnBkZgogICAgIGh0dHBzOi8vd3d3LnVzZW5peC5vcmcvY29uZmVyZW5jZS9hdGMxNC90ZWNobmlj YWwtc2Vzc2lvbnMvcHJlc2VudGF0aW9uL3RpYW4KCgpOb3RlOgoKVGhlIEtWTUdUIHByb2plY3Qg c2hvdWxkIGJlIGNvbnNpZGVyZWQgYSB3b3JrIGluIHByb2dyZXNzLiBBcyBzdWNoIGl0IGlzIG5v dCBhIGNvbXBsZXRlIHByb2R1Y3Qgbm9yIHNob3VsZCBpdCBiZSBjb25zaWRlcmVkIG9uZS4gRXh0 cmEgY2FyZSBzaG91bGQgYmUgdGFrZW4gd2hlbiB0ZXN0aW5nIGFuZCBjb25maWd1cmluZyBhIHN5 c3RlbSB0byB1c2UgdGhlIEtWTUdUIHByb2plY3QuCgoKLS0KVGhhbmtzLApKaWtlCgpPbiAxMi8w NC8yMDE0IDEwOjI0IEFNLCBKaWtlIFNvbmcgd3JvdGU6Cj4gSGkgYWxsLAo+Cj4gICAgV2UgYXJl IHBsZWFzZWQgdG8gYW5ub3VuY2UgdGhlIGZpcnN0IHJlbGVhc2Ugb2YgS1ZNR1QgcHJvamVjdC4g S1ZNR1QgaXMgdGhlIGltcGxlbWVudGF0aW9uIG9mIEludGVsIEdWVC1nIHRlY2hub2xvZ3ksIGEg ZnVsbCBHUFUgdmlydHVhbGl6YXRpb24gc29sdXRpb24uIFVuZGVyIEludGVsIEdWVC1nLCBhIHZp cnR1YWwgR1BVIGluc3RhbmNlIGlzIG1haW50YWluZWQgZm9yIGVhY2ggVk0sIHdpdGggcGFydCBv ZiBwZXJmb3JtYW5jZSBjcml0aWNhbCByZXNvdXJjZXMgZGlyZWN0bHkgYXNzaWduZWQuIFRoZSBj YXBhYmlsaXR5IG9mIHJ1bm5pbmcgbmF0aXZlIGdyYXBoaWNzIGRyaXZlciBpbnNpZGUgYSBWTSwg d2l0aG91dCBoeXBlcnZpc29yIGludGVydmVudGlvbiBpbiBwZXJmb3JtYW5jZSBjcml0aWNhbCBw YXRocywgYWNoaWV2ZXMgYSBnb29kIGJhbGFuY2Ugb2YgcGVyZm9ybWFuY2UsIGZlYXR1cmUsIGFu ZCBzaGFyaW5nIGNhcGFiaWxpdHkuCj4KPgo+ICAgIEtWTUdUIGlzIHN0aWxsIGluIHRoZSBlYXJs eSBzdGFnZToKPgo+ICAgICAtIEJhc2ljIGZ1bmN0aW9ucyBvZiBmdWxsIEdQVSB2aXJ0dWFsaXph dGlvbiB3b3JrcywgZ3Vlc3QgY2FuIHNlZSBhIGZ1bGwtZmVhdHVyZWQgdkdQVS4KPiAgICAgICBX ZSByYW4gc2V2ZXJhbCAzRCB3b3JrbG9hZHMgc3VjaCBhcyBsaWdodHNtYXJrLCBuZXh1aXosIHVy YmFudGVycm9yIGFuZCB3YXJzb3cuCj4KPiAgICAgLSBPbmx5IExpbnV4IGd1ZXN0IHN1cHBvcnRl ZCBzbyBmYXIsIGFuZCBQUEdUVCBtdXN0IGJlIGRpc2FibGVkIGluIGd1ZXN0IHRocm91Z2ggYQo+ ICAgICAgIGtlcm5lbCBwYXJhbWV0ZXIoc2VlIFJFQURNRS5rdm1ndCBpbiBRRU1VKS4KPgo+ICAg ICAtIFRoaXMgZHJvcCBhbHNvIGluY2x1ZGVzIHNvbWUgWGVuIHNwZWNpZmljIGNoYW5nZXMsIHdo aWNoIHdpbGwgYmUgY2xlYW5lZCB1cCBsYXRlci4KPgo+ICAgICAtIE91ciBlbmQgZ29hbCBpcyB0 byB1cHN0cmVhbSBib3RoIFhlbkdUIGFuZCBLVk1HVCwgd2hpY2ggc2hhcmVzIH45MCUgbG9naWMg Zm9yIHZHUFUKPiAgICAgICBkZXZpY2UgbW9kZWwgKHdpbGwgYmUgcGFydCBvZiBpOTE1IGRyaXZl ciksIHdpdGggb25seSBkaWZmZXJlbmNlIGluIGh5cGVydmlzb3IKPiAgICAgICBzcGVjaWZpYyBz ZXJ2aWNlcwo+Cj4gICAgIC0gaW5zdWZmaWNpZW50IHRlc3QgY292ZXJhZ2UsIHNvIHBsZWFzZSBi ZWFyIHdpdGggc3RhYmlsaXR5IGlzc3VlcyA6KQo+Cj4KPgo+ICAgIFRoZXJlIGFyZSB0aGluZ3Mg bmVlZCB0byBiZSBpbXByb3ZlZCwgZXNwLiB0aGUgS1ZNIGludGVyZmFjaW5nIHBhcnQ6Cj4KPiAJ MQlhIGRvbWlkIHdhcyBhZGRlZCB0byBlYWNoIEtWTUdUIGd1ZXN0Cj4KPiAJCUFuIElEIGlzIG5l ZWRlZCBmb3IgZm9yZWdyb3VuZCBPUyBzd2l0Y2hpbmcsIGUuZy4KPgo+IAkJCSMgZWNobyA8ZG9t aWQ+CT4JL3N5cy9rZXJuZWwvdmd0L2NvbnRyb2wvZm9yZWdyb3VuZF92bQo+Cj4gCQlkb21pZCAw IGlzIHJlc2VydmVkIGZvciBob3N0IE9TLgo+Cj4KPiAgICAJMglTUkNVIHdvcmthcm91bmRzLgo+ Cj4gCQlTb21lIEtWTSBmdW5jdGlvbnMsIHN1Y2ggYXM6Cj4KPiAJCQkJa3ZtX2lvX2J1c19yZWdp c3Rlcl9kZXYKPiAJCQkJaW5zdGFsbF9uZXdfbWVtc2xvdHMKPgo+IAkJbXVzdCBiZSBjYWxsZWQg KndpdGhvdXQqICZrdm0tPnNyY3UgcmVhZC1sb2NrZWQuIE90aGVyd2lzZSBpdCBoYW5ncy4KPgo+ IAkJSW4gS1ZNR1QsIHdlIG5lZWQgdG8gcmVnaXN0ZXIgYW4gaW9kZXYgb25seSAqYWZ0ZXIqIEJB UiByZWdpc3RlcnMgYXJlCj4gCQl3cml0dGVuIGJ5IGd1ZXN0LiBUaGF0IG1lYW5zLCB3ZSBhbHJl YWR5IGhhdmUgJmt2bS0+c3JjdSBob2xkIC0KPiAJCXRyYXBwaW5nL2VtdWxhdGluZyBQSU8oQkFS IHJlZ2lzdGVycykgbWFrZXMgdXMgaW4gc3VjaCBhIGNvbmRpdGlvbi4KPiAJCVRoYXQgd2lsbCBt YWtlIGt2bV9pb19idXNfcmVnaXN0ZXJfZGV2IGhhbmdzLgo+Cj4gCQlDdXJyZW50bHkgd2UgaGF2 ZSB0byBkaXNhYmxlIHJjdV9hc3NpZ25fcG9pbnRlcigpIGluIHN1Y2ggZnVuY3Rpb25zLgo+Cj4g CQlUaGVzZSB3ZXJlIGRpcnR5IHdvcmthcm91bmRzLCB5b3VyIHN1Z2dlc3Rpb25zIGFyZSBoaWdo IHdlbGNvbWUhCj4KPgo+IAkzCXN5c2NhbGxzIHdlcmUgY2FsbGVkIHRvIGFjY2VzcyAiL2Rldi9t ZW0iIGZyb20ga2VybmVsCj4KPiAJCUFuIGluLWtlcm5lbCBtZW1zbG90IHdhcyBhZGRlZCBmb3Ig YXBlcnR1cmUsIGJ1dCB1c2luZyBzeXNjYWxscyBsaWtlCj4gCQlvcGVuIGFuZCBtbWFwIHRvIG9w ZW4gYW5kIGFjY2VzcyB0aGUgY2hhcmFjdGVyIGRldmljZSAiL2Rldi9tZW0iLAo+IAkJZm9yIHBh c3MtdGhyb3VnaC4KPgo+Cj4KPgo+IFRoZSBzb3VyY2UgY29kZXMoa2VybmVsLCBxZW11IGFzIHdl bGwgYXMgc2VhYmlvcykgYXJlIGF2YWlsYWJsZSBhdCBnaXRodWI6Cj4KPiAJZ2l0Oi8vZ2l0aHVi LmNvbS8wMW9yZy9LVk1HVC1rZXJuZWwKPiAJZ2l0Oi8vZ2l0aHViLmNvbS8wMW9yZy9LVk1HVC1x ZW11Cj4gCWdpdDovL2dpdGh1Yi5jb20vMDFvcmcvS1ZNR1Qtc2VhYmlvcwo+Cj4gSW4gdGhlIEtW TUdULXFlbXUgcmVwb3NpdG9yeSwgdGhlcmUgaXMgYSAiUkVBRE1FLmt2bWd0IiB0byBiZSByZWZl cnJlZC4KPgo+Cj4KPiBNb3JlIGluZm9ybWF0aW9uIGFib3V0IEludGVsIEdWVC1nIGFuZCBLVk1H VCBjYW4gYmUgZm91bmQgYXQ6Cj4KPiAJaHR0cHM6Ly93d3cudXNlbml4Lm9yZy9jb25mZXJlbmNl L2F0YzE0L3RlY2huaWNhbC1zZXNzaW9ucy9wcmVzZW50YXRpb24vdGlhbgo+IAlodHRwOi8vZXZl bnRzLmxpbnV4Zm91bmRhdGlvbi5vcmcvc2l0ZXMvZXZlbnRzL2ZpbGVzL3NsaWRlcy9LVk1HVC1h JTIwRnVsbCUyMEdQVSUyMFZpcnR1YWxpemF0aW9uJTIwU29sdXRpb25fMS5wZGYKPgo+Cj4gQXBw cmVjaWF0ZSB5b3VyIGNvbW1lbnRzLCBCVUcgcmVwb3J0cywgYW5kIGNvbnRyaWJ1dGlvbnMhCj4K Pgo+Cj4KPiAtLQo+IFRoYW5rcywKPiBKaWtlCj4KX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX18KSW50ZWwtZ2Z4IG1haWxpbmcgbGlzdApJbnRlbC1nZnhAbGlz dHMuZnJlZWRlc2t0b3Aub3JnCmh0dHA6Ly9saXN0cy5mcmVlZGVza3RvcC5vcmcvbWFpbG1hbi9s aXN0aW5mby9pbnRlbC1nZngK From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753577AbbJ0Jge (ORCPT ); Tue, 27 Oct 2015 05:36:34 -0400 Received: from mga09.intel.com ([134.134.136.24]:4664 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751572AbbJ0Jgc (ORCPT ); Tue, 27 Oct 2015 05:36:32 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.20,204,1444719600"; d="scan'208";a="836410572" Message-ID: <562F45B4.70004@intel.com> Date: Tue, 27 Oct 2015 17:36:52 +0800 From: Jike Song User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: kvm@vger.kernel.org, igvt-g@ml01.01.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org CC: "Tian, Kevin" , "Cowperthwaite, David J" , "Dong, Eddie" , "Li, Susie" , "Downs, Mike" , "Reddy, Raghuveer" Subject: Re: [ANNOUNCE][RFC] KVMGT - the implementation of Intel GVT-g(full GPU virtualization) for KVM References: <547FC5DE.4010701@intel.com> In-Reply-To: <547FC5DE.4010701@intel.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi all, We are pleased to announce another update of Intel GVT-g for KVM. Intel GVT-g is a full GPU virtualization solution with mediated pass-through, starting from 4th generation Intel Core(TM) processors with Intel Graphics processors. A virtual GPU instance is maintained for each VM, with part of performance critical resources directly assigned. The capability of running native graphics driver inside a VM, without hypervisor intervention in performance critical paths, achieves a good balance among performance, feature, and sharing capability. KVM is supported by Intel GVT-g(a.k.a. KVMGT). Repositories Kernel: https://github.com/01org/igvtg-kernel (2015q3-3.18.0 branch) Qemu: https://github.com/01org/igvtg-qemu (kvmgt_public2015q3 branch) This update consists of: - KVMGT is now merged with XenGT in unified repositories(kernel and qemu), but currently different branches for qemu. KVMGT and XenGT share same iGVT-g core logic. - PPGTT supported, hence the Windows guest support - KVMGT now supports both 4th generation (Haswell) and 5th generation (Broadwell) Intel Core(TM) processors - 2D/3D/Media decoding have been validated on Ubuntu 14.04 and Windows7/Windows 8.1 Next update will be around early Jan, 2016. Known issues: - At least 2GB memory is suggested for VM to run most 3D workloads. - 3Dmark06 running in Windows VM may have some stability issue. - Using VLC to play .ogg file may cause mosaic or slow response. Please subscribe the mailing list to report BUGs, discuss, and/or contribute: https://lists.01.org/mailman/listinfo/igvt-g More information about Intel GVT-g background, architecture, etc can be found at(may not be up-to-date): https://01.org/igvt-g http://www.linux-kvm.org/images/f/f3/01x08b-KVMGT-a.pdf https://www.usenix.org/conference/atc14/technical-sessions/presentation/tian Note: The KVMGT project should be considered a work in progress. As such it is not a complete product nor should it be considered one. Extra care should be taken when testing and configuring a system to use the KVMGT project. -- Thanks, Jike On 12/04/2014 10:24 AM, Jike Song wrote: > Hi all, > > We are pleased to announce the first release of KVMGT project. KVMGT is the implementation of Intel GVT-g technology, a full GPU virtualization solution. Under Intel GVT-g, a virtual GPU instance is maintained for each VM, with part of performance critical resources directly assigned. The capability of running native graphics driver inside a VM, without hypervisor intervention in performance critical paths, achieves a good balance of performance, feature, and sharing capability. > > > KVMGT is still in the early stage: > > - Basic functions of full GPU virtualization works, guest can see a full-featured vGPU. > We ran several 3D workloads such as lightsmark, nexuiz, urbanterror and warsow. > > - Only Linux guest supported so far, and PPGTT must be disabled in guest through a > kernel parameter(see README.kvmgt in QEMU). > > - This drop also includes some Xen specific changes, which will be cleaned up later. > > - Our end goal is to upstream both XenGT and KVMGT, which shares ~90% logic for vGPU > device model (will be part of i915 driver), with only difference in hypervisor > specific services > > - insufficient test coverage, so please bear with stability issues :) > > > > There are things need to be improved, esp. the KVM interfacing part: > > 1 a domid was added to each KVMGT guest > > An ID is needed for foreground OS switching, e.g. > > # echo > /sys/kernel/vgt/control/foreground_vm > > domid 0 is reserved for host OS. > > > 2 SRCU workarounds. > > Some KVM functions, such as: > > kvm_io_bus_register_dev > install_new_memslots > > must be called *without* &kvm->srcu read-locked. Otherwise it hangs. > > In KVMGT, we need to register an iodev only *after* BAR registers are > written by guest. That means, we already have &kvm->srcu hold - > trapping/emulating PIO(BAR registers) makes us in such a condition. > That will make kvm_io_bus_register_dev hangs. > > Currently we have to disable rcu_assign_pointer() in such functions. > > These were dirty workarounds, your suggestions are high welcome! > > > 3 syscalls were called to access "/dev/mem" from kernel > > An in-kernel memslot was added for aperture, but using syscalls like > open and mmap to open and access the character device "/dev/mem", > for pass-through. > > > > > The source codes(kernel, qemu as well as seabios) are available at github: > > git://github.com/01org/KVMGT-kernel > git://github.com/01org/KVMGT-qemu > git://github.com/01org/KVMGT-seabios > > In the KVMGT-qemu repository, there is a "README.kvmgt" to be referred. > > > > More information about Intel GVT-g and KVMGT can be found at: > > https://www.usenix.org/conference/atc14/technical-sessions/presentation/tian > http://events.linuxfoundation.org/sites/events/files/slides/KVMGT-a%20Full%20GPU%20Virtualization%20Solution_1.pdf > > > Appreciate your comments, BUG reports, and contributions! > > > > > -- > Thanks, > Jike >