From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: multipart/mixed; boundary="===============0396950122655663140==" MIME-Version: 1.0 From: E.W.Z. Subject: Re: [SPDK] core_mask and spawning new threads Date: Wed, 21 Feb 2018 19:40:54 +0200 Message-ID: <5a8daf25.f8aedf0a.6fae.7d87@mx.google.com> In-Reply-To: FBE7E039FA50BF47A673AD0BD3CD56A83758CB23@HASMSX105.ger.corp.intel.com List-ID: To: spdk@lists.01.org --===============0396950122655663140== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable I think I=E2=80=99ve found a way to overcome this. In my test application w= hen thread which were created to produce I/O were spawned from the =E2=80= =9Cmain=E2=80=9D thread in which the SPDK was initialized. Of course the cp= u mask given to SPDK environment was affecting affinity of thread I was cre= ating. So I tried to do the following, create a temporary thread, in this t= hread create an instance of my object which instantiates the SPDK environme= nt, and then the temporary thread is destroyed and I have fully working SPD= K machinery (which in turn spawned its own threads affected by affinity set= in SPDK initialization) with threads doing IO created from main thread, wh= ich, in turn, knows nothing about SPDK machinery and its affinity. It succe= ssfully completed the run. Looks like a hack, but working as intended. What= do you say? Sent from Mail for Windows 10 From: Stojaczyk, DariuszX Sent: Wednesday, February 21, 2018 5:33 PM To: Storage Performance Development Kit Subject: Re: [SPDK] core_mask and spawning new threads > in case I'm integrating the SPDK into my existing application, I have to = change the way I'm spawning thread in the whole application, right? That's correct. Regards, D. From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Ernest Zed Sent: Wednesday, February 21, 2018 1:18 PM To: Storage Performance Development Kit Subject: Re: [SPDK] core_mask and spawning new threads Thanks for the explanation. Just to make it clear, in case I'm integrating = the SPDK into my existing application, I have to change the way I'm spawnin= g thread in the whole application, right? Sincerely, E. On Wed, Feb 21, 2018 at 1:43 PM, Stojaczyk, DariuszX wrote: Hi Ernest, This is expected behavior. SPDK starts a fixed number of affinitized thread= s on startup, and the core mask parameter you've mentioned is what specifie= s the affinity and number of those. Each thread is bound to a single CPU. T= he one thread that called `spdk_env_init` becomes a master thread and is af= finitized to the `opts.master_core`. To spawn additional POSIX threads from= any SPDK-affinitized thread, you should use `spdk_call_unaffinitized()` (i= nclude/spdk/env.h). = =C2=A0 (There's also an `spdk_unaffinitize_thread()`, but it's restricted to very = specific workloads and probably shouldn't be used at all.) Best regards, D. =C2=A0 From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Ernest Zed Sent: Wednesday, February 21, 2018 10:47 AM To: Storage Performance Development Kit Subject: [SPDK] core_mask and spawning new threads =C2=A0 Hi all, I see that all threads I create within my application are running on the sa= me core. Looks like core_mask is the right parameter to adjust, however I d= ont see core usage affected by changing the core_mask on spdk_env_opts The code: =C2=A0 =C2=A0 spdk_env_opts opts; =C2=A0 =C2=A0 spdk_env_opts_init(&opts); =C2=A0 =C2=A0 opts.name =3D "blah"; =C2=A0 =C2=A0 opts.core_mask =3D "0xFFFF"; =C2=A0 =C2=A0 opts.master_core =3D 1; =C2=A0 =C2=A0 opts.shm_id =3D 0; =C2=A0 =C2=A0 if (spdk_env_init(&opts) < 0) { =C2=A0 =C2=A0 =C2=A0 =C2=A0 throw std::runtime_error("Unable to initialize = SPDK env"); =C2=A0 =C2=A0 } Whatever value I assign to the core_mask everything runs on the same core d= efined in master_core. If I set affinity to my threads manually everything = works as expected. Is it a spdk_env_init bug? Am I doing something wrong? Sincerely, Ernest _______________________________________________ SPDK mailing list SPDK(a)lists.01.org https://lists.01.org/mailman/listinfo/spdk --===============0396950122655663140== Content-Type: text/html MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.html" PGh0bWwgeG1sbnM6bz0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6b2ZmaWNlIiB4 bWxuczp3PSJ1cm46c2NoZW1hcy1taWNyb3NvZnQtY29tOm9mZmljZTp3b3JkIiB4bWxuczptPSJo dHRwOi8vc2NoZW1hcy5taWNyb3NvZnQuY29tL29mZmljZS8yMDA0LzEyL29tbWwiIHhtbG5zPSJo dHRwOi8vd3d3LnczLm9yZy9UUi9SRUMtaHRtbDQwIj48aGVhZD48bWV0YSBodHRwLWVxdWl2PUNv bnRlbnQtVHlwZSBjb250ZW50PSJ0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgiPjxtZXRhIG5hbWU9 R2VuZXJhdG9yIGNvbnRlbnQ9Ik1pY3Jvc29mdCBXb3JkIDE1IChmaWx0ZXJlZCBtZWRpdW0pIj48 c3R5bGU+PCEtLQovKiBGb250IERlZmluaXRpb25zICovCkBmb250LWZhY2UKCXtmb250LWZhbWls eToiQ2FtYnJpYSBNYXRoIjsKCXBhbm9zZS0xOjIgNCA1IDMgNSA0IDYgMyAyIDQ7fQpAZm9udC1m YWNlCgl7Zm9udC1mYW1pbHk6Q2FsaWJyaTsKCXBhbm9zZS0xOjIgMTUgNSAyIDIgMiA0IDMgMiA0 O30KLyogU3R5bGUgRGVmaW5pdGlvbnMgKi8KcC5Nc29Ob3JtYWwsIGxpLk1zb05vcm1hbCwgZGl2 Lk1zb05vcm1hbAoJe21hcmdpbjowaW47CgltYXJnaW4tYm90dG9tOi4wMDAxcHQ7Cglmb250LXNp emU6MTEuMHB0OwoJZm9udC1mYW1pbHk6IkNhbGlicmkiLHNhbnMtc2VyaWY7fQphOmxpbmssIHNw YW4uTXNvSHlwZXJsaW5rCgl7bXNvLXN0eWxlLXByaW9yaXR5Ojk5OwoJY29sb3I6Ymx1ZTsKCXRl eHQtZGVjb3JhdGlvbjp1bmRlcmxpbmU7fQphOnZpc2l0ZWQsIHNwYW4uTXNvSHlwZXJsaW5rRm9s bG93ZWQKCXttc28tc3R5bGUtcHJpb3JpdHk6OTk7Cgljb2xvcjojOTU0RjcyOwoJdGV4dC1kZWNv cmF0aW9uOnVuZGVybGluZTt9Ci5Nc29DaHBEZWZhdWx0Cgl7bXNvLXN0eWxlLXR5cGU6ZXhwb3J0 LW9ubHk7fQpAcGFnZSBXb3JkU2VjdGlvbjEKCXtzaXplOjguNWluIDExLjBpbjsKCW1hcmdpbjox LjBpbiAxLjBpbiAxLjBpbiAxLjBpbjt9CmRpdi5Xb3JkU2VjdGlvbjEKCXtwYWdlOldvcmRTZWN0 aW9uMTt9Ci0tPjwvc3R5bGU+PC9oZWFkPjxib2R5IGxhbmc9RU4tVVMgbGluaz1ibHVlIHZsaW5r PSIjOTU0RjcyIj48ZGl2IGNsYXNzPVdvcmRTZWN0aW9uMT48cCBjbGFzcz1Nc29Ob3JtYWw+SSB0 aGluayBJ4oCZdmUgZm91bmQgYSB3YXkgdG8gb3ZlcmNvbWUgdGhpcy4gSW4gbXkgdGVzdCBhcHBs aWNhdGlvbiB3aGVuIHRocmVhZCB3aGljaCB3ZXJlIGNyZWF0ZWQgdG8gcHJvZHVjZSBJL08gd2Vy ZSBzcGF3bmVkIGZyb20gdGhlIOKAnG1haW7igJ0gdGhyZWFkIGluIHdoaWNoIHRoZSBTUERLIHdh cyBpbml0aWFsaXplZC4gT2YgY291cnNlIHRoZSBjcHUgbWFzayBnaXZlbiB0byBTUERLIGVudmly b25tZW50IHdhcyBhZmZlY3RpbmcgYWZmaW5pdHkgb2YgdGhyZWFkIEkgd2FzIGNyZWF0aW5nLiBT byBJIHRyaWVkIHRvIGRvIHRoZSBmb2xsb3dpbmcsIGNyZWF0ZSBhIHRlbXBvcmFyeSB0aHJlYWQs IGluIHRoaXMgdGhyZWFkIGNyZWF0ZSBhbiBpbnN0YW5jZSBvZiBteSBvYmplY3Qgd2hpY2ggaW5z dGFudGlhdGVzIHRoZSBTUERLIGVudmlyb25tZW50LCBhbmQgdGhlbiB0aGUgdGVtcG9yYXJ5IHRo cmVhZCBpcyBkZXN0cm95ZWQgYW5kIEkgaGF2ZSBmdWxseSB3b3JraW5nIFNQREsgbWFjaGluZXJ5 ICh3aGljaCBpbiB0dXJuIHNwYXduZWQgaXRzIG93biB0aHJlYWRzIGFmZmVjdGVkIGJ5IGFmZmlu aXR5IHNldCBpbiBTUERLIGluaXRpYWxpemF0aW9uKSB3aXRoIHRocmVhZHMgZG9pbmcgSU8gY3Jl YXRlZCBmcm9tIG1haW4gdGhyZWFkLCB3aGljaCwgaW4gdHVybiwga25vd3Mgbm90aGluZyBhYm91 dCBTUERLIG1hY2hpbmVyeSBhbmQgaXRzIGFmZmluaXR5LiBJdCBzdWNjZXNzZnVsbHkgY29tcGxl dGVkIHRoZSBydW4uIExvb2tzIGxpa2UgYSBoYWNrLCBidXQgd29ya2luZyBhcyBpbnRlbmRlZC4g V2hhdCBkbyB5b3Ugc2F5PzwvcD48cCBjbGFzcz1Nc29Ob3JtYWw+PG86cD4mbmJzcDs8L286cD48 L3A+PHAgY2xhc3M9TXNvTm9ybWFsPlNlbnQgZnJvbSA8YSBocmVmPSJodHRwczovL2dvLm1pY3Jv c29mdC5jb20vZndsaW5rLz9MaW5rSWQ9NTUwOTg2Ij5NYWlsPC9hPiBmb3IgV2luZG93cyAxMDwv cD48cCBjbGFzcz1Nc29Ob3JtYWw+PG86cD4mbmJzcDs8L286cD48L3A+PGRpdiBzdHlsZT0nbXNv LWVsZW1lbnQ6cGFyYS1ib3JkZXItZGl2O2JvcmRlcjpub25lO2JvcmRlci10b3A6c29saWQgI0Ux RTFFMSAxLjBwdDtwYWRkaW5nOjMuMHB0IDBpbiAwaW4gMGluJz48cCBjbGFzcz1Nc29Ob3JtYWwg c3R5bGU9J2JvcmRlcjpub25lO3BhZGRpbmc6MGluJz48Yj5Gcm9tOiA8L2I+PGEgaHJlZj0ibWFp bHRvOmRhcml1c3p4LnN0b2phY3p5a0BpbnRlbC5jb20iPlN0b2phY3p5aywgRGFyaXVzelg8L2E+ PGJyPjxiPlNlbnQ6IDwvYj5XZWRuZXNkYXksIEZlYnJ1YXJ5IDIxLCAyMDE4IDU6MzMgUE08YnI+ PGI+VG86IDwvYj48YSBocmVmPSJtYWlsdG86c3Bka0BsaXN0cy4wMS5vcmciPlN0b3JhZ2UgUGVy Zm9ybWFuY2UgRGV2ZWxvcG1lbnQgS2l0PC9hPjxicj48Yj5TdWJqZWN0OiA8L2I+UmU6IFtTUERL XSBjb3JlX21hc2sgYW5kIHNwYXduaW5nIG5ldyB0aHJlYWRzPC9wPjwvZGl2PjxwIGNsYXNzPU1z b05vcm1hbD48bzpwPiZuYnNwOzwvbzpwPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21h cmdpbi1ib3R0b206MTIuMHB0Jz48c3BhbiBzdHlsZT0nY29sb3I6IzFGNDk3RCc+Jmd0OyA8L3Nw YW4+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMi4wcHQ7Zm9udC1mYW1pbHk6IlRpbWVzIE5ldyBS b21hbiIsc2VyaWYnPmluIGNhc2UgSSdtIGludGVncmF0aW5nIHRoZSBTUERLIGludG8gbXkgZXhp c3RpbmcgYXBwbGljYXRpb24sIEkgaGF2ZSB0byBjaGFuZ2UgdGhlIHdheSBJJ20gc3Bhd25pbmcg dGhyZWFkIGluIHRoZSB3aG9sZSBhcHBsaWNhdGlvbiwgcmlnaHQ/PG86cD48L286cD48L3NwYW4+ PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBzdHlsZT0nZm9udC1zaXplOjEyLjBwdDtmb250 LWZhbWlseToiVGltZXMgTmV3IFJvbWFuIixzZXJpZic+VGhhdCdzIGNvcnJlY3QuPG86cD48L286 cD48L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBzdHlsZT0nZm9udC1zaXplOjEy LjBwdDtmb250LWZhbWlseToiVGltZXMgTmV3IFJvbWFuIixzZXJpZic+UmVnYXJkcyw8bzpwPjwv bzpwPjwvc3Bhbj48L3A+PHAgY2xhc3M9TXNvTm9ybWFsPjxzcGFuIHN0eWxlPSdmb250LXNpemU6 MTIuMHB0O2ZvbnQtZmFtaWx5OiJUaW1lcyBOZXcgUm9tYW4iLHNlcmlmJz5ELjwvc3Bhbj48c3Bh biBzdHlsZT0nY29sb3I6IzFGNDk3RCc+PG86cD48L286cD48L3NwYW4+PC9wPjxwIGNsYXNzPU1z b05vcm1hbD48c3BhbiBzdHlsZT0nY29sb3I6IzFGNDk3RCc+PG86cD4mbmJzcDs8L286cD48L3Nw YW4+PC9wPjxkaXYgc3R5bGU9J2JvcmRlcjpub25lO2JvcmRlci1sZWZ0OnNvbGlkIGJsdWUgMS41 cHQ7cGFkZGluZzowaW4gMGluIDBpbiA0LjBwdCc+PGRpdj48ZGl2IHN0eWxlPSdib3JkZXI6bm9u ZTtib3JkZXItdG9wOnNvbGlkICNFMUUxRTEgMS4wcHQ7cGFkZGluZzozLjBwdCAwaW4gMGluIDBp bic+PHAgY2xhc3M9TXNvTm9ybWFsPjxhIG5hbWU9Il9fX19fcmVwbHlzZXBhcmF0b3IiPjwvYT48 Yj5Gcm9tOjwvYj4gU1BESyBbbWFpbHRvOnNwZGstYm91bmNlc0BsaXN0cy4wMS5vcmddIDxiPk9u IEJlaGFsZiBPZiA8L2I+RXJuZXN0IFplZDxicj48Yj5TZW50OjwvYj4gV2VkbmVzZGF5LCBGZWJy dWFyeSAyMSwgMjAxOCAxOjE4IFBNPGJyPjxiPlRvOjwvYj4gU3RvcmFnZSBQZXJmb3JtYW5jZSBE ZXZlbG9wbWVudCBLaXQgJmx0O3NwZGtAbGlzdHMuMDEub3JnJmd0Ozxicj48Yj5TdWJqZWN0Ojwv Yj4gUmU6IFtTUERLXSBjb3JlX21hc2sgYW5kIHNwYXduaW5nIG5ldyB0aHJlYWRzPG86cD48L286 cD48L3A+PC9kaXY+PC9kaXY+PHAgY2xhc3M9TXNvTm9ybWFsPjxzcGFuIHN0eWxlPSdmb250LXNp emU6MTIuMHB0O2ZvbnQtZmFtaWx5OiJUaW1lcyBOZXcgUm9tYW4iLHNlcmlmJz48bzpwPiZuYnNw OzwvbzpwPjwvc3Bhbj48L3A+PGRpdj48cCBjbGFzcz1Nc29Ob3JtYWw+PHNwYW4gc3R5bGU9J2Zv bnQtc2l6ZToxMi4wcHQ7Zm9udC1mYW1pbHk6IlRpbWVzIE5ldyBSb21hbiIsc2VyaWYnPlRoYW5r cyBmb3IgdGhlIGV4cGxhbmF0aW9uLiBKdXN0IHRvIG1ha2UgaXQgY2xlYXIsIGluIGNhc2UgSSdt IGludGVncmF0aW5nIHRoZSBTUERLIGludG8gbXkgZXhpc3RpbmcgYXBwbGljYXRpb24sIEkgaGF2 ZSB0byBjaGFuZ2UgdGhlIHdheSBJJ20gc3Bhd25pbmcgdGhyZWFkIGluIHRoZSB3aG9sZSBhcHBs aWNhdGlvbiwgcmlnaHQ/PG86cD48L286cD48L3NwYW4+PC9wPjxkaXY+PHAgY2xhc3M9TXNvTm9y bWFsPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTIuMHB0O2ZvbnQtZmFtaWx5OiJUaW1lcyBOZXcg Um9tYW4iLHNlcmlmJz5TaW5jZXJlbHksPG86cD48L286cD48L3NwYW4+PC9wPjwvZGl2PjxkaXY+ PHAgY2xhc3M9TXNvTm9ybWFsPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTIuMHB0O2ZvbnQtZmFt aWx5OiJUaW1lcyBOZXcgUm9tYW4iLHNlcmlmJz5FLjxvOnA+PC9vOnA+PC9zcGFuPjwvcD48L2Rp dj48L2Rpdj48ZGl2PjxwIGNsYXNzPU1zb05vcm1hbD48c3BhbiBzdHlsZT0nZm9udC1zaXplOjEy LjBwdDtmb250LWZhbWlseToiVGltZXMgTmV3IFJvbWFuIixzZXJpZic+PG86cD4mbmJzcDs8L286 cD48L3NwYW4+PC9wPjxkaXY+PHAgY2xhc3M9TXNvTm9ybWFsPjxzcGFuIHN0eWxlPSdmb250LXNp emU6MTIuMHB0O2ZvbnQtZmFtaWx5OiJUaW1lcyBOZXcgUm9tYW4iLHNlcmlmJz5PbiBXZWQsIEZl YiAyMSwgMjAxOCBhdCAxOjQzIFBNLCBTdG9qYWN6eWssIERhcml1c3pYICZsdDs8YSBocmVmPSJt YWlsdG86ZGFyaXVzenguc3RvamFjenlrQGludGVsLmNvbSIgdGFyZ2V0PSJfYmxhbmsiPmRhcml1 c3p4LnN0b2phY3p5a0BpbnRlbC5jb208L2E+Jmd0OyB3cm90ZTo8bzpwPjwvbzpwPjwvc3Bhbj48 L3A+PGJsb2NrcXVvdGUgc3R5bGU9J2JvcmRlcjpub25lO2JvcmRlci1sZWZ0OnNvbGlkICNDQ0ND Q0MgMS4wcHQ7cGFkZGluZzowaW4gMGluIDBpbiA2LjBwdDttYXJnaW4tbGVmdDo0LjhwdDttYXJn aW4tdG9wOjUuMHB0O21hcmdpbi1yaWdodDowaW47bWFyZ2luLWJvdHRvbTo1LjBwdCc+PGRpdj48 ZGl2PjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNv LW1hcmdpbi1ib3R0b20tYWx0OmF1dG87bWFyZ2luLWxlZnQ6NC44cHQnPjxzcGFuIHN0eWxlPSdj b2xvcjojMUY0OTdEJz5IaSBFcm5lc3QsPC9zcGFuPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTIu MHB0O2ZvbnQtZmFtaWx5OiJUaW1lcyBOZXcgUm9tYW4iLHNlcmlmJz48bzpwPjwvbzpwPjwvc3Bh bj48L3A+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bztt c28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0bzttYXJnaW4tbGVmdDo0LjhwdCc+PHNwYW4gc3R5bGU9 J2NvbG9yOiMxRjQ5N0QnPlRoaXMgaXMgZXhwZWN0ZWQgYmVoYXZpb3IuIFNQREsgc3RhcnRzIGEg Zml4ZWQgbnVtYmVyIG9mIGFmZmluaXRpemVkIHRocmVhZHMgb24gc3RhcnR1cCwgYW5kIHRoZSBj b3JlIG1hc2sgcGFyYW1ldGVyIHlvdSd2ZSBtZW50aW9uZWQgaXMgd2hhdCBzcGVjaWZpZXMgdGhl IGFmZmluaXR5IGFuZCBudW1iZXIgb2YgdGhvc2UuIEVhY2ggdGhyZWFkIGlzIGJvdW5kIHRvIGEg c2luZ2xlIENQVS4gVGhlIG9uZSB0aHJlYWQgdGhhdCBjYWxsZWQgYHNwZGtfZW52X2luaXRgIGJl Y29tZXMgYSBtYXN0ZXIgdGhyZWFkIGFuZCBpcyBhZmZpbml0aXplZCB0byB0aGUgYG9wdHMubWFz dGVyX2NvcmVgLiBUbyBzcGF3biBhZGRpdGlvbmFsIFBPU0lYIHRocmVhZHMgZnJvbSBhbnkgU1BE Sy1hZmZpbml0aXplZCB0aHJlYWQsIHlvdSBzaG91bGQgdXNlIGBzcGRrX2NhbGxfdW5hZmZpbml0 aXplZCgpYCAoaW5jbHVkZS9zcGRrL2Vudi5oKS4gPC9zcGFuPjxzcGFuIHN0eWxlPSdmb250LXNp emU6MTIuMHB0O2ZvbnQtZmFtaWx5OiJUaW1lcyBOZXcgUm9tYW4iLHNlcmlmJz48bzpwPjwvbzpw Pjwvc3Bhbj48L3A+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6 YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0bzttYXJnaW4tbGVmdDo0LjhwdCc+PHNwYW4g c3R5bGU9J2NvbG9yOiMxRjQ5N0QnPiZuYnNwOzwvc3Bhbj48c3BhbiBzdHlsZT0nZm9udC1zaXpl OjEyLjBwdDtmb250LWZhbWlseToiVGltZXMgTmV3IFJvbWFuIixzZXJpZic+PG86cD48L286cD48 L3NwYW4+PC9wPjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1 dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG87bWFyZ2luLWxlZnQ6NC44cHQnPjxzcGFuIHN0 eWxlPSdjb2xvcjojMUY0OTdEJz4oVGhlcmUncyBhbHNvIGFuIGBzcGRrX3VuYWZmaW5pdGl6ZV90 aHJlYWQoKWAsIGJ1dCBpdCdzIHJlc3RyaWN0ZWQgdG8gdmVyeSBzcGVjaWZpYyB3b3JrbG9hZHMg YW5kIHByb2JhYmx5IHNob3VsZG4ndCBiZSB1c2VkIGF0IGFsbC4pPC9zcGFuPjxzcGFuIHN0eWxl PSdmb250LXNpemU6MTIuMHB0O2ZvbnQtZmFtaWx5OiJUaW1lcyBOZXcgUm9tYW4iLHNlcmlmJz48 bzpwPjwvbzpwPjwvc3Bhbj48L3A+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2lu LXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0bzttYXJnaW4tbGVmdDo0Ljhw dCc+PHNwYW4gc3R5bGU9J2NvbG9yOiMxRjQ5N0QnPkJlc3QgcmVnYXJkcyw8L3NwYW4+PHNwYW4g c3R5bGU9J2ZvbnQtc2l6ZToxMi4wcHQ7Zm9udC1mYW1pbHk6IlRpbWVzIE5ldyBSb21hbiIsc2Vy aWYnPjxvOnA+PC9vOnA+PC9zcGFuPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1t YXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvO21hcmdpbi1sZWZ0 OjQuOHB0Jz48c3BhbiBzdHlsZT0nY29sb3I6IzFGNDk3RCc+RC48L3NwYW4+PHNwYW4gc3R5bGU9 J2ZvbnQtc2l6ZToxMi4wcHQ7Zm9udC1mYW1pbHk6IlRpbWVzIE5ldyBSb21hbiIsc2VyaWYnPjxv OnA+PC9vOnA+PC9zcGFuPjwvcD48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4t dG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvO21hcmdpbi1sZWZ0OjQuOHB0 Jz48c3BhbiBzdHlsZT0nY29sb3I6IzFGNDk3RCc+Jm5ic3A7PC9zcGFuPjxzcGFuIHN0eWxlPSdm b250LXNpemU6MTIuMHB0O2ZvbnQtZmFtaWx5OiJUaW1lcyBOZXcgUm9tYW4iLHNlcmlmJz48bzpw PjwvbzpwPjwvc3Bhbj48L3A+PGRpdiBzdHlsZT0nYm9yZGVyOm5vbmU7Ym9yZGVyLWxlZnQ6c29s aWQgYmx1ZSAxLjVwdDtwYWRkaW5nOjBpbiAwaW4gMGluIDQuMHB0Jz48ZGl2PjxkaXYgc3R5bGU9 J2JvcmRlcjpub25lO2JvcmRlci10b3A6c29saWQgI0UxRTFFMSAxLjBwdDtwYWRkaW5nOjMuMHB0 IDBpbiAwaW4gMGluJz48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFs dDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvO21hcmdpbi1sZWZ0OjQuOHB0Jz48YSBu YW1lPSJtXy0yMTA2NzgzOTI2NzI5MDMwMzEwX19fX19fcmVwbHlzZXBhcmF0Ij48L2E+PGI+RnJv bTo8L2I+IFNQREsgW21haWx0bzo8YSBocmVmPSJtYWlsdG86c3Bkay1ib3VuY2VzQGxpc3RzLjAx Lm9yZyIgdGFyZ2V0PSJfYmxhbmsiPnNwZGstYm91bmNlc0BsaXN0cy4wMS5vcmc8L2E+XSA8Yj5P biBCZWhhbGYgT2YgPC9iPkVybmVzdCBaZWQ8YnI+PGI+U2VudDo8L2I+IFdlZG5lc2RheSwgRmVi cnVhcnkgMjEsIDIwMTggMTA6NDcgQU08YnI+PGI+VG86PC9iPiBTdG9yYWdlIFBlcmZvcm1hbmNl IERldmVsb3BtZW50IEtpdCAmbHQ7PGEgaHJlZj0ibWFpbHRvOnNwZGtAbGlzdHMuMDEub3JnIiB0 YXJnZXQ9Il9ibGFuayI+c3Bka0BsaXN0cy4wMS5vcmc8L2E+Jmd0Ozxicj48Yj5TdWJqZWN0Ojwv Yj4gW1NQREtdIGNvcmVfbWFzayBhbmQgc3Bhd25pbmcgbmV3IHRocmVhZHM8c3BhbiBzdHlsZT0n Zm9udC1zaXplOjEyLjBwdDtmb250LWZhbWlseToiVGltZXMgTmV3IFJvbWFuIixzZXJpZic+PG86 cD48L286cD48L3NwYW4+PC9wPjwvZGl2PjwvZGl2PjxkaXY+PGRpdj48cCBjbGFzcz1Nc29Ob3Jt YWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDph dXRvO21hcmdpbi1sZWZ0OjQuOHB0Jz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjEyLjBwdDtmb250 LWZhbWlseToiVGltZXMgTmV3IFJvbWFuIixzZXJpZic+Jm5ic3A7PG86cD48L286cD48L3NwYW4+ PC9wPjxkaXY+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0 bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0bzttYXJnaW4tbGVmdDo0LjhwdCc+PHNwYW4gc3R5 bGU9J2ZvbnQtc2l6ZToxMi4wcHQ7Zm9udC1mYW1pbHk6IlRpbWVzIE5ldyBSb21hbiIsc2VyaWYn PkhpIGFsbCw8bzpwPjwvbzpwPjwvc3Bhbj48L3A+PGRpdj48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5 bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvO21h cmdpbi1sZWZ0OjQuOHB0Jz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjEyLjBwdDtmb250LWZhbWls eToiVGltZXMgTmV3IFJvbWFuIixzZXJpZic+SSBzZWUgdGhhdCBhbGwgdGhyZWFkcyBJIGNyZWF0 ZSB3aXRoaW4gbXkgYXBwbGljYXRpb24gYXJlIHJ1bm5pbmcgb24gdGhlIHNhbWUgY29yZS4gTG9v a3MgbGlrZSBjb3JlX21hc2sgaXMgdGhlIHJpZ2h0IHBhcmFtZXRlciB0byBhZGp1c3QsIGhvd2V2 ZXIgSSBkb250IHNlZSBjb3JlIHVzYWdlIGFmZmVjdGVkIGJ5IGNoYW5naW5nIHRoZSBjb3JlX21h c2sgb24gc3Bka19lbnZfb3B0czxvOnA+PC9vOnA+PC9zcGFuPjwvcD48L2Rpdj48ZGl2PjxwIGNs YXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1i b3R0b20tYWx0OmF1dG87bWFyZ2luLWxlZnQ6NC44cHQnPjxzcGFuIHN0eWxlPSdmb250LXNpemU6 MTIuMHB0O2ZvbnQtZmFtaWx5OiJUaW1lcyBOZXcgUm9tYW4iLHNlcmlmJz5UaGUgY29kZTo8bzpw PjwvbzpwPjwvc3Bhbj48L3A+PC9kaXY+PGRpdj48ZGl2PjxwIGNsYXNzPU1zb05vcm1hbCBzdHls ZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG87bWFy Z2luLWxlZnQ6NC44cHQnPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTIuMHB0O2ZvbnQtZmFtaWx5 OiJUaW1lcyBOZXcgUm9tYW4iLHNlcmlmJz4mbmJzcDsgJm5ic3A7IHNwZGtfZW52X29wdHMgb3B0 czs8bzpwPjwvbzpwPjwvc3Bhbj48L3A+PC9kaXY+PGRpdj48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5 bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvO21h cmdpbi1sZWZ0OjQuOHB0Jz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjEyLjBwdDtmb250LWZhbWls eToiVGltZXMgTmV3IFJvbWFuIixzZXJpZic+Jm5ic3A7ICZuYnNwOyBzcGRrX2Vudl9vcHRzX2lu aXQoJmFtcDtvcHRzKTs8bzpwPjwvbzpwPjwvc3Bhbj48L3A+PC9kaXY+PGRpdj48cCBjbGFzcz1N c29Ob3JtYWwgc3R5bGU9J21zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9t LWFsdDphdXRvO21hcmdpbi1sZWZ0OjQuOHB0Jz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjEyLjBw dDtmb250LWZhbWlseToiVGltZXMgTmV3IFJvbWFuIixzZXJpZic+Jm5ic3A7ICZuYnNwOyA8YSBo cmVmPSJodHRwOi8vb3B0cy5uYW1lIiB0YXJnZXQ9Il9ibGFuayI+b3B0cy5uYW1lPC9hPiA9ICZx dW90O2JsYWgmcXVvdDs7PG86cD48L286cD48L3NwYW4+PC9wPjwvZGl2PjxkaXY+PHAgY2xhc3M9 TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRv bS1hbHQ6YXV0bzttYXJnaW4tbGVmdDo0LjhwdCc+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMi4w cHQ7Zm9udC1mYW1pbHk6IlRpbWVzIE5ldyBSb21hbiIsc2VyaWYnPiZuYnNwOyAmbmJzcDsgb3B0 cy5jb3JlX21hc2sgPSAmcXVvdDsweEZGRkYmcXVvdDs7PG86cD48L286cD48L3NwYW4+PC9wPjwv ZGl2PjxkaXY+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6YXV0 bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0bzttYXJnaW4tbGVmdDo0LjhwdCc+PHNwYW4gc3R5 bGU9J2ZvbnQtc2l6ZToxMi4wcHQ7Zm9udC1mYW1pbHk6IlRpbWVzIE5ldyBSb21hbiIsc2VyaWYn PiZuYnNwOyAmbmJzcDsgb3B0cy5tYXN0ZXJfY29yZSA9IDE7PG86cD48L286cD48L3NwYW4+PC9w PjwvZGl2PjxkaXY+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFyZ2luLXRvcC1hbHQ6 YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0bzttYXJnaW4tbGVmdDo0LjhwdCc+PHNwYW4g c3R5bGU9J2ZvbnQtc2l6ZToxMi4wcHQ7Zm9udC1mYW1pbHk6IlRpbWVzIE5ldyBSb21hbiIsc2Vy aWYnPiZuYnNwOyAmbmJzcDsgb3B0cy5zaG1faWQgPSAwOzxvOnA+PC9vOnA+PC9zcGFuPjwvcD48 L2Rpdj48ZGl2PjxwIGNsYXNzPU1zb05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1 dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG87bWFyZ2luLWxlZnQ6NC44cHQnPjxzcGFuIHN0 eWxlPSdmb250LXNpemU6MTIuMHB0O2ZvbnQtZmFtaWx5OiJUaW1lcyBOZXcgUm9tYW4iLHNlcmlm Jz4mbmJzcDsgJm5ic3A7IGlmIChzcGRrX2Vudl9pbml0KCZhbXA7b3B0cykgJmx0OyAwKSB7PG86 cD48L286cD48L3NwYW4+PC9wPjwvZGl2PjxkaXY+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdt c28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0bzttYXJnaW4t bGVmdDo0LjhwdCc+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMi4wcHQ7Zm9udC1mYW1pbHk6IlRp bWVzIE5ldyBSb21hbiIsc2VyaWYnPiZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyB0aHJvdyBz dGQ6OnJ1bnRpbWVfZXJyb3IoJnF1b3Q7VW5hYmxlIHRvIGluaXRpYWxpemUgU1BESyBlbnYmcXVv dDspOzxvOnA+PC9vOnA+PC9zcGFuPjwvcD48L2Rpdj48ZGl2PjxwIGNsYXNzPU1zb05vcm1hbCBz dHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG87 bWFyZ2luLWxlZnQ6NC44cHQnPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTIuMHB0O2ZvbnQtZmFt aWx5OiJUaW1lcyBOZXcgUm9tYW4iLHNlcmlmJz4mbmJzcDsgJm5ic3A7IH08bzpwPjwvbzpwPjwv c3Bhbj48L3A+PC9kaXY+PC9kaXY+PGRpdj48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21zby1t YXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvO21hcmdpbi1sZWZ0 OjQuOHB0Jz48c3BhbiBzdHlsZT0nZm9udC1zaXplOjEyLjBwdDtmb250LWZhbWlseToiVGltZXMg TmV3IFJvbWFuIixzZXJpZic+V2hhdGV2ZXIgdmFsdWUgSSBhc3NpZ24gdG8gdGhlIGNvcmVfbWFz ayBldmVyeXRoaW5nIHJ1bnMgb24gdGhlIHNhbWUgY29yZSBkZWZpbmVkIGluIG1hc3Rlcl9jb3Jl LiBJZiBJIHNldCBhZmZpbml0eSB0byBteSB0aHJlYWRzIG1hbnVhbGx5IGV2ZXJ5dGhpbmcgd29y a3MgYXMgZXhwZWN0ZWQuIElzIGl0IGEgc3Bka19lbnZfaW5pdCBidWc/IEFtIEkgZG9pbmcgc29t ZXRoaW5nIHdyb25nPzxvOnA+PC9vOnA+PC9zcGFuPjwvcD48L2Rpdj48ZGl2PjxwIGNsYXNzPU1z b05vcm1hbCBzdHlsZT0nbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20t YWx0OmF1dG87bWFyZ2luLWxlZnQ6NC44cHQnPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTIuMHB0 O2ZvbnQtZmFtaWx5OiJUaW1lcyBOZXcgUm9tYW4iLHNlcmlmJz5TaW5jZXJlbHksPG86cD48L286 cD48L3NwYW4+PC9wPjwvZGl2PjxkaXY+PHAgY2xhc3M9TXNvTm9ybWFsIHN0eWxlPSdtc28tbWFy Z2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0bzttYXJnaW4tbGVmdDo0 LjhwdCc+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMi4wcHQ7Zm9udC1mYW1pbHk6IlRpbWVzIE5l dyBSb21hbiIsc2VyaWYnPkVybmVzdDxvOnA+PC9vOnA+PC9zcGFuPjwvcD48L2Rpdj48L2Rpdj48 L2Rpdj48L2Rpdj48L2Rpdj48L2Rpdj48L2Rpdj48cCBjbGFzcz1Nc29Ob3JtYWwgc3R5bGU9J21z by1tYXJnaW4tdG9wLWFsdDowaW47bWFyZ2luLXJpZ2h0OjBpbjttYXJnaW4tYm90dG9tOjEyLjBw dDttYXJnaW4tbGVmdDo0LjhwdCc+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMi4wcHQ7Zm9udC1m YW1pbHk6IlRpbWVzIE5ldyBSb21hbiIsc2VyaWYnPjxicj5fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fXzxicj5TUERLIG1haWxpbmcgbGlzdDxicj48YSBocmVm PSJtYWlsdG86U1BES0BsaXN0cy4wMS5vcmciPlNQREtAbGlzdHMuMDEub3JnPC9hPjxicj48YSBo cmVmPSJodHRwczovL2xpc3RzLjAxLm9yZy9tYWlsbWFuL2xpc3RpbmZvL3NwZGsiIHRhcmdldD0i X2JsYW5rIj5odHRwczovL2xpc3RzLjAxLm9yZy9tYWlsbWFuL2xpc3RpbmZvL3NwZGs8L2E+PG86 cD48L286cD48L3NwYW4+PC9wPjwvYmxvY2txdW90ZT48L2Rpdj48L2Rpdj48L2Rpdj48cCBjbGFz cz1Nc29Ob3JtYWw+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMi4wcHQ7Zm9udC1mYW1pbHk6IlRp bWVzIE5ldyBSb21hbiIsc2VyaWYnPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD48cCBjbGFz cz1Nc29Ob3JtYWw+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMi4wcHQ7Zm9udC1mYW1pbHk6IlRp bWVzIE5ldyBSb21hbiIsc2VyaWYnPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD48L2Rpdj48 L2JvZHk+PC9odG1sPg== --===============0396950122655663140==--