From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: multipart/mixed; boundary="===============4963417543589286307==" MIME-Version: 1.0 From: Daniel Verkamp Subject: Re: [SPDK] About dump/load (or export/edit/import) JSON file by SPDK Date: Fri, 02 Feb 2018 10:24:15 -0700 Message-ID: <6c5f58f0-0dec-b9d8-c8e5-38f17dadeb38@intel.com> In-Reply-To: 43AD6546-E0F8-4B0D-93EB-C214D9FBA046@intel.com List-ID: To: spdk@lists.01.org --===============4963417543589286307== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable On 02/02/2018 09:25 AM, Harris, James R wrote: > *From: *SPDK on behalf of "Wodkowski, Pawel= X" > *Reply-To: *Storage Performance Development Kit > *Date: *Friday, February 2, 2018 at 7:19 AM > *To: *Storage Performance Development Kit > *Subject: *Re: [SPDK] About dump/load (or export/edit/import) JSON file b= y SPDK > = > About your questions: >> - the top layer of a JSON configuration file is a JSON object > = > I vote for this > = > I agree =E2=80=93 making the JSON configuration file a JSON object seems = like a good idea. > = > I would like to consider allowing comments + something like JSON-minify t= hough.=C2=A0 https://github.com/getify/JSON.minify/tree/python > = > I=E2=80=99d love Daniel=E2=80=99s input on this =E2=80=93 50/50 chance he= =E2=80=99ll either like the minify idea, or will flame me for suggesting it. I agree that we definitely want to be able to have comments in the configur= ation file. We already have support for (non-standard) JavaScript-style // and /* */ co= mments in our JSON parser; it just needs to be enabled with the SPDK_JSON_P= ARSE_FLAG_ALLOW_COMMENTS flag. However, if we make the config file parser an external Python tool, we woul= d probably need something like JSON-minify to remove the comments first (or= a JSON parser that has non-standard comment support). >> - the object has the following key-value pairs: > = >>=C2=A0 - key =3D "apptype", value =3D "name of application type", > = > I don=E2=80=99t see any value in adding this. If we decide to merge all a= pps into one capable of serving iSCSI, vhost, > NVMeF what to do with this type of field. It will be dead anyway. > = > I agree with Pawel here.=C2=A0 I could see cases for testing where we may= even describe the configuration using multiple JSON-RPC files.=C2=A0 For e= xample, one configuration file to construct a bunch of block devices that a= re common across a bunch of different tests.=C2=A0 Then each test could pro= vide a second configuration file that only provides specifics for the upper= layer target (iSCSI, NVMe-oF, vhost, etc.) > = >>=C2=A0 - key =3D "jsonrpc", value =3D "2.0", >>=C2=A0 - key =3D "sequence", value =3D an array of request objects which = is made of ID, method, and params >>=C2=A0 - Currently ID of JSON-RPC request is always 1. >>=C2=A0 - Change it to the sequence number of each JSON-RPC request to cal= l JSON-RPC requests in order. > = > User app might want to read many config files as well want to save config= of each subsystem to different file. > = > I would vote against embedding =E2=80=9Cjsonrpc=E2=80=9D, =E2=80=9Cid=E2= =80=9D etc in any JSON config file. Only data needed to fully restore > application configuration should be needed in JSON config file. Agreed, the jsonrpc and id fields shouldn't be necessary in the config file= dump format. "id" is only really useful for interactive RPC use where mul= tiple requests are submitted at once; the "id" can be used to match up resp= onses to requests, but in the config file case, we can submit the requests = one at a time. >> Do you think any general metadata should be included in the file or just= sequence of requests is enough? > = > For other =E2=80=9Cmetadata=E2=80=9D reported by RPC calls there could be= some additional call like =E2=80=9Cget_info=E2=80=9D. > = > I think a sequence of requests is enough =E2=80=93 plus ability to add co= mments in the JSON. > = > I think a new RPC that accepts a batch of other JSON-RPC objects would be= challenging from an error handling perspective.=C2=A0 For example, what ha= ppens if the 4^th RPC out of a batch of 20 fails?=C2=A0 Sure =E2=80=93 we c= ould try to define a protocol for this but I think having the client just s= end one RPC at a time is sufficient for now. I like the idea of having an external tool (possibly written in Python) tha= t loads the JSON config file and sends RPC requests one at a time through t= he existing JSON-RPC interface; that keeps the SPDK library code modificati= ons to a minimum. Thanks, -- Daniel --===============4963417543589286307==--