From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from eggs.gnu.org ([2001:4830:134:3::10]:59221)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <armbru@redhat.com>) id 1V2k4S-00055w-Ja
	for qemu-devel@nongnu.org; Fri, 26 Jul 2013 11:36:45 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <armbru@redhat.com>) id 1V2k4R-0005ju-LJ
	for qemu-devel@nongnu.org; Fri, 26 Jul 2013 11:36:44 -0400
Received: from mx1.redhat.com ([209.132.183.28]:49459)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <armbru@redhat.com>) id 1V2k4R-0005jj-Ch
	for qemu-devel@nongnu.org; Fri, 26 Jul 2013 11:36:43 -0400
From: Markus Armbruster <armbru@redhat.com>
References: <1374842387-17146-1-git-send-email-armbru@redhat.com>
	<87ppu575aa.fsf@codemonkey.ws>
Date: Fri, 26 Jul 2013 17:36:39 +0200
In-Reply-To: <87ppu575aa.fsf@codemonkey.ws> (Anthony Liguori's message of
	"Fri, 26 Jul 2013 09:41:01 -0500")
Message-ID: <87fvv172pk.fsf@blackfin.pond.sub.org>
MIME-Version: 1.0
Content-Type: text/plain
Subject: Re: [Qemu-devel] [PATCH 0/9] Our QAPI parser is a hack, replace it
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=subscribe>
To: Anthony Liguori <aliguori@us.ibm.com>
Cc: akong@redhat.com, qemu-devel@nongnu.org, mdroth@linux.vnet.ibm.com

Anthony Liguori <aliguori@us.ibm.com> writes:

> Markus Armbruster <armbru@redhat.com> writes:
>
>> If you think I'm exaggerating, check out the list of issues in PATCH
>> 3/9.
>
> You are not.
>
> However, I think we can drop the whole thing and just use the JSON
> module in Python.  The bit below seems to work:
>
>   import json.decoder, re
>   from ordereddict import OrderedDict
>   
>   WHITESPACE = re.compile(r'(#.*\n|[ \r\t\n]*)*', re.MULTILINE)
>   
>   def make_object(pairs):
>       return OrderedDict(pairs)
>   
>   def qapi_parse(data):
>       _w = WHITESPACE.match
>       idx = 0
>       while idx < len(data):
>           idx = _w(data, idx).end()
>           if idx == len(data):
>               break
>           decoder = json.decoder.JSONDecoder(object_pairs_hook=make_object)
>           obj, idx = decoder.raw_decode(data, idx)
>           yield obj
>   
>   if __name__ == '__main__':
>       with open('qapi-schema.json', 'r') as fp:
>           data = fp.read().replace("'", '"')
>   
>       exprs = list(qapi_parse(data))
>       print exprs

I tried to find a way to use JSONDecoder, but not hard enough,
apparently.

The fp.read().replace("'", '"') is no good, because it blindly replaces
within strings, such as 'the cat\'s meow'.

Can your code handle comments between arbitrary tokens?  I suspect they
work only between top-level expressions, but I could be wrong; Python
isn't my strongest language, and I didn't test this.