From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38174) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1V2m7f-00018B-0t for qemu-devel@nongnu.org; Fri, 26 Jul 2013 13:48:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1V2m7e-0002rP-0p for qemu-devel@nongnu.org; Fri, 26 Jul 2013 13:48:10 -0400 Received: from e38.co.us.ibm.com ([32.97.110.159]:36549) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1V2m7d-0002rK-PT for qemu-devel@nongnu.org; Fri, 26 Jul 2013 13:48:09 -0400 Received: from /spool/local by e38.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 26 Jul 2013 11:48:05 -0600 Received: from d03relay04.boulder.ibm.com (d03relay04.boulder.ibm.com [9.17.195.106]) by d03dlp02.boulder.ibm.com (Postfix) with ESMTP id 2806A3E4003E for ; Fri, 26 Jul 2013 11:47:39 -0600 (MDT) Received: from d03av06.boulder.ibm.com (d03av06.boulder.ibm.com [9.17.195.245]) by d03relay04.boulder.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id r6QHm1FL386322 for ; Fri, 26 Jul 2013 11:48:01 -0600 Received: from d03av06.boulder.ibm.com (loopback [127.0.0.1]) by d03av06.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id r6QHoeS8007107 for ; Fri, 26 Jul 2013 11:50:41 -0600 From: Anthony Liguori In-Reply-To: <87fvv172pk.fsf@blackfin.pond.sub.org> References: <1374842387-17146-1-git-send-email-armbru@redhat.com> <87ppu575aa.fsf@codemonkey.ws> <87fvv172pk.fsf@blackfin.pond.sub.org> Date: Fri, 26 Jul 2013 12:47:57 -0500 Message-ID: <87a9l99prm.fsf@codemonkey.ws> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: Re: [Qemu-devel] [PATCH 0/9] Our QAPI parser is a hack, replace it List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Markus Armbruster Cc: akong@redhat.com, qemu-devel@nongnu.org, mdroth@linux.vnet.ibm.com Markus Armbruster writes: > Anthony Liguori writes: > >> Markus Armbruster writes: >> >>> If you think I'm exaggerating, check out the list of issues in PATCH >>> 3/9. >> >> You are not. >> >> However, I think we can drop the whole thing and just use the JSON >> module in Python. The bit below seems to work: >> >> import json.decoder, re >> from ordereddict import OrderedDict >> >> WHITESPACE = re.compile(r'(#.*\n|[ \r\t\n]*)*', re.MULTILINE) >> >> def make_object(pairs): >> return OrderedDict(pairs) >> >> def qapi_parse(data): >> _w = WHITESPACE.match >> idx = 0 >> while idx < len(data): >> idx = _w(data, idx).end() >> if idx == len(data): >> break >> decoder = json.decoder.JSONDecoder(object_pairs_hook=make_object) >> obj, idx = decoder.raw_decode(data, idx) >> yield obj >> >> if __name__ == '__main__': >> with open('qapi-schema.json', 'r') as fp: >> data = fp.read().replace("'", '"') >> >> exprs = list(qapi_parse(data)) >> print exprs > > I tried to find a way to use JSONDecoder, but not hard enough, > apparently. > > The fp.read().replace("'", '"') is no good, because it blindly replaces > within strings, such as 'the cat\'s meow'. > > Can your code handle comments between arbitrary tokens? I suspect they > work only between top-level expressions, but I could be wrong; Python > isn't my strongest language, and I didn't test this. It cannot. The python JSON module has an optimized C implementation which is less flexible than the python one. Unfortunately it looks like at least in my copy of python, the python version has bitrotted. OTOH, the warnings are very clear when you attempt to do this. Regards, Anthony Liguori