[Qemu-devel] Registering buffers with a qdict

[Qemu-devel] Registering buffers with a qdict

[Jan Kiszka]

Hi Luiz,

what is the recommended way of pushing larger buffers (up to 64K so far)
into a qdict? QLIST of QINT (one per byte) looks a bit heavy. I thought
about hex-encoding the content first (series of "%02X"), then
registering it as QSTRING. Or should we introduce a new type, QBUFFER?

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

[Qemu-devel] Re: Registering buffers with a qdict

[Luiz Capitulino]

On Fri, 07 May 2010 13:45:03 +0200
Jan Kiszka <jan.kiszka@siemens.com> wrote:

> Hi Luiz,
> 
> what is the recommended way of pushing larger buffers (up to 64K so far)
> into a qdict? QLIST of QINT (one per byte) looks a bit heavy. I thought
> about hex-encoding the content first (series of "%02X"), then
> registering it as QSTRING. Or should we introduce a new type, QBUFFER?

 I don't think that hex-encoding the contents is so bad if your use case is
very specific and isolated.

 On the other hand, I do prefer a QBuffer type, specially because we can
have buffer operations.

[Qemu-devel] Re: Registering buffers with a qdict

[Jan Kiszka]

Luiz Capitulino wrote:
> On Fri, 07 May 2010 13:45:03 +0200
> Jan Kiszka <jan.kiszka@siemens.com> wrote:
> 
>> Hi Luiz,
>>
>> what is the recommended way of pushing larger buffers (up to 64K so far)
>> into a qdict? QLIST of QINT (one per byte) looks a bit heavy. I thought
>> about hex-encoding the content first (series of "%02X"), then
>> registering it as QSTRING. Or should we introduce a new type, QBUFFER?
> 
>  I don't think that hex-encoding the contents is so bad if your use case is
> very specific and isolated.

The focus will be first on visualizing the buffer (user_print), but who
knows what happens once the services is also exposed via QMP.

> 
>  On the other hand, I do prefer a QBuffer type, specially because we can
> have buffer operations.

The q<type>.c files look sufficiently simply, guess I will add a buffer
type. Still, hex-encoding is probably the best representation for QMP.

Thanks,
Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

[Qemu-devel] Re: Registering buffers with a qdict

[Luiz Capitulino]

On Fri, 07 May 2010 16:21:13 +0200
Jan Kiszka <jan.kiszka@siemens.com> wrote:

> Luiz Capitulino wrote:
> > On Fri, 07 May 2010 13:45:03 +0200
> > Jan Kiszka <jan.kiszka@siemens.com> wrote:
> > 
> >> Hi Luiz,
> >>
> >> what is the recommended way of pushing larger buffers (up to 64K so far)
> >> into a qdict? QLIST of QINT (one per byte) looks a bit heavy. I thought
> >> about hex-encoding the content first (series of "%02X"), then
> >> registering it as QSTRING. Or should we introduce a new type, QBUFFER?
> > 
> >  I don't think that hex-encoding the contents is so bad if your use case is
> > very specific and isolated.
> 
> The focus will be first on visualizing the buffer (user_print), but who
> knows what happens once the services is also exposed via QMP.
> 
> > 
> >  On the other hand, I do prefer a QBuffer type, specially because we can
> > have buffer operations.
> 
> The q<type>.c files look sufficiently simply, guess I will add a buffer
> type. Still, hex-encoding is probably the best representation for QMP.

 Yes, either as a string or (as suggested by Markus) an array of numbers,
if you wish to expose this via QMP you (or any of us) will have to update
the parser to support it.

[Qemu-devel] Re: Registering buffers with a qdict

[Jan Kiszka]

Luiz Capitulino wrote:
> On Fri, 07 May 2010 16:21:13 +0200
> Jan Kiszka <jan.kiszka@siemens.com> wrote:
> 
>> Luiz Capitulino wrote:
>>> On Fri, 07 May 2010 13:45:03 +0200
>>> Jan Kiszka <jan.kiszka@siemens.com> wrote:
>>>
>>>> Hi Luiz,
>>>>
>>>> what is the recommended way of pushing larger buffers (up to 64K so far)
>>>> into a qdict? QLIST of QINT (one per byte) looks a bit heavy. I thought
>>>> about hex-encoding the content first (series of "%02X"), then
>>>> registering it as QSTRING. Or should we introduce a new type, QBUFFER?
>>>  I don't think that hex-encoding the contents is so bad if your use case is
>>> very specific and isolated.
>> The focus will be first on visualizing the buffer (user_print), but who
>> knows what happens once the services is also exposed via QMP.
>>
>>>  On the other hand, I do prefer a QBuffer type, specially because we can
>>> have buffer operations.
>> The q<type>.c files look sufficiently simply, guess I will add a buffer
>> type. Still, hex-encoding is probably the best representation for QMP.
> 
>  Yes, either as a string or (as suggested by Markus) an array of numbers,
> if you wish to expose this via QMP you (or any of us) will have to update
> the parser to support it.

>From a quick glance at the JSON spec, there is no room for a new type. I
think we have to overload an existing one and convert that into a
QBuffer (typically, we know the actual semantic). Hex string encoding is
most compact, so I went this road. But I'm open to change it into a true
type if JSON actually allows it (or we are fine with breaking it).

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

[Qemu-devel] Re: Registering buffers with a qdict

[Luiz Capitulino]

On Mon, 10 May 2010 12:59:47 +0200
Jan Kiszka <jan.kiszka@siemens.com> wrote:

> Luiz Capitulino wrote:
> > On Fri, 07 May 2010 16:21:13 +0200
> > Jan Kiszka <jan.kiszka@siemens.com> wrote:
> > 
> >> Luiz Capitulino wrote:
> >>> On Fri, 07 May 2010 13:45:03 +0200
> >>> Jan Kiszka <jan.kiszka@siemens.com> wrote:
> >>>
> >>>> Hi Luiz,
> >>>>
> >>>> what is the recommended way of pushing larger buffers (up to 64K so far)
> >>>> into a qdict? QLIST of QINT (one per byte) looks a bit heavy. I thought
> >>>> about hex-encoding the content first (series of "%02X"), then
> >>>> registering it as QSTRING. Or should we introduce a new type, QBUFFER?
> >>>  I don't think that hex-encoding the contents is so bad if your use case is
> >>> very specific and isolated.
> >> The focus will be first on visualizing the buffer (user_print), but who
> >> knows what happens once the services is also exposed via QMP.
> >>
> >>>  On the other hand, I do prefer a QBuffer type, specially because we can
> >>> have buffer operations.
> >> The q<type>.c files look sufficiently simply, guess I will add a buffer
> >> type. Still, hex-encoding is probably the best representation for QMP.
> > 
> >  Yes, either as a string or (as suggested by Markus) an array of numbers,
> > if you wish to expose this via QMP you (or any of us) will have to update
> > the parser to support it.
> 
> From a quick glance at the JSON spec, there is no room for a new type. I
> think we have to overload an existing one and convert that into a
> QBuffer (typically, we know the actual semantic). Hex string encoding is
> most compact, so I went this road. But I'm open to change it into a true
> type if JSON actually allows it (or we are fine with breaking it).

 We have capabilities support but I don't think we should use it for this
case. We can have QBuffer internally and the parser can convert it to
a valid json type.

Re: [Qemu-devel] Re: Registering buffers with a qdict

[Avi Kivity]

On 05/10/2010 01:59 PM, Jan Kiszka wrote:
>
>  From a quick glance at the JSON spec, there is no room for a new type. I
> think we have to overload an existing one and convert that into a
> QBuffer (typically, we know the actual semantic). Hex string encoding is
> most compact, so I went this road.

Base64 is even more compact.

> But I'm open to change it into a true
> type if JSON actually allows it (or we are fine with breaking it).
>    

That ruins any possibility of using a standard json encoder/decoder on 
the other end.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Re: Registering buffers with a qdict

[Jan Kiszka]

Avi Kivity wrote:
> On 05/10/2010 01:59 PM, Jan Kiszka wrote:
>>  From a quick glance at the JSON spec, there is no room for a new type. I
>> think we have to overload an existing one and convert that into a
>> QBuffer (typically, we know the actual semantic). Hex string encoding is
>> most compact, so I went this road.
> 
> Base64 is even more compact.

For sure, still room for improvements. There is just no encode() service
in current qemu, so I went the lazy way so far. :)

> 
>> But I'm open to change it into a true
>> type if JSON actually allows it (or we are fine with breaking it).
>>    
> 
> That ruins any possibility of using a standard json encoder/decoder on 
> the other end.
> 

That was my concern as well. Such decoders would not able to tell
strings apart from buffers as that can only be derived from the context.
Still, they could visualize the result and/or forward it to some libqmp
for proper interpretation.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Qemu-devel] Re: Registering buffers with a qdict

[Avi Kivity]

On 05/10/2010 04:43 PM, Jan Kiszka wrote:
> Avi Kivity wrote:
>    
>> On 05/10/2010 01:59 PM, Jan Kiszka wrote:
>>      
>>>    From a quick glance at the JSON spec, there is no room for a new type. I
>>> think we have to overload an existing one and convert that into a
>>> QBuffer (typically, we know the actual semantic). Hex string encoding is
>>> most compact, so I went this road.
>>>        
>> Base64 is even more compact.
>>      
> For sure, still room for improvements. There is just no encode() service
> in current qemu, so I went the lazy way so far. :)
>    

Well, if we expose these encoded buffers via QMP, we can't unlazy the 
implementation.  It becomes an ABI, so we better think (and document) 
this through.

>>> But I'm open to change it into a true
>>> type if JSON actually allows it (or we are fine with breaking it).
>>>
>>>        
>> That ruins any possibility of using a standard json encoder/decoder on
>> the other end.
>>
>>      
> That was my concern as well. Such decoders would not able to tell
> strings apart from buffers as that can only be derived from the context.
> Still, they could visualize the result and/or forward it to some libqmp
> for proper interpretation.
>    

That's fine; the documentation for a command that accepts or returns 
buffers would mention that the value is a hex (or base64) encoded string.

-- 
error compiling committee.c: too many arguments to function