qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: John Snow <jsnow@redhat.com>
To: Markus Armbruster <armbru@redhat.com>
Cc: Stefan Hajnoczi <stefanha@gmail.com>,
	QEMU Developers <qemu-devel@nongnu.org>,
	Stefan Hajnoczi <stefanha@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>
Subject: Re: Call for Google Summer of Code 2021 project ideas
Date: Thu, 14 Jan 2021 11:36:23 -0500	[thread overview]
Message-ID: <c26786ac-159e-149a-aa5e-dd08f418d11e@redhat.com> (raw)
In-Reply-To: <87h7njsnui.fsf@dusky.pond.sub.org>

On 1/14/21 7:29 AM, Markus Armbruster wrote:
> John Snow <jsnow@redhat.com> writes:
> 
>> On 1/13/21 4:19 AM, Markus Armbruster wrote:
>>> John Snow <jsnow@redhat.com> writes:
>>>
>>>> On 1/11/21 6:47 AM, Stefan Hajnoczi wrote:
>>>>> Dear QEMU, KVM, and rust-vmm community,
>>>>> QEMU will apply for Google Summer of Code
>>>>> (https://summerofcode.withgoogle.com/) again this year.  This internship
>>>>> program offers paid, 10-week, remote work internships for
>>>>> contributing to open source.  QEMU can act as an umbrella organization
>>>>> for KVM kernel and rust-vmm projects too.
>>>>> Please post project ideas on the QEMU wiki before February 14th:
>>>>> https://wiki.qemu.org/Google_Summer_of_Code_2021
>>>>> What's new this year:
>>>>>     * The number of internship hours has been halved to 175 hours over
>>>>>       10 weeks. Project ideas must be smaller to fit and students will have
>>>>>       more flexibility with their working hours.
>>>>>     * Eligibility has been expanded to include "licensed coding school or
>>>>>       similar type of program".
>>>>> Good project ideas are suitable for 175 hours (10 weeks half-day)
>>>>> work by a
>>>>> competent programmer who is not yet familiar with the codebase.  In
>>>>> addition, they are:
>>>>>     * Well-defined - the scope is clear
>>>>>     * Self-contained - there are few dependencies
>>>>>     * Uncontroversial - they are acceptable to the community
>>>>>     * Incremental - they produce deliverables along the way
>>>>> Feel free to post ideas even if you are unable to mentor the
>>>>> project.
>>>>> It doesn't hurt to share the idea!
>>>>>
>>>>
>>>> I have one that is probably way too ambitious, but requires a
>>>> particular skillset that might be of good interest to a student that
>>>> has some experience in the area already.
>>>>
>>>> The idea is for a TUI qmp-shell (maybe using urwid?) to create an
>>>> irssi-like REPL interface for QMP. The idea would be to mimic the
>>>> mitmproxy TUI interface (Check it out if you haven't!)
>>>>
>>>> All the ideas below are extremely tentative to give a basic gist of
>>>> what I mean; exact layouts/hotkeys/etc are for the sake of explanation
>>>> only.
>>>>
>>>> Essentially, I want an interface like this:
>>>>
>>>> -----------------------------------------------------------
>>>> | QMP Mode                                                |
>>>> |=========================================================|
>>>> |                                                         |
>>>> | Welcome to the Qemu Machine Protocol shell.             |
>>>> | Please type /help or Ctrl+H to see available commands.  |
>>>> |                                                         |
>>>> |                                                         |
>>>> |                                                         |
>>>> |---------------------------------------------------------|
>>>> | >                                                       |
>>>> -----------------------------------------------------------
>>>>
>>>> commands are entered in the bottom and appear in a log window above,
>>>> appearing most-recent last, like irssi works.
>>>>
>>>> As an example, let's say we issue block-dirty-bitmap-add:
>>>>
>>>> --------------------------------------------------------
>>>> | > block-dirty-bitmap-add node=ide0hd0 name=mybitmap0 |
>>>> --------------------------------------------------------
>>>>
>>>> (...syntax up for debate...! We want something easy to parse, but easy
>>>> to type. For structured data, that's Hard.)
>>> "Hard" is a red flag for such a brief internship.  We need to solve
>>> or
>>> somehow exclude this design problem before we hand the task to the
>>> student.
>>> [...]
>>>
>>
>> Yes, there is a reason I capitalized "Hard". :~)
>>
>> This task CAN be split out, though. For starters, we can use an
>> extremely simplified parsing strategy while we work on the REPL and
>> and UI elements.
>>
>> We can accept things of the form:
>> command arg=value arg2=value
>>
>> and we can worry about how to input structured data later.
> 
> Bzzzt!  You just designed a language.  The fact that it is obviously
> inadequate for the task does not make this not language design.
> 

Did *I* design a language, or did qmp-shell authors design a language? O:-)

(Oh, I'm one of the authors. Well then.)

> Two outcomes:
> 
> 1. The project fails for other reasons.  The language doesn't matter.
> 
> 2. The project succeeds, i.e. it becomes useful to people.  Very quickly
> these people will demand you evolve your inadequate language into a less
> inadequate one, compatibly.  Your evasion of the language design problem
> (because hard) has now added accidental constraints, making the design
> problem even harder.
> 

There's no reason that an interactive shell has to stay backwards 
compatible. Version 0.1 and 0.2 might offer different input paradigms. 
It's not nice, but it's fine for beta-level stuff.

Still, I appreciate that "Beta! Might change!" is an oft-ignored 
warning. This isn't a library or API though, so I still think it's okay 
to take some liberty here.

> The sane way to evade the language design problem is to use the existing
> QMP language.
> 

I wouldn't mind implementing this for version 0.1 -- just allow 
copy-pasting JSON into the input bar -- it's a feature I wanted anyway.

Some things I would like to see, but suspect are not reasonably possible 
to implement quickly:

1. A shorthand syntax mode similar to qmp-shell that is reasonably 
sufficient for most commands. This involves solving a fairly difficult 
language design problem on how to facilitate the input of structured 
data. The biggest pain point here is nested data; we allow JSON as an 
input here but require that it has no spaces. We do not offer a "native" 
shorthand for the nested data.

We *could* co-opt the dotted key syntax, but that's cumbersome for some 
constructs because it involves a lot of repeated parent keys.

As you say, the language design aspect is hard.


2. a multi-line editor mode that facilitates typing QMP messages that 
offers syntax highlighting, field and type validation, autocomplete, 
etc. I have no idea how to implement this -- it's basically an entire 
bespoke editor.


That's *very* hard stuff, I expect, so I won't go there for starters.

So I see two possible options for "not inventing a language":

1. Raw QMP
2. The existing qmp-shell syntax, warts and all.

I don't see a tremendous problem with doing both; but we can start with 
raw QMP. The existing qmp-shell syntax is at least definitely very easy 
to write a new parser for, even if it's kind of ugly and insufficient. I 
still see value in designing a new TUI with the old syntax.

> The project then aims to build a tool that adds useful features over
> "socat "READLINE,history=$HOME/.qmp_history,prompt=QMP>"
> UNIX-CONNECT:/path/to/socket".
> 
> If it succeeds, you can still design and implement a "better" language,
> and let users choose the one they prefer.  Or you could add features to
> help with typing QMP.
> 
>>                                                             I don't
>> think it's a blocker to have someone work on the TUI and asynchronous
>> dispatch elements. I think even just keeping our current parsing but
>> adding some of the features outlined in the proposal would be a big
>> usability win.
> 
> I don't feel this particular itch, but I'm certainly not objecting to
> anyone scratching.
> 

It's something I'd like to see so that I can walk non-QEMU devs through 
interacting with QEMU at a low level for the purposes of debugging, 
reproducing problems, prototyping features, etc.

I use qmp-shell all the time for debugging things myself, I find it 
easier to use than copy-pasting things directly into socat. I wouldn't 
mind the shell getting a little smarter to help me out -- the ability to 
see async events and reconnect on disconnect would already be a massive 
improvement to *my* quality of life.

So much so that I spent a lot of time in December to write an async qmp 
library O:-)

--js



  parent reply	other threads:[~2021-01-14 17:05 UTC|newest]

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-01-11 11:47 Call for Google Summer of Code 2021 project ideas Stefan Hajnoczi
2021-01-12 21:10 ` John Snow
2021-01-13  8:53   ` Stefan Hajnoczi
2021-01-13 18:59     ` qmp-shell TUI (was: Re: Call for Google Summer of Code 2021 project ideas) John Snow
2021-01-14 13:52       ` Stefan Hajnoczi
2021-01-14 13:59         ` Daniel P. Berrangé
2021-01-14 15:02           ` Kevin Wolf
2021-01-14 15:22             ` Daniel P. Berrangé
2021-01-14 16:49               ` Stefan Hajnoczi
2021-01-14 16:55                 ` Daniel P. Berrangé
2021-01-14 17:14                   ` Stefan Hajnoczi
2021-01-14 17:24                     ` Daniel P. Berrangé
2021-01-14 16:48             ` Stefan Hajnoczi
2021-01-14 17:48         ` John Snow
2021-01-13  9:19   ` Call for Google Summer of Code 2021 project ideas Markus Armbruster
2021-01-13 19:05     ` John Snow
2021-01-14 12:29       ` Markus Armbruster
2021-01-14 14:57         ` Kevin Wolf
2021-01-14 16:36         ` John Snow [this message]
2021-01-15 16:31           ` Kashyap Chamarthy
2021-02-15 11:05           ` Paolo Bonzini
2021-02-15 21:47             ` John Snow
2021-02-12 13:22 ` [Rust-VMM] " Florescu, Andreea
2021-02-12 13:51   ` Sergio Lopez
2021-02-17 11:20   ` Stefan Hajnoczi
2021-02-18 11:49     ` Andreea Florescu
2021-02-18 17:43       ` Stefan Hajnoczi

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=c26786ac-159e-149a-aa5e-dd08f418d11e@redhat.com \
    --to=jsnow@redhat.com \
    --cc=armbru@redhat.com \
    --cc=ehabkost@redhat.com \
    --cc=qemu-devel@nongnu.org \
    --cc=stefanha@gmail.com \
    --cc=stefanha@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).