From: John Snow <jsnow@redhat.com>
To: Markus Armbruster <armbru@redhat.com>
Cc: Stefan Hajnoczi <stefanha@gmail.com>,
QEMU Developers <qemu-devel@nongnu.org>,
Stefan Hajnoczi <stefanha@redhat.com>,
Eduardo Habkost <ehabkost@redhat.com>
Subject: Re: Call for Google Summer of Code 2021 project ideas
Date: Thu, 14 Jan 2021 11:36:23 -0500 [thread overview]
Message-ID: <c26786ac-159e-149a-aa5e-dd08f418d11e@redhat.com> (raw)
In-Reply-To: <87h7njsnui.fsf@dusky.pond.sub.org>
On 1/14/21 7:29 AM, Markus Armbruster wrote:
> John Snow <jsnow@redhat.com> writes:
>
>> On 1/13/21 4:19 AM, Markus Armbruster wrote:
>>> John Snow <jsnow@redhat.com> writes:
>>>
>>>> On 1/11/21 6:47 AM, Stefan Hajnoczi wrote:
>>>>> Dear QEMU, KVM, and rust-vmm community,
>>>>> QEMU will apply for Google Summer of Code
>>>>> (https://summerofcode.withgoogle.com/) again this year. This internship
>>>>> program offers paid, 10-week, remote work internships for
>>>>> contributing to open source. QEMU can act as an umbrella organization
>>>>> for KVM kernel and rust-vmm projects too.
>>>>> Please post project ideas on the QEMU wiki before February 14th:
>>>>> https://wiki.qemu.org/Google_Summer_of_Code_2021
>>>>> What's new this year:
>>>>> * The number of internship hours has been halved to 175 hours over
>>>>> 10 weeks. Project ideas must be smaller to fit and students will have
>>>>> more flexibility with their working hours.
>>>>> * Eligibility has been expanded to include "licensed coding school or
>>>>> similar type of program".
>>>>> Good project ideas are suitable for 175 hours (10 weeks half-day)
>>>>> work by a
>>>>> competent programmer who is not yet familiar with the codebase. In
>>>>> addition, they are:
>>>>> * Well-defined - the scope is clear
>>>>> * Self-contained - there are few dependencies
>>>>> * Uncontroversial - they are acceptable to the community
>>>>> * Incremental - they produce deliverables along the way
>>>>> Feel free to post ideas even if you are unable to mentor the
>>>>> project.
>>>>> It doesn't hurt to share the idea!
>>>>>
>>>>
>>>> I have one that is probably way too ambitious, but requires a
>>>> particular skillset that might be of good interest to a student that
>>>> has some experience in the area already.
>>>>
>>>> The idea is for a TUI qmp-shell (maybe using urwid?) to create an
>>>> irssi-like REPL interface for QMP. The idea would be to mimic the
>>>> mitmproxy TUI interface (Check it out if you haven't!)
>>>>
>>>> All the ideas below are extremely tentative to give a basic gist of
>>>> what I mean; exact layouts/hotkeys/etc are for the sake of explanation
>>>> only.
>>>>
>>>> Essentially, I want an interface like this:
>>>>
>>>> -----------------------------------------------------------
>>>> | QMP Mode |
>>>> |=========================================================|
>>>> | |
>>>> | Welcome to the Qemu Machine Protocol shell. |
>>>> | Please type /help or Ctrl+H to see available commands. |
>>>> | |
>>>> | |
>>>> | |
>>>> |---------------------------------------------------------|
>>>> | > |
>>>> -----------------------------------------------------------
>>>>
>>>> commands are entered in the bottom and appear in a log window above,
>>>> appearing most-recent last, like irssi works.
>>>>
>>>> As an example, let's say we issue block-dirty-bitmap-add:
>>>>
>>>> --------------------------------------------------------
>>>> | > block-dirty-bitmap-add node=ide0hd0 name=mybitmap0 |
>>>> --------------------------------------------------------
>>>>
>>>> (...syntax up for debate...! We want something easy to parse, but easy
>>>> to type. For structured data, that's Hard.)
>>> "Hard" is a red flag for such a brief internship. We need to solve
>>> or
>>> somehow exclude this design problem before we hand the task to the
>>> student.
>>> [...]
>>>
>>
>> Yes, there is a reason I capitalized "Hard". :~)
>>
>> This task CAN be split out, though. For starters, we can use an
>> extremely simplified parsing strategy while we work on the REPL and
>> and UI elements.
>>
>> We can accept things of the form:
>> command arg=value arg2=value
>>
>> and we can worry about how to input structured data later.
>
> Bzzzt! You just designed a language. The fact that it is obviously
> inadequate for the task does not make this not language design.
>
Did *I* design a language, or did qmp-shell authors design a language? O:-)
(Oh, I'm one of the authors. Well then.)
> Two outcomes:
>
> 1. The project fails for other reasons. The language doesn't matter.
>
> 2. The project succeeds, i.e. it becomes useful to people. Very quickly
> these people will demand you evolve your inadequate language into a less
> inadequate one, compatibly. Your evasion of the language design problem
> (because hard) has now added accidental constraints, making the design
> problem even harder.
>
There's no reason that an interactive shell has to stay backwards
compatible. Version 0.1 and 0.2 might offer different input paradigms.
It's not nice, but it's fine for beta-level stuff.
Still, I appreciate that "Beta! Might change!" is an oft-ignored
warning. This isn't a library or API though, so I still think it's okay
to take some liberty here.
> The sane way to evade the language design problem is to use the existing
> QMP language.
>
I wouldn't mind implementing this for version 0.1 -- just allow
copy-pasting JSON into the input bar -- it's a feature I wanted anyway.
Some things I would like to see, but suspect are not reasonably possible
to implement quickly:
1. A shorthand syntax mode similar to qmp-shell that is reasonably
sufficient for most commands. This involves solving a fairly difficult
language design problem on how to facilitate the input of structured
data. The biggest pain point here is nested data; we allow JSON as an
input here but require that it has no spaces. We do not offer a "native"
shorthand for the nested data.
We *could* co-opt the dotted key syntax, but that's cumbersome for some
constructs because it involves a lot of repeated parent keys.
As you say, the language design aspect is hard.
2. a multi-line editor mode that facilitates typing QMP messages that
offers syntax highlighting, field and type validation, autocomplete,
etc. I have no idea how to implement this -- it's basically an entire
bespoke editor.
That's *very* hard stuff, I expect, so I won't go there for starters.
So I see two possible options for "not inventing a language":
1. Raw QMP
2. The existing qmp-shell syntax, warts and all.
I don't see a tremendous problem with doing both; but we can start with
raw QMP. The existing qmp-shell syntax is at least definitely very easy
to write a new parser for, even if it's kind of ugly and insufficient. I
still see value in designing a new TUI with the old syntax.
> The project then aims to build a tool that adds useful features over
> "socat "READLINE,history=$HOME/.qmp_history,prompt=QMP>"
> UNIX-CONNECT:/path/to/socket".
>
> If it succeeds, you can still design and implement a "better" language,
> and let users choose the one they prefer. Or you could add features to
> help with typing QMP.
>
>> I don't
>> think it's a blocker to have someone work on the TUI and asynchronous
>> dispatch elements. I think even just keeping our current parsing but
>> adding some of the features outlined in the proposal would be a big
>> usability win.
>
> I don't feel this particular itch, but I'm certainly not objecting to
> anyone scratching.
>
It's something I'd like to see so that I can walk non-QEMU devs through
interacting with QEMU at a low level for the purposes of debugging,
reproducing problems, prototyping features, etc.
I use qmp-shell all the time for debugging things myself, I find it
easier to use than copy-pasting things directly into socat. I wouldn't
mind the shell getting a little smarter to help me out -- the ability to
see async events and reconnect on disconnect would already be a massive
improvement to *my* quality of life.
So much so that I spent a lot of time in December to write an async qmp
library O:-)
--js
next prev parent reply other threads:[~2021-01-14 17:05 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-11 11:47 Call for Google Summer of Code 2021 project ideas Stefan Hajnoczi
2021-01-12 21:10 ` John Snow
2021-01-13 8:53 ` Stefan Hajnoczi
2021-01-13 18:59 ` qmp-shell TUI (was: Re: Call for Google Summer of Code 2021 project ideas) John Snow
2021-01-14 13:52 ` Stefan Hajnoczi
2021-01-14 13:59 ` Daniel P. Berrangé
2021-01-14 15:02 ` Kevin Wolf
2021-01-14 15:22 ` Daniel P. Berrangé
2021-01-14 16:49 ` Stefan Hajnoczi
2021-01-14 16:55 ` Daniel P. Berrangé
2021-01-14 17:14 ` Stefan Hajnoczi
2021-01-14 17:24 ` Daniel P. Berrangé
2021-01-14 16:48 ` Stefan Hajnoczi
2021-01-14 17:48 ` John Snow
2021-01-13 9:19 ` Call for Google Summer of Code 2021 project ideas Markus Armbruster
2021-01-13 19:05 ` John Snow
2021-01-14 12:29 ` Markus Armbruster
2021-01-14 14:57 ` Kevin Wolf
2021-01-14 16:36 ` John Snow [this message]
2021-01-15 16:31 ` Kashyap Chamarthy
2021-02-15 11:05 ` Paolo Bonzini
2021-02-15 21:47 ` John Snow
2021-02-12 13:22 ` [Rust-VMM] " Florescu, Andreea
2021-02-12 13:51 ` Sergio Lopez
2021-02-17 11:20 ` Stefan Hajnoczi
2021-02-18 11:49 ` Andreea Florescu
2021-02-18 17:43 ` Stefan Hajnoczi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=c26786ac-159e-149a-aa5e-dd08f418d11e@redhat.com \
--to=jsnow@redhat.com \
--cc=armbru@redhat.com \
--cc=ehabkost@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=stefanha@gmail.com \
--cc=stefanha@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).