Re: Modern HMP

["Daniel P. Berrangé" <berrange@redhat.com>]

On Thu, Jan 22, 2026 at 01:07:43PM +0100, Markus Armbruster wrote:
> Daniel P. Berrangé <berrange@redhat.com> writes:
> 
> > On Thu, Jan 22, 2026 at 10:38:28AM +0100, Markus Armbruster wrote:
> >> Let's start the discussion with your nicely written Wiki page:
> >> 
> >>     === External HMP Implementation via QMP ===
> >> 
> >>     '''Summary:''' Implement a standalone HMP-compatible monitor as an
> >>     external binary (Python or Rust) that communicates with QEMU
> >>     exclusively through QMP, enabling future decoupling of the built-in
> >>     HMP from QEMU core.
> >> 
> >>     QEMU provides two monitor interfaces:
> >>     * '''QMP''' (QEMU Machine Protocol): A JSON-based machine-readable
> >>       protocol for programmatic control
> >>     * '''HMP''' (Human Monitor Protocol): A text-based interactive
> >>       interface for human operators
> >> 
> >>     Currently, HMP is tightly integrated into QEMU, with commands
> >>     defined in `hmp-commands.hx` and `hmp-commands-info.hx`. Most HMP
> >>     commands already delegate to QMP internally (e.g., `hmp_quit()`
> >>     calls `qmp_quit()`), but HMP parsing, formatting, and command
> >>     dispatch are compiled into the QEMU binary.
> >
> > First of all, I love the idea. An external HMP impl that consumes
> > QMP from outside QEMU so a concept I've suggested many times over
> > 10+ years hoping someone would take the bait and impl it :-)
> >
> >> Also line editing and completion.
> >> 
> >> Most HMP commands cleanly wrap around QMP command handlers such as
> >> qmp_quit().  Wrapping them around QMP commands instead is a
> >> straightforward problem.  I'm more concerned about HMP stuff that uses
> >> other internal interfaces.  Replacing them may require new QMP
> >> interfaces, or maybe a careful culling of inessential HMP features.
> >> Known such stuff: completion does not wrap around QMP command handlers.
> >> It is provided by the HMP core.
> >> 
> >> Risk: this can easily become the 10% that take the other 90% of the
> >> time, or even the 5% that sink the project.
> >
> > IMHO this is essentially guaranteed.
> >
> > 4 years ago I tried to move us closer to this world by introducing
> > "HumanReadableText" and documenting that all remaining & future
> > "info xxx" commands should be backed by a QMP command that just
> > returns human formatted text. The intent was to eliminate the
> > roadblock of having to define formal QAPI types for all the
> > complex data.
> >
> > I converted a bunch of commands, but that indeed became do 90%
> > of the work, leave the other 90% of the work for later victim^H^H^H
> > contributor.
> >
> > None of this means that the GSoc project idea is invalid. We just
> > have to figure out a credible end goal is for the project, ideally
> > with staged delivery.
> 
> Makes sense.
> 
> If we see the GSoC project as a first step towards replacing the
> built-in HMP by an standalone program talking QMP ("Modern HMP"), we
> better scope it carefully, so it (1) has achievable success criteria,
> and (2) makes real progress towards the actual finish line.
> 
> >> Risk: serious code duplication until we can get rid of built-in HMP.
> >> Fine if the goal is to explore and learn by building a prototype, and we
> >> simply throw away the prototype afterwards.
> >
> > IMHO that isn't a risk, that's a guarantee. I can't imagine converting
> > all remaining HMP commands to have a QMP backing, AND doing an external
> > HMP impl all within the GSoc timeline.  That's two largely independent
> > projects, each of which are probably longer than the GSoC time wnidow.
> >
> > Again that doesn't mean the idea is invalid for GSoc, just that we must
> > be honest about likely deliverables, and how follow up work will happen
> > after GSoc to maximise benefit for QEMU.
> >
> > What I would not want to see is a bunch of work done that is then
> > abandoned because it couldn't get used as it wasn't feature complete.
> > Whatever subset is achieved ought to be intended as a stepping stone
> > we can integrate and carry on working with.
> 
> Does the GSoC project make sense without a firm commitment to followup
> work to finish the job?
> 
> Specifically, commitment by whom to do what?
> 
> >>     '''Add `CONFIG_HMP` build option''':
> >>     * Create a new Meson configuration option to disable built-in HMP
> >>     * Allow QEMU to be built without HMP
> >>     * Facilitate testing of external HMP as a replacement
> >> 
> >>     '''Create an external HMP implementation''' in Python or Rust that:
> >>     * Connects to QEMU via QMP socket
> >>     * Parses HMP command syntax and translates to QMP calls
> >>     * Formats QMP responses as human-readable HMP output
> >>     * Supports command completion and help text
> >> 
> >>     '''Use `hmp-commands.hx` for code generation''':
> >>     * Parse the existing `.hx` files to extract command definitions
> >>     * Generate boilerplate code (command tables, argument parsing, help
> >>       text)
> >>     * Produce a report of implemented vs. unimplemented commands
> >>     * Enable tracking of HMP/QMP parity
> >> 
> >> .hx is C source code with ReST snippets.  scripts/hxtool strips out the
> >> ReST.  docs/sphinx/hxtool.py ignores the C source code, and processes
> >> the ReST.  It works.  Not a fan.
> >> 
> >> If we succeed in replacing built-in HMP by an external one, and the
> >> external one isn't written in C, then having C source code in .hx no
> >> longer makes sense.  Parsing it will be wasted effort.  It may still
> >> make sense initially.
> >
> > Indeed, we should clarify language intended as it would influence
> > the approach for the project.  If it is a clean room Rust impl,
> > then it would be completely independent of existing HMP C code.
> > More work initially to ensure we retain the same data formatting
> > of each command, but likely nicer long term, and saying Rust will
> > probably attract more candidates to the idea.
> 
> In theory, we could make the same HMP code work in both contexts,
> built-in HMP and standaline HMP.  I doubt this is feasible, at least not
> at reasonable cost.  Too much disentangling.  I could be wrong.
> 
> If we use separate HMP code, i.e. accept code duplication until we
> retire built-in HMP, then picking a different language for the new copy
> won't add all that much to the bother of having to maintain two copies.
> 
> HMP is not a stable interface.  We make reasonable efforts not to change
> the output without a good reason.  I don't think identical data
> formatting is a requirement.  Just make a reasonable effort.
> 
> Marc-André proposed Python or Rust.  Anyone got a preference backed by
> reasons?

My suggestion would be Rust, as it allows the possibility to embed
that Rust impl inside the current QEMU binaries, to fully replace
the C code and retain broadly the same functionality.

We might never do it, but it feels like a good idea to keep the
door option.  Python rules that out entirely meaning we keep the
current C code forever, unless we do a full break with command
line compatibility at some point.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|