[SPDK] Re: A few thoughts about spdk_top

[SPDK] Re: A few thoughts about spdk_top

[Stojaczyk, Dariusz]

[-- Attachment #1: Type: text/plain, Size: 4100 bytes --]



> -----Original Message-----
> From: Harris, James R <james.r.harris(a)intel.com>
> Sent: Wednesday, April 8, 2020 12:14 AM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>; Szwed, Maciej
> <maciej.szwed(a)intel.com>
> Subject: [SPDK] Re: A few thoughts about spdk_top
> 
> Hi Darek,
> 
> On 4/7/20, 3:30 AM, "Stojaczyk, Dariusz" <dariusz.stojaczyk(a)intel.com> wrote:
> 
>     I like the idea of spdk_top application, although I'm slightly concerned about
> our implementation. Is it a right choice to develop it in C? It's already a huge
> amount of code and it already has a huge amount of limitations, not even
> mentioning possible bugs. I actually spent a lot of time developing in curses,
> GTK, GTKmm and even WinAPI and I don't do it anymore because it's difficult
> and not worth it. I switched to javascript as well as server-side javascript for my
> other projects and I find it incredibly easier to write user apps there. I'm not
> trying to push on javascript specifically, but could we consider writing spdk_top
> in a higher level language, where we don't care about memory allocation, json
> parsing, and printing data to the screen?
> 
> [Jim] I think there's room for multiple user interfaces. Personally, I like a
> terminal-based application like this - I typically run tmux with 4 panes, and
> having something like this in one pane while I'm running a target in the
> foreground in another works well for me.  Maybe something in Python could
> end up as a bit less code?  It's possible, but I look at something like spdkcli and
> it has quite a bit of code too.

Agreed. I prefer CLI apps where possible, although what's the good way to develop them?
I know spdkcli needs to fit in the targetcli-like framework in Python, so there's a fair amount of framework specific code. Maybe for spdk_top we don't necessarily need a framework, just a library? This could use a research to see what's available.

I heard DPDK develops a similar application in Go, although I can't find the code anywhere. There's a presentation on the internet [1] with some screenshots and I have to say it does look impressive. They even draw graphs in CLI, however I slightly question usability of that in particular.

I just don't see our C implementation ever getting as functional as the DPDK one.

> 
> [Jim] Regarding json parsing, I think it is nice to have another use case for our
> client-side JSON RPC APIs.  But you're right, there's about 300 lines of code in
> there specific to issuing RPCs to the app.  Potentially some of that long-term
> could be moved into a common library for other applications that have a need
> for issuing RPCs from a C application.

That's a good point.

> 
>     Currently spdk_top seems to require the same, detailed review as any spdk
> patch and honestly I'm a bit reluctant to review it. Moreover, I find one major
> feature missing there - a view with all pollers on a specific, single thread and
> busy time % for each poller, so that I can see what my thread is most busy with.
> I think it would be the first view I check when doing any spdk optimization. Yet I
> see it's quite a bit of effort to add it to the current spdk_top code, which brings
> me back to my first question.
> 
> [Jim] I think now's the time to provide that feedback on what might be missing.
> But the basic infrastructure is all in place, and knowing Maciek I'm sure he's
> open to suggestions!  I'd encourage everyone to pull down the code and kick
> the proverbial tires.  The end of the series can be found at
> https://review.spdk.io/gerrit/c/spdk/spdk/+/1717/4.  I've also started reviewing
> individual patches and am suggesting areas where the code could be simplified
> a bit.
> 
> 
> Regards,
> 
> -Jim
> 
> 
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org


[1] https://www.dpdk.org/wp-content/uploads/sites/35/2019/10/MeasuringSoftwarePerformance.pdf (slide 19+)
D.

[SPDK] Re: A few thoughts about spdk_top

[Harris, James R]

[-- Attachment #1: Type: text/plain, Size: 2663 bytes --]

Hi Darek,

On 4/7/20, 3:30 AM, "Stojaczyk, Dariusz" <dariusz.stojaczyk(a)intel.com> wrote:

    I like the idea of spdk_top application, although I'm slightly concerned about our implementation. Is it a right choice to develop it in C? It's already a huge amount of code and it already has a huge amount of limitations, not even mentioning possible bugs. I actually spent a lot of time developing in curses, GTK, GTKmm and even WinAPI and I don't do it anymore because it's difficult and not worth it. I switched to javascript as well as server-side javascript for my other projects and I find it incredibly easier to write user apps there. I'm not trying to push on javascript specifically, but could we consider writing spdk_top in a higher level language, where we don't care about memory allocation, json parsing, and printing data to the screen?

[Jim] I think there's room for multiple user interfaces. Personally, I like a terminal-based application like this - I typically run tmux with 4 panes, and having something like this in one pane while I'm running a target in the foreground in another works well for me.  Maybe something in Python could end up as a bit less code?  It's possible, but I look at something like spdkcli and it has quite a bit of code too.

[Jim] Regarding json parsing, I think it is nice to have another use case for our client-side JSON RPC APIs.  But you're right, there's about 300 lines of code in there specific to issuing RPCs to the app.  Potentially some of that long-term could be moved into a common library for other applications that have a need for issuing RPCs from a C application.
    
    Currently spdk_top seems to require the same, detailed review as any spdk patch and honestly I'm a bit reluctant to review it. Moreover, I find one major feature missing there - a view with all pollers on a specific, single thread and busy time % for each poller, so that I can see what my thread is most busy with. I think it would be the first view I check when doing any spdk optimization. Yet I see it's quite a bit of effort to add it to the current spdk_top code, which brings me back to my first question.

[Jim] I think now's the time to provide that feedback on what might be missing. But the basic infrastructure is all in place, and knowing Maciek I'm sure he's open to suggestions!  I'd encourage everyone to pull down the code and kick the proverbial tires.  The end of the series can be found at https://review.spdk.io/gerrit/c/spdk/spdk/+/1717/4.  I've also started reviewing individual patches and am suggesting areas where the code could be simplified a bit.  


Regards,

-Jim