[SPDK] Re: Preparation for SPDK 19.10 release

[Sasha Kotchubievsky <sashakot at dev.mellanox.co.il>]

[-- Attachment #1: Type: text/plain, Size: 7399 bytes --]

Hi Jim,

PSB [SK]

-----Original Message-----
From: Harris, James R <james.r.harris(a)intel.com> 
Sent: Wednesday, October 16, 2019 6:22 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: [SPDK] Re: Preparation for SPDK 19.10 release

Hi Sasha,

The CHANGELOG is built during the months leading up to the release, and Tomek typically does some cleanups just before the release.  If you'd like to help with finalizing the CHANGELOG before the release, please push some patches!  Documentation is always an area where the project could use more help.

[SK] Changelog is good. I'm talking about "Release notes"
https://github.com/spdk/spdk/releases/tag/v19.07
I think, that's first point to see that's new in the release

For the RDMA testing, I have some of the same questions as Paul.  If there are gaps in upstream per-patch or nightly tests, please help us get them closed.  Can you describe the tests you're running internally, how often they take, and how often you are running them?

[SK] Correct me if I'm wrong, but most of existing tests in SPDK use SoftRoce, or single host without actual transferring data over network.  Runtime behavior on real HW can be different. 
Our verification system today tests solutions which include SPDK as building block. We work with stable version of SPDK and don't update it frequently. It mostly focus on Mellanox's parts and less on SPDK.
From functional perspective, we test complex scenarios like OS install/boot to/from distributed storage.  We test reboots and reconnects. In addition,  we test on non-x86 platform too.
For upstream SPDK (reacting on each patch) we have limited number of sanity tests.
From my point of view, for network part the verification system should test:
- Matrix of HW, SW stacks (OS,  OFED)
- Matrix of protocols: RDMA, TCP 
- Corner cases like link failure, network congestion , SW/HW failures during real traffic
- Stress testing
- Long tests 



Thanks,

-Jim

On 10/16/19, 7:47 AM, "Luse, Paul E" <paul.e.luse(a)intel.com> wrote:

    Hi Sasha,
    
    I'm sure you'll get some other opinions as well but as the changelog now pretty much only contains interface and parameter changes sorted by module, are you just suggesting re-organizing it to make it clearer per module where interface changes have occurred vs parameter changes?
    
    Wrt testing, I love that you guys are testing this so thoroughly! I have to ask though, what additional testing are you proposing that isn't already in either the main CI or your own CI? Is there a possibility to get these tests unstreamed? We really want to continue with a model of continuous integration and test without any additional testing nearing a release.  The freeze window is to provide time to prepare the release, make sure we've got everything in it that was intended, go through some Intel security checklist stuff, etc.  Ben said it best once at a summit, our releases are really just a bookkeeping activity.
    
    You can most definitely use that freeze window to do testing on your own and submit a bug that could potentially halt the release, but I'd wouldn't support making that part of the process.
    
    Thx
    Paul
    
    On 10/16/19, 4:55 AM, "Sasha Kotchubievsky" <sashakot(a)dev.mellanox.co.il> wrote:
    
        Hi Tomek,
        
        That's great news. 
        
        I have couple questions:
        
        - Some of latest SPDK releases (18.10, 19.04, 19.07) were released with
        serious bugs in NVME-OF RDMA. Especially latest 19.07. Is it possible to
        wait with formal release till our group in Mellanox will verify release
        candidate in internal verification system? I think, couple of days will be
        enough for us provide a feedback to the community.
        -  Can we add couple special sections to SPDK release notes
        (https://github.com/spdk/spdk/releases/tag/v19.07) ? 
         	- Changes in interfaces
        	- Changes in configuration parameters (including defaults)
        	- Known bugs/issues
        For customers building solution on SPDK and considering upgrade, it's
        important to see changes in interfaces (including configuration parameters).
        
        
        Example of changes in an interface in coming 19.10 is 
        https://github.com/spdk/spdk/commit/9522ed36f8474c9314b76b2618e8d2d9de56afee
        #diff-2c205ae4170cbd2bbf219969f03c9c8d
        This commit adds a new  event to bdev interface.
        
        One of commits changed defaults was
        https://github.com/spdk/spdk/commit/b6b0a0ba59a21faf1a320dd3ea7079b282f9fc31
        It changes default IOUnit size from 4K to 8K and affected existing
        configurations.
        
        From my experience, having a table/list in release notes with such changes
        (interfaces, parameters) can help to users to understand side-effects of the
        upgrade.
        I would suggest to maintain the table in Changelog and ask for contributors
        to update it is a part of patch. At release, this table can be just copied
        to release notes.  
        
        Best regards
        Sasha
        
        -----Original Message-----
        From: Zawadzki, Tomasz <tomasz.zawadzki(a)intel.com> 
        Sent: Wednesday, October 9, 2019 5:41 PM
        To: Storage Performance Development Kit <spdk(a)lists.01.org>
        Subject: [SPDK] Preparation for SPDK 19.10 release
        
        Hello all,
        
        The merge window for SPDK 19.10 release will close by October 24th.
        Please ensure all patches you believe should be included in the release are
        merged to master branch by this date.
        You can do it by adding a hashtag '19.10' in GerritHub on those patches.
        
        The current set of patches that are tagged and need to be reviewed can be
        seen here:
        https://review.gerrithub.io/q/hashtag:%252219.10%2522+(status:open)
        
        Starting with this release, process used so far for Maintenance releases
        (ex. 19.04.1) will be applied to main releases. On October 24th new branch
        'v19.10.x' will be created, and a patch on it will be tagged as release
        candidate.
        Then, on October 31th, a formal release will take place tagging the last
        patch on the branch as SPDK 19.10.
        
        Between release candidate and formal release, only critical fixes shall be
        backported to the 'v19.10.x' branch.
        
        Development can continue without disruptions on 'master' branch.
        
        Thanks,
        Tomek
        _______________________________________________
        SPDK mailing list -- spdk(a)lists.01.org
        To unsubscribe send an email to spdk-leave(a)lists.01.org
        _______________________________________________
        SPDK mailing list -- spdk(a)lists.01.org
        To unsubscribe send an email to spdk-leave(a)lists.01.org
        
    
    _______________________________________________
    SPDK mailing list -- spdk(a)lists.01.org
    To unsubscribe send an email to spdk-leave(a)lists.01.org
    

_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org