Re: WinLink E850-96: WARNING: at block/blk-settings.c:204 blk_validate_limits

Re: WinLink E850-96: WARNING: at block/blk-settings.c:204 blk_validate_limits

[Arnd Bergmann]

On Thu, Feb 29, 2024, at 15:14, Naresh Kamboju wrote:
> The arm64 WinLink E850-96 Board boot failed with 16K and 64K page size builds
> Please find the below warning log on Linux next-20240229.
> First noticed on the next-20240220 tag.
>
> This issue arises only when one of these Kconfig options is enabled.
>   CONFIG_ARM64_16K_PAGES=y
>   CONFIG_ARM64_64K_PAGES=y
>
> Reported-by: Linux Kernel Functional Testing <lkft@linaro.org>
>
> Warning log:
> -------
> [    2.231008] mmc_host mmc0: Bus speed (slot 0) = 49968750Hz (slot
> req 52000000Hz, actual 49968750HZ div = 0)
> [    2.231714] mmc_host mmc0: Bus speed (slot 0) = 399750000Hz (slot
> req 200000000Hz, actual 199875000HZ div = 1)
> [    2.241961] mmc0: new HS400 Enhanced strobe MMC card at address 0001
> [    2.249182] ------------[ cut here ]------------
> [    2.252371] WARNING: CPU: 3 PID: 90 at block/blk-settings.c:204
> blk_validate_limits (block/blk-settings.c:204 (discriminator 1))


The warning was added with commit d690cb8ae14b ("block: add
an API to atomically update queue limits")

+               if (!lim->max_segment_size)
+                       lim->max_segment_size = BLK_MAX_SEGMENT_SIZE;
+               if (WARN_ON_ONCE(lim->max_segment_size < PAGE_SIZE))
+                       return -EINVAL;

Whereas mmc_alloc_disk sets the limit as

        /*
         * Setting a virt_boundary implicity sets a max_segment_size, so try
         * to set the hardware one here.
         */
        if (host->can_dma_map_merge) {
                lim.virt_boundary_mask = dma_get_merge_boundary(mmc_dev(host));
                lim.max_segments = MMC_DMA_MAP_MERGE_SEGMENTS;
        } else {
                lim.max_segment_size =
                        round_down(host->max_seg_size, lim.logical_block_size);
                lim.max_segments = host->max_segs;
        }

and max_seg_size gets initialized to either one less
than 64k, or to exactly 4k in the dwmmc driver here:

        /* Useful defaults if platform data is unset. */
        if (host->use_dma == TRANS_MODE_IDMAC) {
                mmc->max_segs = host->ring_size;
                mmc->max_blk_size = 65535;
                mmc->max_seg_size = 0x1000;
                mmc->max_req_size = mmc->max_seg_size * host->ring_size;
                mmc->max_blk_count = mmc->max_req_size / 512;
        } else if (host->use_dma == TRANS_MODE_EDMAC) {
                mmc->max_segs = 64;
                mmc->max_blk_size = 65535;
                mmc->max_blk_count = 65535;
                mmc->max_req_size =
                                mmc->max_blk_size * mmc->max_blk_count;
                mmc->max_seg_size = mmc->max_req_size;
        } else {
                /* TRANS_MODE_PIO */
                mmc->max_segs = 64;
                mmc->max_blk_size = 65535; /* BLKSIZ is 16 bits */
                mmc->max_blk_count = 512;
                mmc->max_req_size = mmc->max_blk_size *
                                    mmc->max_blk_count;
                mmc->max_seg_size = mmc->max_req_size;
        }

Adding the mmc and dw-mmc maintainers to cc for clarification.

     Arnd

> [    2.261056] Modules linked in:
> [    2.264082] CPU: 3 PID: 90 Comm: kworker/3:1 Not tainted
> 6.8.0-rc6-next-20240229 #1
> [    2.271724] Hardware name: WinLink E850-96 board (DT)
> [    2.276756] Workqueue: events_freezable mmc_rescan
> [    2.281534] pstate: 80400005 (Nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> [    2.288479] pc : blk_validate_limits (block/blk-settings.c:204
> (discriminator 1))
> [    2.293076] lr : blk_set_default_limits (block/blk-settings.c:241)
> <trim>
> [    2.337595] PM: genpd: Disabling unused power domains
> [    2.338361] xhci-hcd xhci-hcd.0.auto: remove, state 84
> [    2.338391] usb usb1: USB disconnect, device number 1
> [    2.343322]  x1 : 0000000000001000 x0 : ffff8000a504f888
> [    2.343335] Call trace:
> [    2.343339] blk_validate_limits (block/blk-settings.c:204 (discriminator 1))
> [    2.343350] blk_alloc_queue (block/blk-core.c:421)
> [    2.343362] blk_mq_alloc_queue (block/blk-mq.c:4083 (discriminator 4))
> [    2.350571] ALSA device list:
> [    2.352700] xhci-hcd xhci-hcd.0.auto: USB bus 1 deregistered
> [    2.355651] __blk_mq_alloc_disk (block/blk-mq.c:4130)
> [    2.355664] mmc_alloc_disk (drivers/mmc/core/queue.c:379)
> [    2.355678] mmc_init_queue (drivers/mmc/core/queue.c:463)
> [    2.359577]   No soundcards found.
> [    2.362942] mmc_blk_alloc_req (drivers/mmc/core/block.c:2500
> (discriminator 1))
> [    2.362951] mmc_blk_probe (drivers/mmc/core/block.c:2603
> drivers/mmc/core/block.c:3025)
> [    2.446877] mmc_bus_probe (drivers/mmc/core/bus.c:132)
> [    2.446892] really_probe (drivers/base/dd.c:579 drivers/base/dd.c:658)
> [    2.454016] __driver_probe_device (drivers/base/dd.c:800)
> [    2.458346] driver_probe_device (drivers/base/dd.c:830)
> [    2.462513] __device_attach_driver (drivers/base/dd.c:959)
> [    2.466939] bus_for_each_drv (drivers/base/bus.c:457)
> [    2.470761] __device_attach (drivers/base/dd.c:1032)
> [    2.474579] device_initial_probe (drivers/base/dd.c:1080)
> [    2.478746] bus_probe_device (drivers/base/bus.c:532)
> [    2.482563] device_add (drivers/base/core.c:3646)
> [    2.486037] mmc_add_card (drivers/mmc/core/bus.c:371)
> [    2.489683] mmc_attach_mmc (drivers/mmc/core/mmc.c:2353)
> [    2.493500] mmc_rescan (drivers/mmc/core/core.c:2097 (discriminator
> 1) drivers/mmc/core/core.c:2259 (discriminator 1))
> [    2.496974] process_one_work (kernel/workqueue.c:3253)
> [    2.500966] worker_thread (kernel/workqueue.c:3322 (discriminator
> 2) kernel/workqueue.c:3409 (discriminator 2))
> [    2.504700] kthread (kernel/kthread.c:388)
> [    2.507914] ret_from_fork (arch/arm64/kernel/entry.S:861)
> [    2.511473] ---[ end trace 0000000000000000 ]---
> [    2.516435] mmcblk: probe of mmc0:0001 failed with error -22
> [    2.522114] Waiting for root device /dev/mmcblk0p12...
>
> Kconfig:
>  - 
> https://storage.tuxsuite.com/public/linaro/lkft/builds/2ccVUNsQjaWBcREYI0wFRjzG3oa/config
>
> Steps to reproduce:
>  - 
> https://tuxapi.tuxsuite.com/v1/groups/linaro/projects/lkft/tests/2ccVWjLtBJ3IuDSKLkyaFholQ67/reproducer
>
> Links:
>  - 
> https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20240229/testrun/22884460/suite/boot/test/gcc-13-lkftconfig-16k_page_size/log
>  - 
> https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20240229/testrun/22884435/suite/boot/test/gcc-13-lkftconfig-64k_page_size/details/
>  - 
> https://tuxapi.tuxsuite.com/v1/groups/linaro/projects/lkft/tests/2d2JxLaSCZ6cTNGMeKdCbAjk9Sh
>  - 
> https://storage.tuxsuite.com/public/linaro/lkft/builds/2d2JuyeiEHCtWGJYV80n1tF77lh/
>
>
> --
> Linaro LKFT
> https://lkft.linaro.org

Re: WinLink E850-96: WARNING: at block/blk-settings.c:204 blk_validate_limits

[Sam Protsenko]

On Thu, Feb 29, 2024 at 8:56 AM Arnd Bergmann <arnd@arndb.de> wrote:
>
> On Thu, Feb 29, 2024, at 15:14, Naresh Kamboju wrote:
> > The arm64 WinLink E850-96 Board boot failed with 16K and 64K page size builds
> > Please find the below warning log on Linux next-20240229.
> > First noticed on the next-20240220 tag.
> >
> > This issue arises only when one of these Kconfig options is enabled.
> >   CONFIG_ARM64_16K_PAGES=y
> >   CONFIG_ARM64_64K_PAGES=y
> >
> > Reported-by: Linux Kernel Functional Testing <lkft@linaro.org>
> >
> > Warning log:
> > -------
> > [    2.231008] mmc_host mmc0: Bus speed (slot 0) = 49968750Hz (slot
> > req 52000000Hz, actual 49968750HZ div = 0)
> > [    2.231714] mmc_host mmc0: Bus speed (slot 0) = 399750000Hz (slot
> > req 200000000Hz, actual 199875000HZ div = 1)
> > [    2.241961] mmc0: new HS400 Enhanced strobe MMC card at address 0001
> > [    2.249182] ------------[ cut here ]------------
> > [    2.252371] WARNING: CPU: 3 PID: 90 at block/blk-settings.c:204
> > blk_validate_limits (block/blk-settings.c:204 (discriminator 1))
>
>
> The warning was added with commit d690cb8ae14b ("block: add
> an API to atomically update queue limits")
>
> +               if (!lim->max_segment_size)
> +                       lim->max_segment_size = BLK_MAX_SEGMENT_SIZE;
> +               if (WARN_ON_ONCE(lim->max_segment_size < PAGE_SIZE))
> +                       return -EINVAL;
>
> Whereas mmc_alloc_disk sets the limit as
>
>         /*
>          * Setting a virt_boundary implicity sets a max_segment_size, so try
>          * to set the hardware one here.
>          */
>         if (host->can_dma_map_merge) {
>                 lim.virt_boundary_mask = dma_get_merge_boundary(mmc_dev(host));
>                 lim.max_segments = MMC_DMA_MAP_MERGE_SEGMENTS;
>         } else {
>                 lim.max_segment_size =
>                         round_down(host->max_seg_size, lim.logical_block_size);
>                 lim.max_segments = host->max_segs;
>         }
>
> and max_seg_size gets initialized to either one less
> than 64k, or to exactly 4k in the dwmmc driver here:
>
>         /* Useful defaults if platform data is unset. */
>         if (host->use_dma == TRANS_MODE_IDMAC) {
>                 mmc->max_segs = host->ring_size;
>                 mmc->max_blk_size = 65535;

Changing this value here to PAGE_SIZE (instead of 65535) makes E850-96
functional again (tested with CONFIG_ARM64_16K_PAGES=y). I'm sure this
is just a quick hack, but maybe it can be helpful in further analysis.
If anybody wants me to run some tests on E850-96 -- please let me
know.

[snip]

Re: WinLink E850-96: WARNING: at block/blk-settings.c:204 blk_validate_limits

[Sam Protsenko]

On Fri, Mar 1, 2024 at 2:51 PM Sam Protsenko <semen.protsenko@linaro.org> wrote:
>
> On Thu, Feb 29, 2024 at 8:56 AM Arnd Bergmann <arnd@arndb.de> wrote:
> >
> > On Thu, Feb 29, 2024, at 15:14, Naresh Kamboju wrote:
> > > The arm64 WinLink E850-96 Board boot failed with 16K and 64K page size builds
> > > Please find the below warning log on Linux next-20240229.
> > > First noticed on the next-20240220 tag.
> > >
> > > This issue arises only when one of these Kconfig options is enabled.
> > >   CONFIG_ARM64_16K_PAGES=y
> > >   CONFIG_ARM64_64K_PAGES=y
> > >
> > > Reported-by: Linux Kernel Functional Testing <lkft@linaro.org>
> > >
> > > Warning log:
> > > -------
> > > [    2.231008] mmc_host mmc0: Bus speed (slot 0) = 49968750Hz (slot
> > > req 52000000Hz, actual 49968750HZ div = 0)
> > > [    2.231714] mmc_host mmc0: Bus speed (slot 0) = 399750000Hz (slot
> > > req 200000000Hz, actual 199875000HZ div = 1)
> > > [    2.241961] mmc0: new HS400 Enhanced strobe MMC card at address 0001
> > > [    2.249182] ------------[ cut here ]------------
> > > [    2.252371] WARNING: CPU: 3 PID: 90 at block/blk-settings.c:204
> > > blk_validate_limits (block/blk-settings.c:204 (discriminator 1))
> >
> >
> > The warning was added with commit d690cb8ae14b ("block: add
> > an API to atomically update queue limits")
> >
> > +               if (!lim->max_segment_size)
> > +                       lim->max_segment_size = BLK_MAX_SEGMENT_SIZE;
> > +               if (WARN_ON_ONCE(lim->max_segment_size < PAGE_SIZE))
> > +                       return -EINVAL;
> >
> > Whereas mmc_alloc_disk sets the limit as
> >
> >         /*
> >          * Setting a virt_boundary implicity sets a max_segment_size, so try
> >          * to set the hardware one here.
> >          */
> >         if (host->can_dma_map_merge) {
> >                 lim.virt_boundary_mask = dma_get_merge_boundary(mmc_dev(host));
> >                 lim.max_segments = MMC_DMA_MAP_MERGE_SEGMENTS;
> >         } else {
> >                 lim.max_segment_size =
> >                         round_down(host->max_seg_size, lim.logical_block_size);
> >                 lim.max_segments = host->max_segs;
> >         }
> >
> > and max_seg_size gets initialized to either one less
> > than 64k, or to exactly 4k in the dwmmc driver here:
> >
> >         /* Useful defaults if platform data is unset. */
> >         if (host->use_dma == TRANS_MODE_IDMAC) {
> >                 mmc->max_segs = host->ring_size;
> >                 mmc->max_blk_size = 65535;
>
> Changing this value here to PAGE_SIZE (instead of 65535) makes E850-96
> functional again (tested with CONFIG_ARM64_16K_PAGES=y). I'm sure this
> is just a quick hack, but maybe it can be helpful in further analysis.
> If anybody wants me to run some tests on E850-96 -- please let me
> know.
>

Sorry, just noticed I commented on the wrong line. Here is the change I made:

-               mmc->max_seg_size = 0x1000;
+               mmc->max_seg_size = PAGE_SIZE;

for (host->use_dma == TRANS_MODE_IDMAC) case.

> [snip]

Re: WinLink E850-96: WARNING: at block/blk-settings.c:204 blk_validate_limits

[Sam Protsenko]

On Fri, Mar 1, 2024 at 3:18 PM Sam Protsenko <semen.protsenko@linaro.org> wrote:
>

[snip]

>
> Sorry, just noticed I commented on the wrong line. Here is the change I made:
>
> -               mmc->max_seg_size = 0x1000;
> +               mmc->max_seg_size = PAGE_SIZE;
>
> for (host->use_dma == TRANS_MODE_IDMAC) case.
>

Just submitted the fix [1]. Please review.

[1] https://lore.kernel.org/all/20240306232052.21317-1-semen.protsenko@linaro.org/T/#u

> > [snip]

Re: WinLink E850-96: WARNING: at block/blk-settings.c:204 blk_validate_limits

[Arnd Bergmann]

On Fri, Mar 1, 2024, at 22:18, Sam Protsenko wrote:
> On Fri, Mar 1, 2024 at 2:51 PM Sam Protsenko <semen.protsenko@linaro.org> wrote:
>> On Thu, Feb 29, 2024 at 8:56 AM Arnd Bergmann <arnd@arndb.de> wrote:

>
> Sorry, just noticed I commented on the wrong line. Here is the change I made:
>
> -               mmc->max_seg_size = 0x1000;
> +               mmc->max_seg_size = PAGE_SIZE;

I went over all MMC drivers to see what else sets a max_seg_size
smaller than a page and found these:

drivers/mmc/host/alcor.c:       mmc->max_seg_size = AU6601_MAX_DMA_BLOCK_SIZE; // 0x1000
drivers/mmc/host/au1xmmc.c:             mmc->max_seg_size = AU1100_MMC_DESCRIPTOR_SIZE; // 64K-1
drivers/mmc/host/dw_mmc.c:              mmc->max_seg_size = 0x1000;
drivers/mmc/host/meson-gx-mmc.c:        mmc->max_seg_size = mmc->max_req_size; //  1536 bytes
drivers/mmc/host/mmci_stm32_sdmmc.c:            host->mmc->max_seg_size = host->variant->stm32_idmabsize_mask; //  GENMASK(12, 5),
drivers/mmc/host/sunxi-mmc.c:   mmc->max_seg_size       = (1 << host->cfg->idma_des_size_bits); // 1 << 13, only on arm32
drivers/mmc/host/wmt-sdmmc.c:   .max_seg_size = 65024, 

I've tried to add the maintainers to Cc here, these likely all
need attention to work with large page sizes, in case of
meson-gx-mmc it even seems like the limit is less than a 4KB
page, so it will stop working entirely.

There are also a couple of drivers that look like they have
an off-by-one error and pass a segment size of one less than
a power-off-two number, e.g.:

drivers/mmc/host/davinci_mmc.c: mmc->max_seg_size       = MAX_CCNT * rw_threshold;  // (64k-1) * 32
drivers/mmc/host/atmel-mci.c:           mmc->max_seg_size = mmc->max_blk_size * mmc->max_segs; // 4095*256

I think some of these are intentional, while others are
probably bugs.

     Arnd