linux-arm-kernel.lists.infradead.org archive mirror
 help / color / mirror / Atom feed
* All OMAP platforms: MMC is broken
@ 2015-09-24  9:00 Russell King - ARM Linux
  2015-09-24 22:51 ` Grygorii Strashko
  2015-09-24 23:37 ` Tony Lindgren
  0 siblings, 2 replies; 35+ messages in thread
From: Russell King - ARM Linux @ 2015-09-24  9:00 UTC (permalink / raw)
  To: linux-arm-kernel

Nightly testing has revealed that both the OMAP3430 LDP and the OMAP4430
SDP fail to boot due to lack of working MMC.  Both platforms fail to
find their rootfs, which is on a SD card.

The breakage occurred somewhere between trees of September 9th (commit
4e4adb2f4628) and September 12th (commit b0a1ea51bda4), so during the
merge window.

Diffing those two trees show no changes to OMAP platform code, but
substantial changes in the MMC code:

 drivers/mmc/card/block.c                           |    17 +
 drivers/mmc/core/core.c                            |    46 +-
 drivers/mmc/core/host.c                            |    42 +-
 drivers/mmc/host/Kconfig                           |     8 +
 drivers/mmc/host/Makefile                          |     1 +
 drivers/mmc/host/android-goldfish.c                |     2 +-
 drivers/mmc/host/atmel-mci.c                       |     1 -
 drivers/mmc/host/dw_mmc-rockchip.c                 |     3 +
 drivers/mmc/host/dw_mmc.c                          |   284 +-
 drivers/mmc/host/omap.c                            |     9 +-
 drivers/mmc/host/omap_hsmmc.c                      |   347 +-
 drivers/mmc/host/pxamci.c                          |   200 +-
 drivers/mmc/host/sdhci-esdhc-imx.c                 |   114 +-
 drivers/mmc/host/sdhci-esdhc.h                     |     3 +-
 drivers/mmc/host/sdhci-msm.c                       |     5 +
 drivers/mmc/host/sdhci-of-arasan.c                 |     4 +
 drivers/mmc/host/sdhci-of-at91.c                   |   191 +
 drivers/mmc/host/sdhci-of-esdhc.c                  |     6 +
 drivers/mmc/host/sdhci-pci.c                       |     1 +
 drivers/mmc/host/sdhci-sirf.c                      |     4 +-
 drivers/mmc/host/sdhci.c                           |   123 +-
 drivers/mmc/host/sdhci.h                           |    10 +-
 drivers/mmc/host/sh_mmcif.c                        |     2 +
 drivers/mmc/host/sunxi-mmc.c                       |     2 +-
 drivers/mmc/host/tmio_mmc_pio.c                    |     6 +-
 drivers/mmc/host/usdhi6rol0.c                      |    15 +-

So this probably results from the MMC update during the merge window:

85579ad7f1df Merge tag 'mmc-v4.3' of git://git.linaro.org/people/ulf.hansson/mmc

The OMAP4430 reports this during it's boot log:

mmc0: new high speed MMC card at address 0001
mmcblk0: mmc0:0001 SEM08G 7.39 GiB
mmcblk0boot0: mmc0:0001 SEM08G partition 1 1.00 MiB
mmcblk0boot1: mmc0:0001 SEM08G partition 2 1.00 MiB
...
VFS: Cannot open root device "mmcblk0p2" or unknown-block(179,2): error -6
b300         7757824 mmcblk0  driver: mmcblk
b310            1024 mmcblk0boot1  (driver?)
b308            1024 mmcblk0boot0  (driver?)
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(179,2)

whereas previous kernels reported:

mmc0: host does not support reading read-only switch, assuming write-enable
mmc0: new high speed SD card at address 0002
mmcblk0: mmc0:0002 00000 971 MiB
 mmcblk0: p1 p2
Waiting 2 sec before mounting root device...
mmc1: new high speed MMC card at address 0001
mmcblk1: mmc1:0001 SEM08G 7.39 GiB
mmcblk1boot0: mmc1:0001 SEM08G partition 1 1.00 MiB
mmcblk1boot1: mmc1:0001 SEM08G partition 2 1.00 MiB
...
VFS: Mounted root (ext3 filesystem) on device 179:2.

OMAP3430 only reports:

VFS: Cannot open root device "mmcblk0p2" or unknown-block(0,0): error -6

where previous kernels reported:

mmc0: host does not support reading read-only switch, assuming write-enable
mmc0: new high speed SD card at address 0002
mmcblk0: mmc0:0002 00000 971 MiB
 mmcblk0: p1 p2
...
VFS: Mounted root (ext2 filesystem) on device 179:2.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-09-24  9:00 All OMAP platforms: MMC is broken Russell King - ARM Linux
@ 2015-09-24 22:51 ` Grygorii Strashko
  2015-09-24 22:53   ` Russell King - ARM Linux
  2015-09-24 23:37 ` Tony Lindgren
  1 sibling, 1 reply; 35+ messages in thread
From: Grygorii Strashko @ 2015-09-24 22:51 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Russell,

+Cc: Lokesh

On 09/24/2015 02:00 AM, Russell King - ARM Linux wrote:
> Nightly testing has revealed that both the OMAP3430 LDP and the OMAP4430
> SDP fail to boot due to lack of working MMC.  Both platforms fail to
> find their rootfs, which is on a SD card.
> 
> The breakage occurred somewhere between trees of September 9th (commit
> 4e4adb2f4628) and September 12th (commit b0a1ea51bda4), so during the
> merge window.
> 
> Diffing those two trees show no changes to OMAP platform code, but
> substantial changes in the MMC code:
> 

[...]

> 
> So this probably results from the MMC update during the merge window:
> 
> 85579ad7f1df Merge tag 'mmc-v4.3' of git://git.linaro.org/people/ulf.hansson/mmc
> 
> The OMAP4430 reports this during it's boot log:
> 
> mmc0: new high speed MMC card at address 0001
> mmcblk0: mmc0:0001 SEM08G 7.39 GiB
> mmcblk0boot0: mmc0:0001 SEM08G partition 1 1.00 MiB
> mmcblk0boot1: mmc0:0001 SEM08G partition 2 1.00 MiB
> ...
> VFS: Cannot open root device "mmcblk0p2" or unknown-block(179,2): error -6
> b300         7757824 mmcblk0  driver: mmcblk
> b310            1024 mmcblk0boot1  (driver?)
> b308            1024 mmcblk0boot0  (driver?)
> Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(179,2)
> 
> whereas previous kernels reported:
> 
> mmc0: host does not support reading read-only switch, assuming write-enable
> mmc0: new high speed SD card at address 0002
> mmcblk0: mmc0:0002 00000 971 MiB
>   mmcblk0: p1 p2
> Waiting 2 sec before mounting root device...
> mmc1: new high speed MMC card at address 0001
> mmcblk1: mmc1:0001 SEM08G 7.39 GiB
> mmcblk1boot0: mmc1:0001 SEM08G partition 1 1.00 MiB
> mmcblk1boot1: mmc1:0001 SEM08G partition 2 1.00 MiB
> ...
> VFS: Mounted root (ext3 filesystem) on device 179:2.
> 
> OMAP3430 only reports:
> 
> VFS: Cannot open root device "mmcblk0p2" or unknown-block(0,0): error -6
> 
> where previous kernels reported:
> 
> mmc0: host does not support reading read-only switch, assuming write-enable
> mmc0: new high speed SD card at address 0002
> mmcblk0: mmc0:0002 00000 971 MiB
>   mmcblk0: p1 p2
> ...
> VFS: Mounted root (ext2 filesystem) on device 179:2.
> 

It looks like mmcX enumeration problem, which is not new, I think :(
Most probably with the last DT+code changes probing of MMC which handles SD
is deferred and as result it will get different ID.

I think Lokesh can comment more on this as he fixed it by switching uboot to 
UUIDs instead of partition names.

http://git.ti.com/ti-u-boot/ti-u-boot/commit/5e31eb59f03a424371c605448b6bbc2f93da787b   

-- 
regards,
-grygorii

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-09-24 22:51 ` Grygorii Strashko
@ 2015-09-24 22:53   ` Russell King - ARM Linux
  0 siblings, 0 replies; 35+ messages in thread
From: Russell King - ARM Linux @ 2015-09-24 22:53 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Sep 24, 2015 at 03:51:35PM -0700, Grygorii Strashko wrote:
> Hi Russell,
> 
> +Cc: Lokesh
> 
> On 09/24/2015 02:00 AM, Russell King - ARM Linux wrote:
> > Nightly testing has revealed that both the OMAP3430 LDP and the OMAP4430
> > SDP fail to boot due to lack of working MMC.  Both platforms fail to
> > find their rootfs, which is on a SD card.
> > 
> > The breakage occurred somewhere between trees of September 9th (commit
> > 4e4adb2f4628) and September 12th (commit b0a1ea51bda4), so during the
> > merge window.
> > 
> > Diffing those two trees show no changes to OMAP platform code, but
> > substantial changes in the MMC code:
> > 
> 
> [...]
> 
> > 
> > So this probably results from the MMC update during the merge window:
> > 
> > 85579ad7f1df Merge tag 'mmc-v4.3' of git://git.linaro.org/people/ulf.hansson/mmc
> > 
> > The OMAP4430 reports this during it's boot log:
> > 
> > mmc0: new high speed MMC card at address 0001
> > mmcblk0: mmc0:0001 SEM08G 7.39 GiB
> > mmcblk0boot0: mmc0:0001 SEM08G partition 1 1.00 MiB
> > mmcblk0boot1: mmc0:0001 SEM08G partition 2 1.00 MiB
> > ...
> > VFS: Cannot open root device "mmcblk0p2" or unknown-block(179,2): error -6
> > b300         7757824 mmcblk0  driver: mmcblk
> > b310            1024 mmcblk0boot1  (driver?)
> > b308            1024 mmcblk0boot0  (driver?)
> > Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(179,2)
> > 
> > whereas previous kernels reported:
> > 
> > mmc0: host does not support reading read-only switch, assuming write-enable
> > mmc0: new high speed SD card at address 0002
> > mmcblk0: mmc0:0002 00000 971 MiB
> >   mmcblk0: p1 p2
> > Waiting 2 sec before mounting root device...
> > mmc1: new high speed MMC card at address 0001
> > mmcblk1: mmc1:0001 SEM08G 7.39 GiB
> > mmcblk1boot0: mmc1:0001 SEM08G partition 1 1.00 MiB
> > mmcblk1boot1: mmc1:0001 SEM08G partition 2 1.00 MiB
> > ...
> > VFS: Mounted root (ext3 filesystem) on device 179:2.
> > 
> > OMAP3430 only reports:
> > 
> > VFS: Cannot open root device "mmcblk0p2" or unknown-block(0,0): error -6
> > 
> > where previous kernels reported:
> > 
> > mmc0: host does not support reading read-only switch, assuming write-enable
> > mmc0: new high speed SD card at address 0002
> > mmcblk0: mmc0:0002 00000 971 MiB
> >   mmcblk0: p1 p2
> > ...
> > VFS: Mounted root (ext2 filesystem) on device 179:2.
> > 
> 
> It looks like mmcX enumeration problem, which is not new, I think :(
> Most probably with the last DT+code changes probing of MMC which handles SD
> is deferred and as result it will get different ID.
> 
> I think Lokesh can comment more on this as he fixed it by switching uboot to 
> UUIDs instead of partition names.

No, that's not the problem.  The SD card is simply _not_ detected in
both cases.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-09-24  9:00 All OMAP platforms: MMC is broken Russell King - ARM Linux
  2015-09-24 22:51 ` Grygorii Strashko
@ 2015-09-24 23:37 ` Tony Lindgren
  2015-09-25  1:13   ` Tony Lindgren
  2015-10-01  9:33   ` Russell King - ARM Linux
  1 sibling, 2 replies; 35+ messages in thread
From: Tony Lindgren @ 2015-09-24 23:37 UTC (permalink / raw)
  To: linux-arm-kernel

* Russell King - ARM Linux <linux@arm.linux.org.uk> [150924 02:04]:
> Nightly testing has revealed that both the OMAP3430 LDP and the OMAP4430
> SDP fail to boot due to lack of working MMC.  Both platforms fail to
> find their rootfs, which is on a SD card.
> 
> The breakage occurred somewhere between trees of September 9th (commit
> 4e4adb2f4628) and September 12th (commit b0a1ea51bda4), so during the
> merge window.

Yes sorry things got messed up in multiple ways :( I've summarized
the mess here earlier:

http://article.gmane.org/gmane.linux.kernel.mmc/33911

And today commit b9c93646fd5c ("regulator: pbias: program pbias register
offset in pbias driver") hit mainline so I'll send a pull request for
the related dts change.

Regards,

Tony

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-09-24 23:37 ` Tony Lindgren
@ 2015-09-25  1:13   ` Tony Lindgren
  2015-09-26  2:54     ` Nishanth Menon
  2015-10-01  9:33   ` Russell King - ARM Linux
  1 sibling, 1 reply; 35+ messages in thread
From: Tony Lindgren @ 2015-09-25  1:13 UTC (permalink / raw)
  To: linux-arm-kernel

* Tony Lindgren <tony@atomide.com> [150924 16:42]:
> * Russell King - ARM Linux <linux@arm.linux.org.uk> [150924 02:04]:
> > Nightly testing has revealed that both the OMAP3430 LDP and the OMAP4430
> > SDP fail to boot due to lack of working MMC.  Both platforms fail to
> > find their rootfs, which is on a SD card.
> > 
> > The breakage occurred somewhere between trees of September 9th (commit
> > 4e4adb2f4628) and September 12th (commit b0a1ea51bda4), so during the
> > merge window.
> 
> Yes sorry things got messed up in multiple ways :( I've summarized
> the mess here earlier:
> 
> http://article.gmane.org/gmane.linux.kernel.mmc/33911
> 
> And today commit b9c93646fd5c ("regulator: pbias: program pbias register
> offset in pbias driver") hit mainline so I'll send a pull request for
> the related dts change.

Kishon, while testing I noticed there are still at least two
issues remaining that seem to need fixing in the MMC driver:

1. The legacy booting without device tree does not probe MMC at
   all on omap3

2. I'm getting the regulator warning on omap4 duovero

Anyways, I'm sending a pull request for the fixes so far as that
gets MMC working again for most platforms.

Regards,

Tony

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-09-25  1:13   ` Tony Lindgren
@ 2015-09-26  2:54     ` Nishanth Menon
  0 siblings, 0 replies; 35+ messages in thread
From: Nishanth Menon @ 2015-09-26  2:54 UTC (permalink / raw)
  To: linux-arm-kernel

On 09/24/2015 08:13 PM, Tony Lindgren wrote:
> * Tony Lindgren <tony@atomide.com> [150924 16:42]:
>> * Russell King - ARM Linux <linux@arm.linux.org.uk> [150924 02:04]:
>>> Nightly testing has revealed that both the OMAP3430 LDP and the OMAP4430
>>> SDP fail to boot due to lack of working MMC.  Both platforms fail to
>>> find their rootfs, which is on a SD card.
>>>
>>> The breakage occurred somewhere between trees of September 9th (commit
>>> 4e4adb2f4628) and September 12th (commit b0a1ea51bda4), so during the
>>> merge window.
>>
>> Yes sorry things got messed up in multiple ways :( I've summarized
>> the mess here earlier:
>>
>> http://article.gmane.org/gmane.linux.kernel.mmc/33911
>>
>> And today commit b9c93646fd5c ("regulator: pbias: program pbias register
>> offset in pbias driver") hit mainline so I'll send a pull request for
>> the related dts change.
> 
> Kishon, while testing I noticed there are still at least two
> issues remaining that seem to need fixing in the MMC driver:
> 
> 1. The legacy booting without device tree does not probe MMC at
>    all on omap3
> 
> 2. I'm getting the regulator warning on omap4 duovero
> 
> Anyways, I'm sending a pull request for the fixes so far as that
> gets MMC working again for most platforms.
> 
> Regards,
> 
With 0925, i think the patches got merged. here is my testing status on
the boards I have access to:

https://github.com/nmenon/kernel-test-logs/tree/next-20150925

Looks like with omap2plus_defconfig, at least
https://github.com/nmenon/kernel-test-logs/blob/next-20150925/omap2plus_defconfig/am335x-evm.txt
somethign is broken there - dunno what yet.

There are a few more farm related failures in various locations, but not
too interesting.


-- 
Regards,
Nishanth Menon

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-09-24 23:37 ` Tony Lindgren
  2015-09-25  1:13   ` Tony Lindgren
@ 2015-10-01  9:33   ` Russell King - ARM Linux
  2015-10-01  9:50     ` Ulf Hansson
  1 sibling, 1 reply; 35+ messages in thread
From: Russell King - ARM Linux @ 2015-10-01  9:33 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Sep 24, 2015 at 04:37:56PM -0700, Tony Lindgren wrote:
> * Russell King - ARM Linux <linux@arm.linux.org.uk> [150924 02:04]:
> > Nightly testing has revealed that both the OMAP3430 LDP and the OMAP4430
> > SDP fail to boot due to lack of working MMC.  Both platforms fail to
> > find their rootfs, which is on a SD card.
> > 
> > The breakage occurred somewhere between trees of September 9th (commit
> > 4e4adb2f4628) and September 12th (commit b0a1ea51bda4), so during the
> > merge window.
> 
> Yes sorry things got messed up in multiple ways :( I've summarized
> the mess here earlier:
> 
> http://article.gmane.org/gmane.linux.kernel.mmc/33911
> 
> And today commit b9c93646fd5c ("regulator: pbias: program pbias register
> offset in pbias driver") hit mainline so I'll send a pull request for
> the related dts change.

It's still broken and untestable.  We're 8 days off it being a full
month worth of failed testing for OMAP3 and OMAP4 platforms.

I think OMAP and MMC people need to do a post-mortem and work out why
this happened, and how to stop it happening again in the future.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-01  9:33   ` Russell King - ARM Linux
@ 2015-10-01  9:50     ` Ulf Hansson
  2015-10-01 10:03       ` Russell King - ARM Linux
  0 siblings, 1 reply; 35+ messages in thread
From: Ulf Hansson @ 2015-10-01  9:50 UTC (permalink / raw)
  To: linux-arm-kernel

On 1 October 2015 at 11:33, Russell King - ARM Linux
<linux@arm.linux.org.uk> wrote:
> On Thu, Sep 24, 2015 at 04:37:56PM -0700, Tony Lindgren wrote:
>> * Russell King - ARM Linux <linux@arm.linux.org.uk> [150924 02:04]:
>> > Nightly testing has revealed that both the OMAP3430 LDP and the OMAP4430
>> > SDP fail to boot due to lack of working MMC.  Both platforms fail to
>> > find their rootfs, which is on a SD card.
>> >
>> > The breakage occurred somewhere between trees of September 9th (commit
>> > 4e4adb2f4628) and September 12th (commit b0a1ea51bda4), so during the
>> > merge window.
>>
>> Yes sorry things got messed up in multiple ways :( I've summarized
>> the mess here earlier:
>>
>> http://article.gmane.org/gmane.linux.kernel.mmc/33911
>>
>> And today commit b9c93646fd5c ("regulator: pbias: program pbias register
>> offset in pbias driver") hit mainline so I'll send a pull request for
>> the related dts change.
>
> It's still broken and untestable.  We're 8 days off it being a full
> month worth of failed testing for OMAP3 and OMAP4 platforms.
>
> I think OMAP and MMC people need to do a post-mortem and work out why
> this happened, and how to stop it happening again in the future.

You are probably right!

I think I should have been more persistent when asking on how to deal
with these patches. Typically they should all have gone together via
one tree, or we needed a slower way forward keeping changes more
standalone.

Anyway, according to kernelci.org, which builds and boot my next
branch omap3/4 seems to boot now...
http://kernelci.org/boot/all/job/ulfh/kernel/v4.3-rc3-27-gf7734d097520/

Kind regards
Uffe

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-01  9:50     ` Ulf Hansson
@ 2015-10-01 10:03       ` Russell King - ARM Linux
  2015-10-05 11:23         ` Tony Lindgren
  0 siblings, 1 reply; 35+ messages in thread
From: Russell King - ARM Linux @ 2015-10-01 10:03 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Oct 01, 2015 at 11:50:00AM +0200, Ulf Hansson wrote:
> On 1 October 2015 at 11:33, Russell King - ARM Linux
> <linux@arm.linux.org.uk> wrote:
> > On Thu, Sep 24, 2015 at 04:37:56PM -0700, Tony Lindgren wrote:
> >> * Russell King - ARM Linux <linux@arm.linux.org.uk> [150924 02:04]:
> >> > Nightly testing has revealed that both the OMAP3430 LDP and the OMAP4430
> >> > SDP fail to boot due to lack of working MMC.  Both platforms fail to
> >> > find their rootfs, which is on a SD card.
> >> >
> >> > The breakage occurred somewhere between trees of September 9th (commit
> >> > 4e4adb2f4628) and September 12th (commit b0a1ea51bda4), so during the
> >> > merge window.
> >>
> >> Yes sorry things got messed up in multiple ways :( I've summarized
> >> the mess here earlier:
> >>
> >> http://article.gmane.org/gmane.linux.kernel.mmc/33911
> >>
> >> And today commit b9c93646fd5c ("regulator: pbias: program pbias register
> >> offset in pbias driver") hit mainline so I'll send a pull request for
> >> the related dts change.
> >
> > It's still broken and untestable.  We're 8 days off it being a full
> > month worth of failed testing for OMAP3 and OMAP4 platforms.
> >
> > I think OMAP and MMC people need to do a post-mortem and work out why
> > this happened, and how to stop it happening again in the future.
> 
> You are probably right!
> 
> I think I should have been more persistent when asking on how to deal
> with these patches. Typically they should all have gone together via
> one tree, or we needed a slower way forward keeping changes more
> standalone.
> 
> Anyway, according to kernelci.org, which builds and boot my next
> branch omap3/4 seems to boot now...
> http://kernelci.org/boot/all/job/ulfh/kernel/v4.3-rc3-27-gf7734d097520/

It continues to fail here.

Okay, sod it, I'm at the point of just not giving a damn about whether
OMAP3 and OMAP4 boot anymore.

I'm taking all OMAP platforms out of my boot farm.  I'm totally fed up
with this kind of regression happening.  It's probably some config option
that someone's recently introduced which defaults to being off, thereby
breaking the ability for people to move forward without constantly having
to re-configure.  This is not acceptable.

STOP BREAKING THE KERNEL.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-01 10:03       ` Russell King - ARM Linux
@ 2015-10-05 11:23         ` Tony Lindgren
  2015-10-05 14:35           ` Tony Lindgren
  0 siblings, 1 reply; 35+ messages in thread
From: Tony Lindgren @ 2015-10-05 11:23 UTC (permalink / raw)
  To: linux-arm-kernel

* Russell King - ARM Linux <linux@arm.linux.org.uk> [151001 03:07]:
> On Thu, Oct 01, 2015 at 11:50:00AM +0200, Ulf Hansson wrote:
> > On 1 October 2015 at 11:33, Russell King - ARM Linux
> > <linux@arm.linux.org.uk> wrote:
> > > On Thu, Sep 24, 2015 at 04:37:56PM -0700, Tony Lindgren wrote:
> > >> * Russell King - ARM Linux <linux@arm.linux.org.uk> [150924 02:04]:
> > >> > Nightly testing has revealed that both the OMAP3430 LDP and the OMAP4430
> > >> > SDP fail to boot due to lack of working MMC.  Both platforms fail to
> > >> > find their rootfs, which is on a SD card.
> > >> >
> > >> > The breakage occurred somewhere between trees of September 9th (commit
> > >> > 4e4adb2f4628) and September 12th (commit b0a1ea51bda4), so during the
> > >> > merge window.
> > >>
> > >> Yes sorry things got messed up in multiple ways :( I've summarized
> > >> the mess here earlier:
> > >>
> > >> http://article.gmane.org/gmane.linux.kernel.mmc/33911
> > >>
> > >> And today commit b9c93646fd5c ("regulator: pbias: program pbias register
> > >> offset in pbias driver") hit mainline so I'll send a pull request for
> > >> the related dts change.
> > >
> > > It's still broken and untestable.  We're 8 days off it being a full
> > > month worth of failed testing for OMAP3 and OMAP4 platforms.
> > >
> > > I think OMAP and MMC people need to do a post-mortem and work out why
> > > this happened, and how to stop it happening again in the future.
> > 
> > You are probably right!
> > 
> > I think I should have been more persistent when asking on how to deal
> > with these patches. Typically they should all have gone together via
> > one tree, or we needed a slower way forward keeping changes more
> > standalone.
> > 
> > Anyway, according to kernelci.org, which builds and boot my next
> > branch omap3/4 seems to boot now...
> > http://kernelci.org/boot/all/job/ulfh/kernel/v4.3-rc3-27-gf7734d097520/
> 
> It continues to fail here.
> 
> Okay, sod it, I'm at the point of just not giving a damn about whether
> OMAP3 and OMAP4 boot anymore.
> 
> I'm taking all OMAP platforms out of my boot farm.  I'm totally fed up
> with this kind of regression happening.  It's probably some config option
> that someone's recently introduced which defaults to being off, thereby
> breaking the ability for people to move forward without constantly having
> to re-configure.  This is not acceptable.
> 
> STOP BREAKING THE KERNEL.

I totally agree, this is unacceptable. And I'm fed up chasing driver
breakage and trying to test everything myself.

If Kihon is not starting to respond to these issues, we better start
reverting some of the MMC patches. And I'd say in the future non-trivial
patches really have to sit in linux next for two weeks before merging.

We have still the omap3 legacy booting issue remaining, and warnings
at least on omap4 overo.

Regards,

Regards,

Tony

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-05 11:23         ` Tony Lindgren
@ 2015-10-05 14:35           ` Tony Lindgren
  2015-10-05 14:51             ` Tony Lindgren
  0 siblings, 1 reply; 35+ messages in thread
From: Tony Lindgren @ 2015-10-05 14:35 UTC (permalink / raw)
  To: linux-arm-kernel

* Tony Lindgren <tony@atomide.com> [151005 04:28]:
> * Russell King - ARM Linux <linux@arm.linux.org.uk> [151001 03:07]:
> > On Thu, Oct 01, 2015 at 11:50:00AM +0200, Ulf Hansson wrote:
> > > On 1 October 2015 at 11:33, Russell King - ARM Linux
> > > <linux@arm.linux.org.uk> wrote:
> > > > On Thu, Sep 24, 2015 at 04:37:56PM -0700, Tony Lindgren wrote:
> > > >> * Russell King - ARM Linux <linux@arm.linux.org.uk> [150924 02:04]:
> > > >> > Nightly testing has revealed that both the OMAP3430 LDP and the OMAP4430
> > > >> > SDP fail to boot due to lack of working MMC.  Both platforms fail to
> > > >> > find their rootfs, which is on a SD card.
> > > >> >
> > > >> > The breakage occurred somewhere between trees of September 9th (commit
> > > >> > 4e4adb2f4628) and September 12th (commit b0a1ea51bda4), so during the
> > > >> > merge window.
> > > >>
> > > >> Yes sorry things got messed up in multiple ways :( I've summarized
> > > >> the mess here earlier:
> > > >>
> > > >> http://article.gmane.org/gmane.linux.kernel.mmc/33911
> > > >>
> > > >> And today commit b9c93646fd5c ("regulator: pbias: program pbias register
> > > >> offset in pbias driver") hit mainline so I'll send a pull request for
> > > >> the related dts change.
> > > >
> > > > It's still broken and untestable.  We're 8 days off it being a full
> > > > month worth of failed testing for OMAP3 and OMAP4 platforms.
> > > >
> > > > I think OMAP and MMC people need to do a post-mortem and work out why
> > > > this happened, and how to stop it happening again in the future.
> > > 
> > > You are probably right!
> > > 
> > > I think I should have been more persistent when asking on how to deal
> > > with these patches. Typically they should all have gone together via
> > > one tree, or we needed a slower way forward keeping changes more
> > > standalone.
> > > 
> > > Anyway, according to kernelci.org, which builds and boot my next
> > > branch omap3/4 seems to boot now...
> > > http://kernelci.org/boot/all/job/ulfh/kernel/v4.3-rc3-27-gf7734d097520/
> > 
> > It continues to fail here.
> > 
> > Okay, sod it, I'm at the point of just not giving a damn about whether
> > OMAP3 and OMAP4 boot anymore.
> > 
> > I'm taking all OMAP platforms out of my boot farm.  I'm totally fed up
> > with this kind of regression happening.  It's probably some config option
> > that someone's recently introduced which defaults to being off, thereby
> > breaking the ability for people to move forward without constantly having
> > to re-configure.  This is not acceptable.
> > 
> > STOP BREAKING THE KERNEL.
> 
> I totally agree, this is unacceptable. And I'm fed up chasing driver
> breakage and trying to test everything myself.
> 
> If Kihon is not starting to respond to these issues, we better start
> reverting some of the MMC patches. And I'd say in the future non-trivial
> patches really have to sit in linux next for two weeks before merging.
> 
> We have still the omap3 legacy booting issue remaining, and warnings
> at least on omap4 overo.

Sorry i mean Kishon instead of Kihon above.

Based on some tests it seems that the duovero unpaired regulator usage
is fixed by reverting:

c55d7a055364 ("mmc: host: omap_hsmmc: use regulator_is_enabled to
find pbias status")

And it seems that omap3 legacy MMC is broken earlier in the
series with:

7d607f917008 ("mmc: host: omap_hsmmc: use
devm_regulator_get_optional() for vmmc")

This one does not cleanly revert so have not yet tried reverting
it.

I'm at ELCE so there may be delays for me debugging stuff. Anybody
see obvious places to fix with those two patches above?

Regards,

Tony

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-05 14:35           ` Tony Lindgren
@ 2015-10-05 14:51             ` Tony Lindgren
  2015-10-05 17:11               ` Tony Lindgren
  0 siblings, 1 reply; 35+ messages in thread
From: Tony Lindgren @ 2015-10-05 14:51 UTC (permalink / raw)
  To: linux-arm-kernel

* Tony Lindgren <tony@atomide.com> [151005 07:44]:
> * Tony Lindgren <tony@atomide.com> [151005 04:28]:
> > * Russell King - ARM Linux <linux@arm.linux.org.uk> [151001 03:07]:
> > > On Thu, Oct 01, 2015 at 11:50:00AM +0200, Ulf Hansson wrote:
> > > > On 1 October 2015 at 11:33, Russell King - ARM Linux
> > > > <linux@arm.linux.org.uk> wrote:
> > > > > On Thu, Sep 24, 2015 at 04:37:56PM -0700, Tony Lindgren wrote:
> > > > >> * Russell King - ARM Linux <linux@arm.linux.org.uk> [150924 02:04]:
> > > > >> > Nightly testing has revealed that both the OMAP3430 LDP and the OMAP4430
> > > > >> > SDP fail to boot due to lack of working MMC.  Both platforms fail to
> > > > >> > find their rootfs, which is on a SD card.
> > > > >> >
> > > > >> > The breakage occurred somewhere between trees of September 9th (commit
> > > > >> > 4e4adb2f4628) and September 12th (commit b0a1ea51bda4), so during the
> > > > >> > merge window.
> > > > >>
> > > > >> Yes sorry things got messed up in multiple ways :( I've summarized
> > > > >> the mess here earlier:
> > > > >>
> > > > >> http://article.gmane.org/gmane.linux.kernel.mmc/33911
> > > > >>
> > > > >> And today commit b9c93646fd5c ("regulator: pbias: program pbias register
> > > > >> offset in pbias driver") hit mainline so I'll send a pull request for
> > > > >> the related dts change.
> > > > >
> > > > > It's still broken and untestable.  We're 8 days off it being a full
> > > > > month worth of failed testing for OMAP3 and OMAP4 platforms.
> > > > >
> > > > > I think OMAP and MMC people need to do a post-mortem and work out why
> > > > > this happened, and how to stop it happening again in the future.
> > > > 
> > > > You are probably right!
> > > > 
> > > > I think I should have been more persistent when asking on how to deal
> > > > with these patches. Typically they should all have gone together via
> > > > one tree, or we needed a slower way forward keeping changes more
> > > > standalone.
> > > > 
> > > > Anyway, according to kernelci.org, which builds and boot my next
> > > > branch omap3/4 seems to boot now...
> > > > http://kernelci.org/boot/all/job/ulfh/kernel/v4.3-rc3-27-gf7734d097520/
> > > 
> > > It continues to fail here.
> > > 
> > > Okay, sod it, I'm at the point of just not giving a damn about whether
> > > OMAP3 and OMAP4 boot anymore.
> > > 
> > > I'm taking all OMAP platforms out of my boot farm.  I'm totally fed up
> > > with this kind of regression happening.  It's probably some config option
> > > that someone's recently introduced which defaults to being off, thereby
> > > breaking the ability for people to move forward without constantly having
> > > to re-configure.  This is not acceptable.
> > > 
> > > STOP BREAKING THE KERNEL.
> > 
> > I totally agree, this is unacceptable. And I'm fed up chasing driver
> > breakage and trying to test everything myself.
> > 
> > If Kihon is not starting to respond to these issues, we better start
> > reverting some of the MMC patches. And I'd say in the future non-trivial
> > patches really have to sit in linux next for two weeks before merging.
> > 
> > We have still the omap3 legacy booting issue remaining, and warnings
> > at least on omap4 overo.
> 
> Sorry i mean Kishon instead of Kihon above.
> 
> Based on some tests it seems that the duovero unpaired regulator usage
> is fixed by reverting:
> 
> c55d7a055364 ("mmc: host: omap_hsmmc: use regulator_is_enabled to
> find pbias status")

With commit c55d7a055364 my guess is that the PBIAS regulator is
already on from an earlier MMC probe and getting re-enabled when
a deferred probe happens?

> And it seems that omap3 legacy MMC is broken earlier in the
> series with:
> 
> 7d607f917008 ("mmc: host: omap_hsmmc: use
> devm_regulator_get_optional() for vmmc")
> 
> This one does not cleanly revert so have not yet tried reverting
> it.

And with commit 7d607f917008 I'm guessing we can't return an
error if the PBIAS regulator does not exist as that's not there
for the legacy booting.

Regards,

Tony

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-05 14:51             ` Tony Lindgren
@ 2015-10-05 17:11               ` Tony Lindgren
  2015-10-05 18:38                 ` Russell King - ARM Linux
  0 siblings, 1 reply; 35+ messages in thread
From: Tony Lindgren @ 2015-10-05 17:11 UTC (permalink / raw)
  To: linux-arm-kernel

* Tony Lindgren <tony@atomide.com> [151005 07:57]:
> * Tony Lindgren <tony@atomide.com> [151005 07:44]:
> > * Tony Lindgren <tony@atomide.com> [151005 04:28]:
> > 
> > Based on some tests it seems that the duovero unpaired regulator usage
> > is fixed by reverting:
> > 
> > c55d7a055364 ("mmc: host: omap_hsmmc: use regulator_is_enabled to
> > find pbias status")
> 
> With commit c55d7a055364 my guess is that the PBIAS regulator is
> already on from an earlier MMC probe and getting re-enabled when
> a deferred probe happens?

Unless somebody has a better fix in mind for the above, I suggest
we revert it for the -rc kernel.

> > And it seems that omap3 legacy MMC is broken earlier in the
> > series with:
> > 
> > 7d607f917008 ("mmc: host: omap_hsmmc: use
> > devm_regulator_get_optional() for vmmc")
> > 
> > This one does not cleanly revert so have not yet tried reverting
> > it.
> 
> And with commit 7d607f917008 I'm guessing we can't return an
> error if the PBIAS regulator does not exist as that's not there
> for the legacy booting.

For omap3 legacy booting, we keep getting -EPROBE_DEFER for
all the optional regulators.

Something like the following might be the minimal fix for the -rc
cycle?

Regards,

Tony

8< ----------------
From: Tony Lindgren <tony@atomide.com>
Date: Mon, 5 Oct 2015 09:37:36 -0700
Subject: [PATCH] mmc: host: omap_hsmmc: Fix MMC for omap3 legacy booting

Starting with commit 7d607f917008 ("mmc: host: omap_hsmmc: use
devm_regulator_get_optional() for vmmc") MMC on omap3 stopped working
for legacy booting.

This is because legacy booting sets up some of the resource in the
platform init code, and for optional regulators always seem to
return -EPROBE_DEFER for the legacy booting.

Let's fix the issue by checking for device tree based booting for
now. Then when omap3 boots in device tree only mode, this patch
can be just reverted.

Fixes: 7d607f917008 ("mmc: host: omap_hsmmc: use
devm_regulator_get_optional() for vmmc")
Signed-off-by: Tony Lindgren <tony@atomide.com>

--- a/drivers/mmc/host/omap_hsmmc.c
+++ b/drivers/mmc/host/omap_hsmmc.c
@@ -478,7 +478,7 @@ static int omap_hsmmc_reg_get(struct omap_hsmmc_host *host)
 	mmc->supply.vmmc = devm_regulator_get_optional(host->dev, "vmmc");
 	if (IS_ERR(mmc->supply.vmmc)) {
 		ret = PTR_ERR(mmc->supply.vmmc);
-		if (ret != -ENODEV)
+		if ((ret != -ENODEV) && host->dev->of_node)
 			return ret;
 		dev_dbg(host->dev, "unable to get vmmc regulator %ld\n",
 			PTR_ERR(mmc->supply.vmmc));
@@ -493,7 +493,7 @@ static int omap_hsmmc_reg_get(struct omap_hsmmc_host *host)
 	mmc->supply.vqmmc = devm_regulator_get_optional(host->dev, "vmmc_aux");
 	if (IS_ERR(mmc->supply.vqmmc)) {
 		ret = PTR_ERR(mmc->supply.vqmmc);
-		if (ret != -ENODEV)
+		if ((ret != -ENODEV) && host->dev->of_node)
 			return ret;
 		dev_dbg(host->dev, "unable to get vmmc_aux regulator %ld\n",
 			PTR_ERR(mmc->supply.vqmmc));
@@ -503,7 +503,7 @@ static int omap_hsmmc_reg_get(struct omap_hsmmc_host *host)
 	host->pbias = devm_regulator_get_optional(host->dev, "pbias");
 	if (IS_ERR(host->pbias)) {
 		ret = PTR_ERR(host->pbias);
-		if (ret != -ENODEV)
+		if ((ret != -ENODEV) && host->dev->of_node)
 			return ret;
 		dev_dbg(host->dev, "unable to get pbias regulator %ld\n",
 			PTR_ERR(host->pbias));

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-05 17:11               ` Tony Lindgren
@ 2015-10-05 18:38                 ` Russell King - ARM Linux
  2015-10-06  9:00                   ` Russell King - ARM Linux
  0 siblings, 1 reply; 35+ messages in thread
From: Russell King - ARM Linux @ 2015-10-05 18:38 UTC (permalink / raw)
  To: linux-arm-kernel

On Mon, Oct 05, 2015 at 10:11:56AM -0700, Tony Lindgren wrote:
> * Tony Lindgren <tony@atomide.com> [151005 07:57]:
> > * Tony Lindgren <tony@atomide.com> [151005 07:44]:
> > > * Tony Lindgren <tony@atomide.com> [151005 04:28]:
> > > 
> > > Based on some tests it seems that the duovero unpaired regulator usage
> > > is fixed by reverting:
> > > 
> > > c55d7a055364 ("mmc: host: omap_hsmmc: use regulator_is_enabled to
> > > find pbias status")
> > 
> > With commit c55d7a055364 my guess is that the PBIAS regulator is
> > already on from an earlier MMC probe and getting re-enabled when
> > a deferred probe happens?
> 
> Unless somebody has a better fix in mind for the above, I suggest
> we revert it for the -rc kernel.

Let me try reverting that in my build tree, and...

> > > And it seems that omap3 legacy MMC is broken earlier in the
> > > series with:
> > > 
> > > 7d607f917008 ("mmc: host: omap_hsmmc: use
> > > devm_regulator_get_optional() for vmmc")
> > > 
> > > This one does not cleanly revert so have not yet tried reverting
> > > it.
> > 
> > And with commit 7d607f917008 I'm guessing we can't return an
> > error if the PBIAS regulator does not exist as that's not there
> > for the legacy booting.
> 
> For omap3 legacy booting, we keep getting -EPROBE_DEFER for
> all the optional regulators.
> 
> Something like the following might be the minimal fix for the -rc
> cycle?

applying this patch.  If that gets things going again, then we
_definitely_ should get both of these to Linus ASAP.  The breakage
has been around far too long already.

> 8< ----------------
> From: Tony Lindgren <tony@atomide.com>
> Date: Mon, 5 Oct 2015 09:37:36 -0700
> Subject: [PATCH] mmc: host: omap_hsmmc: Fix MMC for omap3 legacy booting
> 
> Starting with commit 7d607f917008 ("mmc: host: omap_hsmmc: use
> devm_regulator_get_optional() for vmmc") MMC on omap3 stopped working
> for legacy booting.
> 
> This is because legacy booting sets up some of the resource in the
> platform init code, and for optional regulators always seem to
> return -EPROBE_DEFER for the legacy booting.
> 
> Let's fix the issue by checking for device tree based booting for
> now. Then when omap3 boots in device tree only mode, this patch
> can be just reverted.
> 
> Fixes: 7d607f917008 ("mmc: host: omap_hsmmc: use
> devm_regulator_get_optional() for vmmc")
> Signed-off-by: Tony Lindgren <tony@atomide.com>
> 
> --- a/drivers/mmc/host/omap_hsmmc.c
> +++ b/drivers/mmc/host/omap_hsmmc.c
> @@ -478,7 +478,7 @@ static int omap_hsmmc_reg_get(struct omap_hsmmc_host *host)
>  	mmc->supply.vmmc = devm_regulator_get_optional(host->dev, "vmmc");
>  	if (IS_ERR(mmc->supply.vmmc)) {
>  		ret = PTR_ERR(mmc->supply.vmmc);
> -		if (ret != -ENODEV)
> +		if ((ret != -ENODEV) && host->dev->of_node)
>  			return ret;
>  		dev_dbg(host->dev, "unable to get vmmc regulator %ld\n",
>  			PTR_ERR(mmc->supply.vmmc));
> @@ -493,7 +493,7 @@ static int omap_hsmmc_reg_get(struct omap_hsmmc_host *host)
>  	mmc->supply.vqmmc = devm_regulator_get_optional(host->dev, "vmmc_aux");
>  	if (IS_ERR(mmc->supply.vqmmc)) {
>  		ret = PTR_ERR(mmc->supply.vqmmc);
> -		if (ret != -ENODEV)
> +		if ((ret != -ENODEV) && host->dev->of_node)
>  			return ret;
>  		dev_dbg(host->dev, "unable to get vmmc_aux regulator %ld\n",
>  			PTR_ERR(mmc->supply.vqmmc));
> @@ -503,7 +503,7 @@ static int omap_hsmmc_reg_get(struct omap_hsmmc_host *host)
>  	host->pbias = devm_regulator_get_optional(host->dev, "pbias");
>  	if (IS_ERR(host->pbias)) {
>  		ret = PTR_ERR(host->pbias);
> -		if (ret != -ENODEV)
> +		if ((ret != -ENODEV) && host->dev->of_node)
>  			return ret;
>  		dev_dbg(host->dev, "unable to get pbias regulator %ld\n",
>  			PTR_ERR(host->pbias));

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-05 18:38                 ` Russell King - ARM Linux
@ 2015-10-06  9:00                   ` Russell King - ARM Linux
  2015-10-06  9:44                     ` Tony Lindgren
  0 siblings, 1 reply; 35+ messages in thread
From: Russell King - ARM Linux @ 2015-10-06  9:00 UTC (permalink / raw)
  To: linux-arm-kernel

On Mon, Oct 05, 2015 at 07:38:13PM +0100, Russell King - ARM Linux wrote:
> On Mon, Oct 05, 2015 at 10:11:56AM -0700, Tony Lindgren wrote:
> > * Tony Lindgren <tony@atomide.com> [151005 07:57]:
> > > * Tony Lindgren <tony@atomide.com> [151005 07:44]:
> > > > * Tony Lindgren <tony@atomide.com> [151005 04:28]:
> > > > 
> > > > Based on some tests it seems that the duovero unpaired regulator usage
> > > > is fixed by reverting:
> > > > 
> > > > c55d7a055364 ("mmc: host: omap_hsmmc: use regulator_is_enabled to
> > > > find pbias status")
> > > 
> > > With commit c55d7a055364 my guess is that the PBIAS regulator is
> > > already on from an earlier MMC probe and getting re-enabled when
> > > a deferred probe happens?
> > 
> > Unless somebody has a better fix in mind for the above, I suggest
> > we revert it for the -rc kernel.
> 
> Let me try reverting that in my build tree, and...
> 
> > > > And it seems that omap3 legacy MMC is broken earlier in the
> > > > series with:
> > > > 
> > > > 7d607f917008 ("mmc: host: omap_hsmmc: use
> > > > devm_regulator_get_optional() for vmmc")
> > > > 
> > > > This one does not cleanly revert so have not yet tried reverting
> > > > it.
> > > 
> > > And with commit 7d607f917008 I'm guessing we can't return an
> > > error if the PBIAS regulator does not exist as that's not there
> > > for the legacy booting.
> > 
> > For omap3 legacy booting, we keep getting -EPROBE_DEFER for
> > all the optional regulators.
> > 
> > Something like the following might be the minimal fix for the -rc
> > cycle?
> 
> applying this patch.  If that gets things going again, then we
> _definitely_ should get both of these to Linus ASAP.  The breakage
> has been around far too long already.

Last night's build shows that this fixes the non-DT LDP3430 booting, but
DT-based LDP3430 and SDP4430 both remain broken for the same reason -
neither can find their SD cards.

We're at -rc4.  That means we're could be only three weeks away from 4.3
being released, and DT OMAP has yet to boot _once_ here.

What I find *totally* unacceptable is the lack of reaction from the MMC
and TI people: it's just like "we'll break your crap, and we'll ignore
reports of regressions."  That is *not* acceptable in any shape or form,
and people who do this _should_ have their future patches ignored until
they demonstrate that they understand the need to not break stuff.

I'm at the point of suggesting that there should be a moritorium on all
OMAP related development for 4.4: delay all development to 4.5, and have
4.4 as a pure bug-fixing _only_ cycle for OMAP.  If 4.3 is released and
OMAP is still broken, then I don't think there's an option on that.

Maybe the idea that development work won't hit mainline for another 4
months will help focus the minds on not breaking stuff and then ignoring
regression reports.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-06  9:00                   ` Russell King - ARM Linux
@ 2015-10-06  9:44                     ` Tony Lindgren
  2015-10-06 10:11                       ` Ulf Hansson
  2015-10-06 15:00                       ` Russell King - ARM Linux
  0 siblings, 2 replies; 35+ messages in thread
From: Tony Lindgren @ 2015-10-06  9:44 UTC (permalink / raw)
  To: linux-arm-kernel

* Russell King - ARM Linux <linux@arm.linux.org.uk> [151006 02:04]:
> On Mon, Oct 05, 2015 at 07:38:13PM +0100, Russell King - ARM Linux wrote:
> > On Mon, Oct 05, 2015 at 10:11:56AM -0700, Tony Lindgren wrote:
> > > * Tony Lindgren <tony@atomide.com> [151005 07:57]:
> > > > * Tony Lindgren <tony@atomide.com> [151005 07:44]:
> > > > > * Tony Lindgren <tony@atomide.com> [151005 04:28]:
> > > > > 
> > > > > Based on some tests it seems that the duovero unpaired regulator usage
> > > > > is fixed by reverting:
> > > > > 
> > > > > c55d7a055364 ("mmc: host: omap_hsmmc: use regulator_is_enabled to
> > > > > find pbias status")
> > > > 
> > > > With commit c55d7a055364 my guess is that the PBIAS regulator is
> > > > already on from an earlier MMC probe and getting re-enabled when
> > > > a deferred probe happens?
> > > 
> > > Unless somebody has a better fix in mind for the above, I suggest
> > > we revert it for the -rc kernel.
> > 
> > Let me try reverting that in my build tree, and...
> > 
> > > > > And it seems that omap3 legacy MMC is broken earlier in the
> > > > > series with:
> > > > > 
> > > > > 7d607f917008 ("mmc: host: omap_hsmmc: use
> > > > > devm_regulator_get_optional() for vmmc")
> > > > > 
> > > > > This one does not cleanly revert so have not yet tried reverting
> > > > > it.
> > > > 
> > > > And with commit 7d607f917008 I'm guessing we can't return an
> > > > error if the PBIAS regulator does not exist as that's not there
> > > > for the legacy booting.
> > > 
> > > For omap3 legacy booting, we keep getting -EPROBE_DEFER for
> > > all the optional regulators.
> > > 
> > > Something like the following might be the minimal fix for the -rc
> > > cycle?
> > 
> > applying this patch.  If that gets things going again, then we
> > _definitely_ should get both of these to Linus ASAP.  The breakage
> > has been around far too long already.
> 
> Last night's build shows that this fixes the non-DT LDP3430 booting, but
> DT-based LDP3430 and SDP4430 both remain broken for the same reason -
> neither can find their SD cards.

Hmm DT-based boot finds the MMC card for LDP, dmesg below from DT boot [1].
Looks like you're on on -rc4 and not -rc3. My guess is that MMC is not
working for you with DT-based booting because you don't seem to have
CONFIG_REGULATOR_PBIAS in your seed config for. Care to try enabling that
for both your omap3 and omap4 seed config files?

> We're at -rc4.  That means we're could be only three weeks away from 4.3
> being released, and DT OMAP has yet to boot _once_ here.
> 
> What I find *totally* unacceptable is the lack of reaction from the MMC
> and TI people: it's just like "we'll break your crap, and we'll ignore
> reports of regressions."  That is *not* acceptable in any shape or form,
> and people who do this _should_ have their future patches ignored until
> they demonstrate that they understand the need to not break stuff.
> 
> I'm at the point of suggesting that there should be a moritorium on all
> OMAP related development for 4.4: delay all development to 4.5, and have
> 4.4 as a pure bug-fixing _only_ cycle for OMAP.  If 4.3 is released and
> OMAP is still broken, then I don't think there's an option on that.
> 
> Maybe the idea that development work won't hit mainline for another 4
> months will help focus the minds on not breaking stuff and then ignoring
> regression reports.

I'm thinking along the same lines. In general, I do not and will not
apply any patches that are not fixes until all critical regressions are
out of the way. So not applying anything new for v4.4 until these MMC
regressions are fixed for v4.3.

Regards,

Tony

[1] http://muru.com/for-rmk/dmesg-ldp-v4.3-rc4-mmc-fixes.txt

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-06  9:44                     ` Tony Lindgren
@ 2015-10-06 10:11                       ` Ulf Hansson
  2015-10-06 10:36                         ` Kishon Vijay Abraham I
  2015-10-06 15:00                       ` Russell King - ARM Linux
  1 sibling, 1 reply; 35+ messages in thread
From: Ulf Hansson @ 2015-10-06 10:11 UTC (permalink / raw)
  To: linux-arm-kernel

On 6 October 2015 at 11:44, Tony Lindgren <tony@atomide.com> wrote:
> * Russell King - ARM Linux <linux@arm.linux.org.uk> [151006 02:04]:
>> On Mon, Oct 05, 2015 at 07:38:13PM +0100, Russell King - ARM Linux wrote:
>> > On Mon, Oct 05, 2015 at 10:11:56AM -0700, Tony Lindgren wrote:
>> > > * Tony Lindgren <tony@atomide.com> [151005 07:57]:
>> > > > * Tony Lindgren <tony@atomide.com> [151005 07:44]:
>> > > > > * Tony Lindgren <tony@atomide.com> [151005 04:28]:
>> > > > >
>> > > > > Based on some tests it seems that the duovero unpaired regulator usage
>> > > > > is fixed by reverting:
>> > > > >
>> > > > > c55d7a055364 ("mmc: host: omap_hsmmc: use regulator_is_enabled to
>> > > > > find pbias status")
>> > > >
>> > > > With commit c55d7a055364 my guess is that the PBIAS regulator is
>> > > > already on from an earlier MMC probe and getting re-enabled when
>> > > > a deferred probe happens?
>> > >
>> > > Unless somebody has a better fix in mind for the above, I suggest
>> > > we revert it for the -rc kernel.
>> >
>> > Let me try reverting that in my build tree, and...
>> >
>> > > > > And it seems that omap3 legacy MMC is broken earlier in the
>> > > > > series with:
>> > > > >
>> > > > > 7d607f917008 ("mmc: host: omap_hsmmc: use
>> > > > > devm_regulator_get_optional() for vmmc")
>> > > > >
>> > > > > This one does not cleanly revert so have not yet tried reverting
>> > > > > it.
>> > > >
>> > > > And with commit 7d607f917008 I'm guessing we can't return an
>> > > > error if the PBIAS regulator does not exist as that's not there
>> > > > for the legacy booting.
>> > >
>> > > For omap3 legacy booting, we keep getting -EPROBE_DEFER for
>> > > all the optional regulators.
>> > >
>> > > Something like the following might be the minimal fix for the -rc
>> > > cycle?
>> >
>> > applying this patch.  If that gets things going again, then we
>> > _definitely_ should get both of these to Linus ASAP.  The breakage
>> > has been around far too long already.
>>
>> Last night's build shows that this fixes the non-DT LDP3430 booting, but
>> DT-based LDP3430 and SDP4430 both remain broken for the same reason -
>> neither can find their SD cards.
>
> Hmm DT-based boot finds the MMC card for LDP, dmesg below from DT boot [1].
> Looks like you're on on -rc4 and not -rc3. My guess is that MMC is not
> working for you with DT-based booting because you don't seem to have
> CONFIG_REGULATOR_PBIAS in your seed config for. Care to try enabling that
> for both your omap3 and omap4 seed config files?
>
>> We're at -rc4.  That means we're could be only three weeks away from 4.3
>> being released, and DT OMAP has yet to boot _once_ here.
>>
>> What I find *totally* unacceptable is the lack of reaction from the MMC
>> and TI people: it's just like "we'll break your crap, and we'll ignore
>> reports of regressions."  That is *not* acceptable in any shape or form,
>> and people who do this _should_ have their future patches ignored until
>> they demonstrate that they understand the need to not break stuff.
>>
>> I'm at the point of suggesting that there should be a moritorium on all
>> OMAP related development for 4.4: delay all development to 4.5, and have
>> 4.4 as a pure bug-fixing _only_ cycle for OMAP.  If 4.3 is released and
>> OMAP is still broken, then I don't think there's an option on that.
>>
>> Maybe the idea that development work won't hit mainline for another 4
>> months will help focus the minds on not breaking stuff and then ignoring
>> regression reports.
>
> I'm thinking along the same lines. In general, I do not and will not
> apply any patches that are not fixes until all critical regressions are
> out of the way. So not applying anything new for v4.4 until these MMC
> regressions are fixed for v4.3.
>

Tony, Russell - just wanted to say thanks for putting a big effort in
fixing this. I was expecting support from Kishon, but I guess he's
busy.

Unfortunate, I don't have any of the hardware that fails, otherwise I
would of course be helping out more. The only thing I can think of is
to revert the entire patchset I picked up for 4.3 from Kishon for the
omap_hsmmc driver, but then I am not sure if that would cause other
issues...

Kind regards
Uffe

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-06 10:11                       ` Ulf Hansson
@ 2015-10-06 10:36                         ` Kishon Vijay Abraham I
  2015-10-06 15:07                           ` Russell King - ARM Linux
  0 siblings, 1 reply; 35+ messages in thread
From: Kishon Vijay Abraham I @ 2015-10-06 10:36 UTC (permalink / raw)
  To: linux-arm-kernel

Hi,

On Tuesday 06 October 2015 03:41 PM, Ulf Hansson wrote:
> On 6 October 2015 at 11:44, Tony Lindgren <tony@atomide.com> wrote:
>> * Russell King - ARM Linux <linux@arm.linux.org.uk> [151006 02:04]:
>>> On Mon, Oct 05, 2015 at 07:38:13PM +0100, Russell King - ARM Linux wrote:
>>>> On Mon, Oct 05, 2015 at 10:11:56AM -0700, Tony Lindgren wrote:
>>>>> * Tony Lindgren <tony@atomide.com> [151005 07:57]:
>>>>>> * Tony Lindgren <tony@atomide.com> [151005 07:44]:
>>>>>>> * Tony Lindgren <tony@atomide.com> [151005 04:28]:
>>>>>>>
>>>>>>> Based on some tests it seems that the duovero unpaired regulator usage
>>>>>>> is fixed by reverting:
>>>>>>>
>>>>>>> c55d7a055364 ("mmc: host: omap_hsmmc: use regulator_is_enabled to
>>>>>>> find pbias status")
>>>>>>
>>>>>> With commit c55d7a055364 my guess is that the PBIAS regulator is
>>>>>> already on from an earlier MMC probe and getting re-enabled when
>>>>>> a deferred probe happens?
>>>>>
>>>>> Unless somebody has a better fix in mind for the above, I suggest
>>>>> we revert it for the -rc kernel.
>>>>
>>>> Let me try reverting that in my build tree, and...
>>>>
>>>>>>> And it seems that omap3 legacy MMC is broken earlier in the
>>>>>>> series with:
>>>>>>>
>>>>>>> 7d607f917008 ("mmc: host: omap_hsmmc: use
>>>>>>> devm_regulator_get_optional() for vmmc")
>>>>>>>
>>>>>>> This one does not cleanly revert so have not yet tried reverting
>>>>>>> it.
>>>>>>
>>>>>> And with commit 7d607f917008 I'm guessing we can't return anHi,
>>>>>> error if the PBIAS regulator does not exist as that's not there
>>>>>> for the legacy booting.
>>>>>
>>>>> For omap3 legacy booting, we keep getting -EPROBE_DEFER for
>>>>> all the optional regulators.
>>>>>
>>>>> Something like the following might be the minimal fix for the -rc
>>>>> cycle?
>>>>
>>>> applying this patch.  If that gets things going again, then we
>>>> _definitely_ should get both of these to Linus ASAP.  The breakage
>>>> has been around far too long already.
>>>
>>> Last night's build shows that this fixes the non-DT LDP3430 booting, but
>>> DT-based LDP3430 and SDP4430 both remain broken for the same reason -
>>> neither can find their SD cards.
>>
>> Hmm DT-based boot finds the MMC card for LDP, dmesg below from DT boot [1].
>> Looks like you're on on -rc4 and not -rc3. My guess is that MMC is not
>> working for you with DT-based booting because you don't seem to have
>> CONFIG_REGULATOR_PBIAS in your seed config for. Care to try enabling that
>> for both your omap3 and omap4 seed config files?
>>
>>> We're at -rc4.  That means we're could be only three weeks away from 4.3
>>> being released, and DT OMAP has yet to boot _once_ here.
>>>
>>> What I find *totally* unacceptable is the lack of reaction from the MMC
>>> and TI people: it's just like "we'll break your crap, and we'll ignore
>>> reports of regressions."  That is *not* acceptable in any shape or form,
>>> and people who do this _should_ have their future patches ignored until
>>> they demonstrate that they understand the need to not break stuff.
>>>
>>> I'm at the point of suggesting that there should be a moritorium on all
>>> OMAP related development for 4.4: delay all development to 4.5, and have
>>> 4.4 as a pure bug-fixing _only_ cycle for OMAP.  If 4.3 is released and
>>> OMAP is still broken, then I don't think there's an option on that.
>>>
>>> Maybe the idea that development work won't hit mainline for another 4
>>> months will help focus the minds on not breaking stuff and then ignoring
>>> regression reports.
>>
>> I'm thinking along the same lines. In general, I do not and will not
>> apply any patches that are not fixes until all critical regressions are
>> out of the way. So not applying anything new for v4.4 until these MMC
>> regressions are fixed for v4.3.
>>
> 
> Tony, Russell - just wanted to say thanks for putting a big effort in
> fixing this. I was expecting support from Kishon, but I guess he's
> busy.
> 
> Unfortunate, I don't have any of the hardware that fails, otherwise I
> would of course be helping out more. The only thing I can think of is
> to revert the entire patchset I picked up for 4.3 from Kishon for the
> omap_hsmmc driver, but then I am not sure if that would cause other
> issues...

Please don't do that as that will just mask lot of bugs in the
devicetree and might get MMC working. Indeed I was busy but I'll try to
get this resolved asap. The 2 pending issues as far as I know

1) warn dump while enabling pbias regulator in duovero. As I mentioned
in some other thread it might be because the PBIAS regulator in 4430 is
not setting the status bit properly because of which we can't use the
regulator_is_enabled API to enable/disable the regulator. I don't think
there is any other way to fix this other than reverting c55d7a055364.

2) legacy booting is not working with OMAP3.
I'm not sure what is causing this. I'm right now in ELCE and will be
able to look at this on Friday when I'll be able to get hold of LDP3430.

I really didn't intend to be unresponsive or something but just that the
above 2 boards are not readily accessible to me for the debugging.
Before posting the patches I tested in all the possible boards I had.

Anyways apologies for not having been able to resolved this faster.

Thanks
Kishon

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-06  9:44                     ` Tony Lindgren
  2015-10-06 10:11                       ` Ulf Hansson
@ 2015-10-06 15:00                       ` Russell King - ARM Linux
  2015-10-06 15:37                         ` Tony Lindgren
  1 sibling, 1 reply; 35+ messages in thread
From: Russell King - ARM Linux @ 2015-10-06 15:00 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Oct 06, 2015 at 02:44:25AM -0700, Tony Lindgren wrote:
> * Russell King - ARM Linux <linux@arm.linux.org.uk> [151006 02:04]:
> > On Mon, Oct 05, 2015 at 07:38:13PM +0100, Russell King - ARM Linux wrote:
> > > On Mon, Oct 05, 2015 at 10:11:56AM -0700, Tony Lindgren wrote:
> > > > * Tony Lindgren <tony@atomide.com> [151005 07:57]:
> > > > > * Tony Lindgren <tony@atomide.com> [151005 07:44]:
> > > > > > * Tony Lindgren <tony@atomide.com> [151005 04:28]:
> > > > > > 
> > > > > > Based on some tests it seems that the duovero unpaired regulator usage
> > > > > > is fixed by reverting:
> > > > > > 
> > > > > > c55d7a055364 ("mmc: host: omap_hsmmc: use regulator_is_enabled to
> > > > > > find pbias status")
> > > > > 
> > > > > With commit c55d7a055364 my guess is that the PBIAS regulator is
> > > > > already on from an earlier MMC probe and getting re-enabled when
> > > > > a deferred probe happens?
> > > > 
> > > > Unless somebody has a better fix in mind for the above, I suggest
> > > > we revert it for the -rc kernel.
> > > 
> > > Let me try reverting that in my build tree, and...
> > > 
> > > > > > And it seems that omap3 legacy MMC is broken earlier in the
> > > > > > series with:
> > > > > > 
> > > > > > 7d607f917008 ("mmc: host: omap_hsmmc: use
> > > > > > devm_regulator_get_optional() for vmmc")
> > > > > > 
> > > > > > This one does not cleanly revert so have not yet tried reverting
> > > > > > it.
> > > > > 
> > > > > And with commit 7d607f917008 I'm guessing we can't return an
> > > > > error if the PBIAS regulator does not exist as that's not there
> > > > > for the legacy booting.
> > > > 
> > > > For omap3 legacy booting, we keep getting -EPROBE_DEFER for
> > > > all the optional regulators.
> > > > 
> > > > Something like the following might be the minimal fix for the -rc
> > > > cycle?
> > > 
> > > applying this patch.  If that gets things going again, then we
> > > _definitely_ should get both of these to Linus ASAP.  The breakage
> > > has been around far too long already.
> > 
> > Last night's build shows that this fixes the non-DT LDP3430 booting, but
> > DT-based LDP3430 and SDP4430 both remain broken for the same reason -
> > neither can find their SD cards.
> 
> Hmm DT-based boot finds the MMC card for LDP, dmesg below from DT boot [1].
> Looks like you're on on -rc4 and not -rc3. My guess is that MMC is not
> working for you with DT-based booting because you don't seem to have
> CONFIG_REGULATOR_PBIAS in your seed config for. Care to try enabling that
> for both your omap3 and omap4 seed config files?

This is precisely the kind of crap I'm objecting to.  New kernel versions
come along, and things break because people add extra Kconfig symbols that
previous versions did not rely upon - and there's no communication of
what's required for new kernel versions.

This stuff needs documenting, so that people are aware what changes they
need to make - please put something in Documentation/arm/OMAP which
tracks what new additions are required to the Kconfig to keep things
working.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-06 10:36                         ` Kishon Vijay Abraham I
@ 2015-10-06 15:07                           ` Russell King - ARM Linux
  2015-10-06 19:29                             ` Kishon Vijay Abraham I
  0 siblings, 1 reply; 35+ messages in thread
From: Russell King - ARM Linux @ 2015-10-06 15:07 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Oct 06, 2015 at 04:06:09PM +0530, Kishon Vijay Abraham I wrote:
> Hi,
> 
> On Tuesday 06 October 2015 03:41 PM, Ulf Hansson wrote:
> > On 6 October 2015 at 11:44, Tony Lindgren <tony@atomide.com> wrote:
> >> * Russell King - ARM Linux <linux@arm.linux.org.uk> [151006 02:04]:
> >>> On Mon, Oct 05, 2015 at 07:38:13PM +0100, Russell King - ARM Linux wrote:
> >>>> On Mon, Oct 05, 2015 at 10:11:56AM -0700, Tony Lindgren wrote:
> >>>>> * Tony Lindgren <tony@atomide.com> [151005 07:57]:
> >>>>>> * Tony Lindgren <tony@atomide.com> [151005 07:44]:
> >>>>>>> * Tony Lindgren <tony@atomide.com> [151005 04:28]:
> >>>>>>>
> >>>>>>> Based on some tests it seems that the duovero unpaired regulator usage
> >>>>>>> is fixed by reverting:
> >>>>>>>
> >>>>>>> c55d7a055364 ("mmc: host: omap_hsmmc: use regulator_is_enabled to
> >>>>>>> find pbias status")
> >>>>>>
> >>>>>> With commit c55d7a055364 my guess is that the PBIAS regulator is
> >>>>>> already on from an earlier MMC probe and getting re-enabled when
> >>>>>> a deferred probe happens?
> >>>>>
> >>>>> Unless somebody has a better fix in mind for the above, I suggest
> >>>>> we revert it for the -rc kernel.
> >>>>
> >>>> Let me try reverting that in my build tree, and...
> >>>>
> >>>>>>> And it seems that omap3 legacy MMC is broken earlier in the
> >>>>>>> series with:
> >>>>>>>
> >>>>>>> 7d607f917008 ("mmc: host: omap_hsmmc: use
> >>>>>>> devm_regulator_get_optional() for vmmc")
> >>>>>>>
> >>>>>>> This one does not cleanly revert so have not yet tried reverting
> >>>>>>> it.
> >>>>>>
> >>>>>> And with commit 7d607f917008 I'm guessing we can't return anHi,
> >>>>>> error if the PBIAS regulator does not exist as that's not there
> >>>>>> for the legacy booting.
> >>>>>
> >>>>> For omap3 legacy booting, we keep getting -EPROBE_DEFER for
> >>>>> all the optional regulators.
> >>>>>
> >>>>> Something like the following might be the minimal fix for the -rc
> >>>>> cycle?
> >>>>
> >>>> applying this patch.  If that gets things going again, then we
> >>>> _definitely_ should get both of these to Linus ASAP.  The breakage
> >>>> has been around far too long already.
> >>>
> >>> Last night's build shows that this fixes the non-DT LDP3430 booting, but
> >>> DT-based LDP3430 and SDP4430 both remain broken for the same reason -
> >>> neither can find their SD cards.
> >>
> >> Hmm DT-based boot finds the MMC card for LDP, dmesg below from DT boot [1].
> >> Looks like you're on on -rc4 and not -rc3. My guess is that MMC is not
> >> working for you with DT-based booting because you don't seem to have
> >> CONFIG_REGULATOR_PBIAS in your seed config for. Care to try enabling that
> >> for both your omap3 and omap4 seed config files?
> >>
> >>> We're at -rc4.  That means we're could be only three weeks away from 4.3
> >>> being released, and DT OMAP has yet to boot _once_ here.
> >>>
> >>> What I find *totally* unacceptable is the lack of reaction from the MMC
> >>> and TI people: it's just like "we'll break your crap, and we'll ignore
> >>> reports of regressions."  That is *not* acceptable in any shape or form,
> >>> and people who do this _should_ have their future patches ignored until
> >>> they demonstrate that they understand the need to not break stuff.
> >>>
> >>> I'm at the point of suggesting that there should be a moritorium on all
> >>> OMAP related development for 4.4: delay all development to 4.5, and have
> >>> 4.4 as a pure bug-fixing _only_ cycle for OMAP.  If 4.3 is released and
> >>> OMAP is still broken, then I don't think there's an option on that.
> >>>
> >>> Maybe the idea that development work won't hit mainline for another 4
> >>> months will help focus the minds on not breaking stuff and then ignoring
> >>> regression reports.
> >>
> >> I'm thinking along the same lines. In general, I do not and will not
> >> apply any patches that are not fixes until all critical regressions are
> >> out of the way. So not applying anything new for v4.4 until these MMC
> >> regressions are fixed for v4.3.
> >>
> > 
> > Tony, Russell - just wanted to say thanks for putting a big effort in
> > fixing this. I was expecting support from Kishon, but I guess he's
> > busy.
> > 
> > Unfortunate, I don't have any of the hardware that fails, otherwise I
> > would of course be helping out more. The only thing I can think of is
> > to revert the entire patchset I picked up for 4.3 from Kishon for the
> > omap_hsmmc driver, but then I am not sure if that would cause other
> > issues...
> 
> Please don't do that as that will just mask lot of bugs in the
> devicetree and might get MMC working. Indeed I was busy but I'll try to
> get this resolved asap. The 2 pending issues as far as I know

Sorry, but no.  None of this "mask bugs ... might" crap.  Either they're
bug fixes, or they aren't bug fixes.  This should be a definite decision,
not some vague crap.

If they're bug fixes, why the _hell_ aren't they queued for -rc kernels?

This sounds really screwed up to me, and it's no wonder that OMAP MMC got
broken if this is the kind of thing that's going on.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-06 15:00                       ` Russell King - ARM Linux
@ 2015-10-06 15:37                         ` Tony Lindgren
  2015-10-07 12:45                           ` Russell King - ARM Linux
  0 siblings, 1 reply; 35+ messages in thread
From: Tony Lindgren @ 2015-10-06 15:37 UTC (permalink / raw)
  To: linux-arm-kernel

* Russell King - ARM Linux <linux@arm.linux.org.uk> [151006 08:04]:
> On Tue, Oct 06, 2015 at 02:44:25AM -0700, Tony Lindgren wrote:
> > 
> > Hmm DT-based boot finds the MMC card for LDP, dmesg below from DT boot [1].
> > Looks like you're on on -rc4 and not -rc3. My guess is that MMC is not
> > working for you with DT-based booting because you don't seem to have
> > CONFIG_REGULATOR_PBIAS in your seed config for. Care to try enabling that
> > for both your omap3 and omap4 seed config files?
> 
> This is precisely the kind of crap I'm objecting to.  New kernel versions
> come along, and things break because people add extra Kconfig symbols that
> previous versions did not rely upon - and there's no communication of
> what's required for new kernel versions.
> 
> This stuff needs documenting, so that people are aware what changes they
> need to make - please put something in Documentation/arm/OMAP which
> tracks what new additions are required to the Kconfig to keep things
> working.

Good idea, how about something like the following? AFAIK that's the
only .config option needed as MFD_SYSCON is selected by Kconfig
already.

Regrds,

Tony

8< ----------
From: Tony Lindgren <tony@atomide.com>
Date: Tue, 6 Oct 2015 08:33:16 -0700
Subject: [PATCH] Documentation: ARM: List new omap MMC requirements

Earlier the PBIAS regulator was optional, not so with recent
omap_hsmmc changes. To make things easier for people with
custom .config files, let's add minimal documentation for it
as suggested by Russell King <rmk+kernel@arm.linux.org.uk>.

Signed-off-by: Tony Lindgren <tony@atomide.com>

--- /dev/null
+++ b/Documentation/arm/OMAP/README
@@ -0,0 +1,7 @@
+This file contains documentation for running mainline
+kernel on omaps.
+
+KERNEL		NEW DEPENDENCIES
+v4.3+		Update is needed for custom .config files to make sure
+		CONFIG_REGULATOR_PBIAS is enabled for MMC1 to work
+		properly.

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-06 15:07                           ` Russell King - ARM Linux
@ 2015-10-06 19:29                             ` Kishon Vijay Abraham I
  2015-10-06 19:57                               ` Russell King - ARM Linux
  0 siblings, 1 reply; 35+ messages in thread
From: Kishon Vijay Abraham I @ 2015-10-06 19:29 UTC (permalink / raw)
  To: linux-arm-kernel

Hi,

On Tuesday 06 October 2015 08:37 PM, Russell King - ARM Linux wrote:
> On Tue, Oct 06, 2015 at 04:06:09PM +0530, Kishon Vijay Abraham I wrote:
>> Hi,
>>
>> On Tuesday 06 October 2015 03:41 PM, Ulf Hansson wrote:
>>> On 6 October 2015 at 11:44, Tony Lindgren <tony@atomide.com> wrote:
>>>> * Russell King - ARM Linux <linux@arm.linux.org.uk> [151006 02:04]:
>>>>> On Mon, Oct 05, 2015 at 07:38:13PM +0100, Russell King - ARM Linux wrote:
>>>>>> On Mon, Oct 05, 2015 at 10:11:56AM -0700, Tony Lindgren wrote:
>>>>>>> * Tony Lindgren <tony@atomide.com> [151005 07:57]:
>>>>>>>> * Tony Lindgren <tony@atomide.com> [151005 07:44]:
>>>>>>>>> * Tony Lindgren <tony@atomide.com> [151005 04:28]:
>>>>>>>>>
>>>>>>>>> Based on some tests it seems that the duovero unpaired regulator usage
>>>>>>>>> is fixed by reverting:
>>>>>>>>>
>>>>>>>>> c55d7a055364 ("mmc: host: omap_hsmmc: use regulator_is_enabled to
>>>>>>>>> find pbias status")
>>>>>>>>
>>>>>>>> With commit c55d7a055364 my guess is that the PBIAS regulator is
>>>>>>>> already on from an earlier MMC probe and getting re-enabled when
>>>>>>>> a deferred probe happens?
>>>>>>>
>>>>>>> Unless somebody has a better fix in mind for the above, I suggest
>>>>>>> we revert it for the -rc kernel.
>>>>>>
>>>>>> Let me try reverting that in my build tree, and...
>>>>>>
>>>>>>>>> And it seems that omap3 legacy MMC is broken earlier in the
>>>>>>>>> series with:
>>>>>>>>>
>>>>>>>>> 7d607f917008 ("mmc: host: omap_hsmmc: use
>>>>>>>>> devm_regulator_get_optional() for vmmc")
>>>>>>>>>
>>>>>>>>> This one does not cleanly revert so have not yet tried reverting
>>>>>>>>> it.
>>>>>>>>
>>>>>>>> And with commit 7d607f917008 I'm guessing we can't return anHi,
>>>>>>>> error if the PBIAS regulator does not exist as that's not there
>>>>>>>> for the legacy booting.
>>>>>>>
>>>>>>> For omap3 legacy booting, we keep getting -EPROBE_DEFER for
>>>>>>> all the optional regulators.
>>>>>>>
>>>>>>> Something like the following might be the minimal fix for the -rc
>>>>>>> cycle?
>>>>>>
>>>>>> applying this patch.  If that gets things going again, then we
>>>>>> _definitely_ should get both of these to Linus ASAP.  The breakage
>>>>>> has been around far too long already.
>>>>>
>>>>> Last night's build shows that this fixes the non-DT LDP3430 booting, but
>>>>> DT-based LDP3430 and SDP4430 both remain broken for the same reason -
>>>>> neither can find their SD cards.
>>>>
>>>> Hmm DT-based boot finds the MMC card for LDP, dmesg below from DT boot [1].
>>>> Looks like you're on on -rc4 and not -rc3. My guess is that MMC is not
>>>> working for you with DT-based booting because you don't seem to have
>>>> CONFIG_REGULATOR_PBIAS in your seed config for. Care to try enabling that
>>>> for both your omap3 and omap4 seed config files?
>>>>
>>>>> We're at -rc4.  That means we're could be only three weeks away from 4.3
>>>>> being released, and DT OMAP has yet to boot _once_ here.
>>>>>
>>>>> What I find *totally* unacceptable is the lack of reaction from the MMC
>>>>> and TI people: it's just like "we'll break your crap, and we'll ignore
>>>>> reports of regressions."  That is *not* acceptable in any shape or form,
>>>>> and people who do this _should_ have their future patches ignored until
>>>>> they demonstrate that they understand the need to not break stuff.
>>>>>
>>>>> I'm at the point of suggesting that there should be a moritorium on all
>>>>> OMAP related development for 4.4: delay all development to 4.5, and have
>>>>> 4.4 as a pure bug-fixing _only_ cycle for OMAP.  If 4.3 is released and
>>>>> OMAP is still broken, then I don't think there's an option on that.
>>>>>
>>>>> Maybe the idea that development work won't hit mainline for another 4
>>>>> months will help focus the minds on not breaking stuff and then ignoring
>>>>> regression reports.
>>>>
>>>> I'm thinking along the same lines. In general, I do not and will not
>>>> apply any patches that are not fixes until all critical regressions are
>>>> out of the way. So not applying anything new for v4.4 until these MMC
>>>> regressions are fixed for v4.3.
>>>>
>>>
>>> Tony, Russell - just wanted to say thanks for putting a big effort in
>>> fixing this. I was expecting support from Kishon, but I guess he's
>>> busy.
>>>
>>> Unfortunate, I don't have any of the hardware that fails, otherwise I
>>> would of course be helping out more. The only thing I can think of is
>>> to revert the entire patchset I picked up for 4.3 from Kishon for the
>>> omap_hsmmc driver, but then I am not sure if that would cause other
>>> issues...
>>
>> Please don't do that as that will just mask lot of bugs in the
>> devicetree and might get MMC working. Indeed I was busy but I'll try to
>> get this resolved asap. The 2 pending issues as far as I know
> 
> Sorry, but no.  None of this "mask bugs ... might" crap.  Either they're
> bug fixes, or they aren't bug fixes.  This should be a definite decision,
> not some vague crap.

What I meant is the patches merged by Ulf *did* exposed bugs in device
tree (for instance a dt patch caused PBIAS regulator from not being
registered) and if those patches are reverted then a future dt breakage
might again be not caught.

-Kishon

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-06 19:29                             ` Kishon Vijay Abraham I
@ 2015-10-06 19:57                               ` Russell King - ARM Linux
  2015-10-08  0:46                                 ` Kishon Vijay Abraham I
  0 siblings, 1 reply; 35+ messages in thread
From: Russell King - ARM Linux @ 2015-10-06 19:57 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Oct 07, 2015 at 12:59:29AM +0530, Kishon Vijay Abraham I wrote:
> Hi,
> 
> On Tuesday 06 October 2015 08:37 PM, Russell King - ARM Linux wrote:
> > On Tue, Oct 06, 2015 at 04:06:09PM +0530, Kishon Vijay Abraham I wrote:
> >> Hi,
> >>
> >> On Tuesday 06 October 2015 03:41 PM, Ulf Hansson wrote:
> >>> On 6 October 2015 at 11:44, Tony Lindgren <tony@atomide.com> wrote:
> >>>> * Russell King - ARM Linux <linux@arm.linux.org.uk> [151006 02:04]:
> >>>>> On Mon, Oct 05, 2015 at 07:38:13PM +0100, Russell King - ARM Linux wrote:
> >>>>>> On Mon, Oct 05, 2015 at 10:11:56AM -0700, Tony Lindgren wrote:
> >>>>>>> * Tony Lindgren <tony@atomide.com> [151005 07:57]:
> >>>>>>>> * Tony Lindgren <tony@atomide.com> [151005 07:44]:
> >>>>>>>>> * Tony Lindgren <tony@atomide.com> [151005 04:28]:
> >>>>>>>>>
> >>>>>>>>> Based on some tests it seems that the duovero unpaired regulator usage
> >>>>>>>>> is fixed by reverting:
> >>>>>>>>>
> >>>>>>>>> c55d7a055364 ("mmc: host: omap_hsmmc: use regulator_is_enabled to
> >>>>>>>>> find pbias status")
> >>>>>>>>
> >>>>>>>> With commit c55d7a055364 my guess is that the PBIAS regulator is
> >>>>>>>> already on from an earlier MMC probe and getting re-enabled when
> >>>>>>>> a deferred probe happens?
> >>>>>>>
> >>>>>>> Unless somebody has a better fix in mind for the above, I suggest
> >>>>>>> we revert it for the -rc kernel.
> >>>>>>
> >>>>>> Let me try reverting that in my build tree, and...
> >>>>>>
> >>>>>>>>> And it seems that omap3 legacy MMC is broken earlier in the
> >>>>>>>>> series with:
> >>>>>>>>>
> >>>>>>>>> 7d607f917008 ("mmc: host: omap_hsmmc: use
> >>>>>>>>> devm_regulator_get_optional() for vmmc")
> >>>>>>>>>
> >>>>>>>>> This one does not cleanly revert so have not yet tried reverting
> >>>>>>>>> it.
> >>>>>>>>
> >>>>>>>> And with commit 7d607f917008 I'm guessing we can't return anHi,
> >>>>>>>> error if the PBIAS regulator does not exist as that's not there
> >>>>>>>> for the legacy booting.
> >>>>>>>
> >>>>>>> For omap3 legacy booting, we keep getting -EPROBE_DEFER for
> >>>>>>> all the optional regulators.
> >>>>>>>
> >>>>>>> Something like the following might be the minimal fix for the -rc
> >>>>>>> cycle?
> >>>>>>
> >>>>>> applying this patch.  If that gets things going again, then we
> >>>>>> _definitely_ should get both of these to Linus ASAP.  The breakage
> >>>>>> has been around far too long already.
> >>>>>
> >>>>> Last night's build shows that this fixes the non-DT LDP3430 booting, but
> >>>>> DT-based LDP3430 and SDP4430 both remain broken for the same reason -
> >>>>> neither can find their SD cards.
> >>>>
> >>>> Hmm DT-based boot finds the MMC card for LDP, dmesg below from DT boot [1].
> >>>> Looks like you're on on -rc4 and not -rc3. My guess is that MMC is not
> >>>> working for you with DT-based booting because you don't seem to have
> >>>> CONFIG_REGULATOR_PBIAS in your seed config for. Care to try enabling that
> >>>> for both your omap3 and omap4 seed config files?
> >>>>
> >>>>> We're at -rc4.  That means we're could be only three weeks away from 4.3
> >>>>> being released, and DT OMAP has yet to boot _once_ here.
> >>>>>
> >>>>> What I find *totally* unacceptable is the lack of reaction from the MMC
> >>>>> and TI people: it's just like "we'll break your crap, and we'll ignore
> >>>>> reports of regressions."  That is *not* acceptable in any shape or form,
> >>>>> and people who do this _should_ have their future patches ignored until
> >>>>> they demonstrate that they understand the need to not break stuff.
> >>>>>
> >>>>> I'm at the point of suggesting that there should be a moritorium on all
> >>>>> OMAP related development for 4.4: delay all development to 4.5, and have
> >>>>> 4.4 as a pure bug-fixing _only_ cycle for OMAP.  If 4.3 is released and
> >>>>> OMAP is still broken, then I don't think there's an option on that.
> >>>>>
> >>>>> Maybe the idea that development work won't hit mainline for another 4
> >>>>> months will help focus the minds on not breaking stuff and then ignoring
> >>>>> regression reports.
> >>>>
> >>>> I'm thinking along the same lines. In general, I do not and will not
> >>>> apply any patches that are not fixes until all critical regressions are
> >>>> out of the way. So not applying anything new for v4.4 until these MMC
> >>>> regressions are fixed for v4.3.
> >>>>
> >>>
> >>> Tony, Russell - just wanted to say thanks for putting a big effort in
> >>> fixing this. I was expecting support from Kishon, but I guess he's
> >>> busy.
> >>>
> >>> Unfortunate, I don't have any of the hardware that fails, otherwise I
> >>> would of course be helping out more. The only thing I can think of is
> >>> to revert the entire patchset I picked up for 4.3 from Kishon for the
> >>> omap_hsmmc driver, but then I am not sure if that would cause other
> >>> issues...
> >>
> >> Please don't do that as that will just mask lot of bugs in the
> >> devicetree and might get MMC working. Indeed I was busy but I'll try to
> >> get this resolved asap. The 2 pending issues as far as I know
> > 
> > Sorry, but no.  None of this "mask bugs ... might" crap.  Either they're
> > bug fixes, or they aren't bug fixes.  This should be a definite decision,
> > not some vague crap.
> 
> What I meant is the patches merged by Ulf *did* exposed bugs in device
> tree (for instance a dt patch caused PBIAS regulator from not being
> registered) and if those patches are reverted then a future dt breakage
> might again be not caught.

That makes it even worse then.

What you seem to be saying now is that the old DT files were wrong, and
you've fixed the DT files up in the latest kernel.  You've also changed
the kernel in a way to make the old DT files fail.

That's totally unacceptable.  Users have the right to expect that old DT
files work with later kernels.

It's fine to detect the broken DT files and print a warning, and have
the kernel fix up the breakage, but it is unacceptable to decide that
old DT files are broken and to then change stuff to prevent them working
with later kernels.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-06 15:37                         ` Tony Lindgren
@ 2015-10-07 12:45                           ` Russell King - ARM Linux
  2015-10-07 13:26                             ` Tony Lindgren
  0 siblings, 1 reply; 35+ messages in thread
From: Russell King - ARM Linux @ 2015-10-07 12:45 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Oct 06, 2015 at 08:37:11AM -0700, Tony Lindgren wrote:
> * Russell King - ARM Linux <linux@arm.linux.org.uk> [151006 08:04]:
> > On Tue, Oct 06, 2015 at 02:44:25AM -0700, Tony Lindgren wrote:
> > > 
> > > Hmm DT-based boot finds the MMC card for LDP, dmesg below from DT boot [1].
> > > Looks like you're on on -rc4 and not -rc3. My guess is that MMC is not
> > > working for you with DT-based booting because you don't seem to have
> > > CONFIG_REGULATOR_PBIAS in your seed config for. Care to try enabling that
> > > for both your omap3 and omap4 seed config files?
> > 
> > This is precisely the kind of crap I'm objecting to.  New kernel versions
> > come along, and things break because people add extra Kconfig symbols that
> > previous versions did not rely upon - and there's no communication of
> > what's required for new kernel versions.
> > 
> > This stuff needs documenting, so that people are aware what changes they
> > need to make - please put something in Documentation/arm/OMAP which
> > tracks what new additions are required to the Kconfig to keep things
> > working.
> 
> Good idea, how about something like the following? AFAIK that's the
> only .config option needed as MFD_SYSCON is selected by Kconfig
> already.

Right, that resolves the issue.  Now, what's the story on the revert
and the other patch - have they already been taken for -rc kernels?

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-07 12:45                           ` Russell King - ARM Linux
@ 2015-10-07 13:26                             ` Tony Lindgren
  2015-10-07 13:41                               ` Ulf Hansson
  0 siblings, 1 reply; 35+ messages in thread
From: Tony Lindgren @ 2015-10-07 13:26 UTC (permalink / raw)
  To: linux-arm-kernel

* Russell King - ARM Linux <linux@arm.linux.org.uk> [151007 05:50]:
> On Tue, Oct 06, 2015 at 08:37:11AM -0700, Tony Lindgren wrote:
> > * Russell King - ARM Linux <linux@arm.linux.org.uk> [151006 08:04]:
> > > On Tue, Oct 06, 2015 at 02:44:25AM -0700, Tony Lindgren wrote:
> > > > 
> > > > Hmm DT-based boot finds the MMC card for LDP, dmesg below from DT boot [1].
> > > > Looks like you're on on -rc4 and not -rc3. My guess is that MMC is not
> > > > working for you with DT-based booting because you don't seem to have
> > > > CONFIG_REGULATOR_PBIAS in your seed config for. Care to try enabling that
> > > > for both your omap3 and omap4 seed config files?
> > > 
> > > This is precisely the kind of crap I'm objecting to.  New kernel versions
> > > come along, and things break because people add extra Kconfig symbols that
> > > previous versions did not rely upon - and there's no communication of
> > > what's required for new kernel versions.
> > > 
> > > This stuff needs documenting, so that people are aware what changes they
> > > need to make - please put something in Documentation/arm/OMAP which
> > > tracks what new additions are required to the Kconfig to keep things
> > > working.
> > 
> > Good idea, how about something like the following? AFAIK that's the
> > only .config option needed as MFD_SYSCON is selected by Kconfig
> > already.
> 
> Right, that resolves the issue.  Now, what's the story on the revert
> and the other patch - have they already been taken for -rc kernels?

Not yet, I was waiting for the confirmation from you. I've now posted the
revert with proper description and the fix so Ulf can pick them up and
merged as regression fixes for the -rc series:

http://marc.info/?l=linux-omap&m=144422416621373&w=2
http://marc.info/?l=linux-omap&m=144422416921375&w=2

Regards,

Tony

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-07 13:26                             ` Tony Lindgren
@ 2015-10-07 13:41                               ` Ulf Hansson
  2015-10-07 15:52                                 ` Tony Lindgren
  2015-10-07 17:53                                 ` Russell King - ARM Linux
  0 siblings, 2 replies; 35+ messages in thread
From: Ulf Hansson @ 2015-10-07 13:41 UTC (permalink / raw)
  To: linux-arm-kernel

On 7 October 2015 at 15:26, Tony Lindgren <tony@atomide.com> wrote:
> * Russell King - ARM Linux <linux@arm.linux.org.uk> [151007 05:50]:
>> On Tue, Oct 06, 2015 at 08:37:11AM -0700, Tony Lindgren wrote:
>> > * Russell King - ARM Linux <linux@arm.linux.org.uk> [151006 08:04]:
>> > > On Tue, Oct 06, 2015 at 02:44:25AM -0700, Tony Lindgren wrote:
>> > > >
>> > > > Hmm DT-based boot finds the MMC card for LDP, dmesg below from DT boot [1].
>> > > > Looks like you're on on -rc4 and not -rc3. My guess is that MMC is not
>> > > > working for you with DT-based booting because you don't seem to have
>> > > > CONFIG_REGULATOR_PBIAS in your seed config for. Care to try enabling that
>> > > > for both your omap3 and omap4 seed config files?
>> > >
>> > > This is precisely the kind of crap I'm objecting to.  New kernel versions
>> > > come along, and things break because people add extra Kconfig symbols that
>> > > previous versions did not rely upon - and there's no communication of
>> > > what's required for new kernel versions.
>> > >
>> > > This stuff needs documenting, so that people are aware what changes they
>> > > need to make - please put something in Documentation/arm/OMAP which
>> > > tracks what new additions are required to the Kconfig to keep things
>> > > working.
>> >
>> > Good idea, how about something like the following? AFAIK that's the
>> > only .config option needed as MFD_SYSCON is selected by Kconfig
>> > already.

Similar to MFD_SYSCON, why don't we have REGULATOR_PBIAS to be
selected when omap_hsmmc is being used?

It seems like that should also be a patch for the rc, right!?

>>
>> Right, that resolves the issue.  Now, what's the story on the revert
>> and the other patch - have they already been taken for -rc kernels?

Great!

>
> Not yet, I was waiting for the confirmation from you. I've now posted the
> revert with proper description and the fix so Ulf can pick them up and
> merged as regression fixes for the -rc series:

I am on it!

>
> http://marc.info/?l=linux-omap&m=144422416621373&w=2
> http://marc.info/?l=linux-omap&m=144422416921375&w=2

Russell, may I add your tested by tag for these?

Again, thanks!

Kind regards
Uffe

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-07 13:41                               ` Ulf Hansson
@ 2015-10-07 15:52                                 ` Tony Lindgren
  2015-10-07 19:40                                   ` Ulf Hansson
  2015-10-07 17:53                                 ` Russell King - ARM Linux
  1 sibling, 1 reply; 35+ messages in thread
From: Tony Lindgren @ 2015-10-07 15:52 UTC (permalink / raw)
  To: linux-arm-kernel

* Ulf Hansson <ulf.hansson@linaro.org> [151007 06:46]:
> On 7 October 2015 at 15:26, Tony Lindgren <tony@atomide.com> wrote:
> >> > Good idea, how about something like the following? AFAIK that's the
> >> > only .config option needed as MFD_SYSCON is selected by Kconfig
> >> > already.
> 
> Similar to MFD_SYSCON, why don't we have REGULATOR_PBIAS to be
> selected when omap_hsmmc is being used?
> 
> It seems like that should also be a patch for the rc, right!?

Well selecting CONFIG_REGULATOR is still optional and force selecting
drivers usually leads into randconfig build issues sooner or later.
And we'd like to make everything into loadable modules eventually.

We could print a warning during MMC1 probe if REGULATOR_PBIAS
is not selected and attempt to continue probing?

Regards,

Tony

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-07 13:41                               ` Ulf Hansson
  2015-10-07 15:52                                 ` Tony Lindgren
@ 2015-10-07 17:53                                 ` Russell King - ARM Linux
  1 sibling, 0 replies; 35+ messages in thread
From: Russell King - ARM Linux @ 2015-10-07 17:53 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Oct 07, 2015 at 03:41:46PM +0200, Ulf Hansson wrote:
> > http://marc.info/?l=linux-omap&m=144422416621373&w=2
> > http://marc.info/?l=linux-omap&m=144422416921375&w=2
> 
> Russell, may I add your tested by tag for these?

You may -

Tested-by: Russell King <rmk+kernel@arm.linux.org.uk>

Thanks.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-07 15:52                                 ` Tony Lindgren
@ 2015-10-07 19:40                                   ` Ulf Hansson
  2015-10-07 23:13                                     ` Kishon Vijay Abraham I
  0 siblings, 1 reply; 35+ messages in thread
From: Ulf Hansson @ 2015-10-07 19:40 UTC (permalink / raw)
  To: linux-arm-kernel

On 7 October 2015 at 17:52, Tony Lindgren <tony@atomide.com> wrote:
> * Ulf Hansson <ulf.hansson@linaro.org> [151007 06:46]:
>> On 7 October 2015 at 15:26, Tony Lindgren <tony@atomide.com> wrote:
>> >> > Good idea, how about something like the following? AFAIK that's the
>> >> > only .config option needed as MFD_SYSCON is selected by Kconfig
>> >> > already.
>>
>> Similar to MFD_SYSCON, why don't we have REGULATOR_PBIAS to be
>> selected when omap_hsmmc is being used?
>>
>> It seems like that should also be a patch for the rc, right!?
>
> Well selecting CONFIG_REGULATOR is still optional and force selecting
> drivers usually leads into randconfig build issues sooner or later.
> And we'd like to make everything into loadable modules eventually.

I am not sure I get your point. Perhaps I was too vague in what I suggested.

Unless we express the dependencies via Kconfig files (or perhaps via
updated defconfigs), how do you expect build/boot automated tools to
handle this?

*People* can of course manually poll a README to learn about new
dependencies for each new kernel version, but me personally would
prefer if don't have to.

>
> We could print a warning during MMC1 probe if REGULATOR_PBIAS
> is not selected and attempt to continue probing?

No thanks, it would sprinkle drivers with ugly code :-). I would also
expect that we would get warnings even when we shouldn't, especially
as a cross SoC/board driver may have different dependencies.

To me REGULATOR_PBIAS is a dependency required by a certain SoC/board
when MMC_OMAP_HS is selected. Isn't such dependency easiest dealt with
from SoC/board Kconfig files? Llike for example in the OMAP2 case,
arch/arm/mach-omap2/Kconfig.

Kind regards
Uffe

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-07 19:40                                   ` Ulf Hansson
@ 2015-10-07 23:13                                     ` Kishon Vijay Abraham I
  2015-10-08  8:40                                       ` Tony Lindgren
  0 siblings, 1 reply; 35+ messages in thread
From: Kishon Vijay Abraham I @ 2015-10-07 23:13 UTC (permalink / raw)
  To: linux-arm-kernel

Hi,

On Thursday 08 October 2015 01:10 AM, Ulf Hansson wrote:
> On 7 October 2015 at 17:52, Tony Lindgren <tony@atomide.com> wrote:
>> * Ulf Hansson <ulf.hansson@linaro.org> [151007 06:46]:
>>> On 7 October 2015 at 15:26, Tony Lindgren <tony@atomide.com> wrote:
>>>>>> Good idea, how about something like the following? AFAIK that's the
>>>>>> only .config option needed as MFD_SYSCON is selected by Kconfig
>>>>>> already.
>>>
>>> Similar to MFD_SYSCON, why don't we have REGULATOR_PBIAS to be
>>> selected when omap_hsmmc is being used?
>>>
>>> It seems like that should also be a patch for the rc, right!?
>>
>> Well selecting CONFIG_REGULATOR is still optional and force selecting
>> drivers usually leads into randconfig build issues sooner or later.
>> And we'd like to make everything into loadable modules eventually.
> 
> I am not sure I get your point. Perhaps I was too vague in what I suggested.
> 
> Unless we express the dependencies via Kconfig files (or perhaps via
> updated defconfigs), how do you expect build/boot automated tools to
> handle this?

Both omap2plus_defconfig and multi_v7_defconfig has
CONFIG_PBIAS_REGULATOR enabled [1].

I think by using *depends on* in Kconfig, we'll end up in the same issue
faced by Russell (since even with that CONFIG_PBIAS_REGULATOR won't be
enabled) and using *select* can lead to randconfig errors.

[1] -> https://lkml.org/lkml/2015/9/3/119

Thanks
Kishon

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-06 19:57                               ` Russell King - ARM Linux
@ 2015-10-08  0:46                                 ` Kishon Vijay Abraham I
  0 siblings, 0 replies; 35+ messages in thread
From: Kishon Vijay Abraham I @ 2015-10-08  0:46 UTC (permalink / raw)
  To: linux-arm-kernel

Hi,

On Wednesday 07 October 2015 01:27 AM, Russell King - ARM Linux wrote:
> On Wed, Oct 07, 2015 at 12:59:29AM +0530, Kishon Vijay Abraham I wrote:
>> Hi,
>>
>> On Tuesday 06 October 2015 08:37 PM, Russell King - ARM Linux wrote:
>>> On Tue, Oct 06, 2015 at 04:06:09PM +0530, Kishon Vijay Abraham I wrote:
>>>> Hi,
>>>>
>>>> On Tuesday 06 October 2015 03:41 PM, Ulf Hansson wrote:
>>>>> On 6 October 2015 at 11:44, Tony Lindgren <tony@atomide.com> wrote:
>>>>>> * Russell King - ARM Linux <linux@arm.linux.org.uk> [151006 02:04]:
>>>>>>> On Mon, Oct 05, 2015 at 07:38:13PM +0100, Russell King - ARM Linux wrote:
>>>>>>>> On Mon, Oct 05, 2015 at 10:11:56AM -0700, Tony Lindgren wrote:
>>>>>>>>> * Tony Lindgren <tony@atomide.com> [151005 07:57]:
>>>>>>>>>> * Tony Lindgren <tony@atomide.com> [151005 07:44]:
>>>>>>>>>>> * Tony Lindgren <tony@atomide.com> [151005 04:28]:
>>>>>>>>>>>
>>>>>>>>>>> Based on some tests it seems that the duovero unpaired regulator usage
>>>>>>>>>>> is fixed by reverting:
>>>>>>>>>>>
>>>>>>>>>>> c55d7a055364 ("mmc: host: omap_hsmmc: use regulator_is_enabled to
>>>>>>>>>>> find pbias status")
>>>>>>>>>>
>>>>>>>>>> With commit c55d7a055364 my guess is that the PBIAS regulator is
>>>>>>>>>> already on from an earlier MMC probe and getting re-enabled when
>>>>>>>>>> a deferred probe happens?
>>>>>>>>>
>>>>>>>>> Unless somebody has a better fix in mind for the above, I suggest
>>>>>>>>> we revert it for the -rc kernel.
>>>>>>>>
>>>>>>>> Let me try reverting that in my build tree, and...
>>>>>>>>
>>>>>>>>>>> And it seems that omap3 legacy MMC is broken earlier in the
>>>>>>>>>>> series with:
>>>>>>>>>>>
>>>>>>>>>>> 7d607f917008 ("mmc: host: omap_hsmmc: use
>>>>>>>>>>> devm_regulator_get_optional() for vmmc")
>>>>>>>>>>>
>>>>>>>>>>> This one does not cleanly revert so have not yet tried reverting
>>>>>>>>>>> it.
>>>>>>>>>>
>>>>>>>>>> And with commit 7d607f917008 I'm guessing we can't return anHi,
>>>>>>>>>> error if the PBIAS regulator does not exist as that's not there
>>>>>>>>>> for the legacy booting.
>>>>>>>>>
>>>>>>>>> For omap3 legacy booting, we keep getting -EPROBE_DEFER for
>>>>>>>>> all the optional regulators.
>>>>>>>>>
>>>>>>>>> Something like the following might be the minimal fix for the -rc
>>>>>>>>> cycle?
>>>>>>>>
>>>>>>>> applying this patch.  If that gets things going again, then we
>>>>>>>> _definitely_ should get both of these to Linus ASAP.  The breakage
>>>>>>>> has been around far too long already.
>>>>>>>
>>>>>>> Last night's build shows that this fixes the non-DT LDP3430 booting, but
>>>>>>> DT-based LDP3430 and SDP4430 both remain broken for the same reason -
>>>>>>> neither can find their SD cards.
>>>>>>
>>>>>> Hmm DT-based boot finds the MMC card for LDP, dmesg below from DT boot [1].
>>>>>> Looks like you're on on -rc4 and not -rc3. My guess is that MMC is not
>>>>>> working for you with DT-based booting because you don't seem to have
>>>>>> CONFIG_REGULATOR_PBIAS in your seed config for. Care to try enabling that
>>>>>> for both your omap3 and omap4 seed config files?
>>>>>>
>>>>>>> We're at -rc4.  That means we're could be only three weeks away from 4.3
>>>>>>> being released, and DT OMAP has yet to boot _once_ here.
>>>>>>>
>>>>>>> What I find *totally* unacceptable is the lack of reaction from the MMC
>>>>>>> and TI people: it's just like "we'll break your crap, and we'll ignore
>>>>>>> reports of regressions."  That is *not* acceptable in any shape or form,
>>>>>>> and people who do this _should_ have their future patches ignored until
>>>>>>> they demonstrate that they understand the need to not break stuff.
>>>>>>>
>>>>>>> I'm at the point of suggesting that there should be a moritorium on all
>>>>>>> OMAP related development for 4.4: delay all development to 4.5, and have
>>>>>>> 4.4 as a pure bug-fixing _only_ cycle for OMAP.  If 4.3 is released and
>>>>>>> OMAP is still broken, then I don't think there's an option on that.
>>>>>>>
>>>>>>> Maybe the idea that development work won't hit mainline for another 4
>>>>>>> months will help focus the minds on not breaking stuff and then ignoring
>>>>>>> regression reports.
>>>>>>
>>>>>> I'm thinking along the same lines. In general, I do not and will not
>>>>>> apply any patches that are not fixes until all critical regressions are
>>>>>> out of the way. So not applying anything new for v4.4 until these MMC
>>>>>> regressions are fixed for v4.3.
>>>>>>
>>>>>
>>>>> Tony, Russell - just wanted to say thanks for putting a big effort in
>>>>> fixing this. I was expecting support from Kishon, but I guess he's
>>>>> busy.
>>>>>
>>>>> Unfortunate, I don't have any of the hardware that fails, otherwise I
>>>>> would of course be helping out more. The only thing I can think of is
>>>>> to revert the entire patchset I picked up for 4.3 from Kishon for the
>>>>> omap_hsmmc driver, but then I am not sure if that would cause other
>>>>> issues...
>>>>
>>>> Please don't do that as that will just mask lot of bugs in the
>>>> devicetree and might get MMC working. Indeed I was busy but I'll try to
>>>> get this resolved asap. The 2 pending issues as far as I know
>>>
>>> Sorry, but no.  None of this "mask bugs ... might" crap.  Either they're
>>> bug fixes, or they aren't bug fixes.  This should be a definite decision,
>>> not some vague crap.
>>
>> What I meant is the patches merged by Ulf *did* exposed bugs in device
>> tree (for instance a dt patch caused PBIAS regulator from not being
>> registered) and if those patches are reverted then a future dt breakage
>> might again be not caught.
> 
> That makes it even worse then.
> 
> What you seem to be saying now is that the old DT files were wrong, and
> you've fixed the DT files up in the latest kernel.  You've also changed
> the kernel in a way to make the old DT files fail.

I'll try to explain the changes that went in MMC driver, PBIAS regulator
driver and devicetree.

******
MMC controller in OMAP platforms uses PBIAS to know when the voltage to
the IO lines are stable. So PBIAS regulator device tree node was added
(as a child node of ocp), CONFIG_PBIAS_REGULATOR was added to
omap2plus_defconfig and code to get and enable pbias regulator was added
in MMC driver. But then error checking in MMC driver wasn't correct. It
was something like this

        reg = devm_regulator_get_optional(host->dev, "pbias");
        host->pbias = IS_ERR(reg) ? NULL : reg;

For *any* errors it just sets pbias to NULL and proceeds normally.

With this code the behavior is same if CONFIG_PBIAS_REGULATOR is either
set or unset. The behavior is same if pbias driver gets probed properly
or not.

******
Then certain modifications were done in devicetree in order to correctly
model users of syscon registers. With this pbias regulator device tree
node was made as child node of scm_conf node.

This resulted in pbias device not being created (fixed by [1]), probe of
pbias regulator driver fails while doing platform_get_resource (fixed by
[2]) and pbias regulator enable was failing (fixed by [3])

[1] -> https://lkml.org/lkml/2015/7/27/391
[2] -> https://lkml.org/lkml/2015/9/4/210
[3] -> https://lkml.org/lkml/2015/9/4/205

But even without any of these fixes MMC continued to work because error
checking in MMC driver was not proper.

******
Along with PBIAS fixes, patches were sent to clean-up and fix MMC [4].
With this series MMC fails on real failures like if "pbias-supply"
property is populated in MMC dt node, then MMC driver will fail if it
doesn't get the pbias regulator (which means now if
CONFIG_PBIAS_REGULATOR is not enabled, MMC driver will fail if it finds
pbias-supply property in devicetree node). Since omap2plus_defconfig
already had CONFIG_PBIAS_REGULATOR, it was added for multi_v7_defconfig [5].

However not adding this detail in *Documentation* resulted in MMC
failures and $subject.

[4] -> https://lkml.org/lkml/2015/8/27/122
[5] -> https://lkml.org/lkml/2015/9/3/119

> 
> That's totally unacceptable.  Users have the right to expect that old DT
> files work with later kernels.
> 
> It's fine to detect the broken DT files and print a warning, and have
> the kernel fix up the breakage, but it is unacceptable to decide that
> old DT files are broken and to then change stuff to prevent them working
> with later kernels.

No, we didn't break old DT files. We added new compatible string to
handle newer DT [6]

[6] -> https://lkml.org/lkml/2015/9/3/51

Thanks
Kishon

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-07 23:13                                     ` Kishon Vijay Abraham I
@ 2015-10-08  8:40                                       ` Tony Lindgren
  2015-10-08  9:35                                         ` Russell King - ARM Linux
  0 siblings, 1 reply; 35+ messages in thread
From: Tony Lindgren @ 2015-10-08  8:40 UTC (permalink / raw)
  To: linux-arm-kernel

* Kishon Vijay Abraham I <kishon@ti.com> [151007 16:18]:
> Hi,
> 
> On Thursday 08 October 2015 01:10 AM, Ulf Hansson wrote:
> > On 7 October 2015 at 17:52, Tony Lindgren <tony@atomide.com> wrote:
> >> * Ulf Hansson <ulf.hansson@linaro.org> [151007 06:46]:
> >>> On 7 October 2015 at 15:26, Tony Lindgren <tony@atomide.com> wrote:
> >>>>>> Good idea, how about something like the following? AFAIK that's the
> >>>>>> only .config option needed as MFD_SYSCON is selected by Kconfig
> >>>>>> already.
> >>>
> >>> Similar to MFD_SYSCON, why don't we have REGULATOR_PBIAS to be
> >>> selected when omap_hsmmc is being used?
> >>>
> >>> It seems like that should also be a patch for the rc, right!?
> >>
> >> Well selecting CONFIG_REGULATOR is still optional and force selecting
> >> drivers usually leads into randconfig build issues sooner or later.
> >> And we'd like to make everything into loadable modules eventually.
> > 
> > I am not sure I get your point. Perhaps I was too vague in what I suggested.
> > 
> > Unless we express the dependencies via Kconfig files (or perhaps via
> > updated defconfigs), how do you expect build/boot automated tools to
> > handle this?
> 
> Both omap2plus_defconfig and multi_v7_defconfig has
> CONFIG_PBIAS_REGULATOR enabled [1].
> 
> I think by using *depends on* in Kconfig, we'll end up in the same issue
> faced by Russell (since even with that CONFIG_PBIAS_REGULATOR won't be
> enabled) and using *select* can lead to randconfig errors.

Well the way distros deal with issues like this is have everything
possible as loadable modules. We should get the regulator_pbiaa
loaded automatically in that case as long as it's in the dts.. And
as long as we have the MODULE_DEVICE_TABLE entries right.

Maybe we could also use the composite device to indicate when MMC1
is using PBIAS regulator?

Regrads,

Tony

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-08  8:40                                       ` Tony Lindgren
@ 2015-10-08  9:35                                         ` Russell King - ARM Linux
  2015-10-08  9:56                                           ` Tony Lindgren
  0 siblings, 1 reply; 35+ messages in thread
From: Russell King - ARM Linux @ 2015-10-08  9:35 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Oct 08, 2015 at 01:40:21AM -0700, Tony Lindgren wrote:
> Well the way distros deal with issues like this is have everything
> possible as loadable modules. We should get the regulator_pbiaa
> loaded automatically in that case as long as it's in the dts.. And
> as long as we have the MODULE_DEVICE_TABLE entries right.

Assuming you have a rootfs which doesn't depend on one of those modules,
or an initramfs, and a way to get the built modules into that initial
filesystem.  Automated test boot systems do not have that luxury: the
generation of initramfs is distro specific, and would be very large if
it were to include all possible modules.

Some distros need their initramfs statically configured between "mount
a local rootfs" and "mount a nfs rootfs" and can't be changed once
generated.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-08  9:35                                         ` Russell King - ARM Linux
@ 2015-10-08  9:56                                           ` Tony Lindgren
  2015-10-08 10:00                                             ` Russell King - ARM Linux
  0 siblings, 1 reply; 35+ messages in thread
From: Tony Lindgren @ 2015-10-08  9:56 UTC (permalink / raw)
  To: linux-arm-kernel

* Russell King - ARM Linux <linux@arm.linux.org.uk> [151008 02:40]:
> On Thu, Oct 08, 2015 at 01:40:21AM -0700, Tony Lindgren wrote:
> > Well the way distros deal with issues like this is have everything
> > possible as loadable modules. We should get the regulator_pbiaa
> > loaded automatically in that case as long as it's in the dts.. And
> > as long as we have the MODULE_DEVICE_TABLE entries right.
> 
> Assuming you have a rootfs which doesn't depend on one of those modules,
> or an initramfs, and a way to get the built modules into that initial
> filesystem.  Automated test boot systems do not have that luxury: the
> generation of initramfs is distro specific, and would be very large if
> it were to include all possible modules.

Right, we need to keep the kernel usable in all these configurations
somehow.

For the kernel generated minimal initramfs we could grep for the
needed modules in the dts file.

> Some distros need their initramfs statically configured between "mount
> a local rootfs" and "mount a nfs rootfs" and can't be changed once
> generated.

And some devices need also EHCI for NFSroot :)

Regards,

Tony

^ permalink raw reply	[flat|nested] 35+ messages in thread

* All OMAP platforms: MMC is broken
  2015-10-08  9:56                                           ` Tony Lindgren
@ 2015-10-08 10:00                                             ` Russell King - ARM Linux
  0 siblings, 0 replies; 35+ messages in thread
From: Russell King - ARM Linux @ 2015-10-08 10:00 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Oct 08, 2015 at 02:56:36AM -0700, Tony Lindgren wrote:
> * Russell King - ARM Linux <linux@arm.linux.org.uk> [151008 02:40]:
> > On Thu, Oct 08, 2015 at 01:40:21AM -0700, Tony Lindgren wrote:
> > > Well the way distros deal with issues like this is have everything
> > > possible as loadable modules. We should get the regulator_pbiaa
> > > loaded automatically in that case as long as it's in the dts.. And
> > > as long as we have the MODULE_DEVICE_TABLE entries right.
> > 
> > Assuming you have a rootfs which doesn't depend on one of those modules,
> > or an initramfs, and a way to get the built modules into that initial
> > filesystem.  Automated test boot systems do not have that luxury: the
> > generation of initramfs is distro specific, and would be very large if
> > it were to include all possible modules.
> 
> Right, we need to keep the kernel usable in all these configurations
> somehow.
> 
> For the kernel generated minimal initramfs we could grep for the
> needed modules in the dts file.

Yes, you might be able to do that, but the only thing that can generate
the initramfs is the target system itself, or a similar target system.
For example, you can't sanely generate an initramfs for an ARM Ubuntu
system on an x86 Fedora host.

Distros have the luxury of always being able to do the initramfs
generation on the target system at installation or distro kernel update
time.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

^ permalink raw reply	[flat|nested] 35+ messages in thread

end of thread, other threads:[~2015-10-08 10:00 UTC | newest]

Thread overview: 35+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-09-24  9:00 All OMAP platforms: MMC is broken Russell King - ARM Linux
2015-09-24 22:51 ` Grygorii Strashko
2015-09-24 22:53   ` Russell King - ARM Linux
2015-09-24 23:37 ` Tony Lindgren
2015-09-25  1:13   ` Tony Lindgren
2015-09-26  2:54     ` Nishanth Menon
2015-10-01  9:33   ` Russell King - ARM Linux
2015-10-01  9:50     ` Ulf Hansson
2015-10-01 10:03       ` Russell King - ARM Linux
2015-10-05 11:23         ` Tony Lindgren
2015-10-05 14:35           ` Tony Lindgren
2015-10-05 14:51             ` Tony Lindgren
2015-10-05 17:11               ` Tony Lindgren
2015-10-05 18:38                 ` Russell King - ARM Linux
2015-10-06  9:00                   ` Russell King - ARM Linux
2015-10-06  9:44                     ` Tony Lindgren
2015-10-06 10:11                       ` Ulf Hansson
2015-10-06 10:36                         ` Kishon Vijay Abraham I
2015-10-06 15:07                           ` Russell King - ARM Linux
2015-10-06 19:29                             ` Kishon Vijay Abraham I
2015-10-06 19:57                               ` Russell King - ARM Linux
2015-10-08  0:46                                 ` Kishon Vijay Abraham I
2015-10-06 15:00                       ` Russell King - ARM Linux
2015-10-06 15:37                         ` Tony Lindgren
2015-10-07 12:45                           ` Russell King - ARM Linux
2015-10-07 13:26                             ` Tony Lindgren
2015-10-07 13:41                               ` Ulf Hansson
2015-10-07 15:52                                 ` Tony Lindgren
2015-10-07 19:40                                   ` Ulf Hansson
2015-10-07 23:13                                     ` Kishon Vijay Abraham I
2015-10-08  8:40                                       ` Tony Lindgren
2015-10-08  9:35                                         ` Russell King - ARM Linux
2015-10-08  9:56                                           ` Tony Lindgren
2015-10-08 10:00                                             ` Russell King - ARM Linux
2015-10-07 17:53                                 ` Russell King - ARM Linux

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).