[PATCH v3 0/5] Fix some race conditions that exists between fbmem and sysfb

[PATCH v3 0/5] Fix some race conditions that exists between fbmem and sysfb

[Javier Martinez Canillas]

Hello,

The patches in this series are mostly changes suggested by Daniel Vetter
to fix some race conditions that exists between the fbdev core (fbmem)
and sysfb with regard to device registration and removal.

For example, it is currently possible for sysfb to register a platform
device after a real DRM driver was registered and requested to remove the
conflicting framebuffers.

A symptom of this issue, was worked around with by commit fb561bf9abde
("fbdev: Prevent probing generic drivers if a FB is already registered")
but that's really a hack and should be reverted.

This series attempt to fix it more properly and revert the mentioned hack.
That will also unblock a pending patch to not make the num_registered_fb
variable visible to drivers anymore, since that's internal to fbdev core.

Patch #1 is just a trivial preparatory change.

Patch #2 add sysfb_disable() and sysfb_try_unregister() helpers for fbmem
to use them.

Patch #3 changes how is dealt with conflicting framebuffers unregistering,
rather than having a variable to determine if a lock should be take, it
just drops the lock before unregistering the platform device.

Patch #4 fixes the mentioned race conditions and finally patch #5 is the
revert patch that was posted by Daniel before but he dropped from his set.

The patches were tested on a rpi4 using different video configurations:
(simpledrm -> vc4 both builtin, only vc4 builtin, only simpledrm builtin
and simpledrm builtin with vc4 built as a module).

Best regards,
Javier

Changes in v3:
- Rebase on top of latest drm-misc-next branch.

Changes in v2:
- Rebase on top of latest drm-misc-next and fix conflicts (Daniel Vetter).
- Add kernel-doc comments and include in other_interfaces.rst (Daniel Vetter).
- Explain in the commit message that fbmem has to unregister the device
  as fallback if a driver registered the device itself (Daniel Vetter).
- Also explain that fallback in a comment in the code (Daniel Vetter).
- Don't encode in fbmem the assumption that sysfb will always register
  platform devices (Daniel Vetter).
- Add a FIXME comment about drivers registering devices (Daniel Vetter).
- Drop RFC prefix since patches were already reviewed by Daniel Vetter.
- Add Daniel Reviewed-by tags to the patches.

Daniel Vetter (1):
  Revert "fbdev: Prevent probing generic drivers if a FB is already
    registered"

Javier Martinez Canillas (4):
  firmware: sysfb: Make sysfb_create_simplefb() return a pdev pointer
  firmware: sysfb: Add helpers to unregister a pdev and disable
    registration
  fbdev: Restart conflicting fb removal loop when unregistering devices
  fbdev: Fix some race conditions between fbmem and sysfb

 .../driver-api/firmware/other_interfaces.rst  |  6 ++
 drivers/firmware/sysfb.c                      | 77 +++++++++++++++++--
 drivers/firmware/sysfb_simplefb.c             | 16 ++--
 drivers/video/fbdev/core/fbmem.c              | 62 ++++++++++++---
 drivers/video/fbdev/efifb.c                   | 11 ---
 drivers/video/fbdev/simplefb.c                | 11 ---
 include/linux/fb.h                            |  1 -
 include/linux/sysfb.h                         | 29 +++++--
 8 files changed, 158 insertions(+), 55 deletions(-)

-- 
2.35.1

[PATCH v3 3/5] fbdev: Restart conflicting fb removal loop when unregistering devices

[Javier Martinez Canillas]

Drivers that want to remove registered conflicting framebuffers prior to
register their own framebuffer, calls remove_conflicting_framebuffers().

This function takes the registration_lock mutex, to prevent a races when
drivers register framebuffer devices. But if a conflicting framebuffer
device is found, the underlaying platform device is unregistered and this
will lead to the platform driver .remove callback to be called, which in
turn will call to the unregister_framebuffer() that takes the same lock.

To prevent this, a struct fb_info.forced_out field was used as indication
to unregister_framebuffer() whether the mutex has to be grabbed or not.

A cleaner solution is to drop the lock before platform_device_unregister()
so unregister_framebuffer() can take it when called from the fbdev driver,
and just grab the lock again after the device has been registered and do
a removal loop restart.

Since the framebuffer devices will already be removed, the loop would just
finish when no more conflicting framebuffers are found.

Suggested-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Javier Martinez Canillas <javierm@redhat.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
---

(no changes since v1)

 drivers/video/fbdev/core/fbmem.c | 22 +++++++++++++++-------
 include/linux/fb.h               |  1 -
 2 files changed, 15 insertions(+), 8 deletions(-)

diff --git a/drivers/video/fbdev/core/fbmem.c b/drivers/video/fbdev/core/fbmem.c
index 84427470367b..0bb459258df3 100644
--- a/drivers/video/fbdev/core/fbmem.c
+++ b/drivers/video/fbdev/core/fbmem.c
@@ -1553,6 +1553,7 @@ static void do_remove_conflicting_framebuffers(struct apertures_struct *a,
 {
 	int i;
 
+restart_removal:
 	/* check all firmware fbs and kick off if the base addr overlaps */
 	for_each_registered_fb(i) {
 		struct apertures_struct *gen_aper;
@@ -1585,12 +1586,23 @@ static void do_remove_conflicting_framebuffers(struct apertures_struct *a,
 				pr_warn("fb%d: no device set\n", i);
 				do_unregister_framebuffer(registered_fb[i]);
 			} else if (dev_is_platform(device)) {
-				registered_fb[i]->forced_out = true;
+				/*
+				 * Drop the lock because if the device is unregistered, its
+				 * driver will call to unregister_framebuffer(), that takes
+				 * this lock.
+				 */
+				mutex_unlock(&registration_lock);
 				platform_device_unregister(to_platform_device(device));
+				mutex_lock(&registration_lock);
 			} else {
 				pr_warn("fb%d: cannot remove device\n", i);
 				do_unregister_framebuffer(registered_fb[i]);
 			}
+			/*
+			 * Restart the removal loop now that the device has been
+			 * unregistered and its associated framebuffer gone.
+			 */
+			goto restart_removal;
 		}
 	}
 }
@@ -1897,13 +1909,9 @@ EXPORT_SYMBOL(register_framebuffer);
 void
 unregister_framebuffer(struct fb_info *fb_info)
 {
-	bool forced_out = fb_info->forced_out;
-
-	if (!forced_out)
-		mutex_lock(&registration_lock);
+	mutex_lock(&registration_lock);
 	do_unregister_framebuffer(fb_info);
-	if (!forced_out)
-		mutex_unlock(&registration_lock);
+	mutex_unlock(&registration_lock);
 }
 EXPORT_SYMBOL(unregister_framebuffer);
 
diff --git a/include/linux/fb.h b/include/linux/fb.h
index f95da1af9ff6..b781bc721113 100644
--- a/include/linux/fb.h
+++ b/include/linux/fb.h
@@ -502,7 +502,6 @@ struct fb_info {
 	} *apertures;
 
 	bool skip_vt_switch; /* no VT switch on suspend/resume required */
-	bool forced_out; /* set when being removed by another driver */
 };
 
 static inline struct apertures_struct *alloc_apertures(unsigned int max_num) {
-- 
2.35.1

[PATCH v3 4/5] fbdev: Fix some race conditions between fbmem and sysfb

[Javier Martinez Canillas]

The platform devices registered in sysfb match with a firmware-based fbdev
or DRM driver, that are used to have early graphics using framebuffers set
up by the system firmware.

Real DRM drivers later are probed and remove all conflicting framebuffers,
leading to these platform devices for generic drivers to be unregistered.

But the current solution has two issues that this patch fixes:

1) It is a layering violation for the fbdev core to unregister a device
   that was registered by sysfb.

   Instead, the sysfb_try_unregister() helper function can be called for
   sysfb to attempt unregistering the device if is the one registered.

2) The sysfb_init() function could be called after a DRM driver is probed
   and requested to unregister devices for drivers with a conflicting fb.

   To prevent this, disable any future sysfb platform device registration
   by calling sysfb_disable(), if a driver requested to remove conflicting
   framebuffers with remove_conflicting_framebuffers().

There are video drivers (e.g: vga16fb) that register their own device and
don't use the sysfb infrastructure for that, so an unregistration has to
be forced by fbmem if sysfb_try_unregister() fails to do the unregister.

Suggested-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Javier Martinez Canillas <javierm@redhat.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
---

(no changes since v2)

Changes in v2:
- Explain in the commit message that fbmem has to unregister the device
  as fallback if a driver registered the device itself (Daniel Vetter).
- Also explain that fallback in a comment in the code (Daniel Vetter).
- Don't encode in fbmem the assumption that sysfb will always register
  platform devices (Daniel Vetter).
- Add a FIXME comment about drivers registering devices (Daniel Vetter).

 drivers/video/fbdev/core/fbmem.c | 42 ++++++++++++++++++++++++++++----
 1 file changed, 37 insertions(+), 5 deletions(-)

diff --git a/drivers/video/fbdev/core/fbmem.c b/drivers/video/fbdev/core/fbmem.c
index 0bb459258df3..8098305879f8 100644
--- a/drivers/video/fbdev/core/fbmem.c
+++ b/drivers/video/fbdev/core/fbmem.c
@@ -19,6 +19,7 @@
 #include <linux/kernel.h>
 #include <linux/major.h>
 #include <linux/slab.h>
+#include <linux/sysfb.h>
 #include <linux/mm.h>
 #include <linux/mman.h>
 #include <linux/vt.h>
@@ -1585,18 +1586,38 @@ static void do_remove_conflicting_framebuffers(struct apertures_struct *a,
 			if (!device) {
 				pr_warn("fb%d: no device set\n", i);
 				do_unregister_framebuffer(registered_fb[i]);
-			} else if (dev_is_platform(device)) {
+			} else {
 				/*
 				 * Drop the lock because if the device is unregistered, its
 				 * driver will call to unregister_framebuffer(), that takes
 				 * this lock.
 				 */
 				mutex_unlock(&registration_lock);
-				platform_device_unregister(to_platform_device(device));
+				/*
+				 * First attempt the device to be unregistered by sysfb.
+				 */
+				if (!sysfb_try_unregister(device)) {
+					if (dev_is_platform(device)) {
+						/*
+						 * FIXME: sysfb didn't register this device, is a platform
+						 * device registered by a video driver (e.g: vga16fb), so
+						 * force its unregistration here. A proper fix would be to
+						 * move all device registration to the sysfb infrastructure
+						 * or platform code.
+						 */
+						platform_device_unregister(to_platform_device(device));
+					} else {
+						/*
+						 * If is not a platform device, at least print a warning. A
+						 * fix would add to make the code that registered the device
+						 * to also unregister it.
+						 */
+						pr_warn("fb%d: cannot remove device\n", i);
+						/* call unregister_framebuffer() since the lock was dropped */
+						unregister_framebuffer(registered_fb[i]);
+					}
+				}
 				mutex_lock(&registration_lock);
-			} else {
-				pr_warn("fb%d: cannot remove device\n", i);
-				do_unregister_framebuffer(registered_fb[i]);
 			}
 			/*
 			 * Restart the removal loop now that the device has been
@@ -1762,6 +1783,17 @@ int remove_conflicting_framebuffers(struct apertures_struct *a,
 		do_free = true;
 	}
 
+	/*
+	 * If a driver asked to unregister a platform device registered by
+	 * sysfb, then can be assumed that this is a driver for a display
+	 * that is set up by the system firmware and has a generic driver.
+	 *
+	 * Drivers for devices that don't have a generic driver will never
+	 * ask for this, so let's assume that a real driver for the display
+	 * was already probed and prevent sysfb to register devices later.
+	 */
+	sysfb_disable();
+
 	mutex_lock(&registration_lock);
 	do_remove_conflicting_framebuffers(a, name, primary);
 	mutex_unlock(&registration_lock);
-- 
2.35.1

Re: [PATCH v3 0/5] Fix some race conditions that exists between fbmem and sysfb

[Greg Kroah-Hartman]

On Wed, Apr 20, 2022 at 10:52:58AM +0200, Javier Martinez Canillas wrote:
> Hello,
> 
> The patches in this series are mostly changes suggested by Daniel Vetter
> to fix some race conditions that exists between the fbdev core (fbmem)
> and sysfb with regard to device registration and removal.
> 
> For example, it is currently possible for sysfb to register a platform
> device after a real DRM driver was registered and requested to remove the
> conflicting framebuffers.
> 
> A symptom of this issue, was worked around with by commit fb561bf9abde
> ("fbdev: Prevent probing generic drivers if a FB is already registered")
> but that's really a hack and should be reverted.
> 
> This series attempt to fix it more properly and revert the mentioned hack.
> That will also unblock a pending patch to not make the num_registered_fb
> variable visible to drivers anymore, since that's internal to fbdev core.
> 
> Patch #1 is just a trivial preparatory change.
> 
> Patch #2 add sysfb_disable() and sysfb_try_unregister() helpers for fbmem
> to use them.
> 
> Patch #3 changes how is dealt with conflicting framebuffers unregistering,
> rather than having a variable to determine if a lock should be take, it
> just drops the lock before unregistering the platform device.
> 
> Patch #4 fixes the mentioned race conditions and finally patch #5 is the
> revert patch that was posted by Daniel before but he dropped from his set.
> 
> The patches were tested on a rpi4 using different video configurations:
> (simpledrm -> vc4 both builtin, only vc4 builtin, only simpledrm builtin
> and simpledrm builtin with vc4 built as a module).
> 
> Best regards,
> Javier

Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

Re: [PATCH v3 3/5] fbdev: Restart conflicting fb removal loop when unregistering devices

[Thomas Zimmermann]

[-- Attachment #1.1: Type: text/plain, Size: 4416 bytes --]

Hi

Am 20.04.22 um 10:53 schrieb Javier Martinez Canillas:
> Drivers that want to remove registered conflicting framebuffers prior to
> register their own framebuffer, calls remove_conflicting_framebuffers().
> 
> This function takes the registration_lock mutex, to prevent a races when
> drivers register framebuffer devices. But if a conflicting framebuffer
> device is found, the underlaying platform device is unregistered and this
> will lead to the platform driver .remove callback to be called, which in
> turn will call to the unregister_framebuffer() that takes the same lock.
> 
> To prevent this, a struct fb_info.forced_out field was used as indication
> to unregister_framebuffer() whether the mutex has to be grabbed or not.
> 
> A cleaner solution is to drop the lock before platform_device_unregister()
> so unregister_framebuffer() can take it when called from the fbdev driver,
> and just grab the lock again after the device has been registered and do
> a removal loop restart.

I don't see how this patch improves the situation. So far, 
do_remove_conflicting_framebuffers() had no business in maintaining 
locks. And now it's doing this in in a goto-loop where it keeps 
getting/dropping locks.  That's asking for bugs IMHO.

Best regards
Thomas

> 
> Since the framebuffer devices will already be removed, the loop would just
> finish when no more conflicting framebuffers are found.
> 
> Suggested-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> Signed-off-by: Javier Martinez Canillas <javierm@redhat.com>
> Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> ---
> 
> (no changes since v1)
> 
>   drivers/video/fbdev/core/fbmem.c | 22 +++++++++++++++-------
>   include/linux/fb.h               |  1 -
>   2 files changed, 15 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/video/fbdev/core/fbmem.c b/drivers/video/fbdev/core/fbmem.c
> index 84427470367b..0bb459258df3 100644
> --- a/drivers/video/fbdev/core/fbmem.c
> +++ b/drivers/video/fbdev/core/fbmem.c
> @@ -1553,6 +1553,7 @@ static void do_remove_conflicting_framebuffers(struct apertures_struct *a,
>   {
>   	int i;
>   
> +restart_removal:
>   	/* check all firmware fbs and kick off if the base addr overlaps */
>   	for_each_registered_fb(i) {
>   		struct apertures_struct *gen_aper;
> @@ -1585,12 +1586,23 @@ static void do_remove_conflicting_framebuffers(struct apertures_struct *a,
>   				pr_warn("fb%d: no device set\n", i);
>   				do_unregister_framebuffer(registered_fb[i]);
>   			} else if (dev_is_platform(device)) {
> -				registered_fb[i]->forced_out = true;
> +				/*
> +				 * Drop the lock because if the device is unregistered, its
> +				 * driver will call to unregister_framebuffer(), that takes
> +				 * this lock.
> +				 */
> +				mutex_unlock(&registration_lock);
>   				platform_device_unregister(to_platform_device(device));
> +				mutex_lock(&registration_lock);
>   			} else {
>   				pr_warn("fb%d: cannot remove device\n", i);
>   				do_unregister_framebuffer(registered_fb[i]);
>   			}
> +			/*
> +			 * Restart the removal loop now that the device has been
> +			 * unregistered and its associated framebuffer gone.
> +			 */
> +			goto restart_removal;
>   		}
>   	}
>   }
> @@ -1897,13 +1909,9 @@ EXPORT_SYMBOL(register_framebuffer);
>   void
>   unregister_framebuffer(struct fb_info *fb_info)
>   {
> -	bool forced_out = fb_info->forced_out;
> -
> -	if (!forced_out)
> -		mutex_lock(&registration_lock);
> +	mutex_lock(&registration_lock);
>   	do_unregister_framebuffer(fb_info);
> -	if (!forced_out)
> -		mutex_unlock(&registration_lock);
> +	mutex_unlock(&registration_lock);
>   }
>   EXPORT_SYMBOL(unregister_framebuffer);
>   
> diff --git a/include/linux/fb.h b/include/linux/fb.h
> index f95da1af9ff6..b781bc721113 100644
> --- a/include/linux/fb.h
> +++ b/include/linux/fb.h
> @@ -502,7 +502,6 @@ struct fb_info {
>   	} *apertures;
>   
>   	bool skip_vt_switch; /* no VT switch on suspend/resume required */
> -	bool forced_out; /* set when being removed by another driver */
>   };
>   
>   static inline struct apertures_struct *alloc_apertures(unsigned int max_num) {

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Ivo Totev

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: [PATCH v3 4/5] fbdev: Fix some race conditions between fbmem and sysfb

[Thomas Zimmermann]

[-- Attachment #1.1: Type: text/plain, Size: 5696 bytes --]

Hi

Am 20.04.22 um 10:53 schrieb Javier Martinez Canillas:
> The platform devices registered in sysfb match with a firmware-based fbdev
> or DRM driver, that are used to have early graphics using framebuffers set
> up by the system firmware.
> 
> Real DRM drivers later are probed and remove all conflicting framebuffers,
> leading to these platform devices for generic drivers to be unregistered.
> 
> But the current solution has two issues that this patch fixes:
> 
> 1) It is a layering violation for the fbdev core to unregister a device
>     that was registered by sysfb.

Why? We do this elsewhere and it works nicely.

> 
>     Instead, the sysfb_try_unregister() helper function can be called for
>     sysfb to attempt unregistering the device if is the one registered.

And sysfb_try_unregister() is really just a glorified version of 
platform_device_unregister() IMHO.

> 
> 2) The sysfb_init() function could be called after a DRM driver is probed
>     and requested to unregister devices for drivers with a conflicting fb.
> 
>     To prevent this, disable any future sysfb platform device registration
>     by calling sysfb_disable(), if a driver requested to remove conflicting
>     framebuffers with remove_conflicting_framebuffers().

As I mentioned in another comment, as soon as there's anything else than 
EFI/VESA using the sysfb code the unregistering step is likely to break 
in some way.

Best regards
Thomas

> 
> There are video drivers (e.g: vga16fb) that register their own device and
> don't use the sysfb infrastructure for that, so an unregistration has to
> be forced by fbmem if sysfb_try_unregister() fails to do the unregister.
> 
> Suggested-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> Signed-off-by: Javier Martinez Canillas <javierm@redhat.com>
> Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> ---
> 
> (no changes since v2)
> 
> Changes in v2:
> - Explain in the commit message that fbmem has to unregister the device
>    as fallback if a driver registered the device itself (Daniel Vetter).
> - Also explain that fallback in a comment in the code (Daniel Vetter).
> - Don't encode in fbmem the assumption that sysfb will always register
>    platform devices (Daniel Vetter).
> - Add a FIXME comment about drivers registering devices (Daniel Vetter).
> 
>   drivers/video/fbdev/core/fbmem.c | 42 ++++++++++++++++++++++++++++----
>   1 file changed, 37 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/video/fbdev/core/fbmem.c b/drivers/video/fbdev/core/fbmem.c
> index 0bb459258df3..8098305879f8 100644
> --- a/drivers/video/fbdev/core/fbmem.c
> +++ b/drivers/video/fbdev/core/fbmem.c
> @@ -19,6 +19,7 @@
>   #include <linux/kernel.h>
>   #include <linux/major.h>
>   #include <linux/slab.h>
> +#include <linux/sysfb.h>
>   #include <linux/mm.h>
>   #include <linux/mman.h>
>   #include <linux/vt.h>
> @@ -1585,18 +1586,38 @@ static void do_remove_conflicting_framebuffers(struct apertures_struct *a,
>   			if (!device) {
>   				pr_warn("fb%d: no device set\n", i);
>   				do_unregister_framebuffer(registered_fb[i]);
> -			} else if (dev_is_platform(device)) {
> +			} else {
>   				/*
>   				 * Drop the lock because if the device is unregistered, its
>   				 * driver will call to unregister_framebuffer(), that takes
>   				 * this lock.
>   				 */
>   				mutex_unlock(&registration_lock);
> -				platform_device_unregister(to_platform_device(device));
> +				/*
> +				 * First attempt the device to be unregistered by sysfb.
> +				 */
> +				if (!sysfb_try_unregister(device)) {
> +					if (dev_is_platform(device)) {
> +						/*
> +						 * FIXME: sysfb didn't register this device, is a platform
> +						 * device registered by a video driver (e.g: vga16fb), so
> +						 * force its unregistration here. A proper fix would be to
> +						 * move all device registration to the sysfb infrastructure
> +						 * or platform code.
> +						 */
> +						platform_device_unregister(to_platform_device(device));
> +					} else {
> +						/*
> +						 * If is not a platform device, at least print a warning. A
> +						 * fix would add to make the code that registered the device
> +						 * to also unregister it.
> +						 */
> +						pr_warn("fb%d: cannot remove device\n", i);
> +						/* call unregister_framebuffer() since the lock was dropped */
> +						unregister_framebuffer(registered_fb[i]);
> +					}
> +				}
>   				mutex_lock(&registration_lock);
> -			} else {
> -				pr_warn("fb%d: cannot remove device\n", i);
> -				do_unregister_framebuffer(registered_fb[i]);
>   			}
>   			/*
>   			 * Restart the removal loop now that the device has been
> @@ -1762,6 +1783,17 @@ int remove_conflicting_framebuffers(struct apertures_struct *a,
>   		do_free = true;
>   	}
>   
> +	/*
> +	 * If a driver asked to unregister a platform device registered by
> +	 * sysfb, then can be assumed that this is a driver for a display
> +	 * that is set up by the system firmware and has a generic driver.
> +	 *
> +	 * Drivers for devices that don't have a generic driver will never
> +	 * ask for this, so let's assume that a real driver for the display
> +	 * was already probed and prevent sysfb to register devices later.
> +	 */
> +	sysfb_disable();
> +
>   	mutex_lock(&registration_lock);
>   	do_remove_conflicting_framebuffers(a, name, primary);
>   	mutex_unlock(&registration_lock);

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Ivo Totev

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: [PATCH v3 3/5] fbdev: Restart conflicting fb removal loop when unregistering devices

[Javier Martinez Canillas]

Hello Thomas,

Thanks for the feedback.

On 4/25/22 10:27, Thomas Zimmermann wrote:
> Hi
> 
> Am 20.04.22 um 10:53 schrieb Javier Martinez Canillas:
>> Drivers that want to remove registered conflicting framebuffers prior to
>> register their own framebuffer, calls remove_conflicting_framebuffers().
>>
>> This function takes the registration_lock mutex, to prevent a races when
>> drivers register framebuffer devices. But if a conflicting framebuffer
>> device is found, the underlaying platform device is unregistered and this
>> will lead to the platform driver .remove callback to be called, which in
>> turn will call to the unregister_framebuffer() that takes the same lock.
>>
>> To prevent this, a struct fb_info.forced_out field was used as indication
>> to unregister_framebuffer() whether the mutex has to be grabbed or not.
>>
>> A cleaner solution is to drop the lock before platform_device_unregister()
>> so unregister_framebuffer() can take it when called from the fbdev driver,
>> and just grab the lock again after the device has been registered and do
>> a removal loop restart.
> 
> I don't see how this patch improves the situation. So far, 
> do_remove_conflicting_framebuffers() had no business in maintaining 
> locks. And now it's doing this in in a goto-loop where it keeps 
> getting/dropping locks.  That's asking for bugs IMHO.
>

It's true that do_remove_conflicting_framebuffers() gets more complicated
with all the locks release/re-acquire but OTOH unregister_framebuffer()
doesn't do conditionally locking, and more importantly the drivers .remove
callback isn't called with the lock held, which IMHO is also quite fragile.

-- 
Best regards,

Javier Martinez Canillas
Linux Engineering
Red Hat

Re: [PATCH v3 0/5] Fix some race conditions that exists between fbmem and sysfb

[Thomas Zimmermann]

[-- Attachment #1.1: Type: text/plain, Size: 4794 bytes --]

Hi

Am 20.04.22 um 10:52 schrieb Javier Martinez Canillas:
> Hello,
> 
> The patches in this series are mostly changes suggested by Daniel Vetter
> to fix some race conditions that exists between the fbdev core (fbmem)
> and sysfb with regard to device registration and removal.
> 
> For example, it is currently possible for sysfb to register a platform
> device after a real DRM driver was registered and requested to remove the
> conflicting framebuffers.
> 
> A symptom of this issue, was worked around with by commit fb561bf9abde
> ("fbdev: Prevent probing generic drivers if a FB is already registered")
> but that's really a hack and should be reverted.

As I mentioned on IRC, I think this series should be merged for the 
reasons I give in the other comments.

> 
> This series attempt to fix it more properly and revert the mentioned hack.
> That will also unblock a pending patch to not make the num_registered_fb
> variable visible to drivers anymore, since that's internal to fbdev core.

Here's as far as I understand the problem:

  1) build DRM/fbdev and sysfb code into the kernel
  2) during boot, load the DRM/fbdev modules and have them acquire I/O 
ranges
  3) afterwards load sysfb and have it register platform devices for the 
generic framebuffers
  4) these devices now conflict with the already-registered DRM/fbdev 
devices

If that is the problem here, let's simply set a sysfb_disable flag in 
sysfb code when the first DRM/fbdev driver first loads. With the flag 
set, sysfb won't create any platform devices. We assume that there are 
now DRM/fbdev drivers for the framebuffers and sysfb won't be needed.

We can set the flag internally from drm_aperture_detach_drivers() [1] 
and do_remove_conflicting_framebuffers() [2].

Best regards
Thomas

[1] 
https://elixir.bootlin.com/linux/v5.17.4/source/drivers/gpu/drm/drm_aperture.c#L253
[2] 
https://elixir.bootlin.com/linux/v5.17.4/source/drivers/video/fbdev/core/fbmem.c#L1559

> 
> Patch #1 is just a trivial preparatory change.
> 
> Patch #2 add sysfb_disable() and sysfb_try_unregister() helpers for fbmem
> to use them.
> 
> Patch #3 changes how is dealt with conflicting framebuffers unregistering,
> rather than having a variable to determine if a lock should be take, it
> just drops the lock before unregistering the platform device.
> 
> Patch #4 fixes the mentioned race conditions and finally patch #5 is the
> revert patch that was posted by Daniel before but he dropped from his set.
> 
> The patches were tested on a rpi4 using different video configurations:
> (simpledrm -> vc4 both builtin, only vc4 builtin, only simpledrm builtin
> and simpledrm builtin with vc4 built as a module).
> 
> Best regards,
> Javier
> 
> Changes in v3:
> - Rebase on top of latest drm-misc-next branch.
> 
> Changes in v2:
> - Rebase on top of latest drm-misc-next and fix conflicts (Daniel Vetter).
> - Add kernel-doc comments and include in other_interfaces.rst (Daniel Vetter).
> - Explain in the commit message that fbmem has to unregister the device
>    as fallback if a driver registered the device itself (Daniel Vetter).
> - Also explain that fallback in a comment in the code (Daniel Vetter).
> - Don't encode in fbmem the assumption that sysfb will always register
>    platform devices (Daniel Vetter).
> - Add a FIXME comment about drivers registering devices (Daniel Vetter).
> - Drop RFC prefix since patches were already reviewed by Daniel Vetter.
> - Add Daniel Reviewed-by tags to the patches.
> 
> Daniel Vetter (1):
>    Revert "fbdev: Prevent probing generic drivers if a FB is already
>      registered"
> 
> Javier Martinez Canillas (4):
>    firmware: sysfb: Make sysfb_create_simplefb() return a pdev pointer
>    firmware: sysfb: Add helpers to unregister a pdev and disable
>      registration
>    fbdev: Restart conflicting fb removal loop when unregistering devices
>    fbdev: Fix some race conditions between fbmem and sysfb
> 
>   .../driver-api/firmware/other_interfaces.rst  |  6 ++
>   drivers/firmware/sysfb.c                      | 77 +++++++++++++++++--
>   drivers/firmware/sysfb_simplefb.c             | 16 ++--
>   drivers/video/fbdev/core/fbmem.c              | 62 ++++++++++++---
>   drivers/video/fbdev/efifb.c                   | 11 ---
>   drivers/video/fbdev/simplefb.c                | 11 ---
>   include/linux/fb.h                            |  1 -
>   include/linux/sysfb.h                         | 29 +++++--
>   8 files changed, 158 insertions(+), 55 deletions(-)
> 

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Ivo Totev

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: [PATCH v3 0/5] Fix some race conditions that exists between fbmem and sysfb

[Thomas Zimmermann]

[-- Attachment #1.1: Type: text/plain, Size: 5463 bytes --]

Hi

Am 25.04.22 um 10:54 schrieb Thomas Zimmermann:
> Hi
> 
> Am 20.04.22 um 10:52 schrieb Javier Martinez Canillas:
>> Hello,
>>
>> The patches in this series are mostly changes suggested by Daniel Vetter
>> to fix some race conditions that exists between the fbdev core (fbmem)
>> and sysfb with regard to device registration and removal.
>>
>> For example, it is currently possible for sysfb to register a platform
>> device after a real DRM driver was registered and requested to remove the
>> conflicting framebuffers.
>>
>> A symptom of this issue, was worked around with by commit fb561bf9abde
>> ("fbdev: Prevent probing generic drivers if a FB is already registered")
>> but that's really a hack and should be reverted.
> 
> As I mentioned on IRC, I think this series should be merged for the 
> reasons I give in the other comments.
> 
>>
>> This series attempt to fix it more properly and revert the mentioned 
>> hack.
>> That will also unblock a pending patch to not make the num_registered_fb
>> variable visible to drivers anymore, since that's internal to fbdev core.
> 
> Here's as far as I understand the problem:
> 
>   1) build DRM/fbdev and sysfb code into the kernel
>   2) during boot, load the DRM/fbdev modules and have them acquire I/O 
> ranges
>   3) afterwards load sysfb and have it register platform devices for the 
> generic framebuffers
>   4) these devices now conflict with the already-registered DRM/fbdev 
> devices
> 
> If that is the problem here, let's simply set a sysfb_disable flag in 
> sysfb code when the first DRM/fbdev driver first loads. With the flag 
> set, sysfb won't create any platform devices. We assume that there are 
> now DRM/fbdev drivers for the framebuffers and sysfb won't be needed.
> 
> We can set the flag internally from drm_aperture_detach_drivers() [1] 
> and do_remove_conflicting_framebuffers() [2].

And further thinking about it, it would be better to set such a flag 
after successfully registering a DRM/fbdev device.  So we know that 
there's at least one working display in the system. We don't have to 
rely on generic framebuffers after that.

Best regards
Thomas

> 
> Best regards
> Thomas
> 
> [1] 
> https://elixir.bootlin.com/linux/v5.17.4/source/drivers/gpu/drm/drm_aperture.c#L253 
> 
> [2] 
> https://elixir.bootlin.com/linux/v5.17.4/source/drivers/video/fbdev/core/fbmem.c#L1559 
> 
> 
>>
>> Patch #1 is just a trivial preparatory change.
>>
>> Patch #2 add sysfb_disable() and sysfb_try_unregister() helpers for fbmem
>> to use them.
>>
>> Patch #3 changes how is dealt with conflicting framebuffers 
>> unregistering,
>> rather than having a variable to determine if a lock should be take, it
>> just drops the lock before unregistering the platform device.
>>
>> Patch #4 fixes the mentioned race conditions and finally patch #5 is the
>> revert patch that was posted by Daniel before but he dropped from his 
>> set.
>>
>> The patches were tested on a rpi4 using different video configurations:
>> (simpledrm -> vc4 both builtin, only vc4 builtin, only simpledrm builtin
>> and simpledrm builtin with vc4 built as a module).
>>
>> Best regards,
>> Javier
>>
>> Changes in v3:
>> - Rebase on top of latest drm-misc-next branch.
>>
>> Changes in v2:
>> - Rebase on top of latest drm-misc-next and fix conflicts (Daniel 
>> Vetter).
>> - Add kernel-doc comments and include in other_interfaces.rst (Daniel 
>> Vetter).
>> - Explain in the commit message that fbmem has to unregister the device
>>    as fallback if a driver registered the device itself (Daniel Vetter).
>> - Also explain that fallback in a comment in the code (Daniel Vetter).
>> - Don't encode in fbmem the assumption that sysfb will always register
>>    platform devices (Daniel Vetter).
>> - Add a FIXME comment about drivers registering devices (Daniel Vetter).
>> - Drop RFC prefix since patches were already reviewed by Daniel Vetter.
>> - Add Daniel Reviewed-by tags to the patches.
>>
>> Daniel Vetter (1):
>>    Revert "fbdev: Prevent probing generic drivers if a FB is already
>>      registered"
>>
>> Javier Martinez Canillas (4):
>>    firmware: sysfb: Make sysfb_create_simplefb() return a pdev pointer
>>    firmware: sysfb: Add helpers to unregister a pdev and disable
>>      registration
>>    fbdev: Restart conflicting fb removal loop when unregistering devices
>>    fbdev: Fix some race conditions between fbmem and sysfb
>>
>>   .../driver-api/firmware/other_interfaces.rst  |  6 ++
>>   drivers/firmware/sysfb.c                      | 77 +++++++++++++++++--
>>   drivers/firmware/sysfb_simplefb.c             | 16 ++--
>>   drivers/video/fbdev/core/fbmem.c              | 62 ++++++++++++---
>>   drivers/video/fbdev/efifb.c                   | 11 ---
>>   drivers/video/fbdev/simplefb.c                | 11 ---
>>   include/linux/fb.h                            |  1 -
>>   include/linux/sysfb.h                         | 29 +++++--
>>   8 files changed, 158 insertions(+), 55 deletions(-)
>>
> 

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Ivo Totev

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: [PATCH v3 0/5] Fix some race conditions that exists between fbmem and sysfb

[Javier Martinez Canillas]

Hello Thomas,

Thanks for the feedback. It was very useful.

On 4/25/22 11:15, Thomas Zimmermann wrote:
> Hi
> 
> Am 25.04.22 um 10:54 schrieb Thomas Zimmermann:
>> Hi
>>
>> Am 20.04.22 um 10:52 schrieb Javier Martinez Canillas:
>>> Hello,
>>>
>>> The patches in this series are mostly changes suggested by Daniel Vetter
>>> to fix some race conditions that exists between the fbdev core (fbmem)
>>> and sysfb with regard to device registration and removal.
>>>
>>> For example, it is currently possible for sysfb to register a platform
>>> device after a real DRM driver was registered and requested to remove the
>>> conflicting framebuffers.
>>>
>>> A symptom of this issue, was worked around with by commit fb561bf9abde
>>> ("fbdev: Prevent probing generic drivers if a FB is already registered")
>>> but that's really a hack and should be reverted.
>>
>> As I mentioned on IRC, I think this series should be merged for the 
>> reasons I give in the other comments.
>>

You meant that should *not* get merged, as we discussed over IRC.

>>>
>>> This series attempt to fix it more properly and revert the mentioned 
>>> hack.
>>> That will also unblock a pending patch to not make the num_registered_fb
>>> variable visible to drivers anymore, since that's internal to fbdev core.
>>
>> Here's as far as I understand the problem:
>>
>>   1) build DRM/fbdev and sysfb code into the kernel
>>   2) during boot, load the DRM/fbdev modules and have them acquire I/O 
>> ranges
>>   3) afterwards load sysfb and have it register platform devices for the 
>> generic framebuffers
>>   4) these devices now conflict with the already-registered DRM/fbdev 
>> devices
>>

That's correct, yes.

>> If that is the problem here, let's simply set a sysfb_disable flag in 
>> sysfb code when the first DRM/fbdev driver first loads. With the flag 
>> set, sysfb won't create any platform devices. We assume that there are 
>> now DRM/fbdev drivers for the framebuffers and sysfb won't be needed.
>>
>> We can set the flag internally from drm_aperture_detach_drivers() [1] 
>> and do_remove_conflicting_framebuffers() [2].
> 
> And further thinking about it, it would be better to set such a flag 
> after successfully registering a DRM/fbdev device.  So we know that 
> there's at least one working display in the system. We don't have to 
> rely on generic framebuffers after that.
>

Exactly, should be done when the device is registered rather than when
the driver is registered or a call is made to remove the conflicting FB.

I'll rework this series with only the bits for sysfb_disable() and drop
the rest. We can go back to the discussion of the remaining parts later
if that makes sense (I still think that patch 3/5 is a better approach,
but let's defer that for a different series).

-- 
Best regards,

Javier Martinez Canillas
Linux Engineering
Red Hat

Re: [PATCH v3 0/5] Fix some race conditions that exists between fbmem and sysfb

[Daniel Vetter]

On Mon, Apr 25, 2022 at 11:49:13AM +0200, Javier Martinez Canillas wrote:
> Hello Thomas,
> 
> Thanks for the feedback. It was very useful.
> 
> On 4/25/22 11:15, Thomas Zimmermann wrote:
> > Hi
> > 
> > Am 25.04.22 um 10:54 schrieb Thomas Zimmermann:
> >> Hi
> >>
> >> Am 20.04.22 um 10:52 schrieb Javier Martinez Canillas:
> >>> Hello,
> >>>
> >>> The patches in this series are mostly changes suggested by Daniel Vetter
> >>> to fix some race conditions that exists between the fbdev core (fbmem)
> >>> and sysfb with regard to device registration and removal.
> >>>
> >>> For example, it is currently possible for sysfb to register a platform
> >>> device after a real DRM driver was registered and requested to remove the
> >>> conflicting framebuffers.
> >>>
> >>> A symptom of this issue, was worked around with by commit fb561bf9abde
> >>> ("fbdev: Prevent probing generic drivers if a FB is already registered")
> >>> but that's really a hack and should be reverted.
> >>
> >> As I mentioned on IRC, I think this series should be merged for the 
> >> reasons I give in the other comments.
> >>
> 
> You meant that should *not* get merged, as we discussed over IRC.
> 
> >>>
> >>> This series attempt to fix it more properly and revert the mentioned 
> >>> hack.
> >>> That will also unblock a pending patch to not make the num_registered_fb
> >>> variable visible to drivers anymore, since that's internal to fbdev core.
> >>
> >> Here's as far as I understand the problem:
> >>
> >>   1) build DRM/fbdev and sysfb code into the kernel
> >>   2) during boot, load the DRM/fbdev modules and have them acquire I/O 
> >> ranges
> >>   3) afterwards load sysfb and have it register platform devices for the 
> >> generic framebuffers
> >>   4) these devices now conflict with the already-registered DRM/fbdev 
> >> devices
> >>
> 
> That's correct, yes.
> 
> >> If that is the problem here, let's simply set a sysfb_disable flag in 
> >> sysfb code when the first DRM/fbdev driver first loads. With the flag 
> >> set, sysfb won't create any platform devices. We assume that there are 
> >> now DRM/fbdev drivers for the framebuffers and sysfb won't be needed.
> >>
> >> We can set the flag internally from drm_aperture_detach_drivers() [1] 
> >> and do_remove_conflicting_framebuffers() [2].
> > 
> > And further thinking about it, it would be better to set such a flag 
> > after successfully registering a DRM/fbdev device.  So we know that 
> > there's at least one working display in the system. We don't have to 
> > rely on generic framebuffers after that.
> >
> 
> Exactly, should be done when the device is registered rather than when
> the driver is registered or a call is made to remove the conflicting FB.
> 
> I'll rework this series with only the bits for sysfb_disable() and drop
> the rest. We can go back to the discussion of the remaining parts later
> if that makes sense (I still think that patch 3/5 is a better approach,
> but let's defer that for a different series).

We need to kill sysfb _before_ the driver loads, otherwise you can have
two drivers fighting over each another. And yes that means you might end
up with black screen if the driver load goes wrong, but the two drivers
fighting over each another can also result in black screens. And the
latter isn't fixable any other way (in general at least) than by making
sure the fw stuff is gone before driver load starts in earnest.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v3 0/5] Fix some race conditions that exists between fbmem and sysfb

[Javier Martinez Canillas]

Hello Daniel,

On 4/29/22 09:47, Daniel Vetter wrote:

[snip]

>>
>> Exactly, should be done when the device is registered rather than when
>> the driver is registered or a call is made to remove the conflicting FB.
>>
>> I'll rework this series with only the bits for sysfb_disable() and drop
>> the rest. We can go back to the discussion of the remaining parts later
>> if that makes sense (I still think that patch 3/5 is a better approach,
>> but let's defer that for a different series).
> 
> We need to kill sysfb _before_ the driver loads, otherwise you can have
> two drivers fighting over each another. And yes that means you might end
> up with black screen if the driver load goes wrong, but the two drivers
> fighting over each another can also result in black screens. And the
> latter isn't fixable any other way (in general at least) than by making
> sure the fw stuff is gone before driver load starts in earnest.

Yes, you are correct. I didn't realize all the possible cases when agreed
with Thomas about doing this but tried and found that it's not enough.

I've a full patch-set now and will post as a RFC so we can discuss more.

> -Daniel

-- 
Best regards,

Javier Martinez Canillas
Linux Engineering
Red Hat