* drm/ssd130x: stale pixels when the GPU renders into the framebuffer @ 2026-08-13 4:27 Fabio Piparo 2026-08-14 18:34 ` Javier Martinez Canillas 2026-09-14 8:43 ` Thomas Zimmermann 0 siblings, 2 replies; 17+ messages in thread From: Fabio Piparo @ 2026-08-13 4:27 UTC (permalink / raw) To: dri-devel; +Cc: Javier Martinez Canillas The symptom =========== I am working with a Raspberry Pi 5 driving a 128x32 SSD1306 over I2C through ssd130x, brought up with dtoverlay=ssd1306, and a GLES client rendering into it. The GPU renders straight into the panel's framebuffer. The panel shows a blocky pattern that is affected by CPU load. The kernel is 6.18.39. A capture: https://files.fabiopiparo.com/ssd130-line-glitch.webp This is a simple trail traveling left to right, and the busier the Pi is, the shorter the trail (link above). On an idle machine the leftovers build up into a blocky fog. That was the clue that pointed me at the CPU cache, since the lifetime of the artifact tracks memory pressure. The trace ========= I made a test program that flips between two alternating solid frames while i2c_write payloads are traced. 15 of the 31 flushes carry bytes from both frames in one payload, mixed at 64 byte granularity. What appears to happen ====================== My reading: the driver's XRGB conversion reads the framebuffer through a cached mapping while the GPU writes the same memory directly, so what reaches the panel is whatever lines the CPU still holds. The 64 byte granularity and the load dependence both fit that, and write-combining the mapping makes it stop. What stops it ============= Setting shmem->map_wc in a .gem_create_object hook makes all the traced payloads come out as expected. Whether the exporter is the right place for this, I do not know. I am happy to test anything on this hardware and report back. Disclosure ========== The panel, the symptom and the i2c trace come from my own hardware. The analysis above and the change came out of a long debugging session with an AI assistant. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: drm/ssd130x: stale pixels when the GPU renders into the framebuffer 2026-08-13 4:27 drm/ssd130x: stale pixels when the GPU renders into the framebuffer Fabio Piparo @ 2026-08-14 18:34 ` Javier Martinez Canillas 2026-09-14 8:43 ` Thomas Zimmermann 1 sibling, 0 replies; 17+ messages in thread From: Javier Martinez Canillas @ 2026-08-14 18:34 UTC (permalink / raw) To: Fabio Piparo, dri-devel Fabio Piparo <holofermes@gmail.com> writes: Hello Fabio, > The symptom > =========== > > I am working with a Raspberry Pi 5 driving a 128x32 SSD1306 over I2C > through ssd130x, brought up with dtoverlay=ssd1306, and a GLES client > rendering into it. The GPU renders straight into the panel's framebuffer. > The panel shows a blocky pattern that is affected by CPU load. The kernel > is 6.18.39. > Thanks for the detailed report. I'm on vacation until September 4, without access to a SSD1306 and limited connectivity. But I'll take a look to your email in detail once I'm back. -- Best regards, Javier Martinez Canillas Core Platforms Red Hat ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: drm/ssd130x: stale pixels when the GPU renders into the framebuffer 2026-08-13 4:27 drm/ssd130x: stale pixels when the GPU renders into the framebuffer Fabio Piparo 2026-08-14 18:34 ` Javier Martinez Canillas @ 2026-09-14 8:43 ` Thomas Zimmermann 2026-09-23 16:00 ` Fabio Piparo 1 sibling, 1 reply; 17+ messages in thread From: Thomas Zimmermann @ 2026-09-14 8:43 UTC (permalink / raw) To: Fabio Piparo, dri-devel; +Cc: Javier Martinez Canillas Hi Am 13.08.26 um 06:27 schrieb Fabio Piparo: > The symptom > =========== > > I am working with a Raspberry Pi 5 driving a 128x32 SSD1306 over I2C > through ssd130x, brought up with dtoverlay=ssd1306, and a GLES client > rendering into it. The GPU renders straight into the panel's framebuffer. > The panel shows a blocky pattern that is affected by CPU load. The kernel > is 6.18.39. > > A capture: > > https://files.fabiopiparo.com/ssd130-line-glitch.webp > > This is a simple trail traveling left to right, and the busier the Pi is, > the shorter the trail (link above). On an idle machine the leftovers build > up into a blocky fog. That was the clue that pointed me at the CPU cache, > since the lifetime of the artifact tracks memory pressure. > > The trace > ========= > > I made a test program that flips between two alternating solid frames > while i2c_write payloads are traced. 15 of the 31 flushes carry bytes > from both frames in one payload, mixed at 64 byte granularity. I assume you write these frames quickly one after the other? As i2c is really slow, you might write to buffers that are still being transferred in the background. Do you read back the status of the page flips? DRM should tell you when it has completed transferring a frame. See [1] [1] https://elixir.bootlin.com/linux/v7.2.5/source/drivers/gpu/drm/drm_file.c#L85 Best regards Thomas > > What appears to happen > ====================== > > My reading: the driver's XRGB conversion reads the framebuffer through a > cached mapping while the GPU writes the same memory directly, so what > reaches the panel is whatever lines the CPU still holds. The 64 byte > granularity and the load dependence both fit that, and write-combining > the mapping makes it stop. > > What stops it > ============= > > Setting shmem->map_wc in a .gem_create_object hook makes all the traced > payloads come out as expected. > > Whether the exporter is the right place for this, I do not know. > > I am happy to test anything on this hardware and report back. > > Disclosure > ========== > > The panel, the symptom and the i2c trace come from my own hardware. The > analysis above and the change came out of a long debugging session with > an AI assistant. -- -- Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Frankenstr. 146, 90461 Nürnberg, Germany, www.suse.com GF: Jochen Jaser, Andrew McDonald, (HRB 36809, AG Nürnberg) ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: drm/ssd130x: stale pixels when the GPU renders into the framebuffer 2026-09-14 8:43 ` Thomas Zimmermann @ 2026-09-23 16:00 ` Fabio Piparo 2026-09-29 9:04 ` Thomas Zimmermann 0 siblings, 1 reply; 17+ messages in thread From: Fabio Piparo @ 2026-09-23 16:00 UTC (permalink / raw) To: Thomas Zimmermann; +Cc: Javier Martinez Canillas, dri-devel, Fabio Piparo Hi Thomas, Thanks for taking a look. On 14.09.26 10:43, Thomas Zimmermann wrote: > I assume you write these frames quickly one after the other? As i2c is > really slow, you might write to buffers that are still being transferred > in the background. > > Do you read back the status of the page flips? DRM should tell you when > it has completed transferring a frame. See [1] The test renders with the GPU: two dumb buffers allocated on the ssd130x device and shared with v3d over PRIME. It runs at 10 fps. Before each flip it calls glFinish(), then flips with DRM_MODE_PAGE_FLIP_EVENT and waits for the event before drawing into the other buffer. The same test drawing with the CPU instead comes out clean. I'm happy to send the test program if it would help. Best regards, Fabio ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: drm/ssd130x: stale pixels when the GPU renders into the framebuffer 2026-09-23 16:00 ` Fabio Piparo @ 2026-09-29 9:04 ` Thomas Zimmermann 2026-09-29 21:50 ` Maíra Canal 0 siblings, 1 reply; 17+ messages in thread From: Thomas Zimmermann @ 2026-09-29 9:04 UTC (permalink / raw) To: Fabio Piparo, Melissa Wen, Maíra Canal Cc: Javier Martinez Canillas, dri-devel (cc'ing v3d maintainers Maira and Melissa) Hi Am 23.09.26 um 18:00 schrieb Fabio Piparo: > Hi Thomas, > > Thanks for taking a look. > > On 14.09.26 10:43, Thomas Zimmermann wrote: >> I assume you write these frames quickly one after the other? As i2c is >> really slow, you might write to buffers that are still being transferred >> in the background. >> >> Do you read back the status of the page flips? DRM should tell you when >> it has completed transferring a frame. See [1] > The test renders with the GPU: two dumb buffers allocated on the > ssd130x device and shared with v3d over PRIME. It runs at 10 fps. > Before each flip it calls glFinish(), then flips with > DRM_MODE_PAGE_FLIP_EVENT and waits for the event before drawing into > the other buffer. The same test drawing with the CPU instead comes out > clean. A number of things come to my mind. - Dumb buffers (from ssd130x) are not meant for HW rendering. They are generally for software rendering only. - Hence, the ideomatic use for sharing graphics buffers is to model a producer-consumer relationship. Export the buffer from the device that generates the frame and import it to the buffer that displays it. - Since you're compositor's sharing happens in the opposite direction, ssd130x might not sync correctly. (I'm not sure of v3d requires a dedicated flush.) - Or maybe v3d needs to do a final cache flush after drawing to imported buffers. If nothing else helpers, we could use map_wc for shared buffers in ssd130x, but it seems like papering over something else. Best regards Thomas > > I'm happy to send the test program if it would help. > > Best regards, > Fabio -- -- Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Frankenstr. 146, 90461 Nürnberg, Germany, www.suse.com GF: Stefan Gaiser, Jochen Jaser, Abhinav Puri, (HRB 36809, AG Nürnberg) ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: drm/ssd130x: stale pixels when the GPU renders into the framebuffer 2026-09-29 9:04 ` Thomas Zimmermann @ 2026-09-29 21:50 ` Maíra Canal 2026-09-30 6:48 ` Thomas Zimmermann 0 siblings, 1 reply; 17+ messages in thread From: Maíra Canal @ 2026-09-29 21:50 UTC (permalink / raw) To: Thomas Zimmermann, Fabio Piparo, Melissa Wen Cc: Javier Martinez Canillas, dri-devel Hi Thomas, On 29/09/26 06:04, Thomas Zimmermann wrote: > (cc'ing v3d maintainers Maira and Melissa) > > Hi > > Am 23.09.26 um 18:00 schrieb Fabio Piparo: >> Hi Thomas, >> >> Thanks for taking a look. >> >> On 14.09.26 10:43, Thomas Zimmermann wrote: >>> I assume you write these frames quickly one after the other? As i2c is >>> really slow, you might write to buffers that are still being transferred >>> in the background. >>> >>> Do you read back the status of the page flips? DRM should tell you when >>> it has completed transferring a frame. See [1] >> The test renders with the GPU: two dumb buffers allocated on the >> ssd130x device and shared with v3d over PRIME. It runs at 10 fps. >> Before each flip it calls glFinish(), then flips with >> DRM_MODE_PAGE_FLIP_EVENT and waits for the event before drawing into >> the other buffer. The same test drawing with the CPU instead comes out >> clean. > > A number of things come to my mind. > > - Dumb buffers (from ssd130x) are not meant for HW rendering. They are > generally for software rendering only. > This reminded me of an issue I recently saw involving GPU rendering to dumb buffers when combining simpledrm and Panthor. It had the same symptom: this trail of glitches. After investigating that issue, I noticed that, on platforms where the GPU isn't DMA-coherent, when a shmem GEM buffer is exported via dma- buf to a GPU for rendering, the CPU can read stale data from its cache when later accessing the buffer, which manifests as rendering artifacts. So, the GPU writes reach memory, but the CPU blit reads stale cache lines. drm_gem_fb_begin_cpu_access() only syncs imported buffers, and GEM dma_buf_ops have no begin/end_cpu_access, so nothing invalidates the CPU cache before the blit and the CPU reads stale cached data. I have a patch implementing begin/end_cpu_access to drm_gem_prime_dmabuf_ops, which fixed the issue. However, I'm not sure we would like to support this use case upstream because, as you mentioned, dumb buffers are usually used for software-rendering only. > - Hence, the ideomatic use for sharing graphics buffers is to model a > producer-consumer relationship. Export the buffer from the device that > generates the frame and import it to the buffer that displays it. > > - Since you're compositor's sharing happens in the opposite direction, > ssd130x might not sync correctly. (I'm not sure of v3d requires a > dedicated flush.) > > - Or maybe v3d needs to do a final cache flush after drawing to imported > buffers. I don't think so... Once the job completes, the GPU writes are in memory. I believe the stale copy is in the CPU cache, so the invalidation should happen on the CPU side. Best regards, - Maíra > > If nothing else helpers, we could use map_wc for shared buffers in > ssd130x, but it seems like papering over something else. > > Best regards > Thomas > > >> >> I'm happy to send the test program if it would help. >> >> Best regards, >> Fabio > ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: drm/ssd130x: stale pixels when the GPU renders into the framebuffer 2026-09-29 21:50 ` Maíra Canal @ 2026-09-30 6:48 ` Thomas Zimmermann 2026-09-30 7:48 ` Javier Martinez Canillas 0 siblings, 1 reply; 17+ messages in thread From: Thomas Zimmermann @ 2026-09-30 6:48 UTC (permalink / raw) To: Maíra Canal, Fabio Piparo, Melissa Wen Cc: Javier Martinez Canillas, dri-devel Hi Am 29.09.26 um 23:50 schrieb Maíra Canal: > Hi Thomas, > > On 29/09/26 06:04, Thomas Zimmermann wrote: >> (cc'ing v3d maintainers Maira and Melissa) >> >> Hi >> >> Am 23.09.26 um 18:00 schrieb Fabio Piparo: >>> Hi Thomas, >>> >>> Thanks for taking a look. >>> >>> On 14.09.26 10:43, Thomas Zimmermann wrote: >>>> I assume you write these frames quickly one after the other? As i2c is >>>> really slow, you might write to buffers that are still being >>>> transferred >>>> in the background. >>>> >>>> Do you read back the status of the page flips? DRM should tell you >>>> when >>>> it has completed transferring a frame. See [1] >>> The test renders with the GPU: two dumb buffers allocated on the >>> ssd130x device and shared with v3d over PRIME. It runs at 10 fps. >>> Before each flip it calls glFinish(), then flips with >>> DRM_MODE_PAGE_FLIP_EVENT and waits for the event before drawing into >>> the other buffer. The same test drawing with the CPU instead comes out >>> clean. >> >> A number of things come to my mind. >> >> - Dumb buffers (from ssd130x) are not meant for HW rendering. They >> are generally for software rendering only. >> > > This reminded me of an issue I recently saw involving GPU rendering to > dumb buffers when combining simpledrm and Panthor. It had the same > symptom: this trail of glitches. > > After investigating that issue, I noticed that, on platforms where the > GPU isn't DMA-coherent, when a shmem GEM buffer is exported via dma- > buf to a GPU for rendering, the CPU can read stale data from its cache > when later accessing the buffer, which manifests as rendering artifacts. Exactly what we're seeing here. Thanks for the analysis. > > So, the GPU writes reach memory, but the CPU blit reads stale cache > lines. drm_gem_fb_begin_cpu_access() only syncs imported buffers, and > GEM dma_buf_ops have no begin/end_cpu_access, so nothing invalidates the > CPU cache before the blit and the CPU reads stale cached data. > > I have a patch implementing begin/end_cpu_access to > drm_gem_prime_dmabuf_ops, which fixed the issue. However, I'm not sure > we would like to support this use case upstream because, as you > mentioned, dumb buffers are usually used for software-rendering only. Before we fix anything, I think we should talk to someone with dma-buf/PRIME credentials. As I outlined, the ideomatic pattern is a producer-consumer relationship and HW rendering into dumb-buffers is not supported. Those buffers should have been allocated on the v3d side. IMHO we should write down these rules in the PRIME documentation and (soft-)enforce them in the implementation. >> - Hence, the ideomatic use for sharing graphics buffers is to model a >> producer-consumer relationship. Export the buffer from the device >> that generates the frame and import it to the buffer that displays it. >> >> - Since you're compositor's sharing happens in the opposite >> direction, ssd130x might not sync correctly. (I'm not sure of v3d >> requires a dedicated flush.) >> >> - Or maybe v3d needs to do a final cache flush after drawing to >> imported buffers. > > I don't think so... Once the job completes, the GPU writes are in > memory. I believe the stale copy is in the CPU cache, so the > invalidation should happen on the CPU side. OTOH it's not the exporter's (ssd130x here, but really _any_ driver) job to take care of it either. :/ Best regards Thomas > > Best regards, > - Maíra > >> >> If nothing else helpers, we could use map_wc for shared buffers in >> ssd130x, but it seems like papering over something else. >> >> Best regards >> Thomas >> >> >>> >>> I'm happy to send the test program if it would help. >>> >>> Best regards, >>> Fabio >> > -- -- Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Frankenstr. 146, 90461 Nürnberg, Germany, www.suse.com GF: Stefan Gaiser, Jochen Jaser, Abhinav Puri, (HRB 36809, AG Nürnberg) ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: drm/ssd130x: stale pixels when the GPU renders into the framebuffer 2026-09-30 6:48 ` Thomas Zimmermann @ 2026-09-30 7:48 ` Javier Martinez Canillas 2026-09-30 8:32 ` Thomas Zimmermann 0 siblings, 1 reply; 17+ messages in thread From: Javier Martinez Canillas @ 2026-09-30 7:48 UTC (permalink / raw) To: Thomas Zimmermann, Maíra Canal, Fabio Piparo, Melissa Wen; +Cc: dri-devel Thomas Zimmermann <tzimmermann@suse.de> writes: Hello Maíra and Thomas, > Hi > > Am 29.09.26 um 23:50 schrieb Maíra Canal: >> Hi Thomas, >> >> On 29/09/26 06:04, Thomas Zimmermann wrote: >>> (cc'ing v3d maintainers Maira and Melissa) >>> >>> Hi >>> >>> Am 23.09.26 um 18:00 schrieb Fabio Piparo: >>>> Hi Thomas, >>>> >>>> Thanks for taking a look. >>>> >>>> On 14.09.26 10:43, Thomas Zimmermann wrote: >>>>> I assume you write these frames quickly one after the other? As i2c is >>>>> really slow, you might write to buffers that are still being >>>>> transferred >>>>> in the background. >>>>> >>>>> Do you read back the status of the page flips? DRM should tell you >>>>> when >>>>> it has completed transferring a frame. See [1] >>>> The test renders with the GPU: two dumb buffers allocated on the >>>> ssd130x device and shared with v3d over PRIME. It runs at 10 fps. >>>> Before each flip it calls glFinish(), then flips with >>>> DRM_MODE_PAGE_FLIP_EVENT and waits for the event before drawing into >>>> the other buffer. The same test drawing with the CPU instead comes out >>>> clean. >>> >>> A number of things come to my mind. >>> >>> - Dumb buffers (from ssd130x) are not meant for HW rendering. They >>> are generally for software rendering only. >>> This is another thing that likely needs to be documented somewhere, for people to know what are the limitations of dumb buffers. >> >> This reminded me of an issue I recently saw involving GPU rendering to >> dumb buffers when combining simpledrm and Panthor. It had the same >> symptom: this trail of glitches. >> >> After investigating that issue, I noticed that, on platforms where the >> GPU isn't DMA-coherent, when a shmem GEM buffer is exported via dma- >> buf to a GPU for rendering, the CPU can read stale data from its cache >> when later accessing the buffer, which manifests as rendering artifacts. > > Exactly what we're seeing here. Thanks for the analysis. > Indeed, the issue is not specific to ssd130x, but to any display driver that uses dumb buffers. >> >> So, the GPU writes reach memory, but the CPU blit reads stale cache >> lines. drm_gem_fb_begin_cpu_access() only syncs imported buffers, and >> GEM dma_buf_ops have no begin/end_cpu_access, so nothing invalidates the >> CPU cache before the blit and the CPU reads stale cached data. >> Exactly, this what I mentioned to Thomas yesterday over IRC yesterday, since I noticed the same while reading drm_gem_fb_begin_cpu_access() code: https://elixir.bootlin.com/linux/v7.3-rc1/source/drivers/gpu/drm/drm_gem_framebuffer_helper.c#L473 >> I have a patch implementing begin/end_cpu_access to >> drm_gem_prime_dmabuf_ops, which fixed the issue. However, I'm not sure >> we would like to support this use case upstream because, as you >> mentioned, dumb buffers are usually used for software-rendering only. > > Before we fix anything, I think we should talk to someone with > dma-buf/PRIME credentials. As I outlined, the ideomatic pattern is a > producer-consumer relationship and HW rendering into dumb-buffers is not > supported. Those buffers should have been allocated on the v3d side. > > IMHO we should write down these rules in the PRIME documentation and > (soft-)enforce them in the implementation. > Yeah, I think either drm_gem_fb_begin_cpu_access() needs to also sync exported buffers (what Maíra is proposing as a fix) or there should be documentation of the rules for cross-devices buffer sharing through PRIME. As mentioned, allocating buffers through v3d and importing into ssd130x (or simpledrm) would work correctly, since the v3d driver already sets bo->base.map_wc = true. -- Best regards, Javier Martinez Canillas Core Platforms Red Hat ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: drm/ssd130x: stale pixels when the GPU renders into the framebuffer 2026-09-30 7:48 ` Javier Martinez Canillas @ 2026-09-30 8:32 ` Thomas Zimmermann 2026-09-30 13:49 ` Maíra Canal 0 siblings, 1 reply; 17+ messages in thread From: Thomas Zimmermann @ 2026-09-30 8:32 UTC (permalink / raw) To: Javier Martinez Canillas, Maíra Canal, Fabio Piparo, Melissa Wen Cc: dri-devel Hi Am 30.09.26 um 09:48 schrieb Javier Martinez Canillas: [...] >>> I have a patch implementing begin/end_cpu_access to >>> drm_gem_prime_dmabuf_ops, which fixed the issue. However, I'm not sure >>> we would like to support this use case upstream because, as you >>> mentioned, dumb buffers are usually used for software-rendering only. >> Before we fix anything, I think we should talk to someone with >> dma-buf/PRIME credentials. As I outlined, the ideomatic pattern is a >> producer-consumer relationship and HW rendering into dumb-buffers is not >> supported. Those buffers should have been allocated on the v3d side. >> >> IMHO we should write down these rules in the PRIME documentation and >> (soft-)enforce them in the implementation. >> > Yeah, I think either drm_gem_fb_begin_cpu_access() needs to also sync > exported buffers (what Maíra is proposing as a fix) or there should be > documentation of the rules for cross-devices buffer sharing through PRIME. This specific case only happens with v3d and only this driver can know when to flush caches. If we flush caches in begin_cpu_access, we easily end up paying the overhead on all systems. > > As mentioned, allocating buffers through v3d and importing into ssd130x > (or simpledrm) would work correctly, since the v3d driver already sets > bo->base.map_wc = true. Hard enforcement would mean that drivers would reject to render to imported buffers. It is most certainly too late to implement this now. But drivers such as v3d could at least print a warning when users attempt to do it. Best regards Thomas > -- -- Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Frankenstr. 146, 90461 Nürnberg, Germany, www.suse.com GF: Stefan Gaiser, Jochen Jaser, Abhinav Puri, (HRB 36809, AG Nürnberg) ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: drm/ssd130x: stale pixels when the GPU renders into the framebuffer 2026-09-30 8:32 ` Thomas Zimmermann @ 2026-09-30 13:49 ` Maíra Canal 2026-09-30 15:52 ` Javier Martinez Canillas 0 siblings, 1 reply; 17+ messages in thread From: Maíra Canal @ 2026-09-30 13:49 UTC (permalink / raw) To: Thomas Zimmermann, Javier Martinez Canillas, Fabio Piparo, Melissa Wen Cc: dri-devel Hi, On 30/09/26 05:32, Thomas Zimmermann wrote: > Hi > > Am 30.09.26 um 09:48 schrieb Javier Martinez Canillas: > [...] >>>> I have a patch implementing begin/end_cpu_access to >>>> drm_gem_prime_dmabuf_ops, which fixed the issue. However, I'm not sure >>>> we would like to support this use case upstream because, as you >>>> mentioned, dumb buffers are usually used for software-rendering only. >>> Before we fix anything, I think we should talk to someone with >>> dma-buf/PRIME credentials. As I outlined, the ideomatic pattern is a >>> producer-consumer relationship and HW rendering into dumb-buffers is not >>> supported. Those buffers should have been allocated on the v3d side. >>> >>> IMHO we should write down these rules in the PRIME documentation and >>> (soft-)enforce them in the implementation. >>> >> Yeah, I think either drm_gem_fb_begin_cpu_access() needs to also sync >> exported buffers (what Maíra is proposing as a fix) or there should be >> documentation of the rules for cross-devices buffer sharing through >> PRIME. > > This specific case only happens with v3d and only this driver can know > when to flush caches. If we flush caches in begin_cpu_access, we easily > end up paying the overhead on all systems. > I believe that's the main issue. We would pay a reasonably big price to support this uncommon (and maybe even wrong) use case. P.S.: To be clear, I wasn't proposing a fix. I should have called my patch a "hack", because although it works, I don't believe we should support this use case for the reasons stated by Thomas. Best regards, - Maíra > >> >> As mentioned, allocating buffers through v3d and importing into ssd130x >> (or simpledrm) would work correctly, since the v3d driver already sets >> bo->base.map_wc = true. > > Hard enforcement would mean that drivers would reject to render to > imported buffers. It is most certainly too late to implement this now. > But drivers such as v3d could at least print a warning when users > attempt to do it. > > Best regards > Thomas > > >> > ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: drm/ssd130x: stale pixels when the GPU renders into the framebuffer 2026-09-30 13:49 ` Maíra Canal @ 2026-09-30 15:52 ` Javier Martinez Canillas 2026-09-30 18:45 ` Fabio Piparo 2026-09-30 19:57 ` Maíra Canal 0 siblings, 2 replies; 17+ messages in thread From: Javier Martinez Canillas @ 2026-09-30 15:52 UTC (permalink / raw) To: Maíra Canal, Thomas Zimmermann, Fabio Piparo, Melissa Wen; +Cc: dri-devel Maíra Canal <mcanal@igalia.com> writes: Hello Maíra, > Hi, > > On 30/09/26 05:32, Thomas Zimmermann wrote: >> Hi >> >> Am 30.09.26 um 09:48 schrieb Javier Martinez Canillas: >> [...] >>>>> I have a patch implementing begin/end_cpu_access to >>>>> drm_gem_prime_dmabuf_ops, which fixed the issue. However, I'm not sure >>>>> we would like to support this use case upstream because, as you >>>>> mentioned, dumb buffers are usually used for software-rendering only. >>>> Before we fix anything, I think we should talk to someone with >>>> dma-buf/PRIME credentials. As I outlined, the ideomatic pattern is a >>>> producer-consumer relationship and HW rendering into dumb-buffers is not >>>> supported. Those buffers should have been allocated on the v3d side. >>>> >>>> IMHO we should write down these rules in the PRIME documentation and >>>> (soft-)enforce them in the implementation. >>>> >>> Yeah, I think either drm_gem_fb_begin_cpu_access() needs to also sync >>> exported buffers (what Maíra is proposing as a fix) or there should be >>> documentation of the rules for cross-devices buffer sharing through >>> PRIME. >> >> This specific case only happens with v3d and only this driver can know >> when to flush caches. If we flush caches in begin_cpu_access, we easily >> end up paying the overhead on all systems. >> > > I believe that's the main issue. We would pay a reasonably big price to > support this uncommon (and maybe even wrong) use case. > > P.S.: To be clear, I wasn't proposing a fix. I should have called my > patch a "hack", because although it works, I don't believe we should > support this use case for the reasons stated by Thomas. > Yes, I agree with you. Making v3d to warn as Thomas suggested seems to be the best compromise. > Best regards, > - Maíra > -- Best regards, Javier Martinez Canillas Core Platforms Red Hat ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: drm/ssd130x: stale pixels when the GPU renders into the framebuffer 2026-09-30 15:52 ` Javier Martinez Canillas @ 2026-09-30 18:45 ` Fabio Piparo 2026-10-02 8:18 ` Thomas Zimmermann 2026-09-30 19:57 ` Maíra Canal 1 sibling, 1 reply; 17+ messages in thread From: Fabio Piparo @ 2026-09-30 18:45 UTC (permalink / raw) To: Javier Martinez Canillas Cc: Maíra Canal, Thomas Zimmermann, Melissa Wen, dri-devel Hi all, On Wed, Sep 30, 2026 at 2:48 AM Thomas Zimmermann wrote: > Those buffers should have been allocated on the v3d side. Ah! Now I think I see what's happening. Some context on where I'm coming from, since I think I explained it badly. My program opens the panel's card and lets Mesa handle the buffers. fd = open("/dev/dri/card2", O_RDWR); /* ssd130x */ gbm = gbm_create_device(fd); surf = gbm_surface_create(gbm, 128, 32, GBM_FORMAT_XRGB8888, GBM_BO_USE_SCANOUT | GBM_BO_USE_RENDERING); bo = gbm_surface_lock_front_buffer(surf); On a Pi 4/5 Mesa hands me dumb buffers from ssd130x with v3d drawing into them. On a Pi 0-3 it goes the other way, the GPU makes the buffers and ssd130x imports them, and that works fine. Reading this thread convinced me that what I was doing was wrong for my use case, and that my program should own the buffers itself. So now it does the producer-consumer thing: gpu = gbm_create_device(open("/dev/dri/renderD128", O_RDWR)); /* v3d */ bo = gbm_bo_create(gpu, 128, 32, GBM_FORMAT_XRGB8888, GBM_BO_USE_RENDERING | GBM_BO_USE_LINEAR); drmPrimeFDToHandle(fd, gbm_bo_get_fd(bo), &handle); /* AddFB2 with that handle, draw into bo through an EGLImage, page flip */ That's clean on a stock kernel and stock Mesa. By the way, I found one other report that looks like the same thing, from March 2024: https://github.com/notro/gud/issues/22 Thanks, Fabio/ ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: drm/ssd130x: stale pixels when the GPU renders into the framebuffer 2026-09-30 18:45 ` Fabio Piparo @ 2026-10-02 8:18 ` Thomas Zimmermann 0 siblings, 0 replies; 17+ messages in thread From: Thomas Zimmermann @ 2026-10-02 8:18 UTC (permalink / raw) To: Fabio Piparo, Javier Martinez Canillas Cc: Maíra Canal, Melissa Wen, dri-devel Hi Am 30.09.26 um 20:45 schrieb Fabio Piparo: > Hi all, > > On Wed, Sep 30, 2026 at 2:48 AM Thomas Zimmermann wrote: >> Those buffers should have been allocated on the v3d side. > Ah! Now I think I see what's happening. Some context on where I'm > coming from, since I think I explained it badly. > > My program opens the panel's card and lets Mesa handle the buffers. > > fd = open("/dev/dri/card2", O_RDWR); /* ssd130x */ > gbm = gbm_create_device(fd); > surf = gbm_surface_create(gbm, 128, 32, GBM_FORMAT_XRGB8888, > GBM_BO_USE_SCANOUT | GBM_BO_USE_RENDERING); > bo = gbm_surface_lock_front_buffer(surf); > > On a Pi 4/5 Mesa hands me dumb buffers from ssd130x with v3d drawing > into them. On a Pi 0-3 it goes the other way, the GPU makes the > buffers and ssd130x imports them, and that works fine. > > Reading this thread convinced me that what I was doing was wrong for > my use case, and that my program should own the buffers itself. So now > it does the producer-consumer thing: > > gpu = gbm_create_device(open("/dev/dri/renderD128", O_RDWR)); /* v3d */ > bo = gbm_bo_create(gpu, 128, 32, GBM_FORMAT_XRGB8888, > GBM_BO_USE_RENDERING | GBM_BO_USE_LINEAR); > drmPrimeFDToHandle(fd, gbm_bo_get_fd(bo), &handle); > /* AddFB2 with that handle, draw into bo through an EGLImage, page flip */ > > That's clean on a stock kernel and stock Mesa. Great to hear that it works now Best regards Thomas > > By the way, I found one other report that looks like the same thing, > from March 2024: https://github.com/notro/gud/issues/22 > > Thanks, > Fabio/ -- -- Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Frankenstr. 146, 90461 Nürnberg, Germany, www.suse.com GF: Stefan Gaiser, Jochen Jaser, Abhinav Puri, (HRB 36809, AG Nürnberg) ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: drm/ssd130x: stale pixels when the GPU renders into the framebuffer 2026-09-30 15:52 ` Javier Martinez Canillas 2026-09-30 18:45 ` Fabio Piparo @ 2026-09-30 19:57 ` Maíra Canal 2026-10-02 8:12 ` Thomas Zimmermann 1 sibling, 1 reply; 17+ messages in thread From: Maíra Canal @ 2026-09-30 19:57 UTC (permalink / raw) To: Javier Martinez Canillas, Thomas Zimmermann, Fabio Piparo, Melissa Wen, Chema Casanova Cc: dri-devel + cc Chema (Mesa developer in V3D) Hi, On 30/09/26 12:52, Javier Martinez Canillas wrote: > Maíra Canal <mcanal@igalia.com> writes: > > Hello Maíra, > >> Hi, >> >> On 30/09/26 05:32, Thomas Zimmermann wrote: >>> Hi >>> >>> Am 30.09.26 um 09:48 schrieb Javier Martinez Canillas: >>> [...] >>>>>> I have a patch implementing begin/end_cpu_access to >>>>>> drm_gem_prime_dmabuf_ops, which fixed the issue. However, I'm not sure >>>>>> we would like to support this use case upstream because, as you >>>>>> mentioned, dumb buffers are usually used for software-rendering only. >>>>> Before we fix anything, I think we should talk to someone with >>>>> dma-buf/PRIME credentials. As I outlined, the ideomatic pattern is a >>>>> producer-consumer relationship and HW rendering into dumb-buffers is not >>>>> supported. Those buffers should have been allocated on the v3d side. >>>>> >>>>> IMHO we should write down these rules in the PRIME documentation and >>>>> (soft-)enforce them in the implementation. >>>>> >>>> Yeah, I think either drm_gem_fb_begin_cpu_access() needs to also sync >>>> exported buffers (what Maíra is proposing as a fix) or there should be >>>> documentation of the rules for cross-devices buffer sharing through >>>> PRIME. >>> >>> This specific case only happens with v3d and only this driver can know >>> when to flush caches. If we flush caches in begin_cpu_access, we easily >>> end up paying the overhead on all systems. >>> >> >> I believe that's the main issue. We would pay a reasonably big price to >> support this uncommon (and maybe even wrong) use case. >> >> P.S.: To be clear, I wasn't proposing a fix. I should have called my >> patch a "hack", because although it works, I don't believe we should >> support this use case for the reasons stated by Thomas. >> > > Yes, I agree with you. Making v3d to warn as Thomas suggested seems to be > the best compromise. I was thinking about Thomas' suggestion, and I might be missing something. If v3d warns when rendering to imported buffers, it will warn on every RPi 4/5, as Mesa renders the display path into dumb buffers imported from vc4. That works because vc4 buffers are WC and scanned out by hardware, so the CPU never reads them [1]. From my point of view, the issue here is cross-device sharing of a dumb buffer that the exporter reads with the CPU through a cached mapping. [1] Right after writing this paragraph, I decided to read the documentation in drm_dumb_object.c and I found: * Note that dumb objects may not be used for gpu acceleration, as has been * attempted on some ARM embedded platforms. Such drivers really must have * a hardware-specific ioctl to allocate suitable buffer objects. Does anyone know why Mesa kmsro create dumb buffers for rendering? Maybe I understood the documentation incorrectly, but it looks like we shouldn't be doing that. Best regards, - Maíra > >> Best regards, >> - Maíra >> > ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: drm/ssd130x: stale pixels when the GPU renders into the framebuffer 2026-09-30 19:57 ` Maíra Canal @ 2026-10-02 8:12 ` Thomas Zimmermann 2026-10-02 21:13 ` Maíra Canal 0 siblings, 1 reply; 17+ messages in thread From: Thomas Zimmermann @ 2026-10-02 8:12 UTC (permalink / raw) To: Maíra Canal, Javier Martinez Canillas, Fabio Piparo, Melissa Wen, Chema Casanova Cc: dri-devel Hi Am 30.09.26 um 21:57 schrieb Maíra Canal: > + cc Chema (Mesa developer in V3D) > > Hi, > > On 30/09/26 12:52, Javier Martinez Canillas wrote: >> Maíra Canal <mcanal@igalia.com> writes: >> >> Hello Maíra, >> >>> Hi, >>> >>> On 30/09/26 05:32, Thomas Zimmermann wrote: >>>> Hi >>>> >>>> Am 30.09.26 um 09:48 schrieb Javier Martinez Canillas: >>>> [...] >>>>>>> I have a patch implementing begin/end_cpu_access to >>>>>>> drm_gem_prime_dmabuf_ops, which fixed the issue. However, I'm >>>>>>> not sure >>>>>>> we would like to support this use case upstream because, as you >>>>>>> mentioned, dumb buffers are usually used for software-rendering >>>>>>> only. >>>>>> Before we fix anything, I think we should talk to someone with >>>>>> dma-buf/PRIME credentials. As I outlined, the ideomatic pattern >>>>>> is a >>>>>> producer-consumer relationship and HW rendering into dumb-buffers >>>>>> is not >>>>>> supported. Those buffers should have been allocated on the v3d side. >>>>>> >>>>>> IMHO we should write down these rules in the PRIME documentation and >>>>>> (soft-)enforce them in the implementation. >>>>>> >>>>> Yeah, I think either drm_gem_fb_begin_cpu_access() needs to also sync >>>>> exported buffers (what Maíra is proposing as a fix) or there >>>>> should be >>>>> documentation of the rules for cross-devices buffer sharing through >>>>> PRIME. >>>> >>>> This specific case only happens with v3d and only this driver can know >>>> when to flush caches. If we flush caches in begin_cpu_access, we >>>> easily >>>> end up paying the overhead on all systems. >>>> >>> >>> I believe that's the main issue. We would pay a reasonably big price to >>> support this uncommon (and maybe even wrong) use case. >>> >>> P.S.: To be clear, I wasn't proposing a fix. I should have called my >>> patch a "hack", because although it works, I don't believe we should >>> support this use case for the reasons stated by Thomas. >>> >> >> Yes, I agree with you. Making v3d to warn as Thomas suggested seems >> to be >> the best compromise. > > I was thinking about Thomas' suggestion, and I might be missing > something. If v3d warns when rendering to imported buffers, it will > warn on every RPi 4/5, as Mesa renders the display path into dumb > buffers imported from vc4. That works because vc4 buffers are WC and > scanned out by hardware, so the CPU never reads them [1]. Oh, that's not good. It actually convinces me that v3d (and every other driver) should have warned about this situation. See my example below. > > From my point of view, the issue here is cross-device sharing of a dumb > buffer that the exporter reads with the CPU through a cached mapping. The whole workaround with WC doesn't fully capture the issue. Imagine the renderer would have used discrete video memory (e.g. like on an old PCI device). Rendering to an imported buffer would simply not work at all. After rendering a frame into the video memory, the render driver would have to memcpy the result from the video memory to the imported buffer. This would need to be done on every frame because the importer doesn't know when exactly the exporter requires the image. Sharing buffers in the other direction would work fine. The importer (now ssd130x) can always setup a page mapping from the dma-buf's provided s/g table. The s/g table can refer directly to the video memory on the PCI device. > > [1] Right after writing this paragraph, I decided to read the > documentation in drm_dumb_object.c and I found: > > * Note that dumb objects may not be used for gpu acceleration, as has > been > * attempted on some ARM embedded platforms. Such drivers really must have > * a hardware-specific ioctl to allocate suitable buffer objects. > > Does anyone know why Mesa kmsro create dumb buffers for rendering? Maybe > I understood the documentation incorrectly, but it looks like we > shouldn't be doing that. The first bullet point talks specifically about GPU acceleration. Mesa creates dumb buffers for _software rendering_. Ssd130x scans out the final result and transfers it to the panel. No GPU rendering involved there. Best regards Thomas > > Best regards, > - Maíra > >> >>> Best regards, >>> - Maíra >>> >> > -- -- Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Frankenstr. 146, 90461 Nürnberg, Germany, www.suse.com GF: Stefan Gaiser, Jochen Jaser, Abhinav Puri, (HRB 36809, AG Nürnberg) ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: drm/ssd130x: stale pixels when the GPU renders into the framebuffer 2026-10-02 8:12 ` Thomas Zimmermann @ 2026-10-02 21:13 ` Maíra Canal 2026-10-06 6:50 ` Thomas Zimmermann 0 siblings, 1 reply; 17+ messages in thread From: Maíra Canal @ 2026-10-02 21:13 UTC (permalink / raw) To: Thomas Zimmermann, Javier Martinez Canillas, Fabio Piparo, Melissa Wen, Chema Casanova Cc: dri-devel On 02/10/26 05:12, Thomas Zimmermann wrote: > Hi > > Am 30.09.26 um 21:57 schrieb Maíra Canal: >> + cc Chema (Mesa developer in V3D) >> >> Hi, >> >> On 30/09/26 12:52, Javier Martinez Canillas wrote: >>> Maíra Canal <mcanal@igalia.com> writes: >>> >>> Hello Maíra, >>> >>>> Hi, >>>> >>>> On 30/09/26 05:32, Thomas Zimmermann wrote: >>>>> Hi >>>>> >>>>> Am 30.09.26 um 09:48 schrieb Javier Martinez Canillas: >>>>> [...] >>>>>>>> I have a patch implementing begin/end_cpu_access to >>>>>>>> drm_gem_prime_dmabuf_ops, which fixed the issue. However, I'm >>>>>>>> not sure >>>>>>>> we would like to support this use case upstream because, as you >>>>>>>> mentioned, dumb buffers are usually used for software-rendering >>>>>>>> only. >>>>>>> Before we fix anything, I think we should talk to someone with >>>>>>> dma-buf/PRIME credentials. As I outlined, the ideomatic pattern >>>>>>> is a >>>>>>> producer-consumer relationship and HW rendering into dumb-buffers >>>>>>> is not >>>>>>> supported. Those buffers should have been allocated on the v3d side. >>>>>>> >>>>>>> IMHO we should write down these rules in the PRIME documentation and >>>>>>> (soft-)enforce them in the implementation. >>>>>>> >>>>>> Yeah, I think either drm_gem_fb_begin_cpu_access() needs to also sync >>>>>> exported buffers (what Maíra is proposing as a fix) or there >>>>>> should be >>>>>> documentation of the rules for cross-devices buffer sharing through >>>>>> PRIME. >>>>> >>>>> This specific case only happens with v3d and only this driver can know >>>>> when to flush caches. If we flush caches in begin_cpu_access, we >>>>> easily >>>>> end up paying the overhead on all systems. >>>>> >>>> >>>> I believe that's the main issue. We would pay a reasonably big price to >>>> support this uncommon (and maybe even wrong) use case. >>>> >>>> P.S.: To be clear, I wasn't proposing a fix. I should have called my >>>> patch a "hack", because although it works, I don't believe we should >>>> support this use case for the reasons stated by Thomas. >>>> >>> >>> Yes, I agree with you. Making v3d to warn as Thomas suggested seems >>> to be >>> the best compromise. >> >> I was thinking about Thomas' suggestion, and I might be missing >> something. If v3d warns when rendering to imported buffers, it will >> warn on every RPi 4/5, as Mesa renders the display path into dumb >> buffers imported from vc4. That works because vc4 buffers are WC and >> scanned out by hardware, so the CPU never reads them [1]. > > Oh, that's not good. It actually convinces me that v3d (and every other > driver) should have warned about this situation. See my example below. > > >> >> From my point of view, the issue here is cross-device sharing of a dumb >> buffer that the exporter reads with the CPU through a cached mapping. > > The whole workaround with WC doesn't fully capture the issue. > > Imagine the renderer would have used discrete video memory (e.g. like on > an old PCI device). Rendering to an imported buffer would simply not > work at all. After rendering a frame into the video memory, the render > driver would have to memcpy the result from the video memory to the > imported buffer. This would need to be done on every frame because the > importer doesn't know when exactly the exporter requires the image. > Sharing buffers in the other direction would work fine. The importer > (now ssd130x) can always setup a page mapping from the dma-buf's > provided s/g table. The s/g table can refer directly to the video memory > on the PCI device. > > >> >> [1] Right after writing this paragraph, I decided to read the >> documentation in drm_dumb_object.c and I found: >> >> * Note that dumb objects may not be used for gpu acceleration, as has >> been >> * attempted on some ARM embedded platforms. Such drivers really must have >> * a hardware-specific ioctl to allocate suitable buffer objects. >> >> Does anyone know why Mesa kmsro create dumb buffers for rendering? Maybe >> I understood the documentation incorrectly, but it looks like we >> shouldn't be doing that. > > The first bullet point talks specifically about GPU acceleration. Mesa > creates dumb buffers for _software rendering_. Ssd130x scans out the > final result and transfers it to the panel. No GPU rendering involved > there. > Mesa also uses dumb buffers for rendering. kmsro created dumb buffers as render targets for several render-only GPUs, such as V3D, Panfrost, Etnaviv, and others. You can check that in the function renderonly_create_kms_dumb_buffer_for_resource(). I believe I don't have the full historical background to affirm if dumb buffers should be used or not as render targets. If Mesa is using it for so many years, it's probably not wrong. Best regards, - Maíra ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: drm/ssd130x: stale pixels when the GPU renders into the framebuffer 2026-10-02 21:13 ` Maíra Canal @ 2026-10-06 6:50 ` Thomas Zimmermann 0 siblings, 0 replies; 17+ messages in thread From: Thomas Zimmermann @ 2026-10-06 6:50 UTC (permalink / raw) To: Maíra Canal, Javier Martinez Canillas, Fabio Piparo, Melissa Wen, Chema Casanova Cc: dri-devel Hi Am 02.10.26 um 23:13 schrieb Maíra Canal: [...] >> The first bullet point talks specifically about GPU acceleration. >> Mesa creates dumb buffers for _software rendering_. Ssd130x scans out >> the final result and transfers it to the panel. No GPU rendering >> involved there. >> > > Mesa also uses dumb buffers for rendering. kmsro created dumb buffers as > render targets for several render-only GPUs, such as V3D, Panfrost, > Etnaviv, and others. You can check that in the function > renderonly_create_kms_dumb_buffer_for_resource(). That function should not exist. But some authors of kernel drivers were not aware that they cannot use dumb buffers for rendering (I don't blame them) so they never even added a dedicated ioctl for buffer allocation. It works in practice because the hardware has no special requirements for the rendering buffers. BTW the drivers you refer to do have ioctls for buffer allocation. Fixing Mesa would be appreciated. > > I believe I don't have the full historical background to affirm if dumb > buffers should be used or not as render targets. If Mesa is using it for > so many years, it's probably not wrong. I assure you it is wrong. We have this discussion once per year or so, when someone uses dumb buffers for something they are not for. This mostly happens because we didn't reject unsupported input into dumb buffer early enough. Not we're stuck with user space's de-facto semantics. For example, you can see this [1] discussion from Feb 2025 on color formats. Dumb buffers are meant to return linear RGB buffers, but user space allocates whatever it needs. So now we cannot fully reject invalid parameters without breaking something. Hence, the kernel now semi-whitelists and documents the exceptions. See [2]. There's multi-planar YUV in there! [1] https://lore.kernel.org/dri-devel/dcd59a75-7945-4a2e-99f9-3abbb3e9de14@ideasonboard.com/ [2] https://elixir.bootlin.com/linux/v7.2.8/source/drivers/gpu/drm/drm_dumb_buffers.c#L149 Best regards Thomas > > Best regards, > - Maíra > -- -- Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Frankenstr. 146, 90461 Nürnberg, Germany, www.suse.com GF: Stefan Gaiser, Jochen Jaser, Abhinav Puri, (HRB 36809, AG Nürnberg) ^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2026-10-06 6:50 UTC | newest] Thread overview: 17+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-08-13 4:27 drm/ssd130x: stale pixels when the GPU renders into the framebuffer Fabio Piparo 2026-08-14 18:34 ` Javier Martinez Canillas 2026-09-14 8:43 ` Thomas Zimmermann 2026-09-23 16:00 ` Fabio Piparo 2026-09-29 9:04 ` Thomas Zimmermann 2026-09-29 21:50 ` Maíra Canal 2026-09-30 6:48 ` Thomas Zimmermann 2026-09-30 7:48 ` Javier Martinez Canillas 2026-09-30 8:32 ` Thomas Zimmermann 2026-09-30 13:49 ` Maíra Canal 2026-09-30 15:52 ` Javier Martinez Canillas 2026-09-30 18:45 ` Fabio Piparo 2026-10-02 8:18 ` Thomas Zimmermann 2026-09-30 19:57 ` Maíra Canal 2026-10-02 8:12 ` Thomas Zimmermann 2026-10-02 21:13 ` Maíra Canal 2026-10-06 6:50 ` Thomas Zimmermann
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.