Shared Memory Pages for Same Base Device

Shared Memory Pages for Same Base Device

[Shao Miller]

[-- Attachment #1.1: Type: text/plain, Size: 955 bytes --]

Good day to all.

If I've some block device "base" and I've two derived device-mapper 
devices "derived1" and "derived2" that are copy-on-write layers over top 
of that base, and I mount the filesystems on those derived block devices 
and run the same program "foo" from both, does the "foo" running from 
"derived1" share any [read and execute] memory pages with the "foo" 
running from "derived2", since the underlying sectors are both from the 
same position on "base"?

The more general question would be about mmap, but I hope this example 
is clear.  "Docker"[1] uses device-mapper in scenarios like the above 
example and I'm curious if they benefit from shared pages.

[1] https://www.docker.com/
-- 

Shao Miller
/Network Technician/
/905-836-4442 *ext: 112*/
www.treefrog.ca/shao-miller <http://www.treefrog.ca/shao-miller>
*
* /*Treefrog Inc.*/
905-836-4442
567 Davis Drive, Newmarket, ON
www.treefrog.ca <http://www.treefrog.ca/> - @Treefrog


[-- Attachment #1.2: Type: text/html, Size: 2476 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Shared Memory Pages for Same Base Device

[Mike Snitzer]

On Sat, Sep 20 2014 at 11:50am -0400,
Shao Miller <shao.miller@treefrog.ca> wrote:

> Good day to all.
> 
> If I've some block device "base" and I've two derived device-mapper
> devices "derived1" and "derived2" that are copy-on-write layers over
> top of that base, and I mount the filesystems on those derived block
> devices and run the same program "foo" from both, does the "foo"
> running from "derived1" share any [read and execute] memory pages
> with the "foo" running from "derived2", since the underlying sectors
> are both from the same position on "base"?
> 
> The more general question would be about mmap, but I hope this
> example is clear.  "Docker"[1] uses device-mapper in scenarios like
> the above example and I'm curious if they benefit from shared pages.
> 
> [1] https://www.docker.com/

Unfortunately device-mapper thin provisioning doesn't offer shared
pagecache pages across snapshot volumes.  This is a block layer
limitation (the block layer doesn't allow pages to be shared across
block devices, and dm-thinp snapshot volumes are each a block device).
Modifying the VM, block and DM subsystems to provide this capability is
not an easy task and as such is really not a near-term priority.

Interestingly BTRFS does _not_ offer this page sharing either.  I'm told
that the only emerging solution for this is overlayfs.

Re: Shared Memory Pages for Same Base Device

[Shao Miller]

On 9/22/2014 9:08 AM, Mike Snitzer wrote:
>
> Unfortunately device-mapper thin provisioning doesn't offer shared
> pagecache pages across snapshot volumes.  This is a block layer
> limitation (the block layer doesn't allow pages to be shared across
> block devices, and dm-thinp snapshot volumes are each a block device).
> Modifying the VM, block and DM subsystems to provide this capability is
> not an easy task and as such is really not a near-term priority.

I sincerely appreciate your response, Mike.  I figured as much, but 
hopefully this thread will be useful to future searchers.

> Interestingly BTRFS does _not_ offer this page sharing either.  I'm told
> that the only emerging solution for this is overlayfs.
This is very surprising.  I've just done some testing and /proc/PID/maps 
for the same binary from two different snapshots deriving from the same 
base do show the same block-device major and minor, with the same 
offset.  Do you happen to have a reference for that detail about BTRFS, 
or perhaps an idea for the right venue to learn more about that?  
Unfortunately, I guess it's off-topic for device-mapper.

Also unfortunately, it seems that overlayfs isn't mainline Linux nor 
out-of-the-box for some distributions.  (Like CentOS 7.)

I enjoy device-mapper for CoW iSCSI and AoE.  Keep up the great work, folks!

-- 

Shao Miller
/Network Technician/
/905-836-4442 *ext: 112*/
www.treefrog.ca/shao-miller <http://www.treefrog.ca/shao-miller>
*
* /*Treefrog Inc.*/ 905-836-4442
567 Davis Drive, Newmarket, ON
www.treefrog.ca <http://www.treefrog.ca/> - @Treefrog

Re: Shared Memory Pages for Same Base Device

[Mike Snitzer]

On Mon, Sep 22 2014 at 10:08am -0400,
Shao Miller <shao.miller@treefrog.ca> wrote:

> On 9/22/2014 9:08 AM, Mike Snitzer wrote:
> >
> >Unfortunately device-mapper thin provisioning doesn't offer shared
> >pagecache pages across snapshot volumes.  This is a block layer
> >limitation (the block layer doesn't allow pages to be shared across
> >block devices, and dm-thinp snapshot volumes are each a block device).
> >Modifying the VM, block and DM subsystems to provide this capability is
> >not an easy task and as such is really not a near-term priority.
> 
> I sincerely appreciate your response, Mike.  I figured as much, but
> hopefully this thread will be useful to future searchers.
> 
> >Interestingly BTRFS does _not_ offer this page sharing either.  I'm told
> >that the only emerging solution for this is overlayfs.
> This is very surprising.  I've just done some testing and
> /proc/PID/maps for the same binary from two different snapshots
> deriving from the same base do show the same block-device major and
> minor, with the same offset.  Do you happen to have a reference for
> that detail about BTRFS, or perhaps an idea for the right venue to
> learn more about that?  Unfortunately, I guess it's off-topic for
> device-mapper.

I learned as much from Chris Mason (cc'd) in a FB thread I started on
this very topic.  Chris said:

"It would be awesome if we had a way to share page cache pages cow style
across filesystem snapshots. Every file ends up with its own address
space, so we really need to just share the pages. It's complex though"
 
> Also unfortunately, it seems that overlayfs isn't mainline Linux nor
> out-of-the-box for some distributions.  (Like CentOS 7.)
> 
> I enjoy device-mapper for CoW iSCSI and AoE.  Keep up the great work, folks!

Thanks