Re: zsmalloc defrag (Was: [PATCH] mm: remove compressed copy from zram in-memory)

[Ric Mason <ric.masonn@gmail.com>]

Hi Dan,
On 04/10/2013 04:25 AM, Dan Magenheimer wrote:
>> From: Minchan Kim [mailto:minchan@kernel.org]
>> Subject: Re: zsmalloc defrag (Was: [PATCH] mm: remove compressed copy from zram in-memory)
>>
>> Hi Dan,
>>
>> On Mon, Apr 08, 2013 at 09:32:38AM -0700, Dan Magenheimer wrote:
>>>> From: Minchan Kim [mailto:minchan@kernel.org]
>>>> Sent: Monday, April 08, 2013 12:01 AM
>>>> Subject: [PATCH] mm: remove compressed copy from zram in-memory
>>> (patch removed)
>>>
>>>> Fragment ratio is almost same but memory consumption and compile time
>>>> is better. I am working to add defragment function of zsmalloc.
>>> Hi Minchan --
>>>
>>> I would be very interested in your design thoughts on
>>> how you plan to add defragmentation for zsmalloc.  In
>> What I can say now about is only just a word "Compaction".
>> As you know, zsmalloc has a transparent handle so we can do whatever
>> under user. Of course, there is a tradeoff between performance
>> and memory efficiency. I'm biased to latter for embedded usecase.
> Have you designed or implemented this yet?  I have a couple
> of concerns:
>
> 1) The handle is transparent to the "user", but it is still a form
>     of a "pointer" to a zpage.  Are you planning on walking zram's
>     tables and changing those pointers?  That may be OK for zram
>     but for more complex data structures than tables (as in zswap
>     and zcache) it may not be as easy, due to races, or as efficient
>     because you will have to walk potentially very large trees.
> 2) Compaction in the kernel is heavily dependent on page migration
>     and page migration is dependent on using flags in the struct page.

Which flag?

>     There's a lot of code in those two code modules and there
>     are going to be a lot of implementation differences between
>     compacting pages vs compacting zpages.
>
> I'm also wondering if you will be implementing "variable length
> zspages".  Without that, I'm not sure compaction will help
> enough.  (And that is a good example of the difference between
> the kernel page compaction design/code and zspage compaction.)
>
>>> particular, I am wondering if your design will also
>>> handle the requirements for zcache (especially for
>>> cleancache pages) and perhaps also for ramster.
>> I don't know requirements for cleancache pages but compaction is
>> general as you know well so I expect you can get a benefit from it
>> if you are concern on memory efficiency but not sure it's valuable
>> to compact cleancache pages for getting more slot in RAM.
>> Sometime, just discarding would be much better, IMHO.
> Zcache has page reclaim.  Zswap has zpage reclaim.  I am
> concerned that these continue to work in the presence of
> compaction.   With no reclaim at all, zram is a simpler use
> case but if you implement compaction in a way that can't be
> used by either zcache or zswap, then zsmalloc is essentially
> forking.

I fail to understand "then zsmalloc is essentially forking.", could you 
explain more?

>
>>> In https://lkml.org/lkml/2013/3/27/501 I suggested it
>>> would be good to work together on a common design, but
>>> you didn't reply.  Are you thinking that zsmalloc
>> I saw the thread but explicit agreement is really matter?
>> I believe everybody want it although they didn't reply. :)
>>
>> You can make the design/post it or prototyping/post it.
>> If there are some conflit with something in my brain,
>> I will be happy to feedback. :)
>>
>> Anyway, I think my above statement "COMPACTION" would be enough to
>> express my current thought to avoid duplicated work and you can catch up.
>>
>> I will get around to it after LSF/MM.
>>
>>> improvements should focus only on zram, in which case
>> Just focusing zsmalloc.
> Right.  Again, I am asking if you are changing zsmalloc in
> a way that helps zram but hurts zswap and makes it impossible
> for zcache to ever use the improvements to zsmalloc.
>
> If so, that's fine, but please make it clear that is your goal.
>
>>> we may -- and possibly should -- end up with a different
>>> allocator for frontswap-based/cleancache-based compression
>>> in zcache (and possibly zswap)?
>>> I'm just trying to determine if I should proceed separately
>>> with my design (with Bob Liu, who expressed interest) or if
>>> it would be beneficial to work together.
>> Just posting and if it affects zsmalloc/zram/zswap and goes the way
>> I don't want, I will involve the discussion because our product uses
>> zram heavily and consider zswap, too.
>>
>> I really appreciate your enthusiastic collaboration model to find
>> optimal solution!
> My goal is to have compression be an integral part of Linux
> memory management.  It may be tied to a config option, but
> the goal is that distros turn it on by default.  I don't think
> zsmalloc meets that objective yet, but it may be fine for
> your needs.  If so it would be good to understand exactly why
> it doesn't meet the other zproject needs.
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=ilto:"dont@kvack.org"> email@kvack.org </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>