* [Ocfs2-devel] Why ocfs2 haven't implemented "steal" for local_alloc system files?
@ 2013-07-27 10:27 Younger Liu
2013-08-01 3:33 ` Sunil Mushran
0 siblings, 1 reply; 7+ messages in thread
From: Younger Liu @ 2013-07-27 10:27 UTC (permalink / raw)
To: ocfs2-devel
Hi,
While analyzing ocfs2 block allocation, I found:
When claiming space from inode_alloc (or extent_alloc) system files,
if there is no enough space in inode_alloc (or extent_alloc) and
global_bitmap, it could steal space from other slots.
But when claiming space from local_alloc system files, and no
enough space in local_alloc and global_bitmap, it returns -ENOSPC.
Why ocfs2 haven't implemented "steal" for local_alloc system files?
Is there any some reasons?
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Ocfs2-devel] Why ocfs2 haven't implemented "steal" for local_alloc system files?
2013-07-27 10:27 [Ocfs2-devel] Why ocfs2 haven't implemented "steal" for local_alloc system files? Younger Liu
@ 2013-08-01 3:33 ` Sunil Mushran
2013-08-01 7:20 ` Joel Becker
0 siblings, 1 reply; 7+ messages in thread
From: Sunil Mushran @ 2013-08-01 3:33 UTC (permalink / raw)
To: ocfs2-devel
Because it makes no sense. Unlike inode/extent allocs, local_alloc is a
temporary cache. If you fail to allocate, you fallback to the global bitmap.
On Sat, Jul 27, 2013 at 3:27 AM, Younger Liu <younger.liu@huawei.com> wrote:
> Hi,
> While analyzing ocfs2 block allocation, I found:
> When claiming space from inode_alloc (or extent_alloc) system files,
> if there is no enough space in inode_alloc (or extent_alloc) and
> global_bitmap, it could steal space from other slots.
> But when claiming space from local_alloc system files, and no
> enough space in local_alloc and global_bitmap, it returns -ENOSPC.
>
> Why ocfs2 haven't implemented "steal" for local_alloc system files?
> Is there any some reasons?
>
>
> _______________________________________________
> Ocfs2-devel mailing list
> Ocfs2-devel at oss.oracle.com
> https://oss.oracle.com/mailman/listinfo/ocfs2-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oss.oracle.com/pipermail/ocfs2-devel/attachments/20130731/e1a999fb/attachment.html
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Ocfs2-devel] Why ocfs2 haven't implemented "steal" for local_alloc system files?
2013-08-01 3:33 ` Sunil Mushran
@ 2013-08-01 7:20 ` Joel Becker
2013-08-01 8:38 ` Younger Liu
0 siblings, 1 reply; 7+ messages in thread
From: Joel Becker @ 2013-08-01 7:20 UTC (permalink / raw)
To: ocfs2-devel
Basically, there's so little in the cache that stealing would be too
complex. Really we just want to fall back to the global, and if that is
empty, you're near enough ENOSPC that it doesn't much matter.
Joel
On Wed, Jul 31, 2013 at 08:33:48PM -0700, Sunil Mushran wrote:
> Because it makes no sense. Unlike inode/extent allocs, local_alloc is a
> temporary cache. If you fail to allocate, you fallback to the global bitmap.
>
>
> On Sat, Jul 27, 2013 at 3:27 AM, Younger Liu <younger.liu@huawei.com> wrote:
>
> > Hi,
> > While analyzing ocfs2 block allocation, I found:
> > When claiming space from inode_alloc (or extent_alloc) system files,
> > if there is no enough space in inode_alloc (or extent_alloc) and
> > global_bitmap, it could steal space from other slots.
> > But when claiming space from local_alloc system files, and no
> > enough space in local_alloc and global_bitmap, it returns -ENOSPC.
> >
> > Why ocfs2 haven't implemented "steal" for local_alloc system files?
> > Is there any some reasons?
> >
> >
> > _______________________________________________
> > Ocfs2-devel mailing list
> > Ocfs2-devel at oss.oracle.com
> > https://oss.oracle.com/mailman/listinfo/ocfs2-devel
> >
> _______________________________________________
> Ocfs2-devel mailing list
> Ocfs2-devel at oss.oracle.com
> https://oss.oracle.com/mailman/listinfo/ocfs2-devel
--
"Born under a bad sign.
I been down since I began to crawl.
If it wasn't for bad luck,
I wouldn't have no luck at all."
http://www.jlbec.org/
jlbec at evilplan.org
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Ocfs2-devel] Why ocfs2 haven't implemented "steal" for local_alloc system files?
2013-08-01 7:20 ` Joel Becker
@ 2013-08-01 8:38 ` Younger Liu
2013-08-01 16:49 ` Sunil Mushran
2013-08-01 18:57 ` Srinivas Eeda
0 siblings, 2 replies; 7+ messages in thread
From: Younger Liu @ 2013-08-01 8:38 UTC (permalink / raw)
To: ocfs2-devel
On 2013/8/1 15:20, Joel Becker wrote:
> Basically, there's so little in the cache that stealing would be too
> complex. Really we just want to fall back to the global, and if that is
> empty, you're near enough ENOSPC that it doesn't much matter.
>
> Joel
>
Localalloc is a mount option. When mounting an ocfs2 volume, localalloc can
be set a larger size. For example, 2G or a larger number.
In a multi-nodes cluster, all nodes mount ocfs2 volume with localalloc=2048
for performance consideration, if a node claims space for a file, but there
is no space in the local_alloc and global_bitmap. It would return ENOSPC.
However, other nodes may have lots of space which have been claimed
in local_alloc, so ENOSPC cannot reflect the actual situation.
IMO, if there is no space both in the local_alloc and global_bitmap,
it should steal space from other nodes local_alloc.
Thx,
Younger.
> On Wed, Jul 31, 2013 at 08:33:48PM -0700, Sunil Mushran wrote:
>> Because it makes no sense. Unlike inode/extent allocs, local_alloc is a
>> temporary cache. If you fail to allocate, you fallback to the global bitmap.
>>
>>
>> On Sat, Jul 27, 2013 at 3:27 AM, Younger Liu <younger.liu@huawei.com> wrote:
>>
>>> Hi,
>>> While analyzing ocfs2 block allocation, I found:
>>> When claiming space from inode_alloc (or extent_alloc) system files,
>>> if there is no enough space in inode_alloc (or extent_alloc) and
>>> global_bitmap, it could steal space from other slots.
>>> But when claiming space from local_alloc system files, and no
>>> enough space in local_alloc and global_bitmap, it returns -ENOSPC.
>>>
>>> Why ocfs2 haven't implemented "steal" for local_alloc system files?
>>> Is there any some reasons?
>>>
>>>
>>> _______________________________________________
>>> Ocfs2-devel mailing list
>>> Ocfs2-devel at oss.oracle.com
>>> https://oss.oracle.com/mailman/listinfo/ocfs2-devel
>>>
>
>> _______________________________________________
>> Ocfs2-devel mailing list
>> Ocfs2-devel at oss.oracle.com
>> https://oss.oracle.com/mailman/listinfo/ocfs2-devel
>
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Ocfs2-devel] Why ocfs2 haven't implemented "steal" for local_alloc system files?
2013-08-01 8:38 ` Younger Liu
@ 2013-08-01 16:49 ` Sunil Mushran
2013-08-01 18:57 ` Srinivas Eeda
1 sibling, 0 replies; 7+ messages in thread
From: Sunil Mushran @ 2013-08-01 16:49 UTC (permalink / raw)
To: ocfs2-devel
The complexity is not worth it. 2G*16 nodes is only 32G. That's rounding
error in systems that use this file system. And you are assuming that all
other nodes have all 2G in their cache. If a node runs out of space, a more
realistic scenario is that most other nodes are also close to the end.
On Thu, Aug 1, 2013 at 1:38 AM, Younger Liu <younger.liu@huawei.com> wrote:
> On 2013/8/1 15:20, Joel Becker wrote:
> > Basically, there's so little in the cache that stealing would be too
> > complex. Really we just want to fall back to the global, and if that is
> > empty, you're near enough ENOSPC that it doesn't much matter.
> >
> > Joel
> >
> Localalloc is a mount option. When mounting an ocfs2 volume, localalloc can
> be set a larger size. For example, 2G or a larger number.
>
> In a multi-nodes cluster, all nodes mount ocfs2 volume with localalloc=2048
> for performance consideration, if a node claims space for a file, but there
> is no space in the local_alloc and global_bitmap. It would return ENOSPC.
> However, other nodes may have lots of space which have been claimed
> in local_alloc, so ENOSPC cannot reflect the actual situation.
>
> IMO, if there is no space both in the local_alloc and global_bitmap,
> it should steal space from other nodes local_alloc.
>
> Thx,
> Younger.
>
> > On Wed, Jul 31, 2013 at 08:33:48PM -0700, Sunil Mushran wrote:
> >> Because it makes no sense. Unlike inode/extent allocs, local_alloc is a
> >> temporary cache. If you fail to allocate, you fallback to the global
> bitmap.
> >>
> >>
> >> On Sat, Jul 27, 2013 at 3:27 AM, Younger Liu <younger.liu@huawei.com>
> wrote:
> >>
> >>> Hi,
> >>> While analyzing ocfs2 block allocation, I found:
> >>> When claiming space from inode_alloc (or extent_alloc) system files,
> >>> if there is no enough space in inode_alloc (or extent_alloc) and
> >>> global_bitmap, it could steal space from other slots.
> >>> But when claiming space from local_alloc system files, and no
> >>> enough space in local_alloc and global_bitmap, it returns -ENOSPC.
> >>>
> >>> Why ocfs2 haven't implemented "steal" for local_alloc system files?
> >>> Is there any some reasons?
> >>>
> >>>
> >>> _______________________________________________
> >>> Ocfs2-devel mailing list
> >>> Ocfs2-devel at oss.oracle.com
> >>> https://oss.oracle.com/mailman/listinfo/ocfs2-devel
> >>>
> >
> >> _______________________________________________
> >> Ocfs2-devel mailing list
> >> Ocfs2-devel at oss.oracle.com
> >> https://oss.oracle.com/mailman/listinfo/ocfs2-devel
> >
> >
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oss.oracle.com/pipermail/ocfs2-devel/attachments/20130801/7dde9694/attachment.html
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Ocfs2-devel] Why ocfs2 haven't implemented "steal" for local_alloc system files?
2013-08-01 8:38 ` Younger Liu
2013-08-01 16:49 ` Sunil Mushran
@ 2013-08-01 18:57 ` Srinivas Eeda
2013-08-02 9:47 ` Younger Liu
1 sibling, 1 reply; 7+ messages in thread
From: Srinivas Eeda @ 2013-08-01 18:57 UTC (permalink / raw)
To: ocfs2-devel
On 08/01/2013 01:38 AM, Younger Liu wrote:
> On 2013/8/1 15:20, Joel Becker wrote:
>> Basically, there's so little in the cache that stealing would be too
>> complex. Really we just want to fall back to the global, and if that is
>> empty, you're near enough ENOSPC that it doesn't much matter.
>>
>> Joel
>>
> Localalloc is a mount option. When mounting an ocfs2 volume, localalloc can
> be set a larger size. For example, 2G or a larger number.
>
> In a multi-nodes cluster, all nodes mount ocfs2 volume with localalloc=2048
Is there a proof that localalloc=2048 performs better than default 256Mb
? local alloc currently looks for contiguous chunks. setting it to 2gb
means it has to scan the whole bitmap to find that big of a chunk which
in itself defeats the purpose of setting it to 2gb. I mean it might work
well for a filesystem that just got formatted but as it gets used
finding 2gb chunks is hard.
> for performance consideration, if a node claims space for a file, but there
> is no space in the local_alloc and global_bitmap. It would return ENOSPC.
> However, other nodes may have lots of space which have been claimed
> in local_alloc, so ENOSPC cannot reflect the actual situation.
>
> IMO, if there is no space both in the local_alloc and global_bitmap,
> it should steal space from other nodes local_alloc.
>
> Thx,
> Younger.
>
>> On Wed, Jul 31, 2013 at 08:33:48PM -0700, Sunil Mushran wrote:
>>> Because it makes no sense. Unlike inode/extent allocs, local_alloc is a
>>> temporary cache. If you fail to allocate, you fallback to the global bitmap.
>>>
>>>
>>> On Sat, Jul 27, 2013 at 3:27 AM, Younger Liu <younger.liu@huawei.com> wrote:
>>>
>>>> Hi,
>>>> While analyzing ocfs2 block allocation, I found:
>>>> When claiming space from inode_alloc (or extent_alloc) system files,
>>>> if there is no enough space in inode_alloc (or extent_alloc) and
>>>> global_bitmap, it could steal space from other slots.
>>>> But when claiming space from local_alloc system files, and no
>>>> enough space in local_alloc and global_bitmap, it returns -ENOSPC.
>>>>
>>>> Why ocfs2 haven't implemented "steal" for local_alloc system files?
>>>> Is there any some reasons?
>>>>
>>>>
>>>> _______________________________________________
>>>> Ocfs2-devel mailing list
>>>> Ocfs2-devel at oss.oracle.com
>>>> https://oss.oracle.com/mailman/listinfo/ocfs2-devel
>>>>
>>> _______________________________________________
>>> Ocfs2-devel mailing list
>>> Ocfs2-devel at oss.oracle.com
>>> https://oss.oracle.com/mailman/listinfo/ocfs2-devel
>>
>
>
> _______________________________________________
> Ocfs2-devel mailing list
> Ocfs2-devel at oss.oracle.com
> https://oss.oracle.com/mailman/listinfo/ocfs2-devel
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Ocfs2-devel] Why ocfs2 haven't implemented "steal" for local_alloc system files?
2013-08-01 18:57 ` Srinivas Eeda
@ 2013-08-02 9:47 ` Younger Liu
0 siblings, 0 replies; 7+ messages in thread
From: Younger Liu @ 2013-08-02 9:47 UTC (permalink / raw)
To: ocfs2-devel
On 2013/8/2 2:57, Srinivas Eeda wrote:
> On 08/01/2013 01:38 AM, Younger Liu wrote:
>> On 2013/8/1 15:20, Joel Becker wrote:
>>> Basically, there's so little in the cache that stealing would be too
>>> complex. Really we just want to fall back to the global, and if that is
>>> empty, you're near enough ENOSPC that it doesn't much matter.
>>>
>>> Joel
>>>
>> Localalloc is a mount option. When mounting an ocfs2 volume, localalloc can
>> be set a larger size. For example, 2G or a larger number.
>>
>> In a multi-nodes cluster, all nodes mount ocfs2 volume with localalloc=2048
> Is there a proof that localalloc=2048 performs better than default 256Mb
> ? local alloc currently looks for contiguous chunks. setting it to 2gb
> means it has to scan the whole bitmap to find that big of a chunk which
> in itself defeats the purpose of setting it to 2gb. I mean it might work
> well for a filesystem that just got formatted but as it gets used
> finding 2gb chunks is hard.
yes, I have proved that localalloc=2048 performs better, it improves about 5%.
In the scenario, ocfs2 volume stores virtual images which are very large,
for example, 20G or 50G.
>
>> for performance consideration, if a node claims space for a file, but there
>> is no space in the local_alloc and global_bitmap. It would return ENOSPC.
>> However, other nodes may have lots of space which have been claimed
>> in local_alloc, so ENOSPC cannot reflect the actual situation.
>>
>> IMO, if there is no space both in the local_alloc and global_bitmap,
>> it should steal space from other nodes local_alloc.
>>
>> Thx,
>> Younger.
>>
>>> On Wed, Jul 31, 2013 at 08:33:48PM -0700, Sunil Mushran wrote:
>>>> Because it makes no sense. Unlike inode/extent allocs, local_alloc is a
>>>> temporary cache. If you fail to allocate, you fallback to the global bitmap.
>>>>
>>>>
>>>> On Sat, Jul 27, 2013 at 3:27 AM, Younger Liu <younger.liu@huawei.com> wrote:
>>>>
>>>>> Hi,
>>>>> While analyzing ocfs2 block allocation, I found:
>>>>> When claiming space from inode_alloc (or extent_alloc) system files,
>>>>> if there is no enough space in inode_alloc (or extent_alloc) and
>>>>> global_bitmap, it could steal space from other slots.
>>>>> But when claiming space from local_alloc system files, and no
>>>>> enough space in local_alloc and global_bitmap, it returns -ENOSPC.
>>>>>
>>>>> Why ocfs2 haven't implemented "steal" for local_alloc system files?
>>>>> Is there any some reasons?
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Ocfs2-devel mailing list
>>>>> Ocfs2-devel at oss.oracle.com
>>>>> https://oss.oracle.com/mailman/listinfo/ocfs2-devel
>>>>>
>>>> _______________________________________________
>>>> Ocfs2-devel mailing list
>>>> Ocfs2-devel at oss.oracle.com
>>>> https://oss.oracle.com/mailman/listinfo/ocfs2-devel
>>>
>>
>>
>> _______________________________________________
>> Ocfs2-devel mailing list
>> Ocfs2-devel at oss.oracle.com
>> https://oss.oracle.com/mailman/listinfo/ocfs2-devel
>
>
> _______________________________________________
> Ocfs2-devel mailing list
> Ocfs2-devel at oss.oracle.com
> https://oss.oracle.com/mailman/listinfo/ocfs2-devel
>
>
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2013-08-02 9:47 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-07-27 10:27 [Ocfs2-devel] Why ocfs2 haven't implemented "steal" for local_alloc system files? Younger Liu
2013-08-01 3:33 ` Sunil Mushran
2013-08-01 7:20 ` Joel Becker
2013-08-01 8:38 ` Younger Liu
2013-08-01 16:49 ` Sunil Mushran
2013-08-01 18:57 ` Srinivas Eeda
2013-08-02 9:47 ` Younger Liu
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.