* [Ocfs2-devel] [RFC] ocfs2: Idea to make ocfs2_search_chain high efficiency @ 2015-08-24 12:30 Norton.Zhu 2015-08-24 16:57 ` Srinivas Eeda 2015-08-25 18:49 ` Tariq Saeed 0 siblings, 2 replies; 5+ messages in thread From: Norton.Zhu @ 2015-08-24 12:30 UTC (permalink / raw) To: ocfs2-devel In ocfs2_search_chain, I found it has low efficiency while searching an available gd in some circumstances: 1) The lun has a great many gd(it reads lots of unavailable gd(free bits is zero)); 2) Not too many gd, but the available gd is scattered in the unavailable gd(fragmentation); So I have an idea to optimize the search method: 1) Use the reserved member in the ocfs2_group_desc to make an available chain list(gds in the list are all available, free bits more than zero); 2) At the beginning, the chain list is the same with origin chain list; 3) While do allocation, it searches gd in the available chain list; 4) After each allocation, if some gd's free bits is zero, Remove it from the available chain list; 5) After each reclaim, if some gd's free bits change from zero to positive, Append it to the head of the available chain list; Once started with the basics outlined above, no unavailable gd will be read. Anyone has better ideas or advices? ^ permalink raw reply [flat|nested] 5+ messages in thread
* [Ocfs2-devel] [RFC] ocfs2: Idea to make ocfs2_search_chain high efficiency 2015-08-24 12:30 [Ocfs2-devel] [RFC] ocfs2: Idea to make ocfs2_search_chain high efficiency Norton.Zhu @ 2015-08-24 16:57 ` Srinivas Eeda 2015-08-25 1:47 ` Joel Becker 2015-08-25 18:49 ` Tariq Saeed 1 sibling, 1 reply; 5+ messages in thread From: Srinivas Eeda @ 2015-08-24 16:57 UTC (permalink / raw) To: ocfs2-devel Hi Norton, while the localalloc bmp is enabled the chances of a particular gd ending with zero free bits is very minimal. local alloc bmp will pick next gd once min number of free bits falls below localalloc bmp size. So next gd is picked while the current gd still has free space. Having said that, I understand the problem you are describing. Since we only move the gd with more free bits ahead of the previous unusable gd we end of rescanning unusable gds whenever the newly picked gd becomes unusable. This is inefficient and causes performance problem when fs gets fragmented I am not sure if we will be allowed to use the bg_reserved2 ... but if we are then we could use it to remember next gd we should look at. For example, if we assume there are 'N' GDs and we are currently seeing that GD10 has more free space ... so the order of GDs are GD10, GD1, GD2, ... GDN. Now if we end up using most of the space in GD10, according to current logic we re-read GD1, GD2.. GD9 and then read GD11 and pick that (assuming it has free space). But instead if we had GD10 remembers GD11 as the next one to look at ... then we can skip reading GD1..GD9. Thanks, --Srini On 08/24/2015 05:30 AM, Norton.Zhu wrote: > In ocfs2_search_chain, I found it has low efficiency while searching an available gd in some circumstances: > 1) The lun has a great many gd(it reads lots of unavailable gd(free bits is zero)); > 2) Not too many gd, but the available gd is scattered in the unavailable gd(fragmentation); > > So I have an idea to optimize the search method: > 1) Use the reserved member in the ocfs2_group_desc to make an available chain list(gds in the list are all available, free bits more than zero); > 2) At the beginning, the chain list is the same with origin chain list; > 3) While do allocation, it searches gd in the available chain list; > 4) After each allocation, if some gd's free bits is zero, Remove it from the available chain list; > 5) After each reclaim, if some gd's free bits change from zero to positive, Append it to the head of the available chain list; > > Once started with the basics outlined above, no unavailable gd will be read. > > Anyone has better ideas or advices? > > > _______________________________________________ > Ocfs2-devel mailing list > Ocfs2-devel at oss.oracle.com > https://oss.oracle.com/mailman/listinfo/ocfs2-devel ^ permalink raw reply [flat|nested] 5+ messages in thread
* [Ocfs2-devel] [RFC] ocfs2: Idea to make ocfs2_search_chain high efficiency 2015-08-24 16:57 ` Srinivas Eeda @ 2015-08-25 1:47 ` Joel Becker 2015-08-25 12:08 ` Norton.Zhu 0 siblings, 1 reply; 5+ messages in thread From: Joel Becker @ 2015-08-25 1:47 UTC (permalink / raw) To: ocfs2-devel Srini has summed it up, mostly. We shouldn't be hitting these suboptimal chains of gds unless the localalloc is empty. Hopefully that's not terribly frequent, but it can happen. Your solution does increase the complexity of the commit. Pushing a newly-available gd back on to the available list is cheap, as you're already modifing the chain_rec and the gd. But deleting a full gd from the available list requires you to find the previous entry in the list and edit it as well. The lock ordering is hopefully not terrible. Joel On Mon, Aug 24, 2015 at 09:57:26AM -0700, Srinivas Eeda wrote: > Hi Norton, > > while the localalloc bmp is enabled the chances of a particular gd > ending with zero free bits is very minimal. local alloc bmp will pick > next gd once min number of free bits falls below localalloc bmp size. So > next gd is picked while the current gd still has free space. > > Having said that, I understand the problem you are describing. Since we > only move the gd with more free bits ahead of the previous unusable gd > we end of rescanning unusable gds whenever the newly picked gd becomes > unusable. This is inefficient and causes performance problem when fs > gets fragmented > > I am not sure if we will be allowed to use the bg_reserved2 ... but if > we are then we could use it to remember next gd we should look at. > > For example, if we assume there are 'N' GDs and we are currently seeing > that GD10 has more free space ... so the order of GDs are > > GD10, GD1, GD2, ... GDN. > > Now if we end up using most of the space in GD10, according to current > logic we re-read GD1, GD2.. GD9 and then read GD11 and pick that > (assuming it has free space). But instead if we had GD10 remembers GD11 > as the next one to look at ... then we can skip reading GD1..GD9. > > Thanks, > --Srini > > > On 08/24/2015 05:30 AM, Norton.Zhu wrote: > > In ocfs2_search_chain, I found it has low efficiency while searching an available gd in some circumstances: > > 1) The lun has a great many gd(it reads lots of unavailable gd(free bits is zero)); > > 2) Not too many gd, but the available gd is scattered in the unavailable gd(fragmentation); > > > > So I have an idea to optimize the search method: > > 1) Use the reserved member in the ocfs2_group_desc to make an available chain list(gds in the list are all available, free bits more than zero); > > 2) At the beginning, the chain list is the same with origin chain list; > > 3) While do allocation, it searches gd in the available chain list; > > 4) After each allocation, if some gd's free bits is zero, Remove it from the available chain list; > > 5) After each reclaim, if some gd's free bits change from zero to positive, Append it to the head of the available chain list; > > > > Once started with the basics outlined above, no unavailable gd will be read. > > > > Anyone has better ideas or advices? > > > > > > _______________________________________________ > > Ocfs2-devel mailing list > > Ocfs2-devel at oss.oracle.com > > https://oss.oracle.com/mailman/listinfo/ocfs2-devel > > > _______________________________________________ > Ocfs2-devel mailing list > Ocfs2-devel at oss.oracle.com > https://oss.oracle.com/mailman/listinfo/ocfs2-devel -- ^ permalink raw reply [flat|nested] 5+ messages in thread
* [Ocfs2-devel] [RFC] ocfs2: Idea to make ocfs2_search_chain high efficiency 2015-08-25 1:47 ` Joel Becker @ 2015-08-25 12:08 ` Norton.Zhu 0 siblings, 0 replies; 5+ messages in thread From: Norton.Zhu @ 2015-08-25 12:08 UTC (permalink / raw) To: ocfs2-devel Srini's idea seems good, but I have no idea how to always keep the best GD, or keep it ordered in some way. And localalloc doesn't help sometimes, for example, my test case is: 1) I have formated the lun(size 96T, Cluster=32k, Block=4k); Almost 100 thousands of GDs; 2) Using fallocate 30G each time;(ocfs2_alloc_should_use_local returns 0 each time) When the lun is used about 80%, it costs lots of time to finish the fallcation. I want using GD0 as the available chain list head to keep some important info( the start begining gd). When find an available GD and delete it from the available chain list, If it's in the head, set the head to the head->next; If it's in the middle or tail, set cur->previous to the cur->next; PS, the origin chain list keep same each time(when allocation or deallocation), and the GD0 in the the two chain list is the same. About the overhead: Maybe it indeed increase the complexity of the commit(we need update 1or2or3 GD) after allocation, I'm thinking about how to make it simpler in some way. Any idea? Thanks, --Norton On 2015/8/25 9:47, Joel Becker wrote: > Srini has summed it up, mostly. We shouldn't be hitting these > suboptimal chains of gds unless the localalloc is empty. Hopefully > that's not terribly frequent, but it can happen. > > Your solution does increase the complexity of the commit. Pushing a > newly-available gd back on to the available list is cheap, as you're > already modifing the chain_rec and the gd. But deleting a full gd from > the available list requires you to find the previous entry in the list > and edit it as well. The lock ordering is hopefully not terrible. > > Joel > > On Mon, Aug 24, 2015 at 09:57:26AM -0700, Srinivas Eeda wrote: >> Hi Norton, >> >> while the localalloc bmp is enabled the chances of a particular gd >> ending with zero free bits is very minimal. local alloc bmp will pick >> next gd once min number of free bits falls below localalloc bmp size. So >> next gd is picked while the current gd still has free space. >> >> Having said that, I understand the problem you are describing. Since we >> only move the gd with more free bits ahead of the previous unusable gd >> we end of rescanning unusable gds whenever the newly picked gd becomes >> unusable. This is inefficient and causes performance problem when fs >> gets fragmented >> >> I am not sure if we will be allowed to use the bg_reserved2 ... but if >> we are then we could use it to remember next gd we should look at. >> >> For example, if we assume there are 'N' GDs and we are currently seeing >> that GD10 has more free space ... so the order of GDs are >> >> GD10, GD1, GD2, ... GDN. >> >> Now if we end up using most of the space in GD10, according to current >> logic we re-read GD1, GD2.. GD9 and then read GD11 and pick that >> (assuming it has free space). But instead if we had GD10 remembers GD11 >> as the next one to look at ... then we can skip reading GD1..GD9. >> >> Thanks, >> --Srini >> >> >> On 08/24/2015 05:30 AM, Norton.Zhu wrote: >>> In ocfs2_search_chain, I found it has low efficiency while searching an available gd in some circumstances: >>> 1) The lun has a great many gd(it reads lots of unavailable gd(free bits is zero)); >>> 2) Not too many gd, but the available gd is scattered in the unavailable gd(fragmentation); >>> >>> So I have an idea to optimize the search method: >>> 1) Use the reserved member in the ocfs2_group_desc to make an available chain list(gds in the list are all available, free bits more than zero); >>> 2) At the beginning, the chain list is the same with origin chain list; >>> 3) While do allocation, it searches gd in the available chain list; >>> 4) After each allocation, if some gd's free bits is zero, Remove it from the available chain list; >>> 5) After each reclaim, if some gd's free bits change from zero to positive, Append it to the head of the available chain list; >>> >>> Once started with the basics outlined above, no unavailable gd will be read. >>> >>> Anyone has better ideas or advices? >>> >>> >>> _______________________________________________ >>> Ocfs2-devel mailing list >>> Ocfs2-devel at oss.oracle.com >>> https://oss.oracle.com/mailman/listinfo/ocfs2-devel >> >> >> _______________________________________________ >> Ocfs2-devel mailing list >> Ocfs2-devel at oss.oracle.com >> https://oss.oracle.com/mailman/listinfo/ocfs2-devel > ^ permalink raw reply [flat|nested] 5+ messages in thread
* [Ocfs2-devel] [RFC] ocfs2: Idea to make ocfs2_search_chain high efficiency 2015-08-24 12:30 [Ocfs2-devel] [RFC] ocfs2: Idea to make ocfs2_search_chain high efficiency Norton.Zhu 2015-08-24 16:57 ` Srinivas Eeda @ 2015-08-25 18:49 ` Tariq Saeed 1 sibling, 0 replies; 5+ messages in thread From: Tariq Saeed @ 2015-08-25 18:49 UTC (permalink / raw) To: ocfs2-devel Another idea is to have a background thread to work in the background scanning chains to put the gd with optimum, not necessarily the maximum, number of free clusters in the front (similar to what in-line alloc does today, but do it in background). While a chain is being worked upon by the bg thread, it will be markedi "forbidden" to the normal in-line allocators, who will skip it. Eventually a state will be reached when an allocation request will find the 1st gd to satisfy alloc req, reducing and eliminating in-line traversal looking for a gd with large enough free chunk. . The thread will continuously work in the background on every chain in an endless loop. Thanks -Tariq Saeed On 08/24/2015 05:30 AM, Norton.Zhu wrote: > In ocfs2_search_chain, I found it has low efficiency while searching an available gd in some circumstances: > 1) The lun has a great many gd(it reads lots of unavailable gd(free bits is zero)); > 2) Not too many gd, but the available gd is scattered in the unavailable gd(fragmentation); > > So I have an idea to optimize the search method: > 1) Use the reserved member in the ocfs2_group_desc to make an available chain list(gds in the list are all available, free bits more than zero); > 2) At the beginning, the chain list is the same with origin chain list; > 3) While do allocation, it searches gd in the available chain list; > 4) After each allocation, if some gd's free bits is zero, Remove it from the available chain list; > 5) After each reclaim, if some gd's free bits change from zero to positive, Append it to the head of the available chain list; > > Once started with the basics outlined above, no unavailable gd will be read. > > Anyone has better ideas or advices? > > > _______________________________________________ > Ocfs2-devel mailing list > Ocfs2-devel at oss.oracle.com > https://oss.oracle.com/mailman/listinfo/ocfs2-devel ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2015-08-25 18:49 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2015-08-24 12:30 [Ocfs2-devel] [RFC] ocfs2: Idea to make ocfs2_search_chain high efficiency Norton.Zhu 2015-08-24 16:57 ` Srinivas Eeda 2015-08-25 1:47 ` Joel Becker 2015-08-25 12:08 ` Norton.Zhu 2015-08-25 18:49 ` Tariq Saeed
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.