* jewel backport of https://github.com/ceph/ceph/pull/11326
@ 2016-10-11 9:54 Loic Dachary
2016-10-11 14:11 ` Jason Dillaman
0 siblings, 1 reply; 3+ messages in thread
From: Loic Dachary @ 2016-10-11 9:54 UTC (permalink / raw)
To: Mykola Golub, Jason Dillaman; +Cc: Ceph Development
Hi,
Trying to cherry pick commits instead of resolving conflicts and it looks like I'm going to end up adding a number of commits. Before I go down this path, I'd like a confirmation from you that it's the right call (see below for more information about what I did).
Thanks !
git cherry-pick -x dc77a62 from https://github.com/ceph/ceph/pull/11326 conflicts on
unmerged src/journal/JournalRecorder.cc
@@@ -279,8 -281,7 +276,12 @@@ void JournalRecorder::create_next_objec
new_object_recorder->get_object_number());
}
++<<<<<<< HEAD
+ new_object_recorder->append(append_buffers, false);
+
++=======
+ new_object_recorder->append_unlock(std::move(append_buffers));
++>>>>>>> dc77a62... journal: avoid holding lock while sending journal append
m_object_ptrs[splay_offset] = new_object_recorder;
}
----------------------------------------------------------------------
git log --no-merges --oneline --cherry-mark --left-right ceph/jewel...ceph/master -- src/journal/JournalRecorder.cc
> dc77a62 journal: avoid holding lock while sending journal append
> 7b740f5 journal: make librados call async in ObjectRecorder
> 5c88edd journal: increase concurrency of journal recorder
= b4b9843 journal: improve debug log messages
= 11475f4 journal: improve debug log messages
= 30c7f4d journal: update commit entry object number upon overflow
= 5fbf5f8 journal: update commit entry object number upon overflow
= 18f663d journal: close, advance, and open object set ordering
= 27fba6a journal: new ObjectRecorder closed callback
= de83005 journal: close, advance, and open object set ordering
= bba9143 journal: new ObjectRecorder closed callback
= f1a2e6a journal: prohibit journal entries larger than the soft object size
= d5f307d journal: prohibit journal entries larger than the soft object size
git cherry-pick -x 5c88edd conflicts on
unmerged src/librbd/Journal.cc
@@@ -17,8 -16,11 +17,9 @@@
#include "common/Timer.h"
#include "common/WorkQueue.h"
#include "include/rados/librados.hpp"
-#include "librbd/journal/RemoveRequest.h"
-#include "librbd/journal/CreateRequest.h"
#include <boost/scope_exit.hpp>
+ #include <utility>
#define dout_subsys ceph_subsys_rbd
#undef dout_prefix
@@@ -705,40 -671,56 +706,70 @@@ int Journal<I>::demote()
CephContext *cct = m_image_ctx.cct;
ldout(cct, 20) << __func__ << dendl;
++<<<<<<< HEAD
+ Mutex::Locker locker(m_lock);
+ assert(m_journaler != nullptr && is_tag_owner());
++=======
+ int r;
+ C_SaferCond ctx;
+ Future future;
+ C_SaferCond flush_ctx;
++>>>>>>> 7b740f5... journal: make librados call async in ObjectRecorder
----------------------------------------------------------------------
git log --no-merges --oneline --cherry-mark --left-right ceph/jewel...ceph/master -- src/librbd/Journal.cc
> 953561f librbd: async method to check journal tag owner
> 7b740f5 journal: make librados call async in ObjectRecorder
> 5c88edd journal: increase concurrency of journal recorder
> fd00549 librbd: new journal listener event for force promotion
> dbbcecf librbd: unify journal event callbacks into single interface
> b8eafef librbd: optionally flag "laggy" journal clients disconnected
> 551d7ea librbd: remove use of owner_lock on IO path
= e83866b librbd: do not record journal events if append is disabled by policy
= 1f63307 librbd: optimize header file dependency & modify related file.
= 549aada librbd: new configuration option to restrict journal payload size
= 99195e8 librbd: wait for journal commit op event to be safely recorded
= 91c70f3 journal: helper class for organizing optional settings
= f71dc87 librbd: improve journaling debug log messages
= 14db535 librbd: separate journal event decoding and processing
> ac590e8 librbd: normalize journal promotion/demotion events
> 718befd librbd: separate journal::TagData predecessor data into new struct
> 47d1e62 rbd-mirror: include tag tid in bootstrap debug log messages
= c2b786c rbd: journal: Support for listening updates on client metadata
> 3492153 journaler: remove ununsed synchronous API routines
> d5fa466 librbd / journal: async journal creation state machine
> c4e439d librbd / journal: async journal removal state machine
> 1416f1c librbd / ImageCtx: singleton safetimer instance helper
= 405142c librbd: do not record journal events if append is disabled by policy
= 11d7500 librbd: new configuration option to restrict journal payload size
= 47e0fbf librbd: wait for journal commit op event to be safely recorded
= dad8328 journal: helper class for organizing optional settings
= 270cb74 librbd: improve journaling debug log messages
= 57cd75e librbd: separate journal event decoding and processing
= f3f4a4a librbd: journal callback to interrupt replay
= deb6ca8 librbd: force-remove journal when disabling feature and removing image
= 2185a62 librbd: optimize header file dependency & modify related file.
= 9687e5e librbd: journal callback to interrupt replay
= d83aa1e librbd: force-remove journal when disabling feature and removing image
= 0dd8573 rbd: journal: Support for listening updates on client metadata
= f0bf15e librbd: stop journal recorder before starting external replay
= 0014a9e librbd: flush journal commit positions before starting op
= ef12536 librbd: resize and snap create can hang on journal replay failure
= c8b4cab librbd: missing journal state for flushing/restarting replay
= de29be7 librbd: Journal no longer requires AioCompletion for IO events
= 65556aa librbd: bubble journal error up via existing AioObjectRequests
= 5aa6eb2 librbd: stop journal recorder before starting external replay
= 6361296 librbd: flush journal commit positions before starting op
= 1032f19 librbd: resize and snap create can hang on journal replay failure
= a7f5bc1 librbd: missing journal state for flushing/restarting replay
= 82587a3 librbd: Journal no longer requires AioCompletion for IO events
= 79c9347 librbd: bubble journal error up via existing AioObjectRequests
= bf4c458 librbd: integrate with async journaler shutdown API
= 0780230 rbd-mirror: Added implementation of image-deleter thread
= ad29785 librbd: integrate with async journaler shutdown API
= 6cc6e66 rbd-mirror: Added implementation of image-deleter thread
= 5ec1f79 librbd: split large AioWrite journal events
= f221fed librbd: assertion to ensure no concurrent processing of replay events
= 44827a3 librbd: assertion to ensure no concurrent processing of replay events
= 70b9fbf librbd: split large AioWrite journal events
--
Loïc Dachary, Artisan Logiciel Libre
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: jewel backport of https://github.com/ceph/ceph/pull/11326
2016-10-11 9:54 jewel backport of https://github.com/ceph/ceph/pull/11326 Loic Dachary
@ 2016-10-11 14:11 ` Jason Dillaman
2016-10-11 14:15 ` Loic Dachary
0 siblings, 1 reply; 3+ messages in thread
From: Jason Dillaman @ 2016-10-11 14:11 UTC (permalink / raw)
To: Loic Dachary; +Cc: Mykola Golub, Ceph Development
I think the biggest set of conflicts will come from
https://github.com/ceph/ceph/pull/10445, which will need to be merged
along with #11326. If you'd like, I can take care of this backport
today.
On Tue, Oct 11, 2016 at 5:54 AM, Loic Dachary <loic@dachary.org> wrote:
> Hi,
>
> Trying to cherry pick commits instead of resolving conflicts and it looks like I'm going to end up adding a number of commits. Before I go down this path, I'd like a confirmation from you that it's the right call (see below for more information about what I did).
>
> Thanks !
>
> git cherry-pick -x dc77a62 from https://github.com/ceph/ceph/pull/11326 conflicts on
>
> unmerged src/journal/JournalRecorder.cc
> @@@ -279,8 -281,7 +276,12 @@@ void JournalRecorder::create_next_objec
> new_object_recorder->get_object_number());
> }
>
> ++<<<<<<< HEAD
> + new_object_recorder->append(append_buffers, false);
> +
> ++=======
> + new_object_recorder->append_unlock(std::move(append_buffers));
> ++>>>>>>> dc77a62... journal: avoid holding lock while sending journal append
> m_object_ptrs[splay_offset] = new_object_recorder;
> }
>
> ----------------------------------------------------------------------
>
> git log --no-merges --oneline --cherry-mark --left-right ceph/jewel...ceph/master -- src/journal/JournalRecorder.cc
>
>> dc77a62 journal: avoid holding lock while sending journal append
>> 7b740f5 journal: make librados call async in ObjectRecorder
>> 5c88edd journal: increase concurrency of journal recorder
> = b4b9843 journal: improve debug log messages
> = 11475f4 journal: improve debug log messages
> = 30c7f4d journal: update commit entry object number upon overflow
> = 5fbf5f8 journal: update commit entry object number upon overflow
> = 18f663d journal: close, advance, and open object set ordering
> = 27fba6a journal: new ObjectRecorder closed callback
> = de83005 journal: close, advance, and open object set ordering
> = bba9143 journal: new ObjectRecorder closed callback
> = f1a2e6a journal: prohibit journal entries larger than the soft object size
> = d5f307d journal: prohibit journal entries larger than the soft object size
>
> git cherry-pick -x 5c88edd conflicts on
>
> unmerged src/librbd/Journal.cc
> @@@ -17,8 -16,11 +17,9 @@@
> #include "common/Timer.h"
> #include "common/WorkQueue.h"
> #include "include/rados/librados.hpp"
> -#include "librbd/journal/RemoveRequest.h"
> -#include "librbd/journal/CreateRequest.h"
>
> #include <boost/scope_exit.hpp>
> + #include <utility>
>
> #define dout_subsys ceph_subsys_rbd
> #undef dout_prefix
> @@@ -705,40 -671,56 +706,70 @@@ int Journal<I>::demote()
> CephContext *cct = m_image_ctx.cct;
> ldout(cct, 20) << __func__ << dendl;
>
> ++<<<<<<< HEAD
> + Mutex::Locker locker(m_lock);
> + assert(m_journaler != nullptr && is_tag_owner());
> ++=======
> + int r;
> + C_SaferCond ctx;
> + Future future;
> + C_SaferCond flush_ctx;
> ++>>>>>>> 7b740f5... journal: make librados call async in ObjectRecorder
>
> ----------------------------------------------------------------------
> git log --no-merges --oneline --cherry-mark --left-right ceph/jewel...ceph/master -- src/librbd/Journal.cc
>
>> 953561f librbd: async method to check journal tag owner
>> 7b740f5 journal: make librados call async in ObjectRecorder
>> 5c88edd journal: increase concurrency of journal recorder
>> fd00549 librbd: new journal listener event for force promotion
>> dbbcecf librbd: unify journal event callbacks into single interface
>> b8eafef librbd: optionally flag "laggy" journal clients disconnected
>> 551d7ea librbd: remove use of owner_lock on IO path
> = e83866b librbd: do not record journal events if append is disabled by policy
> = 1f63307 librbd: optimize header file dependency & modify related file.
> = 549aada librbd: new configuration option to restrict journal payload size
> = 99195e8 librbd: wait for journal commit op event to be safely recorded
> = 91c70f3 journal: helper class for organizing optional settings
> = f71dc87 librbd: improve journaling debug log messages
> = 14db535 librbd: separate journal event decoding and processing
>> ac590e8 librbd: normalize journal promotion/demotion events
>> 718befd librbd: separate journal::TagData predecessor data into new struct
>> 47d1e62 rbd-mirror: include tag tid in bootstrap debug log messages
> = c2b786c rbd: journal: Support for listening updates on client metadata
>> 3492153 journaler: remove ununsed synchronous API routines
>> d5fa466 librbd / journal: async journal creation state machine
>> c4e439d librbd / journal: async journal removal state machine
>> 1416f1c librbd / ImageCtx: singleton safetimer instance helper
> = 405142c librbd: do not record journal events if append is disabled by policy
> = 11d7500 librbd: new configuration option to restrict journal payload size
> = 47e0fbf librbd: wait for journal commit op event to be safely recorded
> = dad8328 journal: helper class for organizing optional settings
> = 270cb74 librbd: improve journaling debug log messages
> = 57cd75e librbd: separate journal event decoding and processing
> = f3f4a4a librbd: journal callback to interrupt replay
> = deb6ca8 librbd: force-remove journal when disabling feature and removing image
> = 2185a62 librbd: optimize header file dependency & modify related file.
> = 9687e5e librbd: journal callback to interrupt replay
> = d83aa1e librbd: force-remove journal when disabling feature and removing image
> = 0dd8573 rbd: journal: Support for listening updates on client metadata
> = f0bf15e librbd: stop journal recorder before starting external replay
> = 0014a9e librbd: flush journal commit positions before starting op
> = ef12536 librbd: resize and snap create can hang on journal replay failure
> = c8b4cab librbd: missing journal state for flushing/restarting replay
> = de29be7 librbd: Journal no longer requires AioCompletion for IO events
> = 65556aa librbd: bubble journal error up via existing AioObjectRequests
> = 5aa6eb2 librbd: stop journal recorder before starting external replay
> = 6361296 librbd: flush journal commit positions before starting op
> = 1032f19 librbd: resize and snap create can hang on journal replay failure
> = a7f5bc1 librbd: missing journal state for flushing/restarting replay
> = 82587a3 librbd: Journal no longer requires AioCompletion for IO events
> = 79c9347 librbd: bubble journal error up via existing AioObjectRequests
> = bf4c458 librbd: integrate with async journaler shutdown API
> = 0780230 rbd-mirror: Added implementation of image-deleter thread
> = ad29785 librbd: integrate with async journaler shutdown API
> = 6cc6e66 rbd-mirror: Added implementation of image-deleter thread
> = 5ec1f79 librbd: split large AioWrite journal events
> = f221fed librbd: assertion to ensure no concurrent processing of replay events
> = 44827a3 librbd: assertion to ensure no concurrent processing of replay events
> = 70b9fbf librbd: split large AioWrite journal events
>
> --
> Loïc Dachary, Artisan Logiciel Libre
--
Jason
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: jewel backport of https://github.com/ceph/ceph/pull/11326
2016-10-11 14:11 ` Jason Dillaman
@ 2016-10-11 14:15 ` Loic Dachary
0 siblings, 0 replies; 3+ messages in thread
From: Loic Dachary @ 2016-10-11 14:15 UTC (permalink / raw)
To: dillaman; +Cc: Mykola Golub, Ceph Development
On 11/10/2016 16:11, Jason Dillaman wrote:
> I think the biggest set of conflicts will come from
> https://github.com/ceph/ceph/pull/10445, which will need to be merged
> along with #11326. If you'd like, I can take care of this backport
> today.
That would be fantastic. And then I can take care the other, simpler, backports.
Thanks !
>
> On Tue, Oct 11, 2016 at 5:54 AM, Loic Dachary <loic@dachary.org> wrote:
>> Hi,
>>
>> Trying to cherry pick commits instead of resolving conflicts and it looks like I'm going to end up adding a number of commits. Before I go down this path, I'd like a confirmation from you that it's the right call (see below for more information about what I did).
>>
>> Thanks !
>>
>> git cherry-pick -x dc77a62 from https://github.com/ceph/ceph/pull/11326 conflicts on
>>
>> unmerged src/journal/JournalRecorder.cc
>> @@@ -279,8 -281,7 +276,12 @@@ void JournalRecorder::create_next_objec
>> new_object_recorder->get_object_number());
>> }
>>
>> ++<<<<<<< HEAD
>> + new_object_recorder->append(append_buffers, false);
>> +
>> ++=======
>> + new_object_recorder->append_unlock(std::move(append_buffers));
>> ++>>>>>>> dc77a62... journal: avoid holding lock while sending journal append
>> m_object_ptrs[splay_offset] = new_object_recorder;
>> }
>>
>> ----------------------------------------------------------------------
>>
>> git log --no-merges --oneline --cherry-mark --left-right ceph/jewel...ceph/master -- src/journal/JournalRecorder.cc
>>
>>> dc77a62 journal: avoid holding lock while sending journal append
>>> 7b740f5 journal: make librados call async in ObjectRecorder
>>> 5c88edd journal: increase concurrency of journal recorder
>> = b4b9843 journal: improve debug log messages
>> = 11475f4 journal: improve debug log messages
>> = 30c7f4d journal: update commit entry object number upon overflow
>> = 5fbf5f8 journal: update commit entry object number upon overflow
>> = 18f663d journal: close, advance, and open object set ordering
>> = 27fba6a journal: new ObjectRecorder closed callback
>> = de83005 journal: close, advance, and open object set ordering
>> = bba9143 journal: new ObjectRecorder closed callback
>> = f1a2e6a journal: prohibit journal entries larger than the soft object size
>> = d5f307d journal: prohibit journal entries larger than the soft object size
>>
>> git cherry-pick -x 5c88edd conflicts on
>>
>> unmerged src/librbd/Journal.cc
>> @@@ -17,8 -16,11 +17,9 @@@
>> #include "common/Timer.h"
>> #include "common/WorkQueue.h"
>> #include "include/rados/librados.hpp"
>> -#include "librbd/journal/RemoveRequest.h"
>> -#include "librbd/journal/CreateRequest.h"
>>
>> #include <boost/scope_exit.hpp>
>> + #include <utility>
>>
>> #define dout_subsys ceph_subsys_rbd
>> #undef dout_prefix
>> @@@ -705,40 -671,56 +706,70 @@@ int Journal<I>::demote()
>> CephContext *cct = m_image_ctx.cct;
>> ldout(cct, 20) << __func__ << dendl;
>>
>> ++<<<<<<< HEAD
>> + Mutex::Locker locker(m_lock);
>> + assert(m_journaler != nullptr && is_tag_owner());
>> ++=======
>> + int r;
>> + C_SaferCond ctx;
>> + Future future;
>> + C_SaferCond flush_ctx;
>> ++>>>>>>> 7b740f5... journal: make librados call async in ObjectRecorder
>>
>> ----------------------------------------------------------------------
>> git log --no-merges --oneline --cherry-mark --left-right ceph/jewel...ceph/master -- src/librbd/Journal.cc
>>
>>> 953561f librbd: async method to check journal tag owner
>>> 7b740f5 journal: make librados call async in ObjectRecorder
>>> 5c88edd journal: increase concurrency of journal recorder
>>> fd00549 librbd: new journal listener event for force promotion
>>> dbbcecf librbd: unify journal event callbacks into single interface
>>> b8eafef librbd: optionally flag "laggy" journal clients disconnected
>>> 551d7ea librbd: remove use of owner_lock on IO path
>> = e83866b librbd: do not record journal events if append is disabled by policy
>> = 1f63307 librbd: optimize header file dependency & modify related file.
>> = 549aada librbd: new configuration option to restrict journal payload size
>> = 99195e8 librbd: wait for journal commit op event to be safely recorded
>> = 91c70f3 journal: helper class for organizing optional settings
>> = f71dc87 librbd: improve journaling debug log messages
>> = 14db535 librbd: separate journal event decoding and processing
>>> ac590e8 librbd: normalize journal promotion/demotion events
>>> 718befd librbd: separate journal::TagData predecessor data into new struct
>>> 47d1e62 rbd-mirror: include tag tid in bootstrap debug log messages
>> = c2b786c rbd: journal: Support for listening updates on client metadata
>>> 3492153 journaler: remove ununsed synchronous API routines
>>> d5fa466 librbd / journal: async journal creation state machine
>>> c4e439d librbd / journal: async journal removal state machine
>>> 1416f1c librbd / ImageCtx: singleton safetimer instance helper
>> = 405142c librbd: do not record journal events if append is disabled by policy
>> = 11d7500 librbd: new configuration option to restrict journal payload size
>> = 47e0fbf librbd: wait for journal commit op event to be safely recorded
>> = dad8328 journal: helper class for organizing optional settings
>> = 270cb74 librbd: improve journaling debug log messages
>> = 57cd75e librbd: separate journal event decoding and processing
>> = f3f4a4a librbd: journal callback to interrupt replay
>> = deb6ca8 librbd: force-remove journal when disabling feature and removing image
>> = 2185a62 librbd: optimize header file dependency & modify related file.
>> = 9687e5e librbd: journal callback to interrupt replay
>> = d83aa1e librbd: force-remove journal when disabling feature and removing image
>> = 0dd8573 rbd: journal: Support for listening updates on client metadata
>> = f0bf15e librbd: stop journal recorder before starting external replay
>> = 0014a9e librbd: flush journal commit positions before starting op
>> = ef12536 librbd: resize and snap create can hang on journal replay failure
>> = c8b4cab librbd: missing journal state for flushing/restarting replay
>> = de29be7 librbd: Journal no longer requires AioCompletion for IO events
>> = 65556aa librbd: bubble journal error up via existing AioObjectRequests
>> = 5aa6eb2 librbd: stop journal recorder before starting external replay
>> = 6361296 librbd: flush journal commit positions before starting op
>> = 1032f19 librbd: resize and snap create can hang on journal replay failure
>> = a7f5bc1 librbd: missing journal state for flushing/restarting replay
>> = 82587a3 librbd: Journal no longer requires AioCompletion for IO events
>> = 79c9347 librbd: bubble journal error up via existing AioObjectRequests
>> = bf4c458 librbd: integrate with async journaler shutdown API
>> = 0780230 rbd-mirror: Added implementation of image-deleter thread
>> = ad29785 librbd: integrate with async journaler shutdown API
>> = 6cc6e66 rbd-mirror: Added implementation of image-deleter thread
>> = 5ec1f79 librbd: split large AioWrite journal events
>> = f221fed librbd: assertion to ensure no concurrent processing of replay events
>> = 44827a3 librbd: assertion to ensure no concurrent processing of replay events
>> = 70b9fbf librbd: split large AioWrite journal events
>>
>> --
>> Loïc Dachary, Artisan Logiciel Libre
>
>
>
--
Loïc Dachary, Artisan Logiciel Libre
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2016-10-11 14:20 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-10-11 9:54 jewel backport of https://github.com/ceph/ceph/pull/11326 Loic Dachary
2016-10-11 14:11 ` Jason Dillaman
2016-10-11 14:15 ` Loic Dachary
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).