* ANNOUNCE: consolidated patches for shared snapshots @ 2008-12-08 3:41 Mikulas Patocka 2008-12-08 6:19 ` FUJITA Tomonori 0 siblings, 1 reply; 5+ messages in thread From: Mikulas Patocka @ 2008-12-08 3:41 UTC (permalink / raw) To: Alasdair G Kergon; +Cc: device-mapper development, Milan Broz Hi I released new patches for shared snapshots. The functionality is the same as in previous releases, but the two snapshot exception stores, one created by Fujita Tomonori and one by me, are consolidated and common code is shared. The patches are at: http://people.redhat.com/mpatocka/patches/kernel/new-snapshots/ The patches don't alter existing snapshot implementation, they contain just new files (they alter only Kconfig and Makefile to make the new files compilable). The new architecture is as follows: The module dm-multisnapshot.ko contains common code for both exception stores. It has basically processing and queuing IOs and attaching snapshots and interfacing with device mapper. This module loads the specific module for a given exception store: dm-store-mikulas.ko for my store or dm-store-fujita-daniel.ko for Zumastor store. In the target constructor, there's an argument "mikulas" or "fujita-daniel" that tells which module to load. The interface is what I designed --- the point is that Fujita's code can be converted to use my interface, but my code cannot be converted to use Fujita's interface --- so if we want to drive both exception stores with the same userspace code (we definitely want!), then there's no other option but to use my interface. The difference from Fujita's old interface is that in my interface the kernel selects snapshot ID when creating a new snapshots and in old Fujita's interface the user selected it. For my snapshot store, it is a design requirement that the snapshot ID is seleted by the kernel driver. There is really almost nothing to be shared with the old snapshot code. The data structures and the interface are have different logic --- for the old snapshots, there was one small structure for the origin and several big structures for the snapshots; for the new snapshots, there is one big structure for the origin and several small structures for the snapshots. So I wouldn't recommend to attempt to glue the new code on the old snapshots. For Fujita: You can take the file that I created and continue development on it. I fixed some other bugs or non-portable constructs in the code, I marked them as comments so that you can review the fixes. I took the code from ftp://ftp.kernel.org/pub/linux/kernel/people/tomo/dm-snap/2008-11-26/ and ported it, so if you meanwhile made some other changes your the code, diff your changes against 2008-11-26 version and apply the diff by hand to the file that I release. Mikulas ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: ANNOUNCE: consolidated patches for shared snapshots 2008-12-08 3:41 ANNOUNCE: consolidated patches for shared snapshots Mikulas Patocka @ 2008-12-08 6:19 ` FUJITA Tomonori 2008-12-08 14:02 ` Mikulas Patocka 0 siblings, 1 reply; 5+ messages in thread From: FUJITA Tomonori @ 2008-12-08 6:19 UTC (permalink / raw) To: dm-devel; +Cc: agk, mbroz On Sun, 7 Dec 2008 22:41:19 -0500 (EST) Mikulas Patocka <mpatocka@redhat.com> wrote: > I released new patches for shared snapshots. The functionality is the same > as in previous releases, but the two snapshot exception stores, one > created by Fujita Tomonori and one by me, are consolidated and common code > is shared. > > The patches are at: > http://people.redhat.com/mpatocka/patches/kernel/new-snapshots/ Thanks, As I said before, I have no preference about how to integrate new snapshot implementations into the existing code. I'm happy to follow the maintainer, Alasdair? > The patches don't alter existing snapshot implementation, they contain > just new files (they alter only Kconfig and Makefile to make the new files > compilable). > > The new architecture is as follows: > > The module dm-multisnapshot.ko contains common code for both exception > stores. It has basically processing and queuing IOs and attaching > snapshots and interfacing with device mapper. I've not closely looked at your dm-multisnapshot but after a quick look, I'm not sure about the number of exception I/Os (except for metadata I/Os) with an origin write. For example, I create five snapshots and remove the 1 and 3 snapshots (*1). Now I have 0, 2, 4 snapshots. Then when I update the origin, how many exception I/Os dm-multisnapshot needs to perform? (*1) I know dm-multisnapshot doesn't support deleting a snapshot for now, but I expect that it will support it in the future. ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: ANNOUNCE: consolidated patches for shared snapshots 2008-12-08 6:19 ` FUJITA Tomonori @ 2008-12-08 14:02 ` Mikulas Patocka 2008-12-08 15:52 ` Heinz Mauelshagen 0 siblings, 1 reply; 5+ messages in thread From: Mikulas Patocka @ 2008-12-08 14:02 UTC (permalink / raw) To: device-mapper development; +Cc: agk, mbroz On Mon, 8 Dec 2008, FUJITA Tomonori wrote: > On Sun, 7 Dec 2008 22:41:19 -0500 (EST) > Mikulas Patocka <mpatocka@redhat.com> wrote: > > > I released new patches for shared snapshots. The functionality is the same > > as in previous releases, but the two snapshot exception stores, one > > created by Fujita Tomonori and one by me, are consolidated and common code > > is shared. > > > > The patches are at: > > http://people.redhat.com/mpatocka/patches/kernel/new-snapshots/ > > Thanks, > > As I said before, I have no preference about how to integrate new > snapshot implementations into the existing code. I'm happy to follow > the maintainer, Alasdair? > > > > The patches don't alter existing snapshot implementation, they contain > > just new files (they alter only Kconfig and Makefile to make the new files > > compilable). > > > > The new architecture is as follows: > > > > The module dm-multisnapshot.ko contains common code for both exception > > stores. It has basically processing and queuing IOs and attaching > > snapshots and interfacing with device mapper. > > I've not closely looked at your dm-multisnapshot but after a quick > look, I'm not sure about the number of exception I/Os (except for > metadata I/Os) with an origin write. When writable snapshots will be implemented in my exception store, there will be a situation when the code will need to perform more copies. My implmentation stores snapshot ID ranges into the btree. For example, if I have snapshots 0-10. Someone wrote to snapshot 5. Now someone writes to the origin at this chunk. Two copies need to be performed, one for snapshots 0-4 and one for snapshots 6-10. Kcopyd can perform at most 8 copies efficiently (one read + many concurrent writes). That's why there is this reset_query/query_next_remap/add_next_remap loop. reset_query resets the search state in the snapshot store, query_next_remap asks for next remapping to be done and add_next_remap inserts the remapping into the b-tree. It continues to loop until it fills all 8 kcopyd slots or until query_next_remap reports that there is nothing more to do. For your implementation, the loop executes just once, because you use bitmap of snapshot IDs and you process all the snapshots in one call. > For example, I create five snapshots and remove the 1 and 3 snapshots > (*1). Now I have 0, 2, 4 snapshots. Then when I update the origin, how > many exception I/Os dm-multisnapshot needs to perform? For your implementation, just one. For my implementation, also just one, because I don't resuse snapshot IDs --- so when I implement it, I can just insert one entry with range 0-4. More entries need to be inserted when someone wrote to the snapshots in the middle of the range. > (*1) I know dm-multisnapshot doesn't support deleting a snapshot for > now, but I expect that it will support it in the future. > > -- > dm-devel mailing list > dm-devel@redhat.com > https://www.redhat.com/mailman/listinfo/dm-devel Mikulas ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: ANNOUNCE: consolidated patches for shared snapshots 2008-12-08 14:02 ` Mikulas Patocka @ 2008-12-08 15:52 ` Heinz Mauelshagen 2008-12-08 21:31 ` Mikulas Patocka 0 siblings, 1 reply; 5+ messages in thread From: Heinz Mauelshagen @ 2008-12-08 15:52 UTC (permalink / raw) To: device-mapper development Am Montag, den 08.12.2008, 09:02 -0500 schrieb Mikulas Patocka: > > On Mon, 8 Dec 2008, FUJITA Tomonori wrote: > > > On Sun, 7 Dec 2008 22:41:19 -0500 (EST) > > Mikulas Patocka <mpatocka@redhat.com> wrote: > > > > > I released new patches for shared snapshots. The functionality is the same > > > as in previous releases, but the two snapshot exception stores, one > > > created by Fujita Tomonori and one by me, are consolidated and common code > > > is shared. > > > > > > The patches are at: > > > http://people.redhat.com/mpatocka/patches/kernel/new-snapshots/ > > > > Thanks, > > > > As I said before, I have no preference about how to integrate new > > snapshot implementations into the existing code. I'm happy to follow > > the maintainer, Alasdair? > > > > > > > The patches don't alter existing snapshot implementation, they contain > > > just new files (they alter only Kconfig and Makefile to make the new files > > > compilable). > > > > > > The new architecture is as follows: > > > > > > The module dm-multisnapshot.ko contains common code for both exception > > > stores. It has basically processing and queuing IOs and attaching > > > snapshots and interfacing with device mapper. > > > > I've not closely looked at your dm-multisnapshot but after a quick > > look, I'm not sure about the number of exception I/Os (except for > > metadata I/Os) with an origin write. > > When writable snapshots will be implemented in my exception store, there > will be a situation when the code will need to perform more copies. My > implmentation stores snapshot ID ranges into the btree. For example, if I > have snapshots 0-10. Someone wrote to snapshot 5. Now someone writes to > the origin at this chunk. Two copies need to be performed, one for > snapshots 0-4 and one for snapshots 6-10. Don't you just need a copy of the origin for snapshot 5, because it's being changed and likewise another one for snapshots 0-4,6-10, because the origin changes, hence snapshot 0-4,6-10 refer to that copy ? I.e if any snapshot changes, it gets a private copy to change, either from the origin unless already changed in which case it receives a copy from the already existing origin copy. Heinz > > Kcopyd can perform at most 8 copies efficiently (one read + many > concurrent writes). > > That's why there is this reset_query/query_next_remap/add_next_remap loop. > > reset_query resets the search state in the snapshot store, > query_next_remap asks for next remapping to be done and add_next_remap > inserts the remapping into the b-tree. It continues to loop until it fills > all 8 kcopyd slots or until query_next_remap reports that there is nothing > more to do. > > For your implementation, the loop executes just once, because you use > bitmap of snapshot IDs and you process all the snapshots in one call. > > > For example, I create five snapshots and remove the 1 and 3 snapshots > > (*1). Now I have 0, 2, 4 snapshots. Then when I update the origin, how > > many exception I/Os dm-multisnapshot needs to perform? > > For your implementation, just one. For my implementation, also just one, > because I don't resuse snapshot IDs --- so when I implement it, I can just > insert one entry with range 0-4. More entries need to be inserted when > someone wrote to the snapshots in the middle of the range. > > > (*1) I know dm-multisnapshot doesn't support deleting a snapshot for > > now, but I expect that it will support it in the future. > > > > -- > > dm-devel mailing list > > dm-devel@redhat.com > > https://www.redhat.com/mailman/listinfo/dm-devel > > Mikulas > > -- > dm-devel mailing list > dm-devel@redhat.com > https://www.redhat.com/mailman/listinfo/dm-devel ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: ANNOUNCE: consolidated patches for shared snapshots 2008-12-08 15:52 ` Heinz Mauelshagen @ 2008-12-08 21:31 ` Mikulas Patocka 0 siblings, 0 replies; 5+ messages in thread From: Mikulas Patocka @ 2008-12-08 21:31 UTC (permalink / raw) To: heinzm, device-mapper development On Mon, 8 Dec 2008, Heinz Mauelshagen wrote: > Am Montag, den 08.12.2008, 09:02 -0500 schrieb Mikulas Patocka: > > > > On Mon, 8 Dec 2008, FUJITA Tomonori wrote: > > > > > I've not closely looked at your dm-multisnapshot but after a quick > > > look, I'm not sure about the number of exception I/Os (except for > > > metadata I/Os) with an origin write. > > > > When writable snapshots will be implemented in my exception store, there > > will be a situation when the code will need to perform more copies. My > > implmentation stores snapshot ID ranges into the btree. For example, if I > > have snapshots 0-10. Someone wrote to snapshot 5. Now someone writes to > > the origin at this chunk. Two copies need to be performed, one for > > snapshots 0-4 and one for snapshots 6-10. > > Don't you just need a copy of the origin for snapshot 5, because it's > being changed and likewise another one for snapshots 0-4,6-10, because > the origin changes, hence snapshot 0-4,6-10 refer to that copy ? > > I.e if any snapshot changes, it gets a private copy to change, either > from the origin unless already changed in which case it receives a copy > from the already existing origin copy. > > Heinz I mean is: If you have snapshots 0-10 and a chunk that is allocated in none of the snapshots, then writing the chunk to the origin allocates one record with range (0-10) in the btree and there is just one copy. But if you have a chunk that is allocated in none of the snapshots, then write this chunk to the snapshot 5, it will copy the chunk and create a record with range (5-5) in the btree. Now, if you write to the origin for that chunk, you need to create two records, one with range (0-4) and the other with range (6-10). That's where multiple copies come from. Mikulas ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2008-12-08 21:31 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2008-12-08 3:41 ANNOUNCE: consolidated patches for shared snapshots Mikulas Patocka 2008-12-08 6:19 ` FUJITA Tomonori 2008-12-08 14:02 ` Mikulas Patocka 2008-12-08 15:52 ` Heinz Mauelshagen 2008-12-08 21:31 ` Mikulas Patocka
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.