* [Drbd-dev] New Features @ 2008-09-19 5:24 Morey Roof 2008-09-19 15:53 ` Lars Ellenberg 0 siblings, 1 reply; 10+ messages in thread From: Morey Roof @ 2008-09-19 5:24 UTC (permalink / raw) To: drbd-dev I have been using DRBD and really like but I have wanted to add some new features to it and I wanted to ask you guys what you need and requirements you may have on patches to DRBD. Thanks, Morey ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Drbd-dev] New Features 2008-09-19 5:24 [Drbd-dev] New Features Morey Roof @ 2008-09-19 15:53 ` Lars Ellenberg 2008-09-19 16:19 ` Morey Roof 0 siblings, 1 reply; 10+ messages in thread From: Lars Ellenberg @ 2008-09-19 15:53 UTC (permalink / raw) To: drbd-dev On Thu, Sep 18, 2008 at 11:24:30PM -0600, Morey Roof wrote: > I have been using DRBD and really like but I have wanted to add some new > features to it and I wanted to ask you guys what you need and > requirements you may have on patches to DRBD. Could you tell us about your background, and prior coding experience, and how much time you think you want to spend on this? In the drbd module, we currently do not actively work on new features that would be suitable to be implemented by anyone not very familiar with the drbd code base. If you think you are, let me know... Finding and squashing bugs is always a wellcome contribution. Userland could be improved, e.g. adding an "include" statement to the drbd.conf grammar, which would allow to have the configuration split out in /etc/drbd.conf.d/*.conf. Error messages can be improved, in both kernel and userland. The community would be happy to see some detailed walk-throughs from scratch for drbd integration with pacemaker, including various typically deployed services on top of drbd. I'd be happy to see a recent and solid HA-DRBD-NFS howto, so if you feel that would be a task for you, set up an HA-DRBD-NFSv4 server, and document everything needed to make it behave as expected (document the expectations as well) in face of failover, switchover, node reboot, cluster reboot, replication link failure, other component failures, various administrative tasks. An advanced variation on the theme: "load balancing" HA-DRBD-NFSv4 cluster, using multiple drbd, each with its own file system, fsid, and HA-ip, and then switchover at will, without causing server nor client hickups. -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Drbd-dev] New Features 2008-09-19 15:53 ` Lars Ellenberg @ 2008-09-19 16:19 ` Morey Roof 2008-09-19 22:13 ` Lars Ellenberg 0 siblings, 1 reply; 10+ messages in thread From: Morey Roof @ 2008-09-19 16:19 UTC (permalink / raw) To: drbd-dev I have been programming professionally for about 12 years with C/C++ in various system. I have worked on various parts of Linux for companies but I never submitted work to be included until recently. So I am a bit new to the get stuff out there model, but I am quite experienced working on very large and complex projects. What has allowed me to start working on some open source projects recently is the move to a job for a college where such work is welcomed and encouraged instead of the traditional business models where everything is a closely guarded secret. We have been using DRBD for a while and I have found that there are several itches I would like to scratch, so to speak. Let me know if you need more information. Thanks, Morey On Fri, Sep 19, 2008 at 9:53 AM, Lars Ellenberg <lars.ellenberg@linbit.com> wrote: > On Thu, Sep 18, 2008 at 11:24:30PM -0600, Morey Roof wrote: >> I have been using DRBD and really like but I have wanted to add some new >> features to it and I wanted to ask you guys what you need and >> requirements you may have on patches to DRBD. > > Could you tell us about your background, and prior coding experience, > and how much time you think you want to spend on this? > > In the drbd module, we currently do not actively work on new features > that would be suitable to be implemented by anyone not very familiar > with the drbd code base. If you think you are, let me know... > > Finding and squashing bugs is always a wellcome contribution. > > Userland could be improved, e.g. adding an "include" statement > to the drbd.conf grammar, which would allow to have the configuration > split out in /etc/drbd.conf.d/*.conf. > > Error messages can be improved, in both kernel and userland. > > The community would be happy to see some detailed walk-throughs from > scratch for drbd integration with pacemaker, including various typically > deployed services on top of drbd. > > I'd be happy to see a recent and solid HA-DRBD-NFS howto, > so if you feel that would be a task for you, > set up an HA-DRBD-NFSv4 server, and document everything needed to make > it behave as expected (document the expectations as well) in face of > failover, switchover, node reboot, cluster reboot, replication link failure, > other component failures, various administrative tasks. > An advanced variation on the theme: "load balancing" HA-DRBD-NFSv4 > cluster, using multiple drbd, each with its own file system, fsid, and > HA-ip, and then switchover at will, without causing server nor client > hickups. > > -- > : Lars Ellenberg > : LINBIT | Your Way to High Availability > : DRBD/HA support and consulting http://www.linbit.com > > DRBD(R) and LINBIT(R) are registered trademarks of LINBIT, Austria. > _______________________________________________ > drbd-dev mailing list > drbd-dev@lists.linbit.com > http://lists.linbit.com/mailman/listinfo/drbd-dev > ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Drbd-dev] New Features 2008-09-19 16:19 ` Morey Roof @ 2008-09-19 22:13 ` Lars Ellenberg 2008-09-19 22:28 ` Morey Roof 0 siblings, 1 reply; 10+ messages in thread From: Lars Ellenberg @ 2008-09-19 22:13 UTC (permalink / raw) To: drbd-dev On Fri, Sep 19, 2008 at 10:19:03AM -0600, Morey Roof wrote: > I have been programming professionally for about 12 years with C/C++ > in various system. I have worked on various parts of Linux for > companies but I never submitted work to be included until recently. > So I am a bit new to the get stuff out there model, but I am quite > experienced working on very large and complex projects. > > What has allowed me to start working on some open source projects > recently is the move to a job for a college where such work is > welcomed and encouraged instead of the traditional business models > where everything is a closely guarded secret. We have been using DRBD > for a while and I have found that there are several itches I would > like to scratch, so to speak. > > Let me know if you need more information. sounds promising. do you want to discuss some of the itches? Lars ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Drbd-dev] New Features 2008-09-19 22:13 ` Lars Ellenberg @ 2008-09-19 22:28 ` Morey Roof 2008-09-20 13:18 ` Lars Ellenberg 0 siblings, 1 reply; 10+ messages in thread From: Morey Roof @ 2008-09-19 22:28 UTC (permalink / raw) To: drbd-dev Lars Ellenberg wrote: > On Fri, Sep 19, 2008 at 10:19:03AM -0600, Morey Roof wrote: > >> I have been programming professionally for about 12 years with C/C++ >> in various system. I have worked on various parts of Linux for >> companies but I never submitted work to be included until recently. >> So I am a bit new to the get stuff out there model, but I am quite >> experienced working on very large and complex projects. >> >> What has allowed me to start working on some open source projects >> recently is the move to a job for a college where such work is >> welcomed and encouraged instead of the traditional business models >> where everything is a closely guarded secret. We have been using DRBD >> for a while and I have found that there are several itches I would >> like to scratch, so to speak. >> >> Let me know if you need more information. >> > > sounds promising. > > do you want to discuss some of the itches? > > Lars > _______________________________________________ > drbd-dev mailing list > drbd-dev@lists.linbit.com > http://lists.linbit.com/mailman/listinfo/drbd-dev > The first item on the list is a write back cache. Now don't run screaming to hills yet. In our setup we have the two nodes in two different buildings which are feed by different power supplies from the city grid. The machines themselves have dual power supplies which each power supply going to it's own UPS and circuit. So, we have a good redundant power supply setup. So, I was thinking about creating a caching system in DRBD to allow for dedicating some of the server's RAM to a large read/write-back cache. In the event of communication failure between the peers I wanted it to flush the entire cache and then run in write-through mode and move back to write-back mode after communication is present again and the resync is finished. So that is the basic idea. I was thinking about having it use the ramdisk system so that you can specify a device that is used as the cache. The reason I was thinking about this is that if you happen to purchase one of those Gigabyte ram HDD that is just like a large battery backed write cache you could set it as the cache device as well. What are your thought? -Morey ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Drbd-dev] New Features 2008-09-19 22:28 ` Morey Roof @ 2008-09-20 13:18 ` Lars Ellenberg 2008-09-20 13:48 ` Lars Ellenberg 0 siblings, 1 reply; 10+ messages in thread From: Lars Ellenberg @ 2008-09-20 13:18 UTC (permalink / raw) To: Morey Roof; +Cc: drbd-dev On Fri, Sep 19, 2008 at 04:28:19PM -0600, Morey Roof wrote: > The first item on the list is a write back cache. Now don't run > screaming to hills yet. In our setup we have the two nodes in two > different buildings which are feed by different power supplies from the > city grid. The machines themselves have dual power supplies which each > power supply going to it's own UPS and circuit. So, we have a good > redundant power supply setup. So, I was thinking about creating a > caching system in DRBD to allow for dedicating some of the server's RAM > to a large read/write-back cache. In the event of communication failure > between the peers I wanted it to flush the entire cache and then run in > write-through mode and move back to write-back mode after communication > is present again and the resync is finished. > > So that is the basic idea. I was thinking about having it use the > ramdisk system so that you can specify a device that is used as the > cache. The reason I was thinking about this is that if you happen to > purchase one of those Gigabyte ram HDD that is just like a large battery > backed write cache you could set it as the cache device as well. > > What are your thought? something I though of myself already, but nothing for the curren drbd architechture. I sent you my Linux-Kongress 2008 paper off list, where I scetch what I have in mind for the future. I'd put in a link here, if I had it online already. though I did not mention it, a (possibly non-volatile) ram disk would fit in pretty well as the log device. let me know what you think. -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. __ please don't Cc me, but send to list -- I'm subscribed ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Drbd-dev] New Features 2008-09-20 13:18 ` Lars Ellenberg @ 2008-09-20 13:48 ` Lars Ellenberg 2008-09-20 23:04 ` Morey Roof 0 siblings, 1 reply; 10+ messages in thread From: Lars Ellenberg @ 2008-09-20 13:48 UTC (permalink / raw) To: drbd-dev On Sat, Sep 20, 2008 at 03:18:21PM +0200, Lars Ellenberg wrote: "Write-Back" cache: some things to think of when introducing a write back cache, * need to do some cache coherency protocol * need to track which block is where, so we can read the correct version in case it has not yet been committed to final location * if using a ram buffer as log disk, we need to track the latest position for overwrites. * if we have stages, i.e. ram buffer first, then log disk, then real storage, we are the most flexible. if peers are sufficiently close, we can send_page from the ram buffer (and calculate checksums there, for data integrity). if we use it as ring buffer, we'd not have to worry about inconsistencies resulting from changes to in-flight buffers, as they are all private. * if we can use some efficient combination of digital tree, btree and hash table to track which block is where, we might be able to track a large, staged log device as a sort of log-structured block device, making snapshots after the fact for data generations still covered by the log very easy. * we need a good refcount scheme on the ram buffers. of course we can start out "simple", and just provide a static cache, no ring buffer or anything. this should probably be implemented as a generic device-mapper target, which also makes testing much easier. which would make it possible to even add it to the current drbd by just stacking it in front of the "lower-level device". for the "write-back" to "write-through" change, we only need a minimal change in the current drbd module, which we can enable based on the type of the device directly below us. we could detect whether its a device-mapper target, if so, which one, and access its special methods if any. this still sound a little quirky, so I'd suggest to introduce a special BIO_RW_WRITE_THROUGH (to be defined) bit for the bi_flags. when not using it, that is write back. when using it, it would trigger a flush of any pending requests, and a direct remapping to the lower level device. BIO_RW_BARRIER requests would still need to trigger a flush as well, and to go straight through. in the new architecture, where "drbd" probably becomes just a special implementation and collection of device-mapper targets, communicating with other device mapper targets becomes more easy (I hope). does that make sense? -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. __ please don't Cc me, but send to list -- I'm subscribed ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Drbd-dev] New Features 2008-09-20 13:48 ` Lars Ellenberg @ 2008-09-20 23:04 ` Morey Roof 2008-09-20 23:19 ` [Drbd-dev] New Features: write-back cache mode Lars Ellenberg 0 siblings, 1 reply; 10+ messages in thread From: Morey Roof @ 2008-09-20 23:04 UTC (permalink / raw) To: drbd-dev This is pretty much what I was thinking. For the btree, generations, and ref-count a good example to look at is how btrfs (This is the other project I have started to mess with) works. The design is very efficient and I think we could use a very close match for our setup. I haven't read the paper you sent yet but will get to that today. Let me know how you would like to start and I can start working a proof of concept and we can see how to go from there. -Morey Lars Ellenberg wrote: > On Sat, Sep 20, 2008 at 03:18:21PM +0200, Lars Ellenberg wrote: > > "Write-Back" cache: > > some things to think of when introducing a write back cache, > * need to do some cache coherency protocol > * need to track which block is where, so we can read the correct > version in case it has not yet been committed to final location > * if using a ram buffer as log disk, we need to track the latest > position for overwrites. > * if we have stages, i.e. ram buffer first, then log disk, then real > storage, we are the most flexible. > if peers are sufficiently close, we can send_page from the ram > buffer (and calculate checksums there, for data integrity). > if we use it as ring buffer, we'd not have to worry about > inconsistencies resulting from changes to in-flight buffers, > as they are all private. > * if we can use some efficient combination of digital tree, > btree and hash table to track which block is where, > we might be able to track a large, staged log device > as a sort of log-structured block device, making snapshots after the > fact for data generations still covered by the log very easy. > * we need a good refcount scheme on the ram buffers. > > of course we can start out "simple", and just provide a static cache, no > ring buffer or anything. > > this should probably be implemented as a generic device-mapper target, > which also makes testing much easier. > > which would make it possible to even add it to the current drbd > by just stacking it in front of the "lower-level device". > > for the "write-back" to "write-through" change, > we only need a minimal change in the current drbd module, which we can > enable based on the type of the device directly below us. > we could detect whether its a device-mapper target, > if so, which one, and access its special methods if any. > > this still sound a little quirky, so I'd suggest to introduce a special > BIO_RW_WRITE_THROUGH (to be defined) bit for the bi_flags. > > when not using it, that is write back. > when using it, it would trigger a flush of any pending requests, > and a direct remapping to the lower level device. > > BIO_RW_BARRIER requests would still need to trigger a flush as well, > and to go straight through. > > in the new architecture, where "drbd" probably becomes just a special > implementation and collection of device-mapper targets, communicating > with other device mapper targets becomes more easy (I hope). > > does that make sense? > > ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Drbd-dev] New Features: write-back cache mode 2008-09-20 23:04 ` Morey Roof @ 2008-09-20 23:19 ` Lars Ellenberg 2008-09-30 20:02 ` Lars Ellenberg 0 siblings, 1 reply; 10+ messages in thread From: Lars Ellenberg @ 2008-09-20 23:19 UTC (permalink / raw) To: drbd-dev On Sat, Sep 20, 2008 at 05:04:44PM -0600, Morey Roof wrote: > This is pretty much what I was thinking. For the btree, generations, > and ref-count a good example to look at is how btrfs (This is the other > project I have started to mess with) works. The design is very > efficient and I think we could use a very close match for our setup. I > haven't read the paper you sent yet but will get to that today. > > Let me know how you would like to start and I can start working a proof > of concept and we can see how to go from there. I still think that should be implemented as a generic device-mapper write-back target, not as part of drbd. there is already some caching target out there somewhere, using small local low-latency disks as cache for higher latency "remote" iSCSI or GNBD disks. At least I read something like that on one of the linux thin clients projects a while ago. I did not look at the code, and never used it, though. a 10 second google suggests that http://www.acis.ufl.edu/~ming/dmcache/ is what I mean. maybe you can use that as a starting point, and point it to a ram disk as cache? -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. __ please don't Cc me, but send to list -- I'm subscribed ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Drbd-dev] New Features: write-back cache mode 2008-09-20 23:19 ` [Drbd-dev] New Features: write-back cache mode Lars Ellenberg @ 2008-09-30 20:02 ` Lars Ellenberg 0 siblings, 0 replies; 10+ messages in thread From: Lars Ellenberg @ 2008-09-30 20:02 UTC (permalink / raw) To: drbd-dev On Sun, Sep 21, 2008 at 01:19:38AM +0200, Lars Ellenberg wrote: > On Sat, Sep 20, 2008 at 05:04:44PM -0600, Morey Roof wrote: > > This is pretty much what I was thinking. For the btree, generations, > > and ref-count a good example to look at is how btrfs (This is the other > > project I have started to mess with) works. The design is very > > efficient and I think we could use a very close match for our setup. I > > haven't read the paper you sent yet but will get to that today. > > > > Let me know how you would like to start and I can start working a proof > > of concept and we can see how to go from there. > > I still think that should be implemented as a > generic device-mapper write-back target, > not as part of drbd. > http://www.acis.ufl.edu/~ming/dmcache/ > maybe you can use that as a starting point, > and point it to a ram disk as cache? any news on that? Lars ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2008-09-30 20:02 UTC | newest] Thread overview: 10+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2008-09-19 5:24 [Drbd-dev] New Features Morey Roof 2008-09-19 15:53 ` Lars Ellenberg 2008-09-19 16:19 ` Morey Roof 2008-09-19 22:13 ` Lars Ellenberg 2008-09-19 22:28 ` Morey Roof 2008-09-20 13:18 ` Lars Ellenberg 2008-09-20 13:48 ` Lars Ellenberg 2008-09-20 23:04 ` Morey Roof 2008-09-20 23:19 ` [Drbd-dev] New Features: write-back cache mode Lars Ellenberg 2008-09-30 20:02 ` Lars Ellenberg
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.