* DISCARD support in kernel driver
@ 2014-01-30 9:31 Jean-Tiare LE BIGOT
2014-01-30 15:24 ` Gregory Farnum
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Jean-Tiare LE BIGOT @ 2014-01-30 9:31 UTC (permalink / raw)
To: Ceph Development
Hi,
I started to implement 'DISCARD' support in RBD kernel driver as
described on http://tracker.ceph.com/issues/190
This first (easy) step was to add at the end of
drivers/block/rbd.c:rbd_init_disk
/* Advertise discard support for aligned blocks */
queue_flag_set_unlocked(QUEUE_FLAG_DISCARD, q);
disk->queue->limits.discard_granularity = segment_size;
disk->queue->limits.discard_alignment = segment_size;
With this both 'mount -o discard' and 'fstrim' stopped to complain about
missing 'DISCARD' support. Good !
Next step was to add support on `rbd_request_fn` which I did (sort of).
But here I'm stuck. If I understood well FS drivers uses `REQ_DISCARD`
flag of `rq->cmd_flags` to notify the driver of the discard operation.
But the problem is that (rq->cmd_flags & REQ_DISCARD) never appears to
be set.
I tried copying then removing large files, lots of small files, forcing
fstrim. But I never got it. I must be missing something obvious but I
can't manage to find what ? Do you have any clue what could be wrong ?
Thanks,
--
Jean-Tiare, shared-hosting team
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: DISCARD support in kernel driver
2014-01-30 9:31 DISCARD support in kernel driver Jean-Tiare LE BIGOT
@ 2014-01-30 15:24 ` Gregory Farnum
2014-01-30 15:27 ` Jean-Tiare LE BIGOT
2014-02-01 2:36 ` Alex Elsayed
2014-03-11 16:50 ` Sage Weil
2 siblings, 1 reply; 6+ messages in thread
From: Gregory Farnum @ 2014-01-30 15:24 UTC (permalink / raw)
To: Jean-Tiare LE BIGOT; +Cc: Ceph Development
On Thu, Jan 30, 2014 at 1:31 AM, Jean-Tiare LE BIGOT
<jean-tiare.le-bigot@ovh.net> wrote:
> Hi,
>
> I started to implement 'DISCARD' support in RBD kernel driver as described
> on http://tracker.ceph.com/issues/190
>
> This first (easy) step was to add at the end of
> drivers/block/rbd.c:rbd_init_disk
>
> /* Advertise discard support for aligned blocks */
> queue_flag_set_unlocked(QUEUE_FLAG_DISCARD, q);
> disk->queue->limits.discard_granularity = segment_size;
> disk->queue->limits.discard_alignment = segment_size;
>
> With this both 'mount -o discard' and 'fstrim' stopped to complain about
> missing 'DISCARD' support. Good !
>
> Next step was to add support on `rbd_request_fn` which I did (sort of). But
> here I'm stuck. If I understood well FS drivers uses `REQ_DISCARD` flag of
> `rq->cmd_flags` to notify the driver of the discard operation. But the
> problem is that (rq->cmd_flags & REQ_DISCARD) never appears to be set.
>
> I tried copying then removing large files, lots of small files, forcing
> fstrim. But I never got it. I must be missing something obvious but I can't
> manage to find what ? Do you have any clue what could be wrong ?
I don't know anything about the relevant kernel internal interfaces
here, but don't most FSes need to be told explicitly to do trimming?
Did you do that?
-Greg
Software Engineer #42 @ http://inktank.com | http://ceph.com
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: DISCARD support in kernel driver
2014-01-30 15:24 ` Gregory Farnum
@ 2014-01-30 15:27 ` Jean-Tiare LE BIGOT
2014-01-30 15:38 ` Gregory Farnum
0 siblings, 1 reply; 6+ messages in thread
From: Jean-Tiare LE BIGOT @ 2014-01-30 15:27 UTC (permalink / raw)
To: Gregory Farnum; +Cc: Ceph Development
Hi Greg,
Thanks for your answer. I did
$ mount -t ext4 -o discard /dev/rbd1 /mnt
# did not see any DISCARD op after that
$ fstrim /mnt
# neither
Maybe missing something there ? I expected '-o discard' to be enough ?
On 01/30/14 16:24, Gregory Farnum wrote:
> On Thu, Jan 30, 2014 at 1:31 AM, Jean-Tiare LE BIGOT
> <jean-tiare.le-bigot@ovh.net> wrote:
>> Hi,
>>
>> I started to implement 'DISCARD' support in RBD kernel driver as described
>> on http://tracker.ceph.com/issues/190
>>
>> This first (easy) step was to add at the end of
>> drivers/block/rbd.c:rbd_init_disk
>>
>> /* Advertise discard support for aligned blocks */
>> queue_flag_set_unlocked(QUEUE_FLAG_DISCARD, q);
>> disk->queue->limits.discard_granularity = segment_size;
>> disk->queue->limits.discard_alignment = segment_size;
>>
>> With this both 'mount -o discard' and 'fstrim' stopped to complain about
>> missing 'DISCARD' support. Good !
>>
>> Next step was to add support on `rbd_request_fn` which I did (sort of). But
>> here I'm stuck. If I understood well FS drivers uses `REQ_DISCARD` flag of
>> `rq->cmd_flags` to notify the driver of the discard operation. But the
>> problem is that (rq->cmd_flags & REQ_DISCARD) never appears to be set.
>>
>> I tried copying then removing large files, lots of small files, forcing
>> fstrim. But I never got it. I must be missing something obvious but I can't
>> manage to find what ? Do you have any clue what could be wrong ?
>
> I don't know anything about the relevant kernel internal interfaces
> here, but don't most FSes need to be told explicitly to do trimming?
> Did you do that?
> -Greg
> Software Engineer #42 @ http://inktank.com | http://ceph.com
>
--
Jean-Tiare, shared-hosting team
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: DISCARD support in kernel driver
2014-01-30 15:27 ` Jean-Tiare LE BIGOT
@ 2014-01-30 15:38 ` Gregory Farnum
0 siblings, 0 replies; 6+ messages in thread
From: Gregory Farnum @ 2014-01-30 15:38 UTC (permalink / raw)
To: Jean-Tiare LE BIGOT; +Cc: Ceph Development
My quick Google says that should be enough, but I haven't done it
myself...sorry. :(
On Thu, Jan 30, 2014 at 7:27 AM, Jean-Tiare LE BIGOT
<jean-tiare.le-bigot@ovh.net> wrote:
> Hi Greg,
>
> Thanks for your answer. I did
>
> $ mount -t ext4 -o discard /dev/rbd1 /mnt
> # did not see any DISCARD op after that
> $ fstrim /mnt
> # neither
>
> Maybe missing something there ? I expected '-o discard' to be enough ?
>
>
> On 01/30/14 16:24, Gregory Farnum wrote:
>>
>> On Thu, Jan 30, 2014 at 1:31 AM, Jean-Tiare LE BIGOT
>> <jean-tiare.le-bigot@ovh.net> wrote:
>>>
>>> Hi,
>>>
>>> I started to implement 'DISCARD' support in RBD kernel driver as
>>> described
>>> on http://tracker.ceph.com/issues/190
>>>
>>> This first (easy) step was to add at the end of
>>> drivers/block/rbd.c:rbd_init_disk
>>>
>>> /* Advertise discard support for aligned blocks */
>>> queue_flag_set_unlocked(QUEUE_FLAG_DISCARD, q);
>>> disk->queue->limits.discard_granularity = segment_size;
>>> disk->queue->limits.discard_alignment = segment_size;
>>>
>>> With this both 'mount -o discard' and 'fstrim' stopped to complain about
>>> missing 'DISCARD' support. Good !
>>>
>>> Next step was to add support on `rbd_request_fn` which I did (sort of).
>>> But
>>> here I'm stuck. If I understood well FS drivers uses `REQ_DISCARD` flag
>>> of
>>> `rq->cmd_flags` to notify the driver of the discard operation. But the
>>> problem is that (rq->cmd_flags & REQ_DISCARD) never appears to be set.
>>>
>>> I tried copying then removing large files, lots of small files, forcing
>>> fstrim. But I never got it. I must be missing something obvious but I
>>> can't
>>> manage to find what ? Do you have any clue what could be wrong ?
>>
>>
>> I don't know anything about the relevant kernel internal interfaces
>> here, but don't most FSes need to be told explicitly to do trimming?
>> Did you do that?
>> -Greg
>> Software Engineer #42 @ http://inktank.com | http://ceph.com
>>
>
> --
> Jean-Tiare, shared-hosting team
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: DISCARD support in kernel driver
2014-01-30 9:31 DISCARD support in kernel driver Jean-Tiare LE BIGOT
2014-01-30 15:24 ` Gregory Farnum
@ 2014-02-01 2:36 ` Alex Elsayed
2014-03-11 16:50 ` Sage Weil
2 siblings, 0 replies; 6+ messages in thread
From: Alex Elsayed @ 2014-02-01 2:36 UTC (permalink / raw)
To: ceph-devel
Jean-Tiare LE BIGOT wrote:
> Hi,
>
> I started to implement 'DISCARD' support in RBD kernel driver as
> described on http://tracker.ceph.com/issues/190
>
> This first (easy) step was to add at the end of
> drivers/block/rbd.c:rbd_init_disk
>
> /* Advertise discard support for aligned blocks */
> queue_flag_set_unlocked(QUEUE_FLAG_DISCARD, q);
> disk->queue->limits.discard_granularity = segment_size;
> disk->queue->limits.discard_alignment = segment_size;
>
> With this both 'mount -o discard' and 'fstrim' stopped to complain about
> missing 'DISCARD' support. Good !
>
> Next step was to add support on `rbd_request_fn` which I did (sort of).
> But here I'm stuck. If I understood well FS drivers uses `REQ_DISCARD`
> flag of `rq->cmd_flags` to notify the driver of the discard operation.
> But the problem is that (rq->cmd_flags & REQ_DISCARD) never appears to
> be set.
>
> I tried copying then removing large files, lots of small files, forcing
> fstrim. But I never got it. I must be missing something obvious but I
> can't manage to find what ? Do you have any clue what could be wrong ?
>
> Thanks,
>
Well, you can use the "blkdiscard" command to force a discard of an
arbitrary portion of a block device directly. You can used dd or ddrescue to
copy test patterns (maybe 0xFF) to and from RBD device, and use blkdiscard
to trim parts of that.
If that doesn't work, then the error must be elsewhere.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: DISCARD support in kernel driver
2014-01-30 9:31 DISCARD support in kernel driver Jean-Tiare LE BIGOT
2014-01-30 15:24 ` Gregory Farnum
2014-02-01 2:36 ` Alex Elsayed
@ 2014-03-11 16:50 ` Sage Weil
2 siblings, 0 replies; 6+ messages in thread
From: Sage Weil @ 2014-03-11 16:50 UTC (permalink / raw)
To: Jean-Tiare LE BIGOT; +Cc: Ceph Development
Hi Jean-Tiare,
How is this going?
sage
On Thu, 30 Jan 2014, Jean-Tiare LE BIGOT wrote:
> Hi,
>
> I started to implement 'DISCARD' support in RBD kernel driver as described on
> http://tracker.ceph.com/issues/190
>
> This first (easy) step was to add at the end of
> drivers/block/rbd.c:rbd_init_disk
>
> /* Advertise discard support for aligned blocks */
> queue_flag_set_unlocked(QUEUE_FLAG_DISCARD, q);
> disk->queue->limits.discard_granularity = segment_size;
> disk->queue->limits.discard_alignment = segment_size;
>
> With this both 'mount -o discard' and 'fstrim' stopped to complain about
> missing 'DISCARD' support. Good !
>
> Next step was to add support on `rbd_request_fn` which I did (sort of). But
> here I'm stuck. If I understood well FS drivers uses `REQ_DISCARD` flag of
> `rq->cmd_flags` to notify the driver of the discard operation. But the problem
> is that (rq->cmd_flags & REQ_DISCARD) never appears to be set.
>
> I tried copying then removing large files, lots of small files, forcing
> fstrim. But I never got it. I must be missing something obvious but I can't
> manage to find what ? Do you have any clue what could be wrong ?
>
> Thanks,
>
> --
> Jean-Tiare, shared-hosting team
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2014-03-11 16:50 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-01-30 9:31 DISCARD support in kernel driver Jean-Tiare LE BIGOT
2014-01-30 15:24 ` Gregory Farnum
2014-01-30 15:27 ` Jean-Tiare LE BIGOT
2014-01-30 15:38 ` Gregory Farnum
2014-02-01 2:36 ` Alex Elsayed
2014-03-11 16:50 ` Sage Weil
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.