All of lore.kernel.org
 help / color / mirror / Atom feed
* 10/14/2014 Weekly Ceph Performance Meeting
@ 2014-10-15  3:40 Mark Nelson
  2014-10-15  6:22 ` Alexandre DERUMIER
  2014-10-15 15:50 ` Matt W. Benjamin
  0 siblings, 2 replies; 15+ messages in thread
From: Mark Nelson @ 2014-10-15  3:40 UTC (permalink / raw)
  To: ceph-devel@vger.kernel.org

Hi All,

Just a reminder that the weekly performance meeting is on Wednesdays at 
8AM PST!

Etherpad URL:
http://pad.ceph.com/p/performance_weekly

To join the Meeting:
https://bluejeans.com/268261044

To join via Browser:
https://bluejeans.com/268261044/browser

To join with Lync:
https://bluejeans.com/268261044/lync


To join via Room System:
Video Conferencing System: bjn.vc -or- 199.48.152.152
Meeting ID: 268261044

To join via Phone:
1) Dial:
           +1 408 740 7256
           +1 888 240 2560(US Toll Free)
           +1 408 317 9253(Alternate Number)
           (see all numbers - http://bluejeans.com/numbers)
2) Enter Conference ID: 268261044

Mark

^ permalink raw reply	[flat|nested] 15+ messages in thread
* 10/14/2014 Weekly Ceph Performance Meeting
@ 2014-10-17  5:58 Zhang, Jian
  0 siblings, 0 replies; 15+ messages in thread
From: Zhang, Jian @ 2014-10-17  5:58 UTC (permalink / raw)
  To: ceph-devel@vger.kernel.org,
	Sage Weil <sweil@redhat.com> (sweil@redhat.com),
	Mark Nelson (mark.nelson@inktank.com) (mark.nelson@inktank.com)
  Cc: Duan, Jiangang

Hi,
This is a followup on the OSD wbthrottle slow issue casued by fdatasync discussed on the weekly performance meeting. 
For a 4K random write tests (qd=2) with qem-rbd driver and 50 volumes:
* We are able to get ~3100IOPS on our setup with Firefly (0.80.5) default parameter  
* We can get ~3200 IOPS on the same setup with if we set wbthrottle = 1
* if we replace fdatasync with sync_file_range, we are albe to get 4457 IOPS, 41% improvement compared with all the default parameter. 

Setup configuration: 
* 4 OSD nodes (E3-1275 @3.5GHz processor with 32GB memory)
* each attached 10x Seagate 3TB 7200rpm HDD, with 2x Intel 400GB SSD as Journal
* network is 10GB. 

Wbthrottle tunings:
filestore_wbthrottle_xfs_ios_start_flusher=1
filestore_wbthrottle_xfs_bytes_start_flusher=1
filestore_wbthrottle_xfs_inodes_start_flusher=1

fdatasync -> sync_file_range:

diff codes_backup/WBThrottle.cc ceph-0.80.5/src/os/WBThrottle.cc
163c163
< #ifdef HAVE_FDATASYNC
---
> /*#ifdef HAVE_FDATASYNC
164a165,167
> */
> #ifdef HAVE_SYNC_FILE_RANGE
>     ::sync_file_range(**wb.get<1>(), wb.get<2>().offset, wb.get<2>().len, SYNC_FILE_RANGE_WRITE);
210c213
<   wbiter->second.first.add(nocache, len, 1);
---
>   wbiter->second.first.add(nocache, len, 1, offset, len);
 
 
[root@a-ceph04 opt]# diff codes_backup/WBThrottle.h ceph-0.80.5/src/os/WBThrottle.h
72a73,74
>     uint64_t offset;
>     uint64_t len;
74c76
<     void add(bool _nocache, uint64_t _size, uint64_t _ios) {
---
>     void add(bool _nocache, uint64_t _size, uint64_t _ios, uint64_t _offset, uint64_t _len) {
78a81,82
>       offset = _offset;
>       len = _len;

Thanks
Jian

^ permalink raw reply	[flat|nested] 15+ messages in thread
end of thread, other threads:[~2014-10-23 22:06 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-10-15  3:40 10/14/2014 Weekly Ceph Performance Meeting Mark Nelson
2014-10-15  6:22 ` Alexandre DERUMIER
2014-10-15 13:26   ` Mark Nelson
2014-10-15 14:42     ` Alexandre DERUMIER
2014-10-15 15:54       ` Alexandre DERUMIER
2014-10-15 16:04         ` Alexandre DERUMIER
2014-10-16  3:27         ` Chen, Xiaoxi
2014-10-16  6:25           ` Alexandre DERUMIER
2014-10-16  6:53             ` Chen, Xiaoxi
2014-10-23 21:53               ` Sage Weil
2014-10-23 22:06                 ` Milosz Tanski
2014-10-15 15:50 ` Matt W. Benjamin
2014-10-15 15:58   ` Mark Nelson
2014-10-15 16:44     ` Matt W. Benjamin
  -- strict thread matches above, loose matches on Subject: below --
2014-10-17  5:58 Zhang, Jian

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.