From: Andrei Banu <andrei.banu@redhost.ro>
To: linux-raid@vger.kernel.org
Subject: Re: Incredibly poor performance of mdraid-1 with 2 SSD Samsung 840 PRO
Date: Wed, 24 Apr 2013 11:26:08 +0300 [thread overview]
Message-ID: <51779720.6040109@redhost.ro> (raw)
In-Reply-To: <51775078.3000500@hardwarefreak.com>
Hello,
I am sorry for the irrelevant feedback. Where I misunderstood your
request, I filled in the blanks (poorly).
1. SWAP
root [~]# blkid | grep cef1d19d-2578-43db-9ffc-b6b70e227bfa
/dev/md1: UUID="cef1d19d-2578-43db-9ffc-b6b70e227bfa" TYPE="swap"
So yes, swap is on md1. This *md1 has a size of 2GB*. Isn't this way too
low for a system with 16GB of memory?
2. Let me try again to give you the right test results:
Before the bigfile copy:
root [~]# perf top -U
Samples: 768 of event 'cycles', Event count (approx.): 499088870
18.58% [kernel] [k] port_inb
6.21% [kernel] [k] page_fault
3.36% [kernel] [k] clear_page_c_e
2.82% [kernel] [k] kallsyms_expand_symbol
1.99% [kernel] [k] __mem_cgroup_commit_charge
1.84% [kernel] [k] shmem_getpage_gfp
1.51% [kernel] [k] alloc_pages_vma
1.51% [kernel] [k] __alloc_pages_nodemask
1.46% [kernel] [k] avtab_search_node
1.45% [kernel] [k] format_decode
1.40% [kernel] [k] list_del
1.36% [kernel] [k] get_page_from_freelist
1.35% [kernel] [k] vsnprintf
1.29% [kernel] [k] avc_has_perm_noaudit
1.28% [kernel] [k] number
1.22% [kernel] [k] free_pcppages_bulk
1.21% [kernel] [k] ____pagevec_lru_add
1.14% [kernel] [k] get_page
1.08% [kernel] [k] memcpy
1.07% [kernel] [k] mem_cgroup_update_file_mapped
1.07% [kernel] [k] page_waitqueue
0.98% [kernel] [k] __d_lookup
0.97% [kernel] [k] unmap_vmas
0.91% [kernel] [k] _spin_lock
0.87% [kernel] [k] inode_has_perm
0.81% [kernel] [k] string
0.77% [kernel] [k] page_remove_rmap
0.73% [kernel] [k] __audit_syscall_exit
0.68% [kernel] [k] lookup_page_cgroup
0.61% [kernel] [k] unlock_page
0.61% [kernel] [k] shmem_find_get_pages_and_swap
0.61% [kernel] [k] free_hot_cold_page
0.61% [kernel] [k] release_pages
0.56% [kernel] [k] mem_cgroup_lru_del_list
0.55% [kernel] [k] strncpy_from_user
0.54% [kernel] [k] module_get_kallsym
0.52% [kernel] [k] find_get_page
0.50% [kernel] [k] __do_fault
0.48% [kernel] [k] path_put
0.46% [kernel] [k] __list_add
0.46% [kernel] [k] handle_mm_fault
0.45% [kernel] [k] __wake_up_bit
0.44% [kernel] [k] handle_pte_fault
0.43% [kernel] [k] audit_syscall_entry
0.43% [kernel] [k] thread_return
0.42% [kernel] [k] path_init
0.41% [kernel] [k] dput
0.40% [kernel] [k] task_has_capability
0.40% [kernel] [k] get_task_cred
0.40% [kernel] [k] pointer
0.40% [kernel] [k] _atomic_dec_and_lock
0.39% [kernel] [k] __link_path_walk
0.38% [kernel] [k] memset
0.37% [kernel] [k] do_lookup
0.34% [kernel] [k] radix_tree_lookup_slot
0.34% [kernel] [k] down_read_trylock
0.33% [kernel] [k] kmem_cache_alloc
0.31% [kernel] [k] __set_page_dirty_no_writeback
0.31% [kernel] [k] __inc_zone_state
0.31% [kernel] [k] __mem_cgroup_uncharge_common
root [~]# iotop
Total DISK READ: 0.00 B/s | Total DISK WRITE: 2.33 M/s
TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND
541 be/3 root 0.00 B/s 7.83 K/s 0.00 % 2.27 % [jbd2/md2-8]
8568 be/4 root 0.00 B/s 7.83 K/s 0.00 % 0.00 % lfd - sleeping
1457 be/3 root 0.00 B/s 7.83 K/s 0.00 % 0.00 % auditd
1669 be/4 root 0.00 B/s 3.91 K/s 0.00 % 0.00 % rsyslogd -i
/var/run/syslogd.pid -c 5
1695 be/4 named 0.00 B/s 3.91 K/s 0.00 % 0.00 % named -u named
31391 be/4 mysql 0.00 B/s 23.48 K/s 0.00 % 0.00 % mysqld
--basedir=/ --datadir=/var/lib/mysql --user=mysql --log-error=/var~r
--open-files-limit=50000 --pid-file=/var/lib/mysql/server.pid
1 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % init
2 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [kthreadd]
3 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/0]
4 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [ksoftirqd/0]
5 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/0]
6 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [watchdog/0]
7 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/1]
8 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/1]
9 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [ksoftirqd/1]
10 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [watchdog/1]
11 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/2]
12 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/2]
13 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [ksoftirqd/2]
14 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [watchdog/2]
15 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/3]
16 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/3]
17 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [ksoftirqd/3]
18 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [watchdog/3]
19 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/4]
20 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/4]
21 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [ksoftirqd/4]
22 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [watchdog/4]
23 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/5]
24 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/5]
25 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [ksoftirqd/5]
26 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [watchdog/5]
27 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/6]
28 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/6]
29 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [ksoftirqd/6]
30 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [watchdog/6]
31 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/7]
32 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/7]
33 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [ksoftirqd/7]
34 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [watchdog/7]
35 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [events/0]
36 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [events/1]
37 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [events/2]
38 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [events/3]
39 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [events/4]
40 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [events/5]
41 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [events/6]
42 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [events/7]
43 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [cgroup]
44 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [khelper]
45 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [netns]
46 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [async/mgr]
47 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [pm]
48 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [sync_supers]
49 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [bdi-default]
50 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [kintegrityd/0]
51 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [kintegrityd/1]
52 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [kintegrityd/2]
53 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [kintegrityd/3]
54 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [kintegrityd/4]
Now the file copy with sync:
root [~]# time (cp largefile.tar.gz test05.tmp; sync)
real 1m33.923s
user 0m0.002s
sys 0m0.713s
Large file size: 523MB
BW determination: 523MB / 93.923 seconds = 5.56MB/s
File copy without sync:
root [~]# echo 3 > /proc/sys/vm/drop_caches
root [~]# time cp largefile.tar.gz test07.tmp
real 0m6.452s
user 0m0.007s
sys 0m0.687s
Large file size: 523MB
BW determination: 523MB / 6.452 seconds = 81.06 MB/s
During the copy (near the end: about 70 seconds into the copy - results
with sync):
Samples: 17K of event 'cycles', Event count (approx.): 5067697991
7.48% [kernel] [k] port_inb
5.40% [kernel] [k] page_fault
2.92% [kernel] [k] clear_page_c_e
2.29% [kernel] [k] list_del
2.21% [kernel] [k] _spin_lock
1.99% [kernel] [k] __d_lookup
1.92% [kernel] [k] avtab_search_node
1.64% [kernel] [k] unmap_vmas
1.59% [kernel] [k] get_page_from_freelist
1.55% [kernel] [k] __mem_cgroup_commit_charge
1.22% [kernel] [k] mem_cgroup_update_file_mapped
1.21% [kernel] [k] copy_page_c
1.04% [kernel] [k] find_vma
1.00% [kernel] [k] _spin_lock_irq
0.97% [kernel] [k] __wake_up_bit
0.94% [kernel] [k] __mem_cgroup_uncharge_common
0.92% [kernel] [k] get_page
0.91% [kernel] [k] __alloc_pages_nodemask
0.87% [kernel] [k] handle_mm_fault
0.85% [kernel] [k] __link_path_walk
0.84% [kernel] [k] avc_has_perm_noaudit
0.83% [kernel] [k] alloc_pages_vma
0.81% [kernel] [k] lookup_page_cgroup
0.80% [kernel] [k] __do_page_fault
0.80% [kernel] [k] free_pcppages_bulk
0.77% [kernel] [k] _spin_lock_irqsave
0.75% [kernel] [k] radix_tree_lookup_slot
0.73% [kernel] [k] kmem_cache_alloc
0.68% [ip_tables] [k] ipt_do_table
0.66% [kernel] [k] _atomic_dec_and_lock
0.65% [kernel] [k] release_pages
0.62% [kernel] [k] find_get_page
0.61% [kernel] [k] schedule
0.60% [kernel] [k] inode_has_perm
0.56% [kernel] [k] sidtab_context_to_sid
0.54% [kernel] [k] handle_pte_fault
0.53% [kernel] [k] _spin_unlock_irqrestore
0.53% [kernel] [k] memset
0.52% [kernel] [k] __inc_zone_state
0.51% [kernel] [k] update_curr
0.51% [kernel] [k] kfree
0.50% [kernel] [k] __list_add
0.50% [kernel] [k] __do_fault
0.49% [kernel] [k] shmem_getpage_gfp
0.47% [kernel] [k] filemap_fault
Total DISK READ: 0.00 B/s | Total DISK WRITE: 0.00 B/s
TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND
541 be/3 root 0.00 B/s 0.00 B/s 0.00 % 96.96 % [jbd2/md2-8]
12468 be/4 nobody 0.00 B/s 3.89 K/s 0.00 % 0.00 % httpd -k
start -DSSL
18818 be/4 mysql 0.00 B/s 3.89 K/s 0.00 % 0.00 % mysqld
--basedir=/ --da~sql/server.pid
12333 be/4 nobody 0.00 B/s 3.89 K/s 0.00 % 0.00 % httpd -k
start -DSSL
12560 be/4 nobody 0.00 B/s 3.89 K/s 0.00 % 0.00 % httpd -k
start -DSSL
12568 be/4 nobody 0.00 B/s 3.89 K/s 0.00 % 0.00 % httpd -k
start -DSSL
12281 be/4 nobody 0.00 B/s 3.89 K/s 0.00 % 0.00 % [httpd]
1 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % init
2 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [kthreadd]
3 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/0]
4 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [ksoftirqd/0]
5 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/0]
6 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [watchdog/0]
7 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/1]
8 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/1]
9 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [ksoftirqd/1]
10 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [watchdog/1]
11 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/2]
12 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/2]
13 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [ksoftirqd/2]
14 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [watchdog/2]
15 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/3]
16 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/3]
17 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [ksoftirqd/3]
18 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [watchdog/3]
19 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/4]
20 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/4]
21 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [ksoftirqd/4]
22 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [watchdog/4]
23 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/5]
24 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/5]
25 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [ksoftirqd/5]
26 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [watchdog/5]
27 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/6]
Please let me know if I messed up again so that I can correct it.
@Adam
3. root [~]# fdisk -lu /dev/sd*
Disk /dev/sda: 512.1 GB, 512110190592 bytes
255 heads, 63 sectors/track, 62260 cylinders, total 1000215216 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00026d59
Device Boot Start End Blocks Id System
/dev/sda1 2048 4196351 2097152 fd Linux raid
autodetect
Partition 1 does not end on cylinder boundary.
/dev/sda2 * 4196352 4605951 204800 fd Linux raid
autodetect
Partition 2 does not end on cylinder boundary.
/dev/sda3 4605952 814106623 404750336 fd Linux raid
autodetect
Disk /dev/sda1: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders, total 4194304 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xfffefffe
Disk /dev/sda2: 209 MB, 209715200 bytes
255 heads, 63 sectors/track, 25 cylinders, total 409600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/sda3: 414.5 GB, 414464344064 bytes
255 heads, 63 sectors/track, 50389 cylinders, total 809500672 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/sdb: 512.1 GB, 512110190592 bytes
255 heads, 63 sectors/track, 62260 cylinders, total 1000215216 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0003dede
Device Boot Start End Blocks Id System
/dev/sdb1 2048 4196351 2097152 fd Linux raid
autodetect
Partition 1 does not end on cylinder boundary.
/dev/sdb2 * 4196352 4605951 204800 fd Linux raid
autodetect
Partition 2 does not end on cylinder boundary.
/dev/sdb3 4605952 814106623 404750336 fd Linux raid
autodetect
Disk /dev/sdb1: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders, total 4194304 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xfffefffe
Disk /dev/sdb2: 209 MB, 209715200 bytes
255 heads, 63 sectors/track, 25 cylinders, total 409600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/sdb3: 414.5 GB, 414464344064 bytes
255 heads, 63 sectors/track, 50389 cylinders, total 809500672 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Kind regards!
Andrei Banu
On 4/24/2013 6:24 AM, Stan Hoeppner wrote:
> root [~]# cat /etc/fstab
>> UUID=cef1d19d-2578-43db-9ffc-b6b70e227bfa swap swap defaults 0 0
> I can't discern from UUID where your swap partition is located. Is it a
> partition directly on an SSD or is it a partition atop md1?
>
>> root [/]# echo 3 > /proc/sys/vm/drop_caches
>> root [~]# time cp largefile.tar.gz test03.tmp; time sync;
> You're slowing us down here. Please execute commands as instructed
> without modification. The above is wrong. You don't call time twice.
> If you're worried about sync execution being included time, use:
> $ time (cp src.tmp src.temp; sync)
>
> Though it makes little difference as Linux is pretty good about flushing
> the last few write buffers. But you missed the important part, the math
> for bandwidth determination: 548/real = xx MB/s
>
> This is cp not dd. It's up to you to do the math. Using time allows
> you to do so. 548MB is my example using your previous file size in your
> tests. Modify accordingly if needed.
>
> *Important note* The job of this list is to provide knowledge transfer,
> advice, and assistance. You must do the work, and you must learn along
> the way. We don't fix people's problems, as we don't have access to
> their computers. What we do is *enable* people to fix their problems
> themselves.
>
>> After about 15 seconds the server load started to increase from 1,
>> spiked to 40 in about a minute and then it started decreasing.
> Please stop telling us this. Linux load average is irrelevant.
>
>> 5. The perf top -U output during a dd copy:
> This was supposed to be executed before and simultaneously with the cp
> operation above. Do you know how to use multiple terminal windows?
>
>> 6. iotop
> Again, this was supposed to be run with the cp command, exited toward
> the end of the cp operation, then copy/pasted.
>
> is very dynamic and I am afraid the data I am providing will be
>> unclear but let me give a number of snapshots from during the large file
>> copy and maybe you can make something of it (samples a few seconds apart):
>> !!!!!! 6085 be/4 root 7.69 K/s 1004.85 M/s 0.00 % 0.00 % dd
>> if=largefile.tar.gz of=test10 oflag=sync bs=1G
> This is another example of why you don't use dd for IO testing, and
> especially with a block size of 1GB. dd buffers into RAM up to
> $block_size bytes before it begins flushing to disk. So what you're
> seeing here is that massive push at the beginning of the run. Your SSDs
> in RAID1 peak at ~265MB/s. iotop is showing 1GB/s, 4 times what the
> drives can do. This is obviously not real.
>
> You can get away with oflag=sync using 1GB block size. But if you run
> dd the only way it can be run for realistic results, using bs=4096 which
> matches every filesystem block size including EXTx, XFS, and JFS, then
> using iflag=sync will degrade your performance, an ack is required on
> each block. That's what sync does. With SSD it won't be nearly as
> dramatic as rust, where the difference in runtime is 100-200x slower due
> to rotational latency.
>
>> I appologize for such a lengthy email!
> Don't apologize, just don't send more information than needed,
> especially if you don't know it's relevant. ;) Send only what's
> requested, and as requested, please.
>
next prev parent reply other threads:[~2013-04-24 8:26 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-04-19 22:58 Incredibly poor performance of mdraid-1 with 2 SSD Samsung 840 PRO Andrei Banu
[not found] ` <CAH3kUhEaZGON=fAyVMZOz5fH_DcfKv=hCa96UCeK4pN7k81c_Q@mail.gmail.com>
2013-04-20 23:26 ` Andrei Banu
[not found] ` <51725458.7020109@redhost.ro>
[not found] ` <CAH3kUhHxBiqugFQm=PPJNNe9jOdKy0etUjQNsoDz_LJNUCLCCQ@mail.gmail.com>
2013-04-20 23:25 ` Andrei Banu
2013-04-20 23:26 ` Andrei Banu
2013-04-21 2:48 ` Stan Hoeppner
2013-04-21 12:23 ` Tommy Apel
2013-04-21 16:48 ` Tommy Apel
2013-04-21 19:33 ` Stan Hoeppner
2013-04-21 19:56 ` Tommy Apel
2013-04-22 0:47 ` Stan Hoeppner
2013-04-22 7:51 ` Tommy Apel
2013-04-22 8:29 ` Tommy Apel
2013-04-22 10:26 ` Andrei Banu
2013-04-22 12:02 ` Tommy Apel
2013-04-23 2:59 ` Stan Hoeppner
2013-04-22 23:21 ` Stan Hoeppner
2013-04-25 11:38 ` Thomas Jarosch
2013-04-21 0:10 ` Stan Hoeppner
[not found] ` <51732E2B.6090607@hardwarefreak.com>
2013-04-21 20:46 ` Andrei Banu
2013-04-21 23:17 ` Stan Hoeppner
2013-04-22 10:19 ` Andrei Banu
2013-04-23 2:51 ` Stan Hoeppner
2013-04-23 10:17 ` Andrei Banu
2013-04-24 3:24 ` Stan Hoeppner
2013-04-24 8:26 ` Andrei Banu [this message]
2013-04-24 9:12 ` Adam Goryachev
2013-04-24 10:24 ` Tommy Apel
2013-04-24 21:42 ` Andrei Banu
2013-04-24 21:40 ` Andrei Banu
2013-04-24 16:37 ` Stan Hoeppner
2013-04-24 21:46 ` Andrei Banu
[not found] ` <CAH3kUhHnF0imY=CAHfzaQy4XJuOMgOtbHNp17EYzeSJR2en7Fg@mail.gmail.com>
2013-04-25 10:11 ` Andrei Banu
2013-04-25 10:56 ` Stan Hoeppner
2013-04-22 23:11 ` Andrei Banu
2013-04-23 4:39 ` Stan Hoeppner
2013-04-22 23:25 ` Stan Hoeppner
2013-04-23 4:49 ` Mikael Abrahamsson
2013-04-23 6:01 ` Stan Hoeppner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=51779720.6040109@redhost.ro \
--to=andrei.banu@redhost.ro \
--cc=linux-raid@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.