Defrag operations sometimes don't work.

Defrag operations sometimes don't work.

[erpo41]

[-- Attachment #1: Type: text/plain, Size: 12330 bytes --]

Good afternoon,

First, my apologies if this is a repeat email. My original email
contained an uncompressed copy of dmesg's output and I *believe*
that pushed the message over the 100KB limit, causing the list to
drop it.

When I try to defragment files (or recursively defragment trees of
files) on my btrfs filesystem, I get inconsistent results. In the
following example, I had a 16MB file with 3143 extents. Running btrfs
fi defrag /path/to/file once did nothing. Immediately running the
defrag command a second time reduced the file to 2 extents.

Other times a file may never defragment, regardless of how many times I
run the defrag command or which values I pass to the -t option (0, 1,
10m, 100m, 1g, etc.).

Am I using the defrag tool incorrectly, or should I expect this sort
of behavior while defragging?

Thanks,
Eric


Diagnostic Information
----------------------
uname -a: Linux europa 3.19.0-22-generic #22-Ubuntu SMP Tue Jun 16
17:15:15 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

btrfs --version: Btrfs v3.17

btrfs fi show:
Label: none  uuid: 66df2e33-e4f9-4fe6-aaa8-9137078b3267
    Total devices 1 FS bytes used 456.59GiB
    devid    1 size 2.72TiB used 497.06GiB path /dev/sda2

Btrfs v3.17

btrfs fi df /home:
Data, single: total=492.00GiB, used=454.91GiB
System, DUP: total=32.00MiB, used=64.00KiB
Metadata, DUP: total=2.50GiB, used=1.68GiB
GlobalReserve, single: total=512.00MiB, used=0.00B

dmesg: please see attached

Other potentially useful information:
* This btrfs filesystem was originally created under either Ubuntu
13.04 or 13.10--I can't recall which. Since then the system has been
upgraded to Ubuntu 15.04, and a full rebalance was performed.
* There is a regression in the driver for my WiFi card in Ubuntu 15.04
that caused very poor network performance and dropped connections. As
a workaround, I have set up a cron job to run "iw dev wlan0 set
power_save off" after every boot.


Terminal Transcript
-------------------

eric@europa:~/filefrag/2015-07-09$ filefrag
/home/.ecryptfs/eric/.Private/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1MzDigXwKSteYI4y1XLf7qE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1YygQLPowcpLwoNE.bCX4a---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1SkdyLVwpIlrGQkpNoByI4k--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1xwwXaj3LOfVMtlJJjN6ZQ---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1KA5vCq5.eyeeHNBt.MVbF---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1k2qM8iPdbdF5DWIlAoC0XE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1xNsKJhD8t.i3HCsRyJ998---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1AnqPecL1Vs7HD55Lcv-.CU--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1IpxQ4nE4Q3WQgRcSqpgFD---
/home/.ecryptfs/eric/.Private/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1MzDigXwKSteYI4y1XLf7qE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1YygQLPowcpLwoNE.bCX4a---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1SkdyLVwpIlrGQkpNoByI4k--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1xwwXaj3LOfVMtlJJjN6ZQ---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1KA5vCq5.eyeeHNBt.MVbF---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1k2qM8iPdbdF5DWIlAoC0XE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1xNsKJhD8t.i3HCsRyJ998---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1AnqPecL1Vs7HD55Lcv-.CU--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1IpxQ4nE4Q3WQgRcSqpgFD---:
3143 extents found

eric@europa:~/filefrag/2015-07-09$ btrfs fi defrag
/home/.ecryptfs/eric/.Private/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1MzDigXwKSteYI4y1XLf7qE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1YygQLPowcpLwoNE.bCX4a---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1SkdyLVwpIlrGQkpNoByI4k--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1xwwXaj3LOfVMtlJJjN6ZQ---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1KA5vCq5.eyeeHNBt.MVbF---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1k2qM8iPdbdF5DWIlAoC0XE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1xNsKJhD8t.i3HCsRyJ998---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1AnqPecL1Vs7HD55Lcv-.CU--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1IpxQ4nE4Q3WQgRcSqpgFD---

eric@europa:~/filefrag/2015-07-09$ filefrag
/home/.ecryptfs/eric/.Private/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1MzDigXwKSteYI4y1XLf7qE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1YygQLPowcpLwoNE.bCX4a---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1SkdyLVwpIlrGQkpNoByI4k--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1xwwXaj3LOfVMtlJJjN6ZQ---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1KA5vCq5.eyeeHNBt.MVbF---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1k2qM8iPdbdF5DWIlAoC0XE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1xNsKJhD8t.i3HCsRyJ998---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1AnqPecL1Vs7HD55Lcv-.CU--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1IpxQ4nE4Q3WQgRcSqpgFD---
/home/.ecryptfs/eric/.Private/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1MzDigXwKSteYI4y1XLf7qE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1YygQLPowcpLwoNE.bCX4a---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1SkdyLVwpIlrGQkpNoByI4k--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1xwwXaj3LOfVMtlJJjN6ZQ---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1KA5vCq5.eyeeHNBt.MVbF---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1k2qM8iPdbdF5DWIlAoC0XE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1xNsKJhD8t.i3HCsRyJ998---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1AnqPecL1Vs7HD55Lcv-.CU--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1IpxQ4nE4Q3WQgRcSqpgFD---:
3143 extents found

eric@europa:~/filefrag/2015-07-09$ btrfs fi defrag
/home/.ecryptfs/eric/.Private/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1MzDigXwKSteYI4y1XLf7qE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1YygQLPowcpLwoNE.bCX4a---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1SkdyLVwpIlrGQkpNoByI4k--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1xwwXaj3LOfVMtlJJjN6ZQ---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1KA5vCq5.eyeeHNBt.MVbF---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1k2qM8iPdbdF5DWIlAoC0XE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1xNsKJhD8t.i3HCsRyJ998---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1AnqPecL1Vs7HD55Lcv-.CU--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1IpxQ4nE4Q3WQgRcSqpgFD---

eric@europa:~/filefrag/2015-07-09$ filefrag
/home/.ecryptfs/eric/.Private/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1MzDigXwKSteYI4y1XLf7qE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1YygQLPowcpLwoNE.bCX4a---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1SkdyLVwpIlrGQkpNoByI4k--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1xwwXaj3LOfVMtlJJjN6ZQ---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1KA5vCq5.eyeeHNBt.MVbF---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1k2qM8iPdbdF5DWIlAoC0XE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1xNsKJhD8t.i3HCsRyJ998---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1AnqPecL1Vs7HD55Lcv-.CU--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1IpxQ4nE4Q3WQgRcSqpgFD---
/home/.ecryptfs/eric/.Private/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1MzDigXwKSteYI4y1XLf7qE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1YygQLPowcpLwoNE.bCX4a---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1SkdyLVwpIlrGQkpNoByI4k--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1xwwXaj3LOfVMtlJJjN6ZQ---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1KA5vCq5.eyeeHNBt.MVbF---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1k2qM8iPdbdF5DWIlAoC0XE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1xNsKJhD8t.i3HCsRyJ998---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1AnqPecL1Vs7HD55Lcv-.CU--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1IpxQ4nE4Q3WQgRcSqpgFD---:
2 extents found

eric@europa:~/filefrag/2015-07-09$ stat
/home/.ecryptfs/eric/.Private/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1MzDigXwKSteYI4y1XLf7qE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1YygQLPowcpLwoNE.bCX4a---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1SkdyLVwpIlrGQkpNoByI4k--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1xwwXaj3LOfVMtlJJjN6ZQ---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1KA5vCq5.eyeeHNBt.MVbF---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1k2qM8iPdbdF5DWIlAoC0XE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1xNsKJhD8t.i3HCsRyJ998---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1AnqPecL1Vs7HD55Lcv-.CU--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1IpxQ4nE4Q3WQgRcSqpgFD---

  File: ‘/home/.ecryptfs/eric/.Private/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1MzDigXwKSteYI4y1XLf7qE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1YygQLPowcpLwoNE.bCX4a---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1SkdyLVwpIlrGQkpNoByI4k--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1xwwXaj3LOfVMtlJJjN6ZQ---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1KA5vCq5.eyeeHNBt.MVbF---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1k2qM8iPdbdF5DWIlAoC0XE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1xNsKJhD8t.i3HCsRyJ998---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1AnqPecL1Vs7HD55Lcv-.CU--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1IpxQ4nE4Q3WQgRcSqpgFD---’
  Size: 16629760      Blocks: 32480      IO Block: 4096   regular file
Device: 28h/40d    Inode: 254812      Links: 1
Access: (0600/-rw-------)  Uid: ( 1000/    eric)   Gid: ( 1000/    eric)
Access: 2015-07-08 21:44:34.357334086 -0600
Modify: 2013-09-26 21:30:13.769451000 -0600
Change: 2013-09-26 21:30:13.769451000 -0600
 Birth: -

eric@europa:~/filefrag/2015-07-09$ du -h
/home/.ecryptfs/eric/.Private/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1MzDigXwKSteYI4y1XLf7qE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1YygQLPowcpLwoNE.bCX4a---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1SkdyLVwpIlrGQkpNoByI4k--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1xwwXaj3LOfVMtlJJjN6ZQ---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1KA5vCq5.eyeeHNBt.MVbF---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1k2qM8iPdbdF5DWIlAoC0XE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1xNsKJhD8t.i3HCsRyJ998---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1AnqPecL1Vs7HD55Lcv-.CU--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1IpxQ4nE4Q3WQgRcSqpgFD---
16M    /home/.ecryptfs/eric/.Private/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1MzDigXwKSteYI4y1XLf7qE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1YygQLPowcpLwoNE.bCX4a---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1SkdyLVwpIlrGQkpNoByI4k--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1xwwXaj3LOfVMtlJJjN6ZQ---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1KA5vCq5.eyeeHNBt.MVbF---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1k2qM8iPdbdF5DWIlAoC0XE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1xNsKJhD8t.i3HCsRyJ998---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1AnqPecL1Vs7HD55Lcv-.CU--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1IpxQ4nE4Q3WQgRcSqpgFD---

[-- Attachment #2: dmesg.log.gz --]
[-- Type: application/x-gzip, Size: 19259 bytes --]

Re: Defrag operations sometimes don't work.

[Martin Steigerwald]

Am Freitag, 10. Juli 2015, 19:12:34 schrieb erpo41@gmail.com:
> Good afternoon,

Hi Eric,

> First, my apologies if this is a repeat email. My original email
> contained an uncompressed copy of dmesg's output and I *believe*
> that pushed the message over the 100KB limit, causing the list to
> drop it.
> 
> When I try to defragment files (or recursively defragment trees of
> files) on my btrfs filesystem, I get inconsistent results. In the
> following example, I had a 16MB file with 3143 extents. Running btrfs
> fi defrag /path/to/file once did nothing. Immediately running the
> defrag command a second time reduced the file to 2 extents.

Always do "sync" after a "btrfs fi defrag" and before measuring with 
"filefrag". The kernel may not have written everything. I have seen this 
repeatedly that the extent count drops further after a "sync", following 
"btrfs fi defrag".

Or wait longer before you measure.
 
> Other times a file may never defragment, regardless of how many times I
> run the defrag command or which values I pass to the -t option (0, 1,
> 10m, 100m, 1g, etc.).
> 
> Am I using the defrag tool incorrectly, or should I expect this sort
> of behavior while defragging?
> 
> Thanks,
> Eric
> 
> 
> Diagnostic Information
> ----------------------
> uname -a: Linux europa 3.19.0-22-generic #22-Ubuntu SMP Tue Jun 16
> 17:15:15 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
> 
> btrfs --version: Btrfs v3.17

I recommend to update both :), but for the defragging it should not matter.

[…]
> Terminal Transcript
> -------------------
> 
> eric@europa:~/filefrag/2015-07-09$ filefrag
> /home/.ecryptfs/eric/.Private/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5Uh
[…]

Nice, someone like me searching the correspondending file name in ecryptfs 
for defragmenting :)

Thanks,
-- 
Martin

Re: Defrag operations sometimes don't work.

[erpo41]

On Sat, Jul 11, 2015 at 4:12 AM, Martin Steigerwald <martin@lichtvoll.de> wrote:
> Always do "sync" after a "btrfs fi defrag" and before measuring with
> "filefrag". The kernel may not have written everything. I have seen this
> repeatedly that the extent count drops further after a "sync", following
> "btrfs fi defrag".

First of all, thank you for your help. This fixed the problem for
defragging individual files.

I'm now seeing that recursive defragging doesn't work the way I expect. Running

$ btrfs fi defrag -r /path/to

returns almost immediately and does not reduce the number of fragments
in /path/to/file. However, running

$ btrfs fi defrag /path/to/file

does reduce the number of fragments.


Transcript:
eric@europa:~/filefrag/2015-07-11 09:53:32-06:00$ sync

eric@europa:~/filefrag/2015-07-11 09:53:32-06:00$ filefrag
/home/.ecryptfs/eric/.Private/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1MzDigXwKSteYI4y1XLf7qE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1YygQLPowcpLwoNE.bCX4a---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1rvNb.zifeXTwNQmAhQdkpU--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1oFfB25Ypui-NUlhLrvnCu---/ECRYPTFS_FNEK_ENCRYPTED.FXbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1e.GaDiBl08xl5bZAxYj09kaLyAQQuN1NY-zCYqLw21g-
/home/.ecryptfs/eric/.Private/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1MzDigXwKSteYI4y1XLf7qE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1YygQLPowcpLwoNE.bCX4a---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1rvNb.zifeXTwNQmAhQdkpU--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1oFfB25Ypui-NUlhLrvnCu---/ECRYPTFS_FNEK_ENCRYPTED.FXbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1e.GaDiBl08xl5bZAxYj09kaLyAQQuN1NY-zCYqLw21g-:
1567 extents found

eric@europa:~/filefrag/2015-07-11 09:53:32-06:00$ time btrfs fi defrag
-r /home/.ecryptfs/eric/.Private/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1MzDigXwKSteYI4y1XLf7qE--

real    0m0.006s
user    0m0.000s
sys    0m0.004s

eric@europa:~/filefrag/2015-07-11 09:53:32-06:00$ sync

eric@europa:~/filefrag/2015-07-11 09:53:32-06:00$ filefrag
/home/.ecryptfs/eric/.Private/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1MzDigXwKSteYI4y1XLf7qE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1YygQLPowcpLwoNE.bCX4a---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1rvNb.zifeXTwNQmAhQdkpU--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1oFfB25Ypui-NUlhLrvnCu---/ECRYPTFS_FNEK_ENCRYPTED.FXbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1e.GaDiBl08xl5bZAxYj09kaLyAQQuN1NY-zCYqLw21g-
/home/.ecryptfs/eric/.Private/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1MzDigXwKSteYI4y1XLf7qE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1YygQLPowcpLwoNE.bCX4a---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1rvNb.zifeXTwNQmAhQdkpU--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1oFfB25Ypui-NUlhLrvnCu---/ECRYPTFS_FNEK_ENCRYPTED.FXbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1e.GaDiBl08xl5bZAxYj09kaLyAQQuN1NY-zCYqLw21g-:
1567 extents found

eric@europa:~/filefrag/2015-07-11 09:53:32-06:00$ btrfs fi defrag
/home/.ecryptfs/eric/.Private/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1MzDigXwKSteYI4y1XLf7qE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1YygQLPowcpLwoNE.bCX4a---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1rvNb.zifeXTwNQmAhQdkpU--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1oFfB25Ypui-NUlhLrvnCu---/ECRYPTFS_FNEK_ENCRYPTED.FXbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1e.GaDiBl08xl5bZAxYj09kaLyAQQuN1NY-zCYqLw21g-

eric@europa:~/filefrag/2015-07-11 09:53:32-06:00$ sync

eric@europa:~/filefrag/2015-07-11 09:53:32-06:00$ filefrag
/home/.ecryptfs/eric/.Private/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1MzDigXwKSteYI4y1XLf7qE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1YygQLPowcpLwoNE.bCX4a---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1rvNb.zifeXTwNQmAhQdkpU--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1oFfB25Ypui-NUlhLrvnCu---/ECRYPTFS_FNEK_ENCRYPTED.FXbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1e.GaDiBl08xl5bZAxYj09kaLyAQQuN1NY-zCYqLw21g-
/home/.ecryptfs/eric/.Private/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1MzDigXwKSteYI4y1XLf7qE--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1YygQLPowcpLwoNE.bCX4a---/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1rvNb.zifeXTwNQmAhQdkpU--/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1oFfB25Ypui-NUlhLrvnCu---/ECRYPTFS_FNEK_ENCRYPTED.FXbZ4j3re5lQYkQsG5UhAnMElrqSQB7o7Dt1e.GaDiBl08xl5bZAxYj09kaLyAQQuN1NY-zCYqLw21g-:
15 extents found

>> btrfs --version: Btrfs v3.17
>
> I recommend to update both :), but for the defragging it should not matter.

As long as it doesn't affect defragging, I think I'll stick with what
I've got. There isn't any critical data on this machine that I would
miss.

> […]
>> Terminal Transcript
>> -------------------
>>
>> eric@europa:~/filefrag/2015-07-09$ filefrag
>> /home/.ecryptfs/eric/.Private/ECRYPTFS_FNEK_ENCRYPTED.FWbZ4j3re5lQYkQsG5Uh
> […]
>
> Nice, someone like me searching the correspondending file name in ecryptfs
> for defragmenting :)

I was kind of surprised that ecryptfs-find uses inode numbers and the
find command to locate the encrypted name of a file. It seems like
there should be a better, much faster way to do this.

Thanks,
Eric

Re: Defrag operations sometimes don't work.

[Martin Steigerwald]

Am Samstag, 11. Juli 2015, 10:40:29 schrieb erpo41@gmail.com:
> On Sat, Jul 11, 2015 at 4:12 AM, Martin Steigerwald <martin@lichtvoll.de> 
wrote:
> > Always do "sync" after a "btrfs fi defrag" and before measuring with
> > "filefrag". The kernel may not have written everything. I have seen this
> > repeatedly that the extent count drops further after a "sync", following
> > "btrfs fi defrag".
> 
> First of all, thank you for your help. This fixed the problem for
> defragging individual files.

Note however: Even without "sync" BTRFS will defragment the file. It may 
just take a while till the new extents are written. So there is no need to 
call "sync" after btrfs fi defrag.

Why are you trying to defragment anyway? What are you trying to achieve / 
solve?

> I'm now seeing that recursive defragging doesn't work the way I expect.
> Running
> 
> $ btrfs fi defrag -r /path/to
> 
> returns almost immediately and does not reduce the number of fragments
> in /path/to/file. However, running
> 
> $ btrfs fi defrag /path/to/file
> 
> does reduce the number of fragments.

Well, I have no idea about this one. I have the same behavior with

> >> btrfs --version: Btrfs v3.17

v4.0

Ciao,
-- 
Martin

Re: Defrag operations sometimes don't work.

[erpo41]

On Sun, Jul 12, 2015 at 2:54 AM, Martin Steigerwald <martin@lichtvoll.de> wrote:
> Note however: Even without "sync" BTRFS will defragment the file. It may
> just take a while till the new extents are written. So there is no need to
> call "sync" after btrfs fi defrag.

My purpose in calling sync in that transcript was to ensure each
invocation of filefrag returned the correct results.

> Why are you trying to defragment anyway? What are you trying to achieve /
> solve?

There are several reasons. Some are technical and some are emotional.

Technical reason 1: I sometimes play computer games that are delivered
through Steam. If you're not familiar with it, Steam is a sort of
online storefront application for Linux and other platforms that--I
believe--delivers game patches by overwriting selected ranges in each
game's files. Lots of small overwrites are the main culprit behind
small files with thousand or tens of thousands of fragments on btrfs.
All I know for sure is that some game files have become heavily
fragmented and load times seem excessive, especially given that
they're older games compared to my hardware.

Technical reason 2: Whenever a new Ubuntu release comes out, I make
sure to download and burn the ISO before I upgrade my OS. Usually I
download the ISO using bittorrent, which uses lots of random writes
creating heavily fragmented downloads that severely impact read speeds
from my non-SSD hard disk. This increases my load quite a bit when I
seed the download back to other people. I've solved this problem in
the past by pausing the bittorrent client, copying the file to another
location, deleting the original, moving the copy back into place with
the same name, and resuming the client, but that's a pain. I'd rather
defragment instead.

Emotional reason 1: Heavily fragmented files on rotating media just
feel slovenly to me, like a person wearing dirty clothes or a bathroom
that hasn't been cleaned in a while.

Emotional reason 2: I switched from Windows to Linux in the late 90s.
At that time, RAM was expensive, so I often found myself in situations
where my kernel was dropping pages from the filesystem cache to make
room for application memory. Rereading those files later was very slow
if they were fragmented, so I religiously defragmented my Windows 95
and 98 machines every week. When I switched to Linux, I was dismayed
to learn that ext2 had no defragmentation tool, and I was very
skeptical of claims that "Linux doesn't need defragmentation",
especially considering that most of the people making those claims
probably had no idea how fragmented their own filesystems were. In the
years that followed, I was continually disappointed as stable
defragmentation tools did not appear for ext2 or any other popular
Linux filesystems, so I'm relieved and excited not to be in that
predicament anymore.

>> I'm now seeing that recursive defragging doesn't work the way I expect.
>> Running
>>
>> $ btrfs fi defrag -r /path/to
>>
>> returns almost immediately and does not reduce the number of fragments
>> in /path/to/file. However, running
>>
>> $ btrfs fi defrag /path/to/file
>>
>> does reduce the number of fragments.
>
> Well, I have no idea about this one. I have the same behavior with
>
>> >> btrfs --version: Btrfs v3.17
>
> v4.0

Thanks for checking,
Eric