[Bug 216714] New: Issue with file system image created with mke2fs parameter -E offset

[Bug 216714] New: Issue with file system image created with mke2fs parameter -E offset

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=216714

            Bug ID: 216714
           Summary: Issue with file system image created with mke2fs
                    parameter -E offset
           Product: File System
           Version: 2.5
    Kernel Version: Linux 5.4.0-131-generic
          Hardware: Intel
                OS: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: ext4
          Assignee: fs_ext4@kernel-bugs.osdl.org
          Reporter: tmahmud@iastate.edu
        Regression: No

Overview:
We encountered an issue while using e2fsck to check the file system image
created with the command “mke2fs -t ext4 -E offset=10”.
The issue was observed in e2fsprogs 1.46.5 and 1.45.6. 

According to the manpage of mke2fs, the parameter “-E offset”  is used to
create the file system at an offset from the beginning of the device or file.
This can be useful when creating disk images for virtual machines. No range is
specified for this parameter. 

Below are the steps to reproduce:

truncate -s 500M image
mke2fs -t ext4 -E offset=10 image 500M
e2fsck -yf image

mke2fs output log:
mke2fs 1.46.5 (30-Dec-2021)
Discarding device blocks: done                            
Creating filesystem with 512000 1k blocks and 128016 inodes
Filesystem UUID: 6154bed2-c92b-45fe-b6db-97192ae46155
Superblock backups stored on blocks: 
        8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done


e2fsck output log:
e2fsck 1.46.5 (30-Dec-2021)
ext2fs_open2: Bad magic number in super-block
e2fsck: Superblock invalid, trying backup blocks...
e2fsck: Bad magic number in super-block while trying to open image

The superblock could not be read or does not describe a valid ext2/ext3/ext4
filesystem.  If the device is valid and it really contains an ext2/ext3/ext4
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
    e2fsck -b 8193 <device>
 or
    e2fsck -b 32768 <device>

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 216714] Issue with file system image created with mke2fs parameter -E offset

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=216714

Lukas Czerner (lczerner@redhat.com) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |lczerner@redhat.com

--- Comment #1 from Lukas Czerner (lczerner@redhat.com) ---
Hi,

the -E offset=10 creates a file system that is offset by 10 bytes from the
beginning of the device or a file (as is written in the man page). It is no
wonder then that the tools can't recognize the file system on the device or a
file because the super block is not where it should be - it is offset by 10
bytes.

If you want to use such a file system you have to present it to the other
tools, or even kernel, with the correct offset. It is not going to be
automatic.

So for example this works:

mke2fs -q -t ext4 -E offset=10 image 500M
sudo losetup -o 10 --show -f image
/dev/loop0
sudo e2fsck -f /dev/loop0
e2fsck 1.46.6-rc1 (12-Sep-2022)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/loop0: 11/128016 files (0.0% non-contiguous), 26684/512000 blocks

I hope it helps.

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 216714] Issue with file system image created with mke2fs parameter -E offset

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=216714

Artem S. Tashkinov (aros@gmx.com) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |INVALID

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 216714] Issue with file system image created with mke2fs parameter -E offset

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=216714

--- Comment #2 from tmahmud@iastate.edu ---
Hi Lukas,

Thank you for your reply. It makes sense to me now. I tried using the offset
parameter with losetup and it works for me. 

I have another question related to this. Is there any way to find out the
offset after creating the file system image? For example, after creating the
image with mke2fs, I again have to provide the offset to losetup. Is there any
tool or utility that tells us the offset value after creating the file system
image? It would be very helpful if you could help me regarding this.

Thank you again for your time.

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 216714] Issue with file system image created with mke2fs parameter -E offset

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=216714

--- Comment #3 from Lukas Czerner (lczerner@redhat.com) ---
Hi,

that's a good question. I don't know of any existing tool that could do the
scanning for you. You can either put something together yourself, or you could
use blkid with --offset argument and try to guess the offset. I don't think
there is any better way other than scan the beginning of the device looking for
file system signature stored in superblock.

-Lukas

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 216714] Issue with file system image created with mke2fs parameter -E offset

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=216714

Theodore Tso (tytso@mit.edu) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |tytso@mit.edu

--- Comment #4 from Theodore Tso (tytso@mit.edu) ---
I'm curious --- *why* are you wanting to create file systems with an offset to
begin with?    The original reason why this feature was added was in a highly
specialized case where someone was creating a system image for some embedded
system or for a virtual machine.  In this use case, the image contained a
partition table, and the offset feature was used to create a filesystem at the
appropriate location as specified by the partition table.

A typical use case is as part of an automated build procedure where the system
image (say, for an Android mobile device, or some ARM development board, such
as a Beaglebone, Arduino, etc., or some Virtual machine), using the mke2fs -d
option to pre-populate the file system with the root file system, or some data
partition, etc.   Since Best Practices for such automated build systems involve
creating a reproducible build, there is nothing precious on the file system
that can't be replicated by re-running the the automated build.   So if the
offset is wrong (which is to say, inconsistent with the partition table which
was laid down using the same automated build system), the developer will just
curse to themselves, and can determine the offset by looking at the build image
creation script, and then adjust that offset to match with the offset that was
set in the partition table.

So I'm a bit perplexed about why you were using a random starting offset for
the file system, and why you can't seem to figure out the offset afterwards. 
The typical approach is to RTFS (Read The Fine Shellscript) to determine the
offset, and then to fix the perhaps not-so-fine shell script.  :-)

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.