[PATCH 0/3] fs/ntfs3: Fixes for big endian systems

[PATCH 0/3] fs/ntfs3: Fixes for big endian systems

[Thomas Kühnel]

I tried running the NTFS3 driver on a MIPS big endian system and
noticed various errors when it tried to access bitmaps structures
that are stored as little endian on disk.

These patches were mainly tested in a qemu mips environment.

I'm open for suggestions how to better implement the second patch
"add functions to modify LE bitmaps". It adds copies of two functions
from lib/bitmap.c but modified to work with little endian bitmaps.
Other filesystems seem to have similar functions defined locally like
le_bitmap_set in btrfs or ext4_set_bits in ext4 but no global
implementation exists.

Thomas Kühnel (3):
  fs/ntfs3: fix endian conversion in ni_fname_name
  fs/ntfs3: add functions to modify LE bitmaps
  fs/ntfs3: use _le variants of bitops functions

 fs/ntfs3/bitmap.c  | 56 +++++++++++++++++++++++++++++++++++++++-------
 fs/ntfs3/frecord.c |  4 +++-
 fs/ntfs3/fslog.c   |  4 ++--
 fs/ntfs3/fsntfs.c  |  8 +++----
 fs/ntfs3/index.c   | 14 ++++++------
 fs/ntfs3/ntfs_fs.h |  3 +++
 6 files changed, 67 insertions(+), 22 deletions(-)


base-commit: 0fcfb00b28c0b7884635dacf38e46d60bf3d4eb1
-- 
2.25.1

[PATCH 1/3] fs/ntfs3: fix endian conversion in ni_fname_name

[Thomas Kühnel]

ni_fname_name called ntfs_cmp_names_cpu which assumes that the first
string is in CPU byte order and the second one in little endian.
In this case both strings are little endian so ntfs_cmp_names is the
correct function to call.

Signed-off-by: Thomas Kühnel <thomas.kuehnel@avm.de>
Reviewed-by: Nicolas Schier <n.schier@avm.de>
---
 fs/ntfs3/frecord.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/fs/ntfs3/frecord.c b/fs/ntfs3/frecord.c
index 6f47a9c17f89..f3afdc91af76 100644
--- a/fs/ntfs3/frecord.c
+++ b/fs/ntfs3/frecord.c
@@ -1588,6 +1588,7 @@ struct ATTR_FILE_NAME *ni_fname_name(struct ntfs_inode *ni,
 {
 	struct ATTRIB *attr = NULL;
 	struct ATTR_FILE_NAME *fname;
+	struct le_str *fns;
 
 	*le = NULL;
 
@@ -1610,7 +1611,8 @@ struct ATTR_FILE_NAME *ni_fname_name(struct ntfs_inode *ni,
 	if (uni->len != fname->name_len)
 		goto next;
 
-	if (ntfs_cmp_names_cpu(uni, (struct le_str *)&fname->name_len, NULL,
+	fns = (struct le_str *)&fname->name_len;
+	if (ntfs_cmp_names(uni->name, uni->len, fns->name, fns->len, NULL,
 			       false))
 		goto next;
 
-- 
2.25.1

Re: [PATCH 1/3] fs/ntfs3: fix endian conversion in ni_fname_name

[kernel test robot]

Hi "Thomas,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on 0fcfb00b28c0b7884635dacf38e46d60bf3d4eb1]

url:    https://github.com/0day-ci/linux/commits/Thomas-K-hnel/fs-ntfs3-Fixes-for-big-endian-systems/20211207-184206
base:   0fcfb00b28c0b7884635dacf38e46d60bf3d4eb1
config: arm64-randconfig-s031-20211207 (https://download.01.org/0day-ci/archive/20211207/202112072356.fmLjngs7-lkp@intel.com/config)
compiler: aarch64-linux-gcc (GCC) 11.2.0
reproduce:
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # apt-get install sparse
        # sparse version: v0.6.4-dirty
        # https://github.com/0day-ci/linux/commit/d2fb837ced1828c5a57feac3690d3cc8a36b2fdc
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review Thomas-K-hnel/fs-ntfs3-Fixes-for-big-endian-systems/20211207-184206
        git checkout d2fb837ced1828c5a57feac3690d3cc8a36b2fdc
        # save the config file to linux build tree
        mkdir build_dir
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__' O=build_dir ARCH=arm64 SHELL=/bin/bash fs/ntfs3/

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>


sparse warnings: (new ones prefixed by >>)
>> fs/ntfs3/frecord.c:1615:28: sparse: sparse: incorrect type in argument 1 (different base types) @@     expected restricted __le16 const [usertype] *s1 @@     got unsigned short const * @@
   fs/ntfs3/frecord.c:1615:28: sparse:     expected restricted __le16 const [usertype] *s1
   fs/ntfs3/frecord.c:1615:28: sparse:     got unsigned short const *

vim +1615 fs/ntfs3/frecord.c

  1578	
  1579	/* ni_fname_name
  1580	 *
  1581	 * Return: File name attribute by its value.
  1582	 */
  1583	struct ATTR_FILE_NAME *ni_fname_name(struct ntfs_inode *ni,
  1584					     const struct cpu_str *uni,
  1585					     const struct MFT_REF *home_dir,
  1586					     struct mft_inode **mi,
  1587					     struct ATTR_LIST_ENTRY **le)
  1588	{
  1589		struct ATTRIB *attr = NULL;
  1590		struct ATTR_FILE_NAME *fname;
  1591		struct le_str *fns;
  1592	
  1593		*le = NULL;
  1594	
  1595		/* Enumerate all names. */
  1596	next:
  1597		attr = ni_find_attr(ni, attr, le, ATTR_NAME, NULL, 0, NULL, mi);
  1598		if (!attr)
  1599			return NULL;
  1600	
  1601		fname = resident_data_ex(attr, SIZEOF_ATTRIBUTE_FILENAME);
  1602		if (!fname)
  1603			goto next;
  1604	
  1605		if (home_dir && memcmp(home_dir, &fname->home, sizeof(*home_dir)))
  1606			goto next;
  1607	
  1608		if (!uni)
  1609			goto next;
  1610	
  1611		if (uni->len != fname->name_len)
  1612			goto next;
  1613	
  1614		fns = (struct le_str *)&fname->name_len;
> 1615		if (ntfs_cmp_names(uni->name, uni->len, fns->name, fns->len, NULL,
  1616				       false))
  1617			goto next;
  1618	
  1619		return fname;
  1620	}
  1621	

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org

Re: [PATCH 1/3] fs/ntfs3: fix endian conversion in ni_fname_name

[kernel test robot]

[-- Attachment #1: Type: text/plain, Size: 3227 bytes --]

Hi "Thomas,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on 0fcfb00b28c0b7884635dacf38e46d60bf3d4eb1]

url:    https://github.com/0day-ci/linux/commits/Thomas-K-hnel/fs-ntfs3-Fixes-for-big-endian-systems/20211207-184206
base:   0fcfb00b28c0b7884635dacf38e46d60bf3d4eb1
config: arm64-randconfig-s031-20211207 (https://download.01.org/0day-ci/archive/20211207/202112072356.fmLjngs7-lkp(a)intel.com/config)
compiler: aarch64-linux-gcc (GCC) 11.2.0
reproduce:
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # apt-get install sparse
        # sparse version: v0.6.4-dirty
        # https://github.com/0day-ci/linux/commit/d2fb837ced1828c5a57feac3690d3cc8a36b2fdc
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review Thomas-K-hnel/fs-ntfs3-Fixes-for-big-endian-systems/20211207-184206
        git checkout d2fb837ced1828c5a57feac3690d3cc8a36b2fdc
        # save the config file to linux build tree
        mkdir build_dir
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__' O=build_dir ARCH=arm64 SHELL=/bin/bash fs/ntfs3/

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>


sparse warnings: (new ones prefixed by >>)
>> fs/ntfs3/frecord.c:1615:28: sparse: sparse: incorrect type in argument 1 (different base types) @@     expected restricted __le16 const [usertype] *s1 @@     got unsigned short const * @@
   fs/ntfs3/frecord.c:1615:28: sparse:     expected restricted __le16 const [usertype] *s1
   fs/ntfs3/frecord.c:1615:28: sparse:     got unsigned short const *

vim +1615 fs/ntfs3/frecord.c

  1578	
  1579	/* ni_fname_name
  1580	 *
  1581	 * Return: File name attribute by its value.
  1582	 */
  1583	struct ATTR_FILE_NAME *ni_fname_name(struct ntfs_inode *ni,
  1584					     const struct cpu_str *uni,
  1585					     const struct MFT_REF *home_dir,
  1586					     struct mft_inode **mi,
  1587					     struct ATTR_LIST_ENTRY **le)
  1588	{
  1589		struct ATTRIB *attr = NULL;
  1590		struct ATTR_FILE_NAME *fname;
  1591		struct le_str *fns;
  1592	
  1593		*le = NULL;
  1594	
  1595		/* Enumerate all names. */
  1596	next:
  1597		attr = ni_find_attr(ni, attr, le, ATTR_NAME, NULL, 0, NULL, mi);
  1598		if (!attr)
  1599			return NULL;
  1600	
  1601		fname = resident_data_ex(attr, SIZEOF_ATTRIBUTE_FILENAME);
  1602		if (!fname)
  1603			goto next;
  1604	
  1605		if (home_dir && memcmp(home_dir, &fname->home, sizeof(*home_dir)))
  1606			goto next;
  1607	
  1608		if (!uni)
  1609			goto next;
  1610	
  1611		if (uni->len != fname->name_len)
  1612			goto next;
  1613	
  1614		fns = (struct le_str *)&fname->name_len;
> 1615		if (ntfs_cmp_names(uni->name, uni->len, fns->name, fns->len, NULL,
  1616				       false))
  1617			goto next;
  1618	
  1619		return fname;
  1620	}
  1621	

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

Re: [PATCH 1/3] fs/ntfs3: fix endian conversion in ni_fname_name

[Thomas Kühnel]

On 07.12.21 17:00, kernel test robot wrote:
> Hi "Thomas,
> 
> Thank you for the patch! Perhaps something to improve:
> 
> [auto build test WARNING on 0fcfb00b28c0b7884635dacf38e46d60bf3d4eb1]
> 
> url:    https://github.com/0day-ci/linux/commits/Thomas-K-hnel/fs-ntfs3-Fixes-for-big-endian-systems/20211207-184206
> base:   0fcfb00b28c0b7884635dacf38e46d60bf3d4eb1
> config: arm64-randconfig-s031-20211207 (https://download.01.org/0day-ci/archive/20211207/202112072356.fmLjngs7-lkp@intel.com/config)
> compiler: aarch64-linux-gcc (GCC) 11.2.0
> reproduce:
>          wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
>          chmod +x ~/bin/make.cross
>          # apt-get install sparse
>          # sparse version: v0.6.4-dirty
>          # https://github.com/0day-ci/linux/commit/d2fb837ced1828c5a57feac3690d3cc8a36b2fdc
>          git remote add linux-review https://github.com/0day-ci/linux
>          git fetch --no-tags linux-review Thomas-K-hnel/fs-ntfs3-Fixes-for-big-endian-systems/20211207-184206
>          git checkout d2fb837ced1828c5a57feac3690d3cc8a36b2fdc
>          # save the config file to linux build tree
>          mkdir build_dir
>          COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__' O=build_dir ARCH=arm64 SHELL=/bin/bash fs/ntfs3/
> 
> If you fix the issue, kindly add following tag as appropriate
> Reported-by: kernel test robot <lkp@intel.com>
> 
> 
> sparse warnings: (new ones prefixed by >>)
>>> fs/ntfs3/frecord.c:1615:28: sparse: sparse: incorrect type in argument 1 (different base types) @@     expected restricted __le16 const [usertype] *s1 @@     got unsigned short const * @@
>     fs/ntfs3/frecord.c:1615:28: sparse:     expected restricted __le16 const [usertype] *s1
>     fs/ntfs3/frecord.c:1615:28: sparse:     got unsigned short const *
> 
> vim +1615 fs/ntfs3/frecord.c
> 
>    1578	
>    1579	/* ni_fname_name
>    1580	 *
>    1581	 * Return: File name attribute by its value.
>    1582	 */
>    1583	struct ATTR_FILE_NAME *ni_fname_name(struct ntfs_inode *ni,
>    1584					     const struct cpu_str *uni,


Changing the type of uni to struct le_str * should fix the warning.
I'll change this in the next version of this patch.

>    1585					     const struct MFT_REF *home_dir,
>    1586					     struct mft_inode **mi,
>    1587					     struct ATTR_LIST_ENTRY **le)
>    1588	{
>    1589		struct ATTRIB *attr = NULL;
>    1590		struct ATTR_FILE_NAME *fname;
>    1591		struct le_str *fns;
>    1592	
>    1593		*le = NULL;
>    1594	
>    1595		/* Enumerate all names. */
>    1596	next:
>    1597		attr = ni_find_attr(ni, attr, le, ATTR_NAME, NULL, 0, NULL, mi);
>    1598		if (!attr)
>    1599			return NULL;
>    1600	
>    1601		fname = resident_data_ex(attr, SIZEOF_ATTRIBUTE_FILENAME);
>    1602		if (!fname)
>    1603			goto next;
>    1604	
>    1605		if (home_dir && memcmp(home_dir, &fname->home, sizeof(*home_dir)))
>    1606			goto next;
>    1607	
>    1608		if (!uni)
>    1609			goto next;
>    1610	
>    1611		if (uni->len != fname->name_len)
>    1612			goto next;
>    1613	
>    1614		fns = (struct le_str *)&fname->name_len;
>> 1615		if (ntfs_cmp_names(uni->name, uni->len, fns->name, fns->len, NULL,
>    1616				       false))
>    1617			goto next;
>    1618	
>    1619		return fname;
>    1620	}
>    1621	
> 
> ---
> 0-DAY CI Kernel Test Service, Intel Corporation
> https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
> 

Re: [PATCH 1/3] fs/ntfs3: fix endian conversion in ni_fname_name

[Thomas Kühnel]

[-- Attachment #1: Type: text/plain, Size: 3628 bytes --]



On 07.12.21 17:00, kernel test robot wrote:
> Hi "Thomas,
> 
> Thank you for the patch! Perhaps something to improve:
> 
> [auto build test WARNING on 0fcfb00b28c0b7884635dacf38e46d60bf3d4eb1]
> 
> url:    https://github.com/0day-ci/linux/commits/Thomas-K-hnel/fs-ntfs3-Fixes-for-big-endian-systems/20211207-184206
> base:   0fcfb00b28c0b7884635dacf38e46d60bf3d4eb1
> config: arm64-randconfig-s031-20211207 (https://download.01.org/0day-ci/archive/20211207/202112072356.fmLjngs7-lkp(a)intel.com/config)
> compiler: aarch64-linux-gcc (GCC) 11.2.0
> reproduce:
>          wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
>          chmod +x ~/bin/make.cross
>          # apt-get install sparse
>          # sparse version: v0.6.4-dirty
>          # https://github.com/0day-ci/linux/commit/d2fb837ced1828c5a57feac3690d3cc8a36b2fdc
>          git remote add linux-review https://github.com/0day-ci/linux
>          git fetch --no-tags linux-review Thomas-K-hnel/fs-ntfs3-Fixes-for-big-endian-systems/20211207-184206
>          git checkout d2fb837ced1828c5a57feac3690d3cc8a36b2fdc
>          # save the config file to linux build tree
>          mkdir build_dir
>          COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__' O=build_dir ARCH=arm64 SHELL=/bin/bash fs/ntfs3/
> 
> If you fix the issue, kindly add following tag as appropriate
> Reported-by: kernel test robot <lkp@intel.com>
> 
> 
> sparse warnings: (new ones prefixed by >>)
>>> fs/ntfs3/frecord.c:1615:28: sparse: sparse: incorrect type in argument 1 (different base types) @@     expected restricted __le16 const [usertype] *s1 @@     got unsigned short const * @@
>     fs/ntfs3/frecord.c:1615:28: sparse:     expected restricted __le16 const [usertype] *s1
>     fs/ntfs3/frecord.c:1615:28: sparse:     got unsigned short const *
> 
> vim +1615 fs/ntfs3/frecord.c
> 
>    1578	
>    1579	/* ni_fname_name
>    1580	 *
>    1581	 * Return: File name attribute by its value.
>    1582	 */
>    1583	struct ATTR_FILE_NAME *ni_fname_name(struct ntfs_inode *ni,
>    1584					     const struct cpu_str *uni,


Changing the type of uni to struct le_str * should fix the warning.
I'll change this in the next version of this patch.

>    1585					     const struct MFT_REF *home_dir,
>    1586					     struct mft_inode **mi,
>    1587					     struct ATTR_LIST_ENTRY **le)
>    1588	{
>    1589		struct ATTRIB *attr = NULL;
>    1590		struct ATTR_FILE_NAME *fname;
>    1591		struct le_str *fns;
>    1592	
>    1593		*le = NULL;
>    1594	
>    1595		/* Enumerate all names. */
>    1596	next:
>    1597		attr = ni_find_attr(ni, attr, le, ATTR_NAME, NULL, 0, NULL, mi);
>    1598		if (!attr)
>    1599			return NULL;
>    1600	
>    1601		fname = resident_data_ex(attr, SIZEOF_ATTRIBUTE_FILENAME);
>    1602		if (!fname)
>    1603			goto next;
>    1604	
>    1605		if (home_dir && memcmp(home_dir, &fname->home, sizeof(*home_dir)))
>    1606			goto next;
>    1607	
>    1608		if (!uni)
>    1609			goto next;
>    1610	
>    1611		if (uni->len != fname->name_len)
>    1612			goto next;
>    1613	
>    1614		fns = (struct le_str *)&fname->name_len;
>> 1615		if (ntfs_cmp_names(uni->name, uni->len, fns->name, fns->len, NULL,
>    1616				       false))
>    1617			goto next;
>    1618	
>    1619		return fname;
>    1620	}
>    1621	
> 
> ---
> 0-DAY CI Kernel Test Service, Intel Corporation
> https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
> 

[PATCH 2/3] fs/ntfs3: add functions to modify LE bitmaps

[Thomas Kühnel]

__bitmap_set/__bitmap_clear only works with bitmaps in CPU order.
Define a variant of these functions in ntfs3 to handle modifying bitmaps
read from the filesystem.

Signed-off-by: Thomas Kühnel <thomas.kuehnel@avm.de>
Reviewed-by: Nicolas Schier <n.schier@avm.de>
---
 fs/ntfs3/bitmap.c  | 46 +++++++++++++++++++++++++++++++++++++++++++---
 fs/ntfs3/fslog.c   |  4 ++--
 fs/ntfs3/ntfs_fs.h |  3 +++
 3 files changed, 48 insertions(+), 5 deletions(-)

diff --git a/fs/ntfs3/bitmap.c b/fs/ntfs3/bitmap.c
index aa184407520f..b61cf533b030 100644
--- a/fs/ntfs3/bitmap.c
+++ b/fs/ntfs3/bitmap.c
@@ -741,7 +741,7 @@ int wnd_set_free(struct wnd_bitmap *wnd, size_t bit, size_t bits)
 
 		lock_buffer(bh);
 
-		__bitmap_clear(buf, wbit, op);
+		ntfs_bitmap_clear_le(buf, wbit, op);
 
 		wnd->free_bits[iw] += op;
 
@@ -793,7 +793,7 @@ int wnd_set_used(struct wnd_bitmap *wnd, size_t bit, size_t bits)
 
 		lock_buffer(bh);
 
-		__bitmap_set(buf, wbit, op);
+		ntfs_bitmap_set_le(buf, wbit, op);
 		wnd->free_bits[iw] -= op;
 
 		set_buffer_uptodate(bh);
@@ -1370,7 +1370,7 @@ int wnd_extend(struct wnd_bitmap *wnd, size_t new_bits)
 		lock_buffer(bh);
 		buf = (ulong *)bh->b_data;
 
-		__bitmap_clear(buf, b0, blocksize * 8 - b0);
+		ntfs_bitmap_clear_le(buf, b0, blocksize * 8 - b0);
 		frb = wbits - __bitmap_weight(buf, wbits);
 		wnd->total_zeroes += frb - wnd->free_bits[iw];
 		wnd->free_bits[iw] = frb;
@@ -1489,3 +1489,43 @@ int ntfs_trim_fs(struct ntfs_sb_info *sbi, struct fstrim_range *range)
 
 	return err;
 }
+
+void ntfs_bitmap_set_le(unsigned long *map, unsigned int start, int len)
+{
+	unsigned long *p = map + BIT_WORD(start);
+	const unsigned int size = start + len;
+	int bits_to_set = BITS_PER_LONG - (start % BITS_PER_LONG);
+	unsigned long mask_to_set = cpu_to_le32(BITMAP_FIRST_WORD_MASK(start));
+
+	while (len - bits_to_set >= 0) {
+		*p |= mask_to_set;
+		len -= bits_to_set;
+		bits_to_set = BITS_PER_LONG;
+		mask_to_set = ~0UL;
+		p++;
+	}
+	if (len) {
+		mask_to_set &= cpu_to_le32(BITMAP_LAST_WORD_MASK(size));
+		*p |= mask_to_set;
+	}
+}
+
+void ntfs_bitmap_clear_le(unsigned long *map, unsigned int start, int len)
+{
+	unsigned long *p = map + BIT_WORD(start);
+	const unsigned int size = start + len;
+	int bits_to_clear = BITS_PER_LONG - (start % BITS_PER_LONG);
+	unsigned long mask_to_clear = cpu_to_le32(BITMAP_FIRST_WORD_MASK(start));
+
+	while (len - bits_to_clear >= 0) {
+		*p &= ~mask_to_clear;
+		len -= bits_to_clear;
+		bits_to_clear = BITS_PER_LONG;
+		mask_to_clear = ~0UL;
+		p++;
+	}
+	if (len) {
+		mask_to_clear &= cpu_to_le32(BITMAP_LAST_WORD_MASK(size));
+		*p &= ~mask_to_clear;
+	}
+}
diff --git a/fs/ntfs3/fslog.c b/fs/ntfs3/fslog.c
index 06492f088d60..5ec7dbad3add 100644
--- a/fs/ntfs3/fslog.c
+++ b/fs/ntfs3/fslog.c
@@ -3646,7 +3646,7 @@ static int do_action(struct ntfs_log *log, struct OPEN_ATTR_ENRTY *oe,
 			goto dirty_vol;
 		}
 
-		__bitmap_set(Add2Ptr(buffer_le, roff), bmp_off, bmp_bits);
+		ntfs_bitmap_set_le(Add2Ptr(buffer_le, roff), bmp_off, bmp_bits);
 		a_dirty = true;
 		break;
 
@@ -3660,7 +3660,7 @@ static int do_action(struct ntfs_log *log, struct OPEN_ATTR_ENRTY *oe,
 			goto dirty_vol;
 		}
 
-		__bitmap_clear(Add2Ptr(buffer_le, roff), bmp_off, bmp_bits);
+		ntfs_bitmap_clear_le(Add2Ptr(buffer_le, roff), bmp_off, bmp_bits);
 		a_dirty = true;
 		break;
 
diff --git a/fs/ntfs3/ntfs_fs.h b/fs/ntfs3/ntfs_fs.h
index 8aaec7e0804e..aaecddeff4e5 100644
--- a/fs/ntfs3/ntfs_fs.h
+++ b/fs/ntfs3/ntfs_fs.h
@@ -828,6 +828,9 @@ int wnd_extend(struct wnd_bitmap *wnd, size_t new_bits);
 void wnd_zone_set(struct wnd_bitmap *wnd, size_t Lcn, size_t Len);
 int ntfs_trim_fs(struct ntfs_sb_info *sbi, struct fstrim_range *range);
 
+void ntfs_bitmap_set_le(unsigned long *map, unsigned int start, int len);
+void ntfs_bitmap_clear_le(unsigned long *map, unsigned int start, int len);
+
 /* Globals from upcase.c */
 int ntfs_cmp_names(const __le16 *s1, size_t l1, const __le16 *s2, size_t l2,
 		   const u16 *upcase, bool bothcase);
-- 
2.25.1

Re: [PATCH 2/3] fs/ntfs3: add functions to modify LE bitmaps

[kernel test robot]

Hi "Thomas,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on 0fcfb00b28c0b7884635dacf38e46d60bf3d4eb1]

url:    https://github.com/0day-ci/linux/commits/Thomas-K-hnel/fs-ntfs3-Fixes-for-big-endian-systems/20211207-184206
base:   0fcfb00b28c0b7884635dacf38e46d60bf3d4eb1
config: arm64-randconfig-s031-20211207 (https://download.01.org/0day-ci/archive/20211208/202112080045.huYZOn4p-lkp@intel.com/config)
compiler: aarch64-linux-gcc (GCC) 11.2.0
reproduce:
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # apt-get install sparse
        # sparse version: v0.6.4-dirty
        # https://github.com/0day-ci/linux/commit/2227622e39d3100d10077199481f05ccb9a17204
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review Thomas-K-hnel/fs-ntfs3-Fixes-for-big-endian-systems/20211207-184206
        git checkout 2227622e39d3100d10077199481f05ccb9a17204
        # save the config file to linux build tree
        mkdir build_dir
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__' O=build_dir ARCH=arm64 SHELL=/bin/bash fs/ntfs3/

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>


sparse warnings: (new ones prefixed by >>)
>> fs/ntfs3/bitmap.c:1498:37: sparse: sparse: incorrect type in initializer (different base types) @@     expected unsigned long mask_to_set @@     got restricted __le32 [usertype] @@
   fs/ntfs3/bitmap.c:1498:37: sparse:     expected unsigned long mask_to_set
   fs/ntfs3/bitmap.c:1498:37: sparse:     got restricted __le32 [usertype]
>> fs/ntfs3/bitmap.c:1508:29: sparse: sparse: invalid assignment: &=
>> fs/ntfs3/bitmap.c:1508:29: sparse:    left side has type unsigned long
>> fs/ntfs3/bitmap.c:1508:29: sparse:    right side has type restricted __le32
>> fs/ntfs3/bitmap.c:1518:39: sparse: sparse: incorrect type in initializer (different base types) @@     expected unsigned long mask_to_clear @@     got restricted __le32 [usertype] @@
   fs/ntfs3/bitmap.c:1518:39: sparse:     expected unsigned long mask_to_clear
   fs/ntfs3/bitmap.c:1518:39: sparse:     got restricted __le32 [usertype]
   fs/ntfs3/bitmap.c:1528:31: sparse: sparse: invalid assignment: &=
   fs/ntfs3/bitmap.c:1528:31: sparse:    left side has type unsigned long
   fs/ntfs3/bitmap.c:1528:31: sparse:    right side has type restricted __le32

vim +1498 fs/ntfs3/bitmap.c

  1492	
  1493	void ntfs_bitmap_set_le(unsigned long *map, unsigned int start, int len)
  1494	{
  1495		unsigned long *p = map + BIT_WORD(start);
  1496		const unsigned int size = start + len;
  1497		int bits_to_set = BITS_PER_LONG - (start % BITS_PER_LONG);
> 1498		unsigned long mask_to_set = cpu_to_le32(BITMAP_FIRST_WORD_MASK(start));
  1499	
  1500		while (len - bits_to_set >= 0) {
  1501			*p |= mask_to_set;
  1502			len -= bits_to_set;
  1503			bits_to_set = BITS_PER_LONG;
  1504			mask_to_set = ~0UL;
  1505			p++;
  1506		}
  1507		if (len) {
> 1508			mask_to_set &= cpu_to_le32(BITMAP_LAST_WORD_MASK(size));
  1509			*p |= mask_to_set;
  1510		}
  1511	}
  1512	
  1513	void ntfs_bitmap_clear_le(unsigned long *map, unsigned int start, int len)
  1514	{
  1515		unsigned long *p = map + BIT_WORD(start);
  1516		const unsigned int size = start + len;
  1517		int bits_to_clear = BITS_PER_LONG - (start % BITS_PER_LONG);
> 1518		unsigned long mask_to_clear = cpu_to_le32(BITMAP_FIRST_WORD_MASK(start));

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org

Re: [PATCH 2/3] fs/ntfs3: add functions to modify LE bitmaps

[kernel test robot]

[-- Attachment #1: Type: text/plain, Size: 3790 bytes --]

Hi "Thomas,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on 0fcfb00b28c0b7884635dacf38e46d60bf3d4eb1]

url:    https://github.com/0day-ci/linux/commits/Thomas-K-hnel/fs-ntfs3-Fixes-for-big-endian-systems/20211207-184206
base:   0fcfb00b28c0b7884635dacf38e46d60bf3d4eb1
config: arm64-randconfig-s031-20211207 (https://download.01.org/0day-ci/archive/20211208/202112080045.huYZOn4p-lkp(a)intel.com/config)
compiler: aarch64-linux-gcc (GCC) 11.2.0
reproduce:
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # apt-get install sparse
        # sparse version: v0.6.4-dirty
        # https://github.com/0day-ci/linux/commit/2227622e39d3100d10077199481f05ccb9a17204
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review Thomas-K-hnel/fs-ntfs3-Fixes-for-big-endian-systems/20211207-184206
        git checkout 2227622e39d3100d10077199481f05ccb9a17204
        # save the config file to linux build tree
        mkdir build_dir
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__' O=build_dir ARCH=arm64 SHELL=/bin/bash fs/ntfs3/

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>


sparse warnings: (new ones prefixed by >>)
>> fs/ntfs3/bitmap.c:1498:37: sparse: sparse: incorrect type in initializer (different base types) @@     expected unsigned long mask_to_set @@     got restricted __le32 [usertype] @@
   fs/ntfs3/bitmap.c:1498:37: sparse:     expected unsigned long mask_to_set
   fs/ntfs3/bitmap.c:1498:37: sparse:     got restricted __le32 [usertype]
>> fs/ntfs3/bitmap.c:1508:29: sparse: sparse: invalid assignment: &=
>> fs/ntfs3/bitmap.c:1508:29: sparse:    left side has type unsigned long
>> fs/ntfs3/bitmap.c:1508:29: sparse:    right side has type restricted __le32
>> fs/ntfs3/bitmap.c:1518:39: sparse: sparse: incorrect type in initializer (different base types) @@     expected unsigned long mask_to_clear @@     got restricted __le32 [usertype] @@
   fs/ntfs3/bitmap.c:1518:39: sparse:     expected unsigned long mask_to_clear
   fs/ntfs3/bitmap.c:1518:39: sparse:     got restricted __le32 [usertype]
   fs/ntfs3/bitmap.c:1528:31: sparse: sparse: invalid assignment: &=
   fs/ntfs3/bitmap.c:1528:31: sparse:    left side has type unsigned long
   fs/ntfs3/bitmap.c:1528:31: sparse:    right side has type restricted __le32

vim +1498 fs/ntfs3/bitmap.c

  1492	
  1493	void ntfs_bitmap_set_le(unsigned long *map, unsigned int start, int len)
  1494	{
  1495		unsigned long *p = map + BIT_WORD(start);
  1496		const unsigned int size = start + len;
  1497		int bits_to_set = BITS_PER_LONG - (start % BITS_PER_LONG);
> 1498		unsigned long mask_to_set = cpu_to_le32(BITMAP_FIRST_WORD_MASK(start));
  1499	
  1500		while (len - bits_to_set >= 0) {
  1501			*p |= mask_to_set;
  1502			len -= bits_to_set;
  1503			bits_to_set = BITS_PER_LONG;
  1504			mask_to_set = ~0UL;
  1505			p++;
  1506		}
  1507		if (len) {
> 1508			mask_to_set &= cpu_to_le32(BITMAP_LAST_WORD_MASK(size));
  1509			*p |= mask_to_set;
  1510		}
  1511	}
  1512	
  1513	void ntfs_bitmap_clear_le(unsigned long *map, unsigned int start, int len)
  1514	{
  1515		unsigned long *p = map + BIT_WORD(start);
  1516		const unsigned int size = start + len;
  1517		int bits_to_clear = BITS_PER_LONG - (start % BITS_PER_LONG);
> 1518		unsigned long mask_to_clear = cpu_to_le32(BITMAP_FIRST_WORD_MASK(start));

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

[PATCH 3/3] fs/ntfs3: use _le variants of bitops functions

[Thomas Kühnel]

The functions from bitops.h already have _le variants so use them to
prevent invalid reads/writes of the bitmap on big endian systems.

Signed-off-by: Thomas Kühnel <thomas.kuehnel@avm.de>
---
 fs/ntfs3/bitmap.c | 10 +++++-----
 fs/ntfs3/fsntfs.c |  8 ++++----
 fs/ntfs3/index.c  | 14 +++++++-------
 3 files changed, 16 insertions(+), 16 deletions(-)

diff --git a/fs/ntfs3/bitmap.c b/fs/ntfs3/bitmap.c
index b61cf533b030..e3dc6714ef25 100644
--- a/fs/ntfs3/bitmap.c
+++ b/fs/ntfs3/bitmap.c
@@ -71,7 +71,7 @@ static size_t wnd_scan(const ulong *buf, size_t wbit, u32 wpos, u32 wend,
 	while (wpos < wend) {
 		size_t free_len;
 		u32 free_bits, end;
-		u32 used = find_next_zero_bit(buf, wend, wpos);
+		u32 used = find_next_zero_bit_le(buf, wend, wpos);
 
 		if (used >= wend) {
 			if (*b_len < *prev_tail) {
@@ -97,7 +97,7 @@ static size_t wnd_scan(const ulong *buf, size_t wbit, u32 wpos, u32 wend,
 		 * Now we have a fragment [wpos, wend) staring with 0.
 		 */
 		end = wpos + to_alloc - *prev_tail;
-		free_bits = find_next_bit(buf, min(end, wend), wpos);
+		free_bits = find_next_bit_le(buf, min(end, wend), wpos);
 
 		free_len = *prev_tail + free_bits - wpos;
 
@@ -579,7 +579,7 @@ static int wnd_rescan(struct wnd_bitmap *wnd)
 			wbits = wnd->nbits - wbit;
 
 		do {
-			used = find_next_zero_bit(buf, wbits, wpos);
+			used = find_next_zero_bit_le(buf, wbits, wpos);
 
 			if (used > wpos && prev_tail) {
 				wnd_add_free_ext(wnd, wbit + wpos - prev_tail,
@@ -595,7 +595,7 @@ static int wnd_rescan(struct wnd_bitmap *wnd)
 				break;
 			}
 
-			frb = find_next_bit(buf, wbits, wpos);
+			frb = find_next_bit_le(buf, wbits, wpos);
 			if (frb >= wbits) {
 				/* Keep last free block. */
 				prev_tail += frb - wpos;
@@ -1457,7 +1457,7 @@ int ntfs_trim_fs(struct ntfs_sb_info *sbi, struct fstrim_range *range)
 		buf = (ulong *)bh->b_data;
 
 		for (; wbit < wbits; wbit++) {
-			if (!test_bit(wbit, buf)) {
+			if (!test_bit_le(wbit, buf)) {
 				if (!len)
 					lcn = lcn_wnd + wbit;
 				len += 1;
diff --git a/fs/ntfs3/fsntfs.c b/fs/ntfs3/fsntfs.c
index 4de9acb16968..1f801c9eae67 100644
--- a/fs/ntfs3/fsntfs.c
+++ b/fs/ntfs3/fsntfs.c
@@ -618,13 +618,13 @@ int ntfs_look_free_mft(struct ntfs_sb_info *sbi, CLST *rno, bool mft,
 						 NULL, 0, NULL, NULL))
 					goto next;
 
-				__clear_bit(ir - MFT_REC_RESERVED,
+				__clear_bit_le(ir - MFT_REC_RESERVED,
 					    &sbi->mft.reserved_bitmap);
 			}
 		}
 
 		/* Scan 5 bits for zero. Bit 0 == MFT_REC_RESERVED */
-		zbit = find_next_zero_bit(&sbi->mft.reserved_bitmap,
+		zbit = find_next_zero_bit_le(&sbi->mft.reserved_bitmap,
 					  MFT_REC_FREE, MFT_REC_RESERVED);
 		if (zbit >= MFT_REC_FREE) {
 			sbi->mft.next_reserved = MFT_REC_FREE;
@@ -692,7 +692,7 @@ int ntfs_look_free_mft(struct ntfs_sb_info *sbi, CLST *rno, bool mft,
 	if (*rno >= MFT_REC_FREE)
 		wnd_set_used(wnd, *rno, 1);
 	else if (*rno >= MFT_REC_RESERVED && sbi->mft.reserved_bitmap_inited)
-		__set_bit(*rno - MFT_REC_RESERVED, &sbi->mft.reserved_bitmap);
+		__set_bit_le(*rno - MFT_REC_RESERVED, &sbi->mft.reserved_bitmap);
 
 out:
 	if (!mft)
@@ -718,7 +718,7 @@ void ntfs_mark_rec_free(struct ntfs_sb_info *sbi, CLST rno)
 		else
 			wnd_set_free(wnd, rno, 1);
 	} else if (rno >= MFT_REC_RESERVED && sbi->mft.reserved_bitmap_inited) {
-		__clear_bit(rno - MFT_REC_RESERVED, &sbi->mft.reserved_bitmap);
+		__clear_bit_le(rno - MFT_REC_RESERVED, &sbi->mft.reserved_bitmap);
 	}
 
 	if (rno < wnd_zone_bit(wnd))
diff --git a/fs/ntfs3/index.c b/fs/ntfs3/index.c
index 6f81e3a49abf..9547422200ab 100644
--- a/fs/ntfs3/index.c
+++ b/fs/ntfs3/index.c
@@ -323,7 +323,7 @@ static int indx_mark_used(struct ntfs_index *indx, struct ntfs_inode *ni,
 	if (err)
 		return err;
 
-	__set_bit(bit - bbuf.bit, bbuf.buf);
+	__set_bit_le(bit - bbuf.bit, bbuf.buf);
 
 	bmp_buf_put(&bbuf, true);
 
@@ -343,7 +343,7 @@ static int indx_mark_free(struct ntfs_index *indx, struct ntfs_inode *ni,
 	if (err)
 		return err;
 
-	__clear_bit(bit - bbuf.bit, bbuf.buf);
+	__clear_bit_le(bit - bbuf.bit, bbuf.buf);
 
 	bmp_buf_put(&bbuf, true);
 
@@ -457,7 +457,7 @@ static int scan_nres_bitmap(struct ntfs_inode *ni, struct ATTRIB *bitmap,
 
 static bool scan_for_free(const ulong *buf, u32 bit, u32 bits, size_t *ret)
 {
-	size_t pos = find_next_zero_bit(buf, bits, bit);
+	size_t pos = find_next_zero_bit_le(buf, bits, bit);
 
 	if (pos >= bits)
 		return false;
@@ -489,7 +489,7 @@ static int indx_find_free(struct ntfs_index *indx, struct ntfs_inode *ni,
 
 	if (!b->non_res) {
 		u32 nbits = 8 * le32_to_cpu(b->res.data_size);
-		size_t pos = find_next_zero_bit(resident_data(b), nbits, 0);
+		size_t pos = find_next_zero_bit_le(resident_data(b), nbits, 0);
 
 		if (pos < nbits)
 			*bit = pos;
@@ -505,7 +505,7 @@ static int indx_find_free(struct ntfs_index *indx, struct ntfs_inode *ni,
 
 static bool scan_for_used(const ulong *buf, u32 bit, u32 bits, size_t *ret)
 {
-	size_t pos = find_next_bit(buf, bits, bit);
+	size_t pos = find_next_bit_le(buf, bits, bit);
 
 	if (pos >= bits)
 		return false;
@@ -536,7 +536,7 @@ int indx_used_bit(struct ntfs_index *indx, struct ntfs_inode *ni, size_t *bit)
 
 	if (!b->non_res) {
 		u32 nbits = le32_to_cpu(b->res.data_size) * 8;
-		size_t pos = find_next_bit(resident_data(b), nbits, from);
+		size_t pos = find_next_bit_le(resident_data(b), nbits, from);
 
 		if (pos < nbits)
 			*bit = pos;
@@ -1954,7 +1954,7 @@ static int indx_shrink(struct ntfs_index *indx, struct ntfs_inode *ni,
 		if (bit >= nbits)
 			return 0;
 
-		pos = find_next_bit(bm, nbits, bit);
+		pos = find_next_bit_le(bm, nbits, bit);
 		if (pos < nbits)
 			return 0;
 	} else {
-- 
2.25.1

Re: [PATCH 0/3] fs/ntfs3: Fixes for big endian systems

[Konstantin Komarov]

On 07.12.2021 14:24, Thomas Kühnel wrote:
> I tried running the NTFS3 driver on a MIPS big endian system and
> noticed various errors when it tried to access bitmaps structures
> that are stored as little endian on disk.
>
> These patches were mainly tested in a qemu mips environment.
>
> I'm open for suggestions how to better implement the second patch
> "add functions to modify LE bitmaps". It adds copies of two functions
> from lib/bitmap.c but modified to work with little endian bitmaps.
> Other filesystems seem to have similar functions defined locally like
> le_bitmap_set in btrfs or ext4_set_bits in ext4 but no global
> implementation exists.
>
> Thomas Kühnel (3):
>    fs/ntfs3: fix endian conversion in ni_fname_name
>    fs/ntfs3: add functions to modify LE bitmaps
>    fs/ntfs3: use _le variants of bitops functions
>
>   fs/ntfs3/bitmap.c  | 56 +++++++++++++++++++++++++++++++++++++++-------
>   fs/ntfs3/frecord.c |  4 +++-
>   fs/ntfs3/fslog.c   |  4 ++--
>   fs/ntfs3/fsntfs.c  |  8 +++----
>   fs/ntfs3/index.c   | 14 ++++++------
>   fs/ntfs3/ntfs_fs.h |  3 +++
>   6 files changed, 67 insertions(+), 22 deletions(-)
>
>
> base-commit: 0fcfb00b28c0b7884635dacf38e46d60bf3d4eb1

Hello Thomas,

Thanks for patches, they have been applied long ago.

Sorry for not responding immediately.