[PATCH 0/6] cleanups, parse /proc/meminfo for huge page size

[PATCH 0/6] cleanups, parse /proc/meminfo for huge page size

[Vincent Fu]

Jens, here are four cleanup patches and a pair of patches for enabling
fio to parse /proc/meminfo to obtain the system huge page size.

Vincent

Vincent Fu (6):
  steadystate: delete incorrect comment
  configure: refer to zlib1g-dev package for zlib support
  HOWTO: add blank line for prettier formatting
  t/run-fio-tests: improve json data decoding
  mem: try to parse /proc/meminfo to get huge page size
  docs: update for huge page size parsing

 HOWTO.rst          | 39 ++++++++++++++++++++---------------
 configure          |  2 +-
 fio.1              | 29 ++++++++++++++------------
 memory.c           | 51 ++++++++++++++++++++++++++++++++++++++++++++--
 steadystate.c      |  7 -------
 t/run-fio-tests.py | 20 +++++++-----------
 6 files changed, 96 insertions(+), 52 deletions(-)

-- 
2.25.1

[PATCH 2/6] configure: refer to zlib1g-dev package for zlib support

[Vincent Fu]

Recent Debian-based distributions provide zlib support in the zlib1g-dev
package.

Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
---
 configure | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/configure b/configure
index 95b60bb7..4ee536a0 100755
--- a/configure
+++ b/configure
@@ -3142,7 +3142,7 @@ if test "$libzbc" = "yes" ; then
   output_sym "CONFIG_LIBZBC"
 fi
 if test "$zlib" = "no" ; then
-  echo "Consider installing zlib-dev (zlib-devel, some fio features depend on it."
+  echo "Consider installing zlib1g-dev (zlib-devel) as some fio features depend on it."
   if test "$build_static" = "yes"; then
     echo "Note that some distros have separate packages for static libraries."
   fi
-- 
2.25.1

[PATCH 1/6] steadystate: delete incorrect comment

[Vincent Fu]

Fio actually does not begin collecting steady state data until the
steady state ramp time has expired. Remove the comment that said steady
state data is collected from the start of the job.

Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
---
 steadystate.c | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/steadystate.c b/steadystate.c
index 2e3da1db..ad19318c 100644
--- a/steadystate.c
+++ b/steadystate.c
@@ -250,13 +250,6 @@ int steadystate_check(void)
 		rate_time = mtime_since(&ss->prev_time, &now);
 		memcpy(&ss->prev_time, &now, sizeof(now));
 
-		/*
-		 * Begin monitoring when job starts but don't actually use
-		 * data in checking stopping criterion until ss->ramp_time is
-		 * over. This ensures that we will have a sane value in
-		 * prev_iops/bw the first time through after ss->ramp_time
-		 * is done.
-		 */
 		if (ss->state & FIO_SS_RAMP_OVER) {
 			group_bw += 1000 * (td_bytes - ss->prev_bytes) / rate_time;
 			group_iops += 1000 * (td_iops - ss->prev_iops) / rate_time;
-- 
2.25.1

[PATCH 5/6] mem: try to parse /proc/meminfo to get huge page size

[Vincent Fu]

Instead of relying on a platform-specific default huge page size, try to
parse /proc/meminfo to get the huge page size when it is not explicitly
specified as an option.

Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
---
 memory.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 49 insertions(+), 2 deletions(-)

diff --git a/memory.c b/memory.c
index 6cf73333..3b58e0bc 100644
--- a/memory.c
+++ b/memory.c
@@ -2,6 +2,7 @@
  * Memory helpers
  */
 #include <fcntl.h>
+#include <ctype.h>
 #include <unistd.h>
 #include <sys/mman.h>
 #include <sys/stat.h>
@@ -60,13 +61,59 @@ int fio_pin_memory(struct thread_data *td)
 	return 0;
 }
 
+static unsigned int get_hugepage_size(struct thread_data *td)
+{
+	char buf[2048];
+	FILE *fptr;
+	size_t ret;
+	char *start;
+	unsigned int val;
+
+	if (fio_option_is_set(&td->o, hugepage_size))
+		return td->o.hugepage_size;
+
+	fptr = fopen("/proc/meminfo", "r");
+	if (!fptr)
+		return td->o.hugepage_size;
+
+	ret = fread(buf, 1, sizeof(buf)-1, fptr);
+	if (!ret)
+		goto out;
+	buf[ret] = 0;
+
+	start = strstr(buf, "Hugepagesize:");
+	if (!start)
+		goto out;
+	start += strlen("Hugepagesize:");
+	val = strtoul(start, NULL, 10);
+	if (!val)
+		goto out;
+
+	/* skip all the spaces and digits */
+	while (*start == ' ' || isdigit(*start))
+		start++;
+
+	/* make sure it ends with kB */
+	if (*start++ != 'k')
+		goto out;
+	if (*start++ != 'B')
+		goto out;
+
+	td->o.hugepage_size = val * 1024;
+	dprint(FD_MEM, "parsed hugepagesize from /proc/meminfo %u\n", td->o.hugepage_size);
+
+out:
+	fclose(fptr);
+	return td->o.hugepage_size;
+}
+
 static int alloc_mem_shm(struct thread_data *td, unsigned int total_mem)
 {
 #ifndef CONFIG_NO_SHM
 	int flags = IPC_CREAT | S_IRUSR | S_IWUSR;
 
 	if (td->o.mem_type == MEM_SHMHUGE) {
-		unsigned long mask = td->o.hugepage_size - 1;
+		unsigned long mask = get_hugepage_size(td) - 1;
 
 		flags |= SHM_HUGETLB;
 		total_mem = (total_mem + mask) & ~mask;
@@ -128,7 +175,7 @@ static int alloc_mem_mmap(struct thread_data *td, size_t total_mem)
 	td->mmapfd = -1;
 
 	if (td->o.mem_type == MEM_MMAPHUGE) {
-		unsigned long mask = td->o.hugepage_size - 1;
+		unsigned long mask = get_hugepage_size(td) - 1;
 
 		/* TODO: make sure the file is a real hugetlbfs file */
 		if (!td->o.mmapfile)
-- 
2.25.1

Re: [PATCH 5/6] mem: try to parse /proc/meminfo to get huge page size

[Jens Axboe]

On 5/23/22 11:57 AM, Vincent Fu wrote:
> Instead of relying on a platform-specific default huge page size, try to
> parse /proc/meminfo to get the huge page size when it is not explicitly
> specified as an option.

It's really sad that there isn't a nice way to get this information, but
parsing meminfo will only get you one of them. Eg on my laptop:

axboe@m1 ~> cat /proc/meminfo | grep -i size
Hugepagesize:      32768 kB

But if you look in:

axboe@m1 ~> ls /sys/kernel/mm/hugepages/
hugepages-1048576kB/  hugepages-2048kB/  hugepages-32768kB/

and each directory has entries for increasing the number of them and how
many are currently there.

Parsing /sys/kernel/mm/hugepages/ is probably a better idea than
meminfo.

-- 
Jens Axboe

RE: [PATCH 5/6] mem: try to parse /proc/meminfo to get huge page size

[Vincent Fu]

> -----Original Message-----
> From: Jens Axboe [mailto:axboe@kernel.dk]
> Sent: Monday, May 23, 2022 2:13 PM
> To: Vincent Fu <vincent.fu@samsung.com>; fio@vger.kernel.org
> Subject: Re: [PATCH 5/6] mem: try to parse /proc/meminfo to get huge
> page size
> 
> On 5/23/22 11:57 AM, Vincent Fu wrote:
> > Instead of relying on a platform-specific default huge page size, try to
> > parse /proc/meminfo to get the huge page size when it is not explicitly
> > specified as an option.
> 
> It's really sad that there isn't a nice way to get this information, but
> parsing meminfo will only get you one of them. Eg on my laptop:
> 
> axboe@m1 ~> cat /proc/meminfo | grep -i size
> Hugepagesize:      32768 kB
> 
> But if you look in:
> 
> axboe@m1 ~> ls /sys/kernel/mm/hugepages/
> hugepages-1048576kB/  hugepages-2048kB/  hugepages-32768kB/
> 
> and each directory has entries for increasing the number of them and
> how
> many are currently there.
> 
> Parsing /sys/kernel/mm/hugepages/ is probably a better idea than
> meminfo.
> 

Looking into this more, it seems that to do this 100% correctly will take more
time than I have.

However, I think this patch is still an improvement over a build-time default.
How about if I revise the documentation to say that fio will parse
/proc/meminfo (which reports the default huge page size) but YMMV and users
need to explicitly set the huge page size if they are using something else?

[PATCH 6/6] docs: update for huge page size parsing

[Vincent Fu]

Update the discussion of huge pages to note that fio will try to parse
/proc/meminfo for the hugepage size if it is not specified. Also note
that the default value used when parsing fails is platform specific.

Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
---
 HOWTO.rst | 38 ++++++++++++++++++++++----------------
 fio.1     | 29 ++++++++++++++++-------------
 2 files changed, 38 insertions(+), 29 deletions(-)

diff --git a/HOWTO.rst b/HOWTO.rst
index eee386c1..1e9fa353 100644
--- a/HOWTO.rst
+++ b/HOWTO.rst
@@ -1818,19 +1818,20 @@ Buffers and memory
 			Use GPU memory as the buffers for GPUDirect RDMA benchmark.
 			The :option:`ioengine` must be `rdma`.
 
-	The area allocated is a function of the maximum allowed bs size for the job,
-	multiplied by the I/O depth given. Note that for **shmhuge** and
-	**mmaphuge** to work, the system must have free huge pages allocated. This
-	can normally be checked and set by reading/writing
-	:file:`/proc/sys/vm/nr_hugepages` on a Linux system. Fio assumes a huge page
-	is 4MiB in size. So to calculate the number of huge pages you need for a
-	given job file, add up the I/O depth of all jobs (normally one unless
-	:option:`iodepth` is used) and multiply by the maximum bs set. Then divide
-	that number by the huge page size. You can see the size of the huge pages in
-	:file:`/proc/meminfo`. If no huge pages are allocated by having a non-zero
-	number in `nr_hugepages`, using **mmaphuge** or **shmhuge** will fail. Also
-	see :option:`hugepage-size`.
-
+        The area allocated is a function of the maximum allowed bs size for the job,
+        multiplied by the I/O depth given. Note that for **shmhuge** and
+        **mmaphuge** to work, the system must have free huge pages allocated. This
+        can normally be checked and set by reading/writing
+        :file:`/proc/sys/vm/nr_hugepages` on a Linux system. To calculate the
+        number of huge pages you need for a given job file, add up the I/O
+        depth of all jobs (normally one unless :option:`iodepth` is used) and
+        multiply by the maximum bs set. Then divide that number by the huge
+        page size. You can see the size of huge pages in
+        :file:`/proc/meminfo`. If no huge pages are allocated by having a
+        non-zero number in `nr_hugepages`, using **mmaphuge** or **shmhuge**
+        will fail. See the option :option:`hugepage-size` below to set the
+        hugepage size for fio to use. 
+        
 	**mmaphuge** also needs to have hugetlbfs mounted and the file location
 	should point there. So if it's mounted in :file:`/huge`, you would use
 	`mem=mmaphuge:/huge/somefile`.
@@ -1849,9 +1850,14 @@ Buffers and memory
 .. option:: hugepage-size=int
 
 	Defines the size of a huge page. Must at least be equal to the system
-	setting, see :file:`/proc/meminfo`. Defaults to 4MiB.  Should probably
-	always be a multiple of megabytes, so using ``hugepage-size=Xm`` is the
-	preferred way to set this to avoid setting a non-pow-2 bad value.
+        setting, see :file:`/proc/meminfo`. Should probably always be a
+        multiple of megabytes, so using ``hugepage-size=Xm`` is the preferred
+        way to set this to avoid setting a non-pow-2 bad value. If this option
+        is not specified fio will attempt to parse :file:`/proc/meminfo` to
+        obtain the system huge page size. If this fails fio will use a platform
+        specific default of 2-4MiB. This value is used by fio to ensure that
+        the amount of memory requested for I/O buffers will be a multiple of
+        the huge page size.
 
 .. option:: lockmem=int
 
diff --git a/fio.1 b/fio.1
index ded7bbfc..a7b647bf 100644
--- a/fio.1
+++ b/fio.1
@@ -1629,15 +1629,14 @@ The \fBioengine\fR must be \fBrdma\fR.
 The area allocated is a function of the maximum allowed bs size for the job,
 multiplied by the I/O depth given. Note that for \fBshmhuge\fR and
 \fBmmaphuge\fR to work, the system must have free huge pages allocated. This
-can normally be checked and set by reading/writing
-`/proc/sys/vm/nr_hugepages' on a Linux system. Fio assumes a huge page
-is 4MiB in size. So to calculate the number of huge pages you need for a
-given job file, add up the I/O depth of all jobs (normally one unless
-\fBiodepth\fR is used) and multiply by the maximum bs set. Then divide
-that number by the huge page size. You can see the size of the huge pages in
-`/proc/meminfo'. If no huge pages are allocated by having a non-zero
-number in `nr_hugepages', using \fBmmaphuge\fR or \fBshmhuge\fR will fail. Also
-see \fBhugepage\-size\fR.
+can normally be checked and set by reading/writing `/proc/sys/vm/nr_hugepages'
+on a Linux system.  To calculate the number of huge pages you need for a given
+job file, add up the I/O depth of all jobs (normally one unless \fBiodepth\fR
+is used) and multiply by the maximum bs set. Then divide that number by the
+huge page size. You can see the size of the huge pages in `/proc/meminfo'. If
+no huge pages are allocated by having a non-zero number in `nr_hugepages',
+using \fBmmaphuge\fR or \fBshmhuge\fR will fail. See the \fBhugepage\-size\fR
+option below to set the huge page size for fio to use.
 .P
 \fBmmaphuge\fR also needs to have hugetlbfs mounted and the file location
 should point there. So if it's mounted in `/huge', you would use
@@ -1655,10 +1654,14 @@ of subsequent I/O memory buffers is the sum of the \fBiomem_align\fR and
 \fBbs\fR used.
 .TP
 .BI hugepage\-size \fR=\fPint
-Defines the size of a huge page. Must at least be equal to the system
-setting, see `/proc/meminfo'. Defaults to 4MiB. Should probably
-always be a multiple of megabytes, so using `hugepage\-size=Xm' is the
-preferred way to set this to avoid setting a non-pow-2 bad value.
+Defines the size of a huge page. Must at least be equal to the system setting,
+see `/proc/meminfo'. Should probably always be a multiple of megabytes, so
+using `hugepage\-size=Xm' is the preferred way to set this to avoid setting a
+non-pow-2 bad value. If this option is not specified fio will attempt to parse
+`/proc/meminfo' to obtain the system huge page size. If this fails, fio will
+use a platform-specific default of 2-4MiB. This value is used by fio to ensure
+that the amount of memory requested for I/O buffers will be a multiple of the
+huge page size.
 .TP
 .BI lockmem \fR=\fPint
 Pin the specified amount of memory with \fBmlock\fR\|(2). Can be used to
-- 
2.25.1

[PATCH 3/6] HOWTO: add blank line for prettier formatting

[Vincent Fu]

There needs to be a blank line between the name of an option and its
description in order for the formatting to look nice at
https://fio.readthedocs.io/en/latest/fio_doc.html#cmdoption-arg-ignore-zone-limits,

Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
---
 HOWTO.rst | 1 +
 1 file changed, 1 insertion(+)

diff --git a/HOWTO.rst b/HOWTO.rst
index 84bea5c5..eee386c1 100644
--- a/HOWTO.rst
+++ b/HOWTO.rst
@@ -1064,6 +1064,7 @@ Target file/device
 	thread/process.
 
 .. option:: ignore_zone_limits=bool
+
 	If this option is used, fio will ignore the maximum number of open
 	zones limit of the zoned block device in use, thus allowing the
 	option :option:`max_open_zones` value to be larger than the device
-- 
2.25.1

[PATCH 4/6] t/run-fio-tests: improve json data decoding

[Vincent Fu]

Instead of skipping up to five lines, just skip everything until the
opening curly brace when trying to decode JSON data.

Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
---
 t/run-fio-tests.py | 20 +++++++-------------
 1 file changed, 7 insertions(+), 13 deletions(-)

diff --git a/t/run-fio-tests.py b/t/run-fio-tests.py
index ecceb67e..32cdbc19 100755
--- a/t/run-fio-tests.py
+++ b/t/run-fio-tests.py
@@ -311,21 +311,15 @@ class FioJobTest(FioExeTest):
         #
         # Sometimes fio informational messages are included at the top of the
         # JSON output, especially under Windows. Try to decode output as JSON
-        # data, lopping off up to the first four lines
+        # data, skipping everything until the first {
         #
         lines = file_data.splitlines()
-        for i in range(5):
-            file_data = '\n'.join(lines[i:])
-            try:
-                self.json_data = json.loads(file_data)
-            except json.JSONDecodeError:
-                continue
-            else:
-                logging.debug("Test %d: skipped %d lines decoding JSON data", self.testnum, i)
-                return
-
-        self.failure_reason = "{0} unable to decode JSON data,".format(self.failure_reason)
-        self.passed = False
+        file_data = '\n'.join(lines[lines.index("{"):])
+        try:
+            self.json_data = json.loads(file_data)
+        except json.JSONDecodeError:
+            self.failure_reason = "{0} unable to decode JSON data,".format(self.failure_reason)
+            self.passed = False
 
 
 class FioJobTest_t0005(FioJobTest):
-- 
2.25.1