Buildroot Archive on lore.kernel.org
 help / color / mirror / Atom feed
* [Buildroot] [PATCH 1/1] package/python-ml_dtypes: new package
@ 2023-12-23 23:05 Julien Olivain
  2023-12-25 10:21 ` Thomas Petazzoni via buildroot
  0 siblings, 1 reply; 2+ messages in thread
From: Julien Olivain @ 2023-12-23 23:05 UTC (permalink / raw)
  To: buildroot; +Cc: James Hilliard, Julien Olivain, Thomas Petazzoni, Asaf Kahlon

ml_dtypes is a stand-alone implementation of several NumPy
dtype extensions used in machine learning libraries.

https://github.com/jax-ml/ml_dtypes

Signed-off-by: Julien Olivain <ju.o@free.fr>
---
Patch tested on branch master at commit 9212a71 with commands:

    make check-package
    ...
    0 warnings generated

    support/testing/run-tests \
        -d dl -o output_folder \
        tests.package.test_python_ml_dtypes
    ...
    OK  
 
Note: this package is a dependency of tensorflow-lite >= v2.14.0.
For info, tensorflow-lite v.2.11.0 was proposed in Buildroot in:
https://patchwork.ozlabs.org/project/buildroot/list/?series=346546
---
 DEVELOPERS                                    |  3 ++
 package/Config.in                             |  1 +
 package/python-ml_dtypes/Config.in            | 13 +++++++
 .../python-ml_dtypes/python-ml_dtypes.hash    |  5 +++
 package/python-ml_dtypes/python-ml_dtypes.mk  | 19 +++++++++
 .../tests/package/sample_python_ml_dtypes.py  | 39 +++++++++++++++++++
 .../tests/package/test_python_ml_dtypes.py    | 20 ++++++++++
 7 files changed, 100 insertions(+)
 create mode 100644 package/python-ml_dtypes/Config.in
 create mode 100644 package/python-ml_dtypes/python-ml_dtypes.hash
 create mode 100644 package/python-ml_dtypes/python-ml_dtypes.mk
 create mode 100644 support/testing/tests/package/sample_python_ml_dtypes.py
 create mode 100644 support/testing/tests/package/test_python_ml_dtypes.py

diff --git a/DEVELOPERS b/DEVELOPERS
index bad07d51aa..8b64cb74d1 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -1741,6 +1741,7 @@ F:	package/python-magic-wormhole/
 F:	package/python-magic-wormhole-mailbox-server/
 F:	package/python-magic-wormhole-transit-relay/
 F:	package/python-midiutil/
+F:	package/python-ml_dtypes/
 F:	package/python-pyalsa/
 F:	package/python-spake2/
 F:	package/rdma-core/
@@ -1752,6 +1753,7 @@ F:	support/testing/tests/package/sample_python_distro.py
 F:	support/testing/tests/package/sample_python_gnupg.py
 F:	support/testing/tests/package/sample_python_hwdata.py
 F:	support/testing/tests/package/sample_python_midiutil.py
+F:	support/testing/tests/package/sample_python_ml_dtypes.py
 F:	support/testing/tests/package/sample_python_pyalsa.py
 F:	support/testing/tests/package/sample_python_spake2.py
 F:	support/testing/tests/package/test_acpica.py
@@ -1802,6 +1804,7 @@ F:	support/testing/tests/package/test_python_hkdf.py
 F:	support/testing/tests/package/test_python_hwdata.py
 F:	support/testing/tests/package/test_python_magic_wormhole.py
 F:	support/testing/tests/package/test_python_midiutil.py
+F:	support/testing/tests/package/test_python_ml_dtypes.py
 F:	support/testing/tests/package/test_python_pyalsa.py
 F:	support/testing/tests/package/test_python_spake2.py
 F:	support/testing/tests/package/test_rdma_core.py
diff --git a/package/Config.in b/package/Config.in
index bc723fe018..7442b9d920 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -1170,6 +1170,7 @@ menu "External python modules"
 	source "package/python-mimeparse/Config.in"
 	source "package/python-minimalmodbus/Config.in"
 	source "package/python-mistune/Config.in"
+	source "package/python-ml_dtypes/Config.in"
 	source "package/python-modbus-tk/Config.in"
 	source "package/python-more-itertools/Config.in"
 	source "package/python-mpd2/Config.in"
diff --git a/package/python-ml_dtypes/Config.in b/package/python-ml_dtypes/Config.in
new file mode 100644
index 0000000000..c487692b7a
--- /dev/null
+++ b/package/python-ml_dtypes/Config.in
@@ -0,0 +1,13 @@
+config BR2_PACKAGE_PYTHON_ML_DTYPES
+	bool "python-ml_dtypes"
+	depends on BR2_PACKAGE_PYTHON_NUMPY_ARCH_SUPPORTS
+	depends on BR2_TOOLCHAIN_USES_GLIBC || BR2_TOOLCHAIN_USES_MUSL # python-numpy
+	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_9 # python-numpy
+	depends on BR2_HOST_GCC_AT_LEAST_9 # host-python-numpy
+	select BR2_PACKAGE_PYTHON_NUMPY
+	select BR2_PACKAGE_PYTHON_PYBIND
+	help
+	  ml_dtypes is a stand-alone implementation of several NumPy
+	  dtype extensions used in machine learning libraries.
+
+	  https://github.com/jax-ml/ml_dtypes
diff --git a/package/python-ml_dtypes/python-ml_dtypes.hash b/package/python-ml_dtypes/python-ml_dtypes.hash
new file mode 100644
index 0000000000..a440268e39
--- /dev/null
+++ b/package/python-ml_dtypes/python-ml_dtypes.hash
@@ -0,0 +1,5 @@
+# md5, sha256 from https://pypi.org/pypi/ml_dtypes/json
+md5  6adbb05530819bdb4a78d2372d187fe2  ml_dtypes-0.3.1.tar.gz
+sha256  60778f99194b4c4f36ba42da200b35ef851ce4d4af698aaf70f5b91fe70fc611  ml_dtypes-0.3.1.tar.gz
+# Locally computed sha256 checksums
+sha256  cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30  LICENSE
diff --git a/package/python-ml_dtypes/python-ml_dtypes.mk b/package/python-ml_dtypes/python-ml_dtypes.mk
new file mode 100644
index 0000000000..555632c428
--- /dev/null
+++ b/package/python-ml_dtypes/python-ml_dtypes.mk
@@ -0,0 +1,19 @@
+################################################################################
+#
+# python-ml_dtypes
+#
+################################################################################
+
+PYTHON_ML_DTYPES_VERSION = 0.3.1
+PYTHON_ML_DTYPES_SOURCE = ml_dtypes-$(PYTHON_ML_DTYPES_VERSION).tar.gz
+PYTHON_ML_DTYPES_SITE = https://files.pythonhosted.org/packages/16/6e/9a7a51ee1ca24b8e92109128260c5aec8340c8fe5572e9ceecddae559abe
+PYTHON_ML_DTYPES_LICENSE = Apache-2.0
+PYTHON_ML_DTYPES_LICENSE_FILES = LICENSE
+PYTHON_ML_DTYPES_SETUP_TYPE = distutils
+
+PYTHON_ML_DTYPES_DEPENDENCIES = \
+	host-python-numpy \
+	python-numpy \
+	python-pybind
+
+$(eval $(python-package))
diff --git a/support/testing/tests/package/sample_python_ml_dtypes.py b/support/testing/tests/package/sample_python_ml_dtypes.py
new file mode 100644
index 0000000000..156d54a875
--- /dev/null
+++ b/support/testing/tests/package/sample_python_ml_dtypes.py
@@ -0,0 +1,39 @@
+#! /usr/bin/env python3
+
+# Tests inspired from commands published on project page:
+# https://pypi.org/project/ml-dtypes/
+
+from ml_dtypes import bfloat16
+
+import numpy as np
+
+a = np.zeros(4, dtype=bfloat16)
+assert a.dtype.name == 'bfloat16'
+assert a[0] == 0.0
+
+types = [
+    'bfloat16',
+    'float8_e4m3b11fnuz',
+    'float8_e4m3fn',
+    'float8_e4m3fnuz',
+    'float8_e5m2',
+    'int4',
+    'uint4'
+]
+for t in types:
+    dtype = np.dtype(t)
+    assert dtype.name == t
+
+rng = np.random.default_rng(seed=0)
+vals = rng.uniform(size=10000).astype(bfloat16)
+sum_vals = vals.sum()
+assert sum_vals == 256
+
+b = bfloat16(256) + bfloat16(1)
+assert b == 256
+
+c = np.nextafter(bfloat16(256), bfloat16(np.inf))
+assert c == 258
+
+d = vals.sum(dtype='float32').astype(bfloat16)
+assert d > 4500 and d < 5500
diff --git a/support/testing/tests/package/test_python_ml_dtypes.py b/support/testing/tests/package/test_python_ml_dtypes.py
new file mode 100644
index 0000000000..8d2369576b
--- /dev/null
+++ b/support/testing/tests/package/test_python_ml_dtypes.py
@@ -0,0 +1,20 @@
+from tests.package.test_python import TestPythonPackageBase
+
+
+class TestPythonPy3MlDtypes(TestPythonPackageBase):
+    __test__ = True
+
+    # Note: BR2_PACKAGE_PYTHON3_ZLIB=y is needed as a runtime
+    # dependency because the Bootlin toolchain used for this test is
+    # tainted with zlib (and gets detected by python3/numpy).
+    # See commit 8ce33fed "package/gdb: gdbserver does not need zlib".
+    # This config entry can be removed as soon as the toolchain is
+    # updated without zlib in its sysroot.
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON3_ZLIB=y
+        BR2_PACKAGE_PYTHON_ML_DTYPES=y
+        """
+    sample_scripts = ["tests/package/sample_python_ml_dtypes.py"]
+    timeout = 20
-- 
2.43.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply related	[flat|nested] 2+ messages in thread
* Re: [Buildroot] [PATCH 1/1] package/python-ml_dtypes: new package
  2023-12-23 23:05 [Buildroot] [PATCH 1/1] package/python-ml_dtypes: new package Julien Olivain
@ 2023-12-25 10:21 ` Thomas Petazzoni via buildroot
  0 siblings, 0 replies; 2+ messages in thread
From: Thomas Petazzoni via buildroot @ 2023-12-25 10:21 UTC (permalink / raw)
  To: Julien Olivain; +Cc: James Hilliard, Asaf Kahlon, buildroot

On Sun, 24 Dec 2023 00:05:13 +0100
Julien Olivain <ju.o@free.fr> wrote:

> ml_dtypes is a stand-alone implementation of several NumPy
> dtype extensions used in machine learning libraries.
> 
> https://github.com/jax-ml/ml_dtypes
> 
> Signed-off-by: Julien Olivain <ju.o@free.fr>
> ---
> Patch tested on branch master at commit 9212a71 with commands:

Even though the upstream package is named ml_dtypes with an underscore,
I renamed it to python-ml-dtypes in Buildroot, as we really use - as a
separator in package names. With that changed: applied to master,
thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 2+ messages in thread
end of thread, other threads:[~2023-12-25 10:22 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-12-23 23:05 [Buildroot] [PATCH 1/1] package/python-ml_dtypes: new package Julien Olivain
2023-12-25 10:21 ` Thomas Petazzoni via buildroot

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox