All of lore.kernel.org
 help / color / mirror / Atom feed
* [Buildroot] [PATCH 1/1] package/python-ml_dtypes: new package
@ 2023-12-23 23:05 Julien Olivain
  2023-12-25 10:21 ` Thomas Petazzoni via buildroot
  0 siblings, 1 reply; 2+ messages in thread
From: Julien Olivain @ 2023-12-23 23:05 UTC (permalink / raw)
  To: buildroot; +Cc: James Hilliard, Julien Olivain, Thomas Petazzoni, Asaf Kahlon

ml_dtypes is a stand-alone implementation of several NumPy
dtype extensions used in machine learning libraries.

https://github.com/jax-ml/ml_dtypes

Signed-off-by: Julien Olivain <ju.o@free.fr>
---
Patch tested on branch master at commit 9212a71 with commands:

    make check-package
    ...
    0 warnings generated

    support/testing/run-tests \
        -d dl -o output_folder \
        tests.package.test_python_ml_dtypes
    ...
    OK  
 
Note: this package is a dependency of tensorflow-lite >= v2.14.0.
For info, tensorflow-lite v.2.11.0 was proposed in Buildroot in:
https://patchwork.ozlabs.org/project/buildroot/list/?series=346546
---
 DEVELOPERS                                    |  3 ++
 package/Config.in                             |  1 +
 package/python-ml_dtypes/Config.in            | 13 +++++++
 .../python-ml_dtypes/python-ml_dtypes.hash    |  5 +++
 package/python-ml_dtypes/python-ml_dtypes.mk  | 19 +++++++++
 .../tests/package/sample_python_ml_dtypes.py  | 39 +++++++++++++++++++
 .../tests/package/test_python_ml_dtypes.py    | 20 ++++++++++
 7 files changed, 100 insertions(+)
 create mode 100644 package/python-ml_dtypes/Config.in
 create mode 100644 package/python-ml_dtypes/python-ml_dtypes.hash
 create mode 100644 package/python-ml_dtypes/python-ml_dtypes.mk
 create mode 100644 support/testing/tests/package/sample_python_ml_dtypes.py
 create mode 100644 support/testing/tests/package/test_python_ml_dtypes.py

diff --git a/DEVELOPERS b/DEVELOPERS
index bad07d51aa..8b64cb74d1 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -1741,6 +1741,7 @@ F:	package/python-magic-wormhole/
 F:	package/python-magic-wormhole-mailbox-server/
 F:	package/python-magic-wormhole-transit-relay/
 F:	package/python-midiutil/
+F:	package/python-ml_dtypes/
 F:	package/python-pyalsa/
 F:	package/python-spake2/
 F:	package/rdma-core/
@@ -1752,6 +1753,7 @@ F:	support/testing/tests/package/sample_python_distro.py
 F:	support/testing/tests/package/sample_python_gnupg.py
 F:	support/testing/tests/package/sample_python_hwdata.py
 F:	support/testing/tests/package/sample_python_midiutil.py
+F:	support/testing/tests/package/sample_python_ml_dtypes.py
 F:	support/testing/tests/package/sample_python_pyalsa.py
 F:	support/testing/tests/package/sample_python_spake2.py
 F:	support/testing/tests/package/test_acpica.py
@@ -1802,6 +1804,7 @@ F:	support/testing/tests/package/test_python_hkdf.py
 F:	support/testing/tests/package/test_python_hwdata.py
 F:	support/testing/tests/package/test_python_magic_wormhole.py
 F:	support/testing/tests/package/test_python_midiutil.py
+F:	support/testing/tests/package/test_python_ml_dtypes.py
 F:	support/testing/tests/package/test_python_pyalsa.py
 F:	support/testing/tests/package/test_python_spake2.py
 F:	support/testing/tests/package/test_rdma_core.py
diff --git a/package/Config.in b/package/Config.in
index bc723fe018..7442b9d920 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -1170,6 +1170,7 @@ menu "External python modules"
 	source "package/python-mimeparse/Config.in"
 	source "package/python-minimalmodbus/Config.in"
 	source "package/python-mistune/Config.in"
+	source "package/python-ml_dtypes/Config.in"
 	source "package/python-modbus-tk/Config.in"
 	source "package/python-more-itertools/Config.in"
 	source "package/python-mpd2/Config.in"
diff --git a/package/python-ml_dtypes/Config.in b/package/python-ml_dtypes/Config.in
new file mode 100644
index 0000000000..c487692b7a
--- /dev/null
+++ b/package/python-ml_dtypes/Config.in
@@ -0,0 +1,13 @@
+config BR2_PACKAGE_PYTHON_ML_DTYPES
+	bool "python-ml_dtypes"
+	depends on BR2_PACKAGE_PYTHON_NUMPY_ARCH_SUPPORTS
+	depends on BR2_TOOLCHAIN_USES_GLIBC || BR2_TOOLCHAIN_USES_MUSL # python-numpy
+	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_9 # python-numpy
+	depends on BR2_HOST_GCC_AT_LEAST_9 # host-python-numpy
+	select BR2_PACKAGE_PYTHON_NUMPY
+	select BR2_PACKAGE_PYTHON_PYBIND
+	help
+	  ml_dtypes is a stand-alone implementation of several NumPy
+	  dtype extensions used in machine learning libraries.
+
+	  https://github.com/jax-ml/ml_dtypes
diff --git a/package/python-ml_dtypes/python-ml_dtypes.hash b/package/python-ml_dtypes/python-ml_dtypes.hash
new file mode 100644
index 0000000000..a440268e39
--- /dev/null
+++ b/package/python-ml_dtypes/python-ml_dtypes.hash
@@ -0,0 +1,5 @@
+# md5, sha256 from https://pypi.org/pypi/ml_dtypes/json
+md5  6adbb05530819bdb4a78d2372d187fe2  ml_dtypes-0.3.1.tar.gz
+sha256  60778f99194b4c4f36ba42da200b35ef851ce4d4af698aaf70f5b91fe70fc611  ml_dtypes-0.3.1.tar.gz
+# Locally computed sha256 checksums
+sha256  cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30  LICENSE
diff --git a/package/python-ml_dtypes/python-ml_dtypes.mk b/package/python-ml_dtypes/python-ml_dtypes.mk
new file mode 100644
index 0000000000..555632c428
--- /dev/null
+++ b/package/python-ml_dtypes/python-ml_dtypes.mk
@@ -0,0 +1,19 @@
+################################################################################
+#
+# python-ml_dtypes
+#
+################################################################################
+
+PYTHON_ML_DTYPES_VERSION = 0.3.1
+PYTHON_ML_DTYPES_SOURCE = ml_dtypes-$(PYTHON_ML_DTYPES_VERSION).tar.gz
+PYTHON_ML_DTYPES_SITE = https://files.pythonhosted.org/packages/16/6e/9a7a51ee1ca24b8e92109128260c5aec8340c8fe5572e9ceecddae559abe
+PYTHON_ML_DTYPES_LICENSE = Apache-2.0
+PYTHON_ML_DTYPES_LICENSE_FILES = LICENSE
+PYTHON_ML_DTYPES_SETUP_TYPE = distutils
+
+PYTHON_ML_DTYPES_DEPENDENCIES = \
+	host-python-numpy \
+	python-numpy \
+	python-pybind
+
+$(eval $(python-package))
diff --git a/support/testing/tests/package/sample_python_ml_dtypes.py b/support/testing/tests/package/sample_python_ml_dtypes.py
new file mode 100644
index 0000000000..156d54a875
--- /dev/null
+++ b/support/testing/tests/package/sample_python_ml_dtypes.py
@@ -0,0 +1,39 @@
+#! /usr/bin/env python3
+
+# Tests inspired from commands published on project page:
+# https://pypi.org/project/ml-dtypes/
+
+from ml_dtypes import bfloat16
+
+import numpy as np
+
+a = np.zeros(4, dtype=bfloat16)
+assert a.dtype.name == 'bfloat16'
+assert a[0] == 0.0
+
+types = [
+    'bfloat16',
+    'float8_e4m3b11fnuz',
+    'float8_e4m3fn',
+    'float8_e4m3fnuz',
+    'float8_e5m2',
+    'int4',
+    'uint4'
+]
+for t in types:
+    dtype = np.dtype(t)
+    assert dtype.name == t
+
+rng = np.random.default_rng(seed=0)
+vals = rng.uniform(size=10000).astype(bfloat16)
+sum_vals = vals.sum()
+assert sum_vals == 256
+
+b = bfloat16(256) + bfloat16(1)
+assert b == 256
+
+c = np.nextafter(bfloat16(256), bfloat16(np.inf))
+assert c == 258
+
+d = vals.sum(dtype='float32').astype(bfloat16)
+assert d > 4500 and d < 5500
diff --git a/support/testing/tests/package/test_python_ml_dtypes.py b/support/testing/tests/package/test_python_ml_dtypes.py
new file mode 100644
index 0000000000..8d2369576b
--- /dev/null
+++ b/support/testing/tests/package/test_python_ml_dtypes.py
@@ -0,0 +1,20 @@
+from tests.package.test_python import TestPythonPackageBase
+
+
+class TestPythonPy3MlDtypes(TestPythonPackageBase):
+    __test__ = True
+
+    # Note: BR2_PACKAGE_PYTHON3_ZLIB=y is needed as a runtime
+    # dependency because the Bootlin toolchain used for this test is
+    # tainted with zlib (and gets detected by python3/numpy).
+    # See commit 8ce33fed "package/gdb: gdbserver does not need zlib".
+    # This config entry can be removed as soon as the toolchain is
+    # updated without zlib in its sysroot.
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON3_ZLIB=y
+        BR2_PACKAGE_PYTHON_ML_DTYPES=y
+        """
+    sample_scripts = ["tests/package/sample_python_ml_dtypes.py"]
+    timeout = 20
-- 
2.43.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply related	[flat|nested] 2+ messages in thread
* Re: [Buildroot] [PATCH 1/1] package/python-ml_dtypes: new package
  2023-12-23 23:05 [Buildroot] [PATCH 1/1] package/python-ml_dtypes: new package Julien Olivain
@ 2023-12-25 10:21 ` Thomas Petazzoni via buildroot
  0 siblings, 0 replies; 2+ messages in thread
From: Thomas Petazzoni via buildroot @ 2023-12-25 10:21 UTC (permalink / raw)
  To: Julien Olivain; +Cc: James Hilliard, Asaf Kahlon, buildroot

On Sun, 24 Dec 2023 00:05:13 +0100
Julien Olivain <ju.o@free.fr> wrote:

> ml_dtypes is a stand-alone implementation of several NumPy
> dtype extensions used in machine learning libraries.
> 
> https://github.com/jax-ml/ml_dtypes
> 
> Signed-off-by: Julien Olivain <ju.o@free.fr>
> ---
> Patch tested on branch master at commit 9212a71 with commands:

Even though the upstream package is named ml_dtypes with an underscore,
I renamed it to python-ml-dtypes in Buildroot, as we really use - as a
separator in package names. With that changed: applied to master,
thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 2+ messages in thread
end of thread, other threads:[~2023-12-25 10:22 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-12-23 23:05 [Buildroot] [PATCH 1/1] package/python-ml_dtypes: new package Julien Olivain
2023-12-25 10:21 ` Thomas Petazzoni via buildroot

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.