[Buildroot] [PATCH v1 1/2] package/libwebsockets: add an option to enable lws async dns

[Buildroot] [PATCH v1 1/2] package/libwebsockets: add an option to enable lws async dns

[Thomas Devoogdt]

From: Bart Van Severen <bart.vanseveren@barco.com>

Lws now features optional asynchronous, ie, nonblocking recursive DNS
resolution done on the event loop, enable `-DLWS_WITH_SYS_ASYNC_DNS=1`
at cmake to build it in.

The default libc name resolution is via libc `getaddrinfo()`, which is
blocking, possibly for quite long periods (seconds).  If you are
taking care about latency, but want to create outgoing connections,
you can't tolerate this exception from the rule that everything in
lws is nonblocking.

Lws' asynchronous DNS resolver creates a caching name resolver
that directly queries the configured nameserver itself over UDP,
from the event loop.

https://libwebsockets.org/lws-api-doc-main/html/md_READMEs_2README_8async-dns.html

Signed-off-by: Bart Van Severen <bart.vanseveren@barco.com>
Signed-off-by: Thomas Devoogdt <thomas.devoogdt@barco.com>
---
 package/libwebsockets/Config.in        | 6 ++++++
 package/libwebsockets/libwebsockets.mk | 4 ++++
 2 files changed, 10 insertions(+)

diff --git a/package/libwebsockets/Config.in b/package/libwebsockets/Config.in
index 05c132641e..6116cfecb5 100644
--- a/package/libwebsockets/Config.in
+++ b/package/libwebsockets/Config.in
@@ -19,6 +19,12 @@ config BR2_PACKAGE_LIBWEBSOCKETS_EXT_POLL
 	  unconditionally compiled in for library versions
 	  prior to 3.2.0.
 
+config BR2_PACKAGE_LIBWEBSOCKETS_ASYNC_DNS
+	bool "enable async dns support"
+	help
+	  Enable asynchronous DNS resolver that directly queries
+	  the configured nameserver over UDP, from the event loop.
+
 endif
 
 comment "libwebsockets needs a toolchain w/ dynamic library"
diff --git a/package/libwebsockets/libwebsockets.mk b/package/libwebsockets/libwebsockets.mk
index c00c44cac3..e53febfce4 100644
--- a/package/libwebsockets/libwebsockets.mk
+++ b/package/libwebsockets/libwebsockets.mk
@@ -98,4 +98,8 @@ ifeq ($(BR2_PACKAGE_LIBWEBSOCKETS_EXT_POLL),y)
 LIBWEBSOCKETS_CONF_OPTS += -DLWS_WITH_EXTERNAL_POLL=ON
 endif
 
+ifeq ($(BR2_PACKAGE_LIBWEBSOCKETS_ASYNC_DNS),y)
+LIBWEBSOCKETS_CONF_OPTS += -DLWS_WITH_SYS_ASYNC_DNS=ON
+endif
+
 $(eval $(cmake-package))
-- 
2.43.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH v1 2/2] package/libwebsockets: fix LWS_MAX_SMP when BR2_TOOLCHAIN_HAS_THREADS is set

[Thomas Devoogdt]

From: Bart Van Severen <bart.vanseveren@barco.com>

"If LWS_MAX_SMP=1, then there is no code related to pthreads compiled
 in the library. If unset, LWS_MAX_SMP defaults to 32 and a small
 amount of pthread mutex code is built into the library."

This is a misconception, when unset, LWS_MAX_SMP is actually set to 1,
so mutexes aren't built in.

To fix, set it to 32 explicitly when threads are enabled.

https://libwebsockets.org/lws-api-doc-master/html/md_READMEs_README_8coding.html

Signed-off-by: Bart Van Severen <bart.vanseveren@barco.com>
Signed-off-by: Thomas Devoogdt <thomas.devoogdt@barco.com>
---
 package/libwebsockets/libwebsockets.mk | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/package/libwebsockets/libwebsockets.mk b/package/libwebsockets/libwebsockets.mk
index e53febfce4..90ab5da0c9 100644
--- a/package/libwebsockets/libwebsockets.mk
+++ b/package/libwebsockets/libwebsockets.mk
@@ -18,15 +18,14 @@ LIBWEBSOCKETS_CONF_OPTS = \
 	-DLWS_WITHOUT_EXTENSIONS=OFF
 
 # If LWS_MAX_SMP=1, then there is no code related to pthreads compiled
-# in the library. If unset, LWS_MAX_SMP defaults to 32 and a small
-# amount of pthread mutex code is built into the library.
+# in the library. If unset, LWS_MAX_SMP defaults to 1.
 ifeq ($(BR2_TOOLCHAIN_HAS_THREADS),)
 LIBWEBSOCKETS_CONF_OPTS += \
 	-DLWS_MAX_SMP=1 \
 	-DLWS_WITH_SYS_SMD=OFF
 else
 LIBWEBSOCKETS_CONF_OPTS += \
-	-DLWS_MAX_SMP= \
+	-DLWS_MAX_SMP=32 \
 	-DLWS_WITH_SYS_SMD=ON
 endif
 
-- 
2.43.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v1 1/2] package/libwebsockets: add an option to enable lws async dns

[Thomas Petazzoni via buildroot]

On Fri, 20 Sep 2024 08:20:13 +0200
Thomas Devoogdt <thomas@devoogdt.com> wrote:

> From: Bart Van Severen <bart.vanseveren@barco.com>
> 
> Lws now features optional asynchronous, ie, nonblocking recursive DNS
> resolution done on the event loop, enable `-DLWS_WITH_SYS_ASYNC_DNS=1`
> at cmake to build it in.
> 
> The default libc name resolution is via libc `getaddrinfo()`, which is
> blocking, possibly for quite long periods (seconds).  If you are
> taking care about latency, but want to create outgoing connections,
> you can't tolerate this exception from the rule that everything in
> lws is nonblocking.
> 
> Lws' asynchronous DNS resolver creates a caching name resolver
> that directly queries the configured nameserver itself over UDP,
> from the event loop.
> 
> https://libwebsockets.org/lws-api-doc-main/html/md_READMEs_2README_8async-dns.html
> 
> Signed-off-by: Bart Van Severen <bart.vanseveren@barco.com>
> Signed-off-by: Thomas Devoogdt <thomas.devoogdt@barco.com>
> ---
>  package/libwebsockets/Config.in        | 6 ++++++
>  package/libwebsockets/libwebsockets.mk | 4 ++++
>  2 files changed, 10 insertions(+)

Applied to master, thanks.

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v1 2/2] package/libwebsockets: fix LWS_MAX_SMP when BR2_TOOLCHAIN_HAS_THREADS is set

[Thomas Petazzoni via buildroot]

Hello Thomas/Bart,

On Fri, 20 Sep 2024 08:20:14 +0200
Thomas Devoogdt <thomas@devoogdt.com> wrote:

> From: Bart Van Severen <bart.vanseveren@barco.com>
> 
> "If LWS_MAX_SMP=1, then there is no code related to pthreads compiled
>  in the library. If unset, LWS_MAX_SMP defaults to 32 and a small
>  amount of pthread mutex code is built into the library."
> 
> This is a misconception, when unset, LWS_MAX_SMP is actually set to 1,
> so mutexes aren't built in.
> 
> To fix, set it to 32 explicitly when threads are enabled.
> 
> https://libwebsockets.org/lws-api-doc-master/html/md_READMEs_README_8coding.html
> 
> Signed-off-by: Bart Van Severen <bart.vanseveren@barco.com>
> Signed-off-by: Thomas Devoogdt <thomas.devoogdt@barco.com>
> ---
>  package/libwebsockets/libwebsockets.mk | 5 ++---
>  1 file changed, 2 insertions(+), 3 deletions(-)

The question is then why 32, and not 8, 16, or 64 ?

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v1 2/2] package/libwebsockets: fix LWS_MAX_SMP when BR2_TOOLCHAIN_HAS_THREADS is set

[Thomas Devoogdt]

Hi Thomas

Op za 26 okt 2024 om 16:43 schreef Thomas Petazzoni
<thomas.petazzoni@bootlin.com>:
>
> Hello Thomas/Bart,
>
> On Fri, 20 Sep 2024 08:20:14 +0200
> Thomas Devoogdt <thomas@devoogdt.com> wrote:
>
> > From: Bart Van Severen <bart.vanseveren@barco.com>
> >
> > "If LWS_MAX_SMP=1, then there is no code related to pthreads compiled
> >  in the library. If unset, LWS_MAX_SMP defaults to 32 and a small
> >  amount of pthread mutex code is built into the library."
> >
> > This is a misconception, when unset, LWS_MAX_SMP is actually set to 1,
> > so mutexes aren't built in.
> >
> > To fix, set it to 32 explicitly when threads are enabled.
> >
> > https://libwebsockets.org/lws-api-doc-master/html/md_READMEs_README_8coding.html
> >
> > Signed-off-by: Bart Van Severen <bart.vanseveren@barco.com>
> > Signed-off-by: Thomas Devoogdt <thomas.devoogdt@barco.com>
> > ---
> >  package/libwebsockets/libwebsockets.mk | 5 ++---
> >  1 file changed, 2 insertions(+), 3 deletions(-)
>
> The question is then why 32, and not 8, 16, or 64 ?

The original comment mentioned the number 32, so it makes sense to keep that 32.

Also, see this older manual
https://libwebsockets.org/lws-api-doc-master/html/md_README.coding.html:
"You can control the context basic data allocation for multithreading
from Cmake using -DLWS_MAX_SMP=, if not given it's set to 32."

And this newer manual
https://libwebsockets.org/lws-api-doc-master/html/md_READMEs_README_8coding.html:
"You can control the context basic data allocation for multithreading
from Cmake using -DLWS_MAX_SMP=, if not given it's set to 1. "

So, given that the original manual talks about 32, it makes sense to
use it, even though it is no longer valid.
Other values will also work, as long the minimum is 2. But I guess
that this 32 is a good trade-off.

>
> Thanks!
>
> Thomas
> --
> Thomas Petazzoni, CTO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>

Kr,

Thomas Devoogdt
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v1 2/2] package/libwebsockets: fix LWS_MAX_SMP when BR2_TOOLCHAIN_HAS_THREADS is set

[Thomas Petazzoni via buildroot]

Hello Thomas,

On Sat, 26 Oct 2024 20:25:26 +0200
Thomas Devoogdt <thomas@devoogdt.com> wrote:

> > The question is then why 32, and not 8, 16, or 64 ?  
> 
> The original comment mentioned the number 32, so it makes sense to keep that 32.
> 
> Also, see this older manual
> https://libwebsockets.org/lws-api-doc-master/html/md_README.coding.html:
> "You can control the context basic data allocation for multithreading
> from Cmake using -DLWS_MAX_SMP=, if not given it's set to 32."
> 
> And this newer manual
> https://libwebsockets.org/lws-api-doc-master/html/md_READMEs_README_8coding.html:
> "You can control the context basic data allocation for multithreading
> from Cmake using -DLWS_MAX_SMP=, if not given it's set to 1. "
> 
> So, given that the original manual talks about 32, it makes sense to
> use it, even though it is no longer valid.
> Other values will also work, as long the minimum is 2. But I guess
> that this 32 is a good trade-off.

It felt a bit random, but OK, I guess there's no "good" value there.
Thanks for the feedback!

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v1 2/2] package/libwebsockets: fix LWS_MAX_SMP when BR2_TOOLCHAIN_HAS_THREADS is set

[Thomas Devoogdt]

Hi Thomas, all,


This thread has been a bit abandoned for some time now, any concerns
still to be addressed?
Would be nice to have it for the upcoming 2025.02 release.


Kind regards,

Thomas Devoogdt
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v1 2/2] package/libwebsockets: fix LWS_MAX_SMP when BR2_TOOLCHAIN_HAS_THREADS is set

[Thomas Petazzoni via buildroot]

Hello Thomas,

On Fri, Sep 20, 2024 at 08:20:14AM +0200, Thomas Devoogdt wrote:
> From: Bart Van Severen <bart.vanseveren@barco.com>
> 
> "If LWS_MAX_SMP=1, then there is no code related to pthreads compiled
>  in the library. If unset, LWS_MAX_SMP defaults to 32 and a small
>  amount of pthread mutex code is built into the library."
> 
> This is a misconception, when unset, LWS_MAX_SMP is actually set to 1,
> so mutexes aren't built in.
> 
> To fix, set it to 32 explicitly when threads are enabled.
> 
> https://libwebsockets.org/lws-api-doc-master/html/md_READMEs_README_8coding.html
> 
> Signed-off-by: Bart Van Severen <bart.vanseveren@barco.com>
> Signed-off-by: Thomas Devoogdt <thomas.devoogdt@barco.com>

Thanks for the patch and additional details. I have improved a bit the
commit log with some of the details provided as part of the
discussion, and finally applied your patch... after 1.5 years.

Thanks!

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot