Re: [Buildroot] [PATCH 1/1] support/testing: add aichat runtime test

[Julien Olivain via buildroot <buildroot@buildroot.org>]

Hi,

On 07/01/2026 19:20, Julien Olivain via buildroot wrote:
> Hi Thomas,
> 
> On 07/01/2026 15:07, Thomas Petazzoni wrote:
>> Hello Julien,
>> 
>> Thanks for the patch! Obviously looks good, just one question.
>> 
>> On Wed,  7 Jan 2026 00:11:10 +0100
>> Julien Olivain via buildroot <buildroot@buildroot.org> wrote:
>> 
>>> +    def test_run(self):
>>> +        self.login()
>>> +
>>> +        # Check the program can execute.
>>> +        self.assertRunOk("aichat --version")
>>> +
>>> +        # We define a Hugging Face model to be downloaded.
>>> +        # We choose a relatively small model, for testing.
>>> +        hf_model = "ggml-org/gemma-3-270m-it-GGUF"
>> 
>> Does this mean that the model will be downloaded by llama-server when
>> we run it? I don't think we expect tests to require a network
>> connection to the Internet, and download "random stuff".
>> 
>> Or did I misunderstand your comment?
> 
> Your understanding is correct:
> llama-server has the capability to download the model from Internet
> if not available locally. This is what is happening here.
> 
> I am not sure if there was a strict rule forbidding public network
> to runtime tests.
> 
> I think Yann started to add runtime tests with such public network
> requirement a while back. See [1] [2] [3] [4]. I'm just following
> his steps ;) This is also what motivated commit [5] for example.
> 
> I think the argumentation was that the test runner needs Internet
> connectivity anyway to download source packages. Most runtime tests
> does not require network connection, so I though it was more a fact
> than a strict rule. Some tests (like podman, etc.) will have a better
> coverage if they are working with their online repositories.
> 
> Regarding the data and its source, I would not exactly call that
> "random stuff". Hugging Face [6] is a reference repository used
> by llama.cpp. Also, this model [7] in particular is a smaller
> version of the llama.cpp example in [8]. Note that ggml-org [9]
> is reference library for llama.cpp.
> 
> Do you want me to send a v2 adding this justification?
> 
> Or do you prefer to (re?)open the topic to strictly forbid network
> in runtime tests?

After discussing during the Buildroot Dev Days 2026, we decided
that runtime tests should avoid using network whenever possible.
But in cases where a tool functionality depends on the network,
it could be used.

So for this reason, I applied this patch.

>> Thomas
>> --
>> Thomas Petazzoni, co-owner and CEO, Bootlin
>> Embedded Linux and Kernel engineering and training
>> https://bootlin.com
> 
> Best regards,
> 
> Julien.
> 
> [1] 
> https://gitlab.com/buildroot.org/buildroot/-/blob/2025.11/support/testing/tests/package/test_distribution_registry.py#L80-85
> [2] 
> https://gitlab.com/buildroot.org/buildroot/-/blob/2025.11/support/testing/tests/package/test_docker_compose.py#L42-43
> [3] 
> https://gitlab.com/buildroot.org/buildroot/-/blob/2025.11/support/testing/tests/package/test_podman.py#L97-98
> [4] 
> https://gitlab.com/buildroot.org/buildroot/-/blob/2025.11/support/testing/tests/package/test_skopeo.py#L50-54
> [5] 
> https://gitlab.com/buildroot.org/buildroot/-/commit/cf8641b73e7f1577637bfef0ece78dd519b25d19
> [6] https://huggingface.co/
> [7] https://huggingface.co/ggml-org/gemma-3-270m-it-GGUF
> [8] 
> https://github.com/ggml-org/llama.cpp/blob/b7271/README.md?plain=1#L53
> [9] https://github.com/ggml-org

Best regards,

Julien.
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot