From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F0D9A303C87; Mon, 24 Nov 2025 15:44:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763999073; cv=none; b=PGlTtWZFcILO1kZAS9Z1AyDQBtATRiiXqJT/rJNgBxVmkSnUoADudeJGtDzT7wKZyH6kh3/cis87FGqfG0hd8pWZAyHbFnxKOug9660qGIjdKqccleCB5VX4/RTGWPozn1d05zDY48hm/jUHXTzDNVKyj/MWDHNaVcFC5AhFqCc= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763999073; c=relaxed/simple; bh=nOF/3aYSHUai9rTAQhLQCWdjVUz8rMOro1S9AA568zE=; h=From:To:Cc:Subject:In-Reply-To:References:Date:Message-ID: MIME-Version:Content-Type; b=PUKuZq50K1Jpf61CXqqkyfM+9+b4vWn36ko8IvyZk9wazCQ7tOgd0LfdZDW7KnI6pajUe39Ab0BXdbe0SiXy5mh2cZQ7+HLNopg4iDFvNxwkztZ4VgAIsTjEslGrcHnWIWF2RB1jRnrlkNz47lj2NxIevSa4VLVEY1TU/yNCc/w= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=apVuihVt; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="apVuihVt" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 14CBAC4CEF1; Mon, 24 Nov 2025 15:44:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1763999072; bh=nOF/3aYSHUai9rTAQhLQCWdjVUz8rMOro1S9AA568zE=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=apVuihVtYuxCjeeMj/JSzOsSNsmqURqTths6nw2hVQUUvx99l6YJ4RAe0aYv2Gksa mP9HCq400YqLWqcKU06RzK3HO4KnvpppK6AhCaAzCgKvWld/09HAzq7OGoVwGB+3dR 2nGNtH+gUOw+Doya2YZapOe4o1qD8h2mOsqc5Jf9BMQczchxKoQ2WsGWYMJGPClmAN KUjFkNOghf7nfqcPSj8W9yNwu4yyjl09Thxb4RWNSV0x7xm0bMtAY9cKd5KqXGVRQw 1glJjWtZJGGU6QvD/cAGtw2jvmpD/OWsXtIFkWEHJS2wKu5BKSqLXATB11DT6711PP P9/LxoddpdWZA== From: Pratyush Yadav To: Pasha Tatashin Cc: pratyush@kernel.org, jasonmiu@google.com, graf@amazon.com, rppt@kernel.org, dmatlack@google.com, rientjes@google.com, corbet@lwn.net, rdunlap@infradead.org, ilpo.jarvinen@linux.intel.com, kanie@linux.alibaba.com, ojeda@kernel.org, aliceryhl@google.com, masahiroy@kernel.org, akpm@linux-foundation.org, tj@kernel.org, yoann.congal@smile.fr, mmaurer@google.com, roman.gushchin@linux.dev, chenridong@huawei.com, axboe@kernel.dk, mark.rutland@arm.com, jannh@google.com, vincent.guittot@linaro.org, hannes@cmpxchg.org, dan.j.williams@intel.com, david@redhat.com, joel.granados@kernel.org, rostedt@goodmis.org, anna.schumaker@oracle.com, song@kernel.org, linux@weissschuh.net, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, gregkh@linuxfoundation.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, rafael@kernel.org, dakr@kernel.org, bartosz.golaszewski@linaro.org, cw00.choi@samsung.com, myungjoo.ham@samsung.com, yesanishhere@gmail.com, Jonathan.Cameron@huawei.com, quic_zijuhu@quicinc.com, aleksander.lobakin@intel.com, ira.weiny@intel.com, andriy.shevchenko@linux.intel.com, leon@kernel.org, lukas@wunner.de, bhelgaas@google.com, wagi@kernel.org, djeffery@redhat.com, stuart.w.hayes@gmail.com, lennart@poettering.net, brauner@kernel.org, linux-api@vger.kernel.org, linux-fsdevel@vger.kernel.org, saeedm@nvidia.com, ajayachandra@nvidia.com, jgg@nvidia.com, parav@nvidia.com, leonro@nvidia.com, witu@nvidia.com, hughd@google.com, skhawaja@google.com, chrisl@kernel.org Subject: Re: [PATCH v7 06/22] liveupdate: luo_file: implement file systems callbacks In-Reply-To: <20251122222351.1059049-7-pasha.tatashin@soleen.com> (Pasha Tatashin's message of "Sat, 22 Nov 2025 17:23:33 -0500") References: <20251122222351.1059049-1-pasha.tatashin@soleen.com> <20251122222351.1059049-7-pasha.tatashin@soleen.com> Date: Mon, 24 Nov 2025 16:44:21 +0100 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Precedence: bulk X-Mailing-List: linux-api@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On Sat, Nov 22 2025, Pasha Tatashin wrote: > This patch implements the core mechanism for managing preserved > files throughout the live update lifecycle. It provides the logic to > invoke the file handler callbacks (preserve, unpreserve, freeze, > unfreeze, retrieve, and finish) at the appropriate stages. > > During the reboot phase, luo_file_freeze() serializes the final > metadata for each file (handler compatible string, token, and data > handle) into a memory region preserved by KHO. In the new kernel, > luo_file_deserialize() reconstructs the in-memory file list from this > data, preparing the session for retrieval. > > Signed-off-by: Pasha Tatashin > --- > include/linux/kho/abi/luo.h | 39 +- > include/linux/liveupdate.h | 98 ++++ > kernel/liveupdate/Makefile | 1 + > kernel/liveupdate/luo_file.c | 882 +++++++++++++++++++++++++++++++ > kernel/liveupdate/luo_internal.h | 38 ++ > 5 files changed, 1057 insertions(+), 1 deletion(-) > create mode 100644 kernel/liveupdate/luo_file.c > > diff --git a/include/linux/kho/abi/luo.h b/include/linux/kho/abi/luo.h > index a2d2940eca6b..fc143f243871 100644 > --- a/include/linux/kho/abi/luo.h > +++ b/include/linux/kho/abi/luo.h > @@ -65,6 +65,11 @@ > * Metadata for a single session, including its name and a physical = pointer > * to another preserved memory block containing an array of > * `struct luo_file_ser` for all files in that session. > + * > + * - struct luo_file_ser: > + * Metadata for a single preserved file. Contains the `compatible` s= tring to > + * find the correct handler in the new kernel, a user-provided `toke= n` for > + * identification, and an opaque `data` handle for the handler to us= e. > */ >=20=20 > #ifndef _LINUX_KHO_ABI_LUO_H > @@ -82,13 +87,43 @@ > #define LUO_FDT_COMPATIBLE "luo-v1" > #define LUO_FDT_LIVEUPDATE_NUM "liveupdate-number" >=20=20 > +#define LIVEUPDATE_HNDL_COMPAT_LENGTH 48 > + > +/** > + * struct luo_file_ser - Represents the serialized preserves files. > + * @compatible: File handler compatible string. > + * @data: Private data > + * @token: User provided token for this file > + * > + * If this structure is modified, LUO_SESSION_COMPATIBLE must be updated. > + */ > +struct luo_file_ser { > + char compatible[LIVEUPDATE_HNDL_COMPAT_LENGTH]; > + u64 data; > + u64 token; > +} __packed; > + > +/** > + * struct luo_file_set_ser - Represents the serialized metadata for file= set > + * @files: The physical address of a contiguous memory block that holds > + * the serialized state of files (array of luo_file_ser) in th= is file > + * set. > + * @count: The total number of files that were part of this session du= ring > + * serialization. Used for iteration and validation during > + * restoration. > + */ > +struct luo_file_set_ser { > + u64 files; > + u64 count; > +} __packed; The change to using file_set looks a lot nicer than what the previous version was doing! > + > /* > * LUO FDT session node > * LUO_FDT_SESSION_HEADER: is a u64 physical address of struct > * luo_session_header_ser > */ > #define LUO_FDT_SESSION_NODE_NAME "luo-session" > -#define LUO_FDT_SESSION_COMPATIBLE "luo-session-v1" > +#define LUO_FDT_SESSION_COMPATIBLE "luo-session-v2" > #define LUO_FDT_SESSION_HEADER "luo-session-header" >=20=20 > /** > @@ -110,6 +145,7 @@ struct luo_session_header_ser { > * struct luo_session_ser - Represents the serialized metadata for a LUO= session. > * @name: The unique name of the session, provided by the usersp= ace at > * the time of session creation. > + * @file_set_ser: Serialized files belonging to this session, > * > * This structure is used to package session-specific metadata for trans= fer > * between kernels via Kexec Handover. An array of these structures (one= per > @@ -120,6 +156,7 @@ struct luo_session_header_ser { > */ > struct luo_session_ser { > char name[LIVEUPDATE_SESSION_NAME_LENGTH]; > + struct luo_file_set_ser file_set_ser; > } __packed; >=20=20 > #endif /* _LINUX_KHO_ABI_LUO_H */ > diff --git a/include/linux/liveupdate.h b/include/linux/liveupdate.h > index c6a1d6bd90cb..122ad8f16ff9 100644 > --- a/include/linux/liveupdate.h > +++ b/include/linux/liveupdate.h > @@ -8,8 +8,93 @@ > #define _LINUX_LIVEUPDATE_H >=20=20 > #include > +#include > +#include > #include > #include > +#include > + > +struct liveupdate_file_handler; > +struct file; > + > +/** > + * struct liveupdate_file_op_args - Arguments for file operation callbac= ks. > + * @handler: The file handler being called. > + * @retrieved: The retrieve status for the 'can_finish / finish' > + * operation. > + * @file: The file object. For retrieve: [OUT] The callback = sets > + * this to the new file. For other ops: [IN] The call= er sets > + * this to the file being operated on. > + * @serialized_data: The opaque u64 handle, preserve/prepare/freeze may= update > + * this field. > + * > + * This structure bundles all parameters for the file operation callback= s. > + * The 'data' and 'file' fields are used for both input and output. > + */ > +struct liveupdate_file_op_args { > + struct liveupdate_file_handler *handler; > + bool retrieved; > + struct file *file; > + u64 serialized_data; > +}; > + > +/** > + * struct liveupdate_file_ops - Callbacks for live-updatable files. > + * @can_preserve: Required. Lightweight check to see if this handler is > + * compatible with the given file. > + * @preserve: Required. Performs state-saving for the file. > + * @unpreserve: Required. Cleans up any resources allocated by @preser= ve. > + * @freeze: Optional. Final actions just before kernel transition. > + * @unfreeze: Optional. Undo freeze operations. > + * @retrieve: Required. Restores the file in the new kernel. > + * @can_finish: Optional. Check if this FD can finish, i.e. all restor= ation > + * pre-requirements for this FD are satisfied. Called pri= or to > + * finish, in order to do successful finish calls for all > + * resources in the session. > + * @finish: Required. Final cleanup in the new kernel. > + * @owner: Module reference > + * > + * All operations (except can_preserve) receive a pointer to a > + * 'struct liveupdate_file_op_args' containing the necessary context. > + */ > +struct liveupdate_file_ops { > + bool (*can_preserve)(struct liveupdate_file_handler *handler, > + struct file *file); > + int (*preserve)(struct liveupdate_file_op_args *args); > + void (*unpreserve)(struct liveupdate_file_op_args *args); > + int (*freeze)(struct liveupdate_file_op_args *args); > + void (*unfreeze)(struct liveupdate_file_op_args *args); > + int (*retrieve)(struct liveupdate_file_op_args *args); > + bool (*can_finish)(struct liveupdate_file_op_args *args); > + void (*finish)(struct liveupdate_file_op_args *args); > + struct module *owner; > +}; > + > +/** > + * struct liveupdate_file_handler - Represents a handler for a live-upda= table file type. > + * @ops: Callback functions > + * @compatible: The compatibility string (e.g., "memfd-v1", "vfi= ofd-v1") > + * that uniquely identifies the file type this hand= ler > + * supports. This is matched against the compatible= string > + * associated with individual &struct file instance= s. > + * > + * Modules that want to support live update for specific file types shou= ld > + * register an instance of this structure. LUO uses this registration to > + * determine if a given file can be preserved and to find the appropriate > + * operations to manage its state across the update. > + */ > +struct liveupdate_file_handler { > + const struct liveupdate_file_ops *ops; > + const char compatible[LIVEUPDATE_HNDL_COMPAT_LENGTH]; > + > + /* private: */ > + > + /* > + * Used for linking this handler instance into a global list of > + * registered file handlers. > + */ > + struct list_head __private list; > +}; >=20=20 > #ifdef CONFIG_LIVEUPDATE >=20=20 > @@ -19,6 +104,9 @@ bool liveupdate_enabled(void); > /* Called during kexec to tell LUO that entered into reboot */ > int liveupdate_reboot(void); >=20=20 > +int liveupdate_register_file_handler(struct liveupdate_file_handler *fh); > +int liveupdate_unregister_file_handler(struct liveupdate_file_handler *f= h); > + > #else /* CONFIG_LIVEUPDATE */ >=20=20 > static inline bool liveupdate_enabled(void) > @@ -31,5 +119,15 @@ static inline int liveupdate_reboot(void) > return 0; > } >=20=20 > +static inline int liveupdate_register_file_handler(struct liveupdate_fil= e_handler *fh) > +{ > + return -EOPNOTSUPP; > +} > + > +static inline int liveupdate_unregister_file_handler(struct liveupdate_f= ile_handler *fh) > +{ > + return -EOPNOTSUPP; > +} > + > #endif /* CONFIG_LIVEUPDATE */ > #endif /* _LINUX_LIVEUPDATE_H */ > diff --git a/kernel/liveupdate/Makefile b/kernel/liveupdate/Makefile > index 6af93caa58cf..7cad2eece32d 100644 > --- a/kernel/liveupdate/Makefile > +++ b/kernel/liveupdate/Makefile > @@ -2,6 +2,7 @@ >=20=20 > luo-y :=3D \ > luo_core.o \ > + luo_file.o \ > luo_session.o >=20=20 > obj-$(CONFIG_KEXEC_HANDOVER) +=3D kexec_handover.o > diff --git a/kernel/liveupdate/luo_file.c b/kernel/liveupdate/luo_file.c > new file mode 100644 > index 000000000000..f10d6c37328c > --- /dev/null > +++ b/kernel/liveupdate/luo_file.c > @@ -0,0 +1,882 @@ > +// SPDX-License-Identifier: GPL-2.0 > + > +/* > + * Copyright (c) 2025, Google LLC. > + * Pasha Tatashin > + */ > + > +/** > + * DOC: LUO File Descriptors > + * > + * LUO provides the infrastructure to preserve specific, stateful file > + * descriptors across a kexec-based live update. The primary goal is to = allow > + * workloads, such as virtual machines using vfio, memfd, or iommufd, to > + * retain access to their essential resources without interruption. > + * > + * The framework is built around a callback-based handler model and a we= ll- > + * defined lifecycle for each preserved file. > + * > + * Handler Registration: > + * Kernel modules responsible for a specific file type (e.g., memfd, vfi= o) > + * register a &struct liveupdate_file_handler. This handler provides a s= et of > + * callbacks that LUO invokes at different stages of the update process,= most > + * notably: > + * > + * - can_preserve(): A lightweight check to determine if the handler is > + * compatible with a given 'struct file'. > + * - preserve(): The heavyweight operation that saves the file's state= and > + * returns an opaque u64 handle. This is typically performed while t= he > + * workload is still active to minimize the downtime during the > + * actual reboot transition. > + * - unpreserve(): Cleans up any resources allocated by .preserve(), c= alled > + * if the preservation process is aborted before the reboot (i.e. se= ssion is > + * closed). > + * - freeze(): A final pre-reboot opportunity to prepare the state for= kexec. > + * We are already in reboot syscall, and therefore userspace cannot = mutate > + * the file anymore. > + * - unfreeze(): Undoes the actions of .freeze(), called if the live u= pdate > + * is aborted after the freeze phase. > + * - retrieve(): Reconstructs the file in the new kernel from the pres= erved > + * handle. > + * - finish(): Performs final check and cleanup in the new kernel. Aft= er > + * succesul finish call, LUO gives up ownership to this file. > + * > + * File Preservation Lifecycle happy path: > + * > + * 1. Preserve (Normal Operation): A userspace agent preserves files one= by one > + * via an ioctl. For each file, luo_preserve_file() finds a compatible > + * handler, calls its .preserve() operation, and creates an internal = &struct > + * luo_file to track the live state. > + * > + * 2. Freeze (Pre-Reboot): Just before the kexec, luo_file_freeze() is c= alled. > + * It iterates through all preserved files, calls their respective .f= reeze() > + * operation, and serializes their final metadata (compatible string,= token, > + * and data handle) into a contiguous memory block for KHO. > + * > + * 3. Deserialize: After kexec, luo_file_deserialize() runs when session= gets > + * deserialized (which is when /dev/liveupdate is first opened). It r= eads the > + * serialized data from the KHO memory region and reconstructs the in= -memory > + * list of &struct luo_file instances for the new kernel, linking the= m to > + * their corresponding handlers. > + * > + * 4. Retrieve (New Kernel - Userspace Ready): The userspace agent can n= ow > + * restore file descriptors by providing a token. luo_retrieve_file() > + * searches for the matching token, calls the handler's .retrieve() o= p to > + * re-create the 'struct file', and returns a new FD. Files can be > + * retrieved in ANY order. > + * > + * 5. Finish (New Kernel - Cleanup): Once a session retrival is complete, > + * luo_file_finish() is called. It iterates through all files, invoke= s their > + * .finish() operations for final cleanup, and releases all associate= d kernel > + * resources. > + * > + * File Preservation Lifecycle unhappy paths: > + * > + * 1. Abort Before Reboot: If the userspace agent aborts the live update > + * process before calling reboot (e.g., by closing the session file > + * descriptor), the session's release handler calls > + * luo_file_unpreserve_files(). This invokes the .unpreserve() callba= ck on > + * all preserved files, ensuring all allocated resources are cleaned = up and > + * returning the system to a clean state. > + * > + * 2. Freeze Failure: During the reboot() syscall, if any handler's .fre= eze() > + * op fails, the .unfreeze() op is invoked on all previously *success= ful* > + * freezes to roll back their state. The reboot() syscall then return= s an > + * error to userspace, canceling the live update. > + * > + * 3. Finish Failure: In the new kernel, if a handler's .finish() op fai= ls, > + * the luo_file_finish() operation is aborted. LUO retains ownership = of > + * all files within that session, including those that were not yet > + * processed. The userspace agent can attempt to call the finish oper= ation > + * again later. If the issue cannot be resolved, these resources will= be held > + * by LUO until the next live update cycle, at which point they will = be > + * discarded. > + */ > + > +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include "luo_internal.h" > + > +static LIST_HEAD(luo_file_handler_list); > + > +/* 2 4K pages, give space for 128 files per file_set */ > +#define LUO_FILE_PGCNT 2ul > +#define LUO_FILE_MAX \ > + ((LUO_FILE_PGCNT << PAGE_SHIFT) / sizeof(struct luo_file_ser)) > + > +/** > + * struct luo_file - Represents a single preserved file instance. > + * @fh: Pointer to the &struct liveupdate_file_handler that m= anages > + * this type of file. > + * @file: Pointer to the kernel's &struct file that is being pr= eserved. > + * This is NULL in the new kernel until the file is succ= essfully > + * retrieved. > + * @serialized_data: The opaque u64 handle to the serialized state of th= e file. > + * This handle is passed back to the handler's .freeze(), > + * .retrieve(), and .finish() callbacks, allowing it to = track > + * and update its serialized state across phases. > + * @retrieved: A flag indicating whether a user/kernel in the new ke= rnel has > + * successfully called retrieve() on this file. This pre= vents > + * multiple retrieval attempts. > + * @mutex: A mutex that protects the fields of this specific ins= tance > + * (e.g., @retrieved, @file), ensuring that operations l= ike > + * retrieving or finishing a file are atomic. > + * @list: The list_head linking this instance into its parent > + * file_set's list of preserved files. > + * @token: The user-provided unique token used to identify this = file. > + * > + * This structure is the core in-kernel representation of a single file = being > + * managed through a live update. An instance is created by luo_preserve= _file() > + * to link a 'struct file' to its corresponding handler, a user-provided= token, > + * and the serialized state handle returned by the handler's .preserve() > + * operation. > + * > + * These instances are tracked in a per-file_set list. The @serialized_d= ata > + * field, which holds a handle to the file's serialized state, may be up= dated > + * during the .freeze() callback before being serialized for the next ke= rnel. > + * After reboot, these structures are recreated by luo_file_deserialize(= ) and > + * are finally cleaned up by luo_file_finish(). > + */ > +struct luo_file { > + struct liveupdate_file_handler *fh; > + struct file *file; > + u64 serialized_data; > + bool retrieved; > + struct mutex mutex; > + struct list_head list; > + u64 token; > +}; > + > +static int luo_alloc_files_mem(struct luo_file_set *file_set) > +{ > + size_t size; > + void *mem; > + > + if (file_set->files) > + return 0; > + > + WARN_ON_ONCE(file_set->count); > + > + size =3D LUO_FILE_PGCNT << PAGE_SHIFT; > + mem =3D kho_alloc_preserve(size); > + if (IS_ERR(mem)) > + return PTR_ERR(mem); > + > + file_set->files =3D mem; > + > + return 0; > +} > + > +static void luo_free_files_mem(struct luo_file_set *file_set) > +{ > + /* If file_set has files, no need to free preservation memory */ > + if (file_set->count) > + return; > + > + if (!file_set->files) > + return; > + > + kho_unpreserve_free(file_set->files); > + file_set->files =3D NULL; > +} > + > +static bool luo_token_is_used(struct luo_file_set *file_set, u64 token) > +{ > + struct luo_file *iter; > + > + list_for_each_entry(iter, &file_set->files_list, list) { > + if (iter->token =3D=3D token) > + return true; > + } > + > + return false; > +} > + > +/** > + * luo_preserve_file - Initiate the preservation of a file descriptor. > + * @file_set: The file_set to which the preserved file will be added. > + * @token: A unique, user-provided identifier for the file. > + * @fd: The file descriptor to be preserved. > + * > + * This function orchestrates the first phase of preserving a file. Upon= entry, > + * it takes a reference to the 'struct file' via fget(), effectively mak= ing LUO > + * a co-owner of the file. This reference is held until the file is eith= er > + * unpreserved or successfully finished in the next kernel, preventing t= he file > + * from being prematurely destroyed. > + * > + * This function orchestrates the first phase of preserving a file. It p= erforms > + * the following steps: > + * > + * 1. Validates that the @token is not already in use within the file_se= t. > + * 2. Ensures the file_set's memory for files serialization is allocated > + * (allocates if needed). > + * 3. Iterates through registered handlers, calling can_preserve() to fi= nd one > + * compatible with the given @fd. > + * 4. Calls the handler's .preserve() operation, which saves the file's = state > + * and returns an opaque private data handle. > + * 5. Adds the new instance to the file_set's internal list. > + * > + * On success, LUO takes a reference to the 'struct file' and considers = it > + * under its management until it is unpreserved or finished. > + * > + * In case of any failure, all intermediate allocations (file reference,= memory > + * for the 'luo_file' struct, etc.) are cleaned up before returning an e= rror. > + * > + * Context: Can be called from an ioctl handler during normal system ope= ration. > + * Return: 0 on success. Returns a negative errno on failure: > + * -EEXIST if the token is already used. > + * -EBADF if the file descriptor is invalid. > + * -ENOSPC if the file_set is full. > + * -ENOENT if no compatible handler is found. > + * -ENOMEM on memory allocation failure. > + * Other erros might be returned by .preserve(). > + */ > +int luo_preserve_file(struct luo_file_set *file_set, u64 token, int fd) > +{ > + struct liveupdate_file_op_args args =3D {0}; > + struct liveupdate_file_handler *fh; > + struct luo_file *luo_file; > + struct file *file; > + int err; > + > + if (luo_token_is_used(file_set, token)) > + return -EEXIST; > + > + file =3D fget(fd); > + if (!file) > + return -EBADF; > + > + err =3D luo_alloc_files_mem(file_set); > + if (err) > + goto err_files_mem; Nit: ^^ two spaces here. > + > + if (file_set->count =3D=3D LUO_FILE_MAX) { > + err =3D -ENOSPC; > + goto err_files_mem; > + } > + > + err =3D -ENOENT; > + luo_list_for_each_private(fh, &luo_file_handler_list, list) { > + if (fh->ops->can_preserve(fh, file)) { > + err =3D 0; > + break; > + } > + } > + > + /* err is still -ENOENT if no handler was found */ > + if (err) > + goto err_files_mem; > + > + luo_file =3D kzalloc(sizeof(*luo_file), GFP_KERNEL); > + if (!luo_file) { > + err =3D -ENOMEM; > + goto err_files_mem; > + } > + > + luo_file->file =3D file; > + luo_file->fh =3D fh; > + luo_file->token =3D token; > + luo_file->retrieved =3D false; > + mutex_init(&luo_file->mutex); > + > + args.handler =3D fh; > + args.file =3D file; > + err =3D fh->ops->preserve(&args); > + if (err) > + goto err_kfree; > + > + luo_file->serialized_data =3D args.serialized_data; > + list_add_tail(&luo_file->list, &file_set->files_list); > + file_set->count++; > + > + return 0; > + > +err_kfree: > + mutex_destroy(&luo_file->mutex); > + kfree(luo_file); > +err_files_mem: > + fput(file); > + luo_free_files_mem(file_set); > + > + return err; > +} > + > +/** > + * luo_file_unpreserve_files - Unpreserves all files from a file_set. > + * @file_set: The files to be cleaned up. > + * > + * This function serves as the primary cleanup path for a file_set. It is > + * invoked when the userspace agent closes the file_set's file descripto= r. > + * > + * For each file, it performs the following cleanup actions: > + * 1. Calls the handler's .unpreserve() callback to allow the handler = to > + * release any resources it allocated. > + * 2. Removes the file from the file_set's internal tracking list. > + * 3. Releases the reference to the 'struct file' that was taken by > + * luo_preserve_file() via fput(), returning ownership. > + * 4. Frees the memory associated with the internal 'struct luo_file'. > + * > + * After all individual files are unpreserved, it frees the contiguous m= emory > + * block that was allocated to hold their serialization data. > + */ > +void luo_file_unpreserve_files(struct luo_file_set *file_set) > +{ > + struct luo_file *luo_file; > + > + while (!list_empty(&file_set->files_list)) { > + struct liveupdate_file_op_args args =3D {0}; > + > + luo_file =3D list_last_entry(&file_set->files_list, > + struct luo_file, list); > + > + args.handler =3D luo_file->fh; > + args.file =3D luo_file->file; > + args.serialized_data =3D luo_file->serialized_data; > + luo_file->fh->ops->unpreserve(&args); > + > + list_del(&luo_file->list); > + file_set->count--; > + > + fput(luo_file->file); > + mutex_destroy(&luo_file->mutex); > + kfree(luo_file); > + } > + > + luo_free_files_mem(file_set); > +} > + > +static int luo_file_freeze_one(struct luo_file_set *file_set, > + struct luo_file *luo_file) > +{ > + int err =3D 0; > + > + guard(mutex)(&luo_file->mutex); > + > + if (luo_file->fh->ops->freeze) { > + struct liveupdate_file_op_args args =3D {0}; > + > + args.handler =3D luo_file->fh; > + args.file =3D luo_file->file; > + args.serialized_data =3D luo_file->serialized_data; > + > + err =3D luo_file->fh->ops->freeze(&args); > + if (!err) > + luo_file->serialized_data =3D args.serialized_data; > + } > + > + return err; > +} > + > +static void luo_file_unfreeze_one(struct luo_file_set *file_set, > + struct luo_file *luo_file) > +{ > + guard(mutex)(&luo_file->mutex); > + > + if (luo_file->fh->ops->unfreeze) { > + struct liveupdate_file_op_args args =3D {0}; > + > + args.handler =3D luo_file->fh; > + args.file =3D luo_file->file; > + args.serialized_data =3D luo_file->serialized_data; > + > + luo_file->fh->ops->unfreeze(&args); > + } > + > + luo_file->serialized_data =3D 0; > +} > + > +static void __luo_file_unfreeze(struct luo_file_set *file_set, > + struct luo_file *failed_entry) > +{ > + struct list_head *files_list =3D &file_set->files_list; > + struct luo_file *luo_file; > + > + list_for_each_entry(luo_file, files_list, list) { > + if (luo_file =3D=3D failed_entry) > + break; > + > + luo_file_unfreeze_one(file_set, luo_file); > + } > + > + memset(file_set->files, 0, LUO_FILE_PGCNT << PAGE_SHIFT); > +} > + > +/** > + * luo_file_freeze - Freezes all preserved files and serializes their me= tadata. > + * @file_set: The file_set whose files are to be frozen. > + * @file_set_ser: Where to put the serialized file_set. > + * > + * This function is called from the reboot() syscall path, just before t= he > + * kernel transitions to the new image via kexec. Its purpose is to perf= orm the > + * final preparation and serialization of all preserved files in the fil= e_set. > + * > + * It iterates through each preserved file in FIFO order (the order of > + * preservation) and performs two main actions: > + * > + * 1. Freezes the File: It calls the handler's .freeze() callback for ea= ch > + * file. This gives the handler a final opportunity to quiesce the de= vice or > + * prepare its state for the upcoming reboot. The handler may update = its > + * private data handle during this step. > + * > + * 2. Serializes Metadata: After a successful freeze, it copies the fina= l file > + * metadata=E2=80=94the handler's compatible string, the user token, = and the final > + * private data handle=E2=80=94into the pre-allocated contiguous memo= ry buffer > + * (file_set->files) that will be handed over to the next kernel via = KHO. > + * > + * Error Handling (Rollback): > + * This function is atomic. If any handler's .freeze() operation fails, = the > + * entire live update is aborted. The __luo_file_unfreeze() helper is > + * immediately called to invoke the .unfreeze() op on all files that were > + * successfully frozen before the point of failure, rolling them back to= a > + * running state. The function then returns an error, causing the reboot= () > + * syscall to fail. > + * > + * Context: Called only from the liveupdate_reboot() path. > + * Return: 0 on success, or a negative errno on failure. > + */ > +int luo_file_freeze(struct luo_file_set *file_set, > + struct luo_file_set_ser *file_set_ser) > +{ > + struct luo_file_ser *file_ser =3D file_set->files; > + struct luo_file *luo_file; > + int err; > + int i; > + > + if (!file_set->count) > + return 0; > + > + if (WARN_ON(!file_ser)) > + return -EINVAL; > + > + i =3D 0; > + list_for_each_entry(luo_file, &file_set->files_list, list) { > + err =3D luo_file_freeze_one(file_set, luo_file); > + if (err < 0) { > + pr_warn("Freeze failed for token[%#0llx] handler[%s] err[%pe]\n", > + luo_file->token, luo_file->fh->compatible, > + ERR_PTR(err)); > + goto err_unfreeze; > + } > + > + strscpy(file_ser[i].compatible, luo_file->fh->compatible, > + sizeof(file_ser[i].compatible)); > + file_ser[i].data =3D luo_file->serialized_data; > + file_ser[i].token =3D luo_file->token; > + i++; > + } > + > + file_set_ser->count =3D file_set->count; > + if (file_set->files) > + file_set_ser->files =3D virt_to_phys(file_set->files); > + > + return 0; > + > +err_unfreeze: > + __luo_file_unfreeze(file_set, luo_file); > + > + return err; > +} > + > +/** > + * luo_file_unfreeze - Unfreezes all files in a file_set and clear seria= lization > + * @file_set: The file_set whose files are to be unfrozen. > + * @file_set_ser: Serialized file_set. > + * > + * This function rolls back the state of all files in a file_set after t= he > + * freeze phase has begun but must be aborted. It is the counterpart to > + * luo_file_freeze(). > + * > + * It invokes the __luo_file_unfreeze() helper with a NULL argument, whi= ch > + * signals the helper to iterate through all files in the file_set and c= all > + * their respective .unfreeze() handler callbacks. > + * > + * Context: This is called when the live update is aborted during > + * the reboot() syscall, after luo_file_freeze() has been calle= d. > + */ > +void luo_file_unfreeze(struct luo_file_set *file_set, > + struct luo_file_set_ser *file_set_ser) > +{ > + if (!file_set->count) > + return; > + > + __luo_file_unfreeze(file_set, NULL); > + memset(file_set_ser, 0, sizeof(*file_set_ser)); > +} > + > +/** > + * luo_retrieve_file - Restores a preserved file from a file_set by its = token. > + * @file_set: The file_set from which to retrieve the file. > + * @token: The unique token identifying the file to be restored. > + * @filep: Output parameter; on success, this is populated with a poi= nter > + * to the newly retrieved 'struct file'. > + * > + * This function is the primary mechanism for recreating a file in the n= ew > + * kernel after a live update. It searches the file_set's list of deseri= alized > + * files for an entry matching the provided @token. > + * > + * The operation is idempotent: if a file has already been successfully > + * retrieved, this function will simply return a pointer to the existing > + * 'struct file' and report success without re-executing the retrieve > + * operation. This is handled by checking the 'retrieved' flag under a l= ock. > + * > + * File retrieval can happen in any order; it is not bound by the order = of > + * preservation. > + * > + * Context: Can be called from an ioctl or other in-kernel code in the n= ew > + * kernel. > + * Return: 0 on success. Returns a negative errno on failure: > + * -ENOENT if no file with the matching token is found. > + * Any error code returned by the handler's .retrieve() op. > + */ > +int luo_retrieve_file(struct luo_file_set *file_set, u64 token, > + struct file **filep) > +{ > + struct liveupdate_file_op_args args =3D {0}; > + struct luo_file *luo_file; > + int err; > + > + if (list_empty(&file_set->files_list)) > + return -ENOENT; > + > + list_for_each_entry(luo_file, &file_set->files_list, list) { > + if (luo_file->token =3D=3D token) > + break; > + } > + > + if (luo_file->token !=3D token) > + return -ENOENT; > + > + guard(mutex)(&luo_file->mutex); > + if (luo_file->retrieved) { > + /* > + * Someone is asking for this file again, so get a reference > + * for them. > + */ > + get_file(luo_file->file); > + *filep =3D luo_file->file; > + return 0; > + } > + > + args.handler =3D luo_file->fh; > + args.serialized_data =3D luo_file->serialized_data; > + err =3D luo_file->fh->ops->retrieve(&args); > + if (!err) { > + luo_file->file =3D args.file; > + > + /* Get reference so we can keep this file in LUO until finish */ > + get_file(luo_file->file); > + *filep =3D luo_file->file; > + luo_file->retrieved =3D true; > + } > + > + return err; > +} > + > +static int luo_file_can_finish_one(struct luo_file_set *file_set, > + struct luo_file *luo_file) > +{ > + bool can_finish =3D true; > + > + guard(mutex)(&luo_file->mutex); > + > + if (luo_file->fh->ops->can_finish) { > + struct liveupdate_file_op_args args =3D {0}; > + > + args.handler =3D luo_file->fh; > + args.file =3D luo_file->file; > + args.serialized_data =3D luo_file->serialized_data; > + args.retrieved =3D luo_file->retrieved; > + can_finish =3D luo_file->fh->ops->can_finish(&args); > + } > + > + return can_finish ? 0 : -EBUSY; > +} > + > +static void luo_file_finish_one(struct luo_file_set *file_set, > + struct luo_file *luo_file) > +{ > + struct liveupdate_file_op_args args =3D {0}; > + > + guard(mutex)(&luo_file->mutex); > + > + args.handler =3D luo_file->fh; > + args.file =3D luo_file->file; > + args.serialized_data =3D luo_file->serialized_data; > + args.retrieved =3D luo_file->retrieved; > + > + luo_file->fh->ops->finish(&args); > +} > + > +/** > + * luo_file_finish - Completes the lifecycle for all files in a file_set. > + * @file_set: The file_set to be finalized. > + * > + * This function orchestrates the final teardown of a live update file_s= et in > + * the new kernel. It should be called after all necessary files have be= en > + * retrieved and the userspace agent is ready to release the preserved s= tate. > + * > + * The function iterates through all tracked files. For each file, it pe= rforms > + * the following sequence of cleanup actions: > + * > + * 1. If file is not yet retrieved, retrieves it, and calls can_finish()= on > + * every file in the file_set. If all can_finish return true, continu= e to > + * finish. > + * 2. Calls the handler's .finish() callback (via luo_file_finish_one) to > + * allow for final resource cleanup within the handler. > + * 3. Releases LUO's ownership reference on the 'struct file' via fput()= . This > + * is the counterpart to the get_file() call in luo_retrieve_file(). > + * 4. Removes the 'struct luo_file' from the file_set's internal list. > + * 5. Frees the memory for the 'struct luo_file' instance itself. > + * > + * After successfully finishing all individual files, it frees the > + * contiguous memory block that was used to transfer the serialized meta= data > + * from the previous kernel. > + * > + * Error Handling (Atomic Failure): > + * This operation is atomic. If any handler's .can_finish() op fails, th= e entire > + * function aborts immediately and returns an error. > + * > + * Context: Can be called from an ioctl handler in the new kernel. > + * Return: 0 on success, or a negative errno on failure. > + */ > +int luo_file_finish(struct luo_file_set *file_set) > +{ > + struct list_head *files_list =3D &file_set->files_list; > + struct luo_file *luo_file; > + int err; > + > + if (!file_set->count) > + return 0; > + > + list_for_each_entry(luo_file, files_list, list) { > + err =3D luo_file_can_finish_one(file_set, luo_file); > + if (err) > + return err; > + } > + > + while (!list_empty(&file_set->files_list)) { > + luo_file =3D list_last_entry(&file_set->files_list, > + struct luo_file, list); > + > + luo_file_finish_one(file_set, luo_file); > + > + if (luo_file->file) > + fput(luo_file->file); > + list_del(&luo_file->list); > + file_set->count--; > + mutex_destroy(&luo_file->mutex); > + kfree(luo_file); > + } > + > + if (file_set->files) { > + kho_restore_free(file_set->files); > + file_set->files =3D NULL; > + } > + > + return 0; > +} > + > +/** > + * luo_file_deserialize - Reconstructs the list of preserved files in th= e new kernel. > + * @file_set: The incoming file_set to fill with deserialized data. > + * @file_set_ser: Serialized KHO file_set data from the previous kernel. > + * > + * This function is called during the early boot process of the new kern= el. It > + * takes the raw, contiguous memory block of 'struct luo_file_ser' entri= es, > + * provided by the previous kernel, and transforms it back into a live, > + * in-memory linked list of 'struct luo_file' instances. > + * > + * For each serialized entry, it performs the following steps: > + * 1. Reads the 'compatible' string. > + * 2. Searches the global list of registered file handlers for one that > + * matches the compatible string. > + * 3. Allocates a new 'struct luo_file'. > + * 4. Populates the new structure with the deserialized data (token, p= rivate > + * data handle) and links it to the found handler. The 'file' point= er is > + * initialized to NULL, as the file has not been retrieved yet. > + * 5. Adds the new 'struct luo_file' to the file_set's files_list. > + * > + * This prepares the file_set for userspace, which can later call > + * luo_retrieve_file() to restore the actual file descriptors. > + * > + * Context: Called from session deserialization. > + */ > +int luo_file_deserialize(struct luo_file_set *file_set, > + struct luo_file_set_ser *file_set_ser) > +{ > + struct luo_file_ser *file_ser; > + u64 i; > + > + if (!file_set_ser->files) { > + WARN_ON(file_set_ser->count); > + return 0; > + } > + > + file_set->count =3D file_set_ser->count; > + file_set->files =3D phys_to_virt(file_set_ser->files); > + > + /* > + * Note on error handling: > + * > + * If deserialization fails (e.g., allocation failure or corrupt data), > + * we intentionally skip cleanup of files that were already restored. > + * > + * A partial failure leaves the preserved state inconsistent. > + * Implementing a safe "undo" to unwind complex dependencies (sessions, > + * files, hardware state) is error-prone and provides little value, as > + * the system is effectively in a broken state. > + * > + * We treat these resources as leaked. The expected recovery path is for > + * userspace to detect the failure and trigger a reboot, which will > + * reliably reset devices and reclaim memory. > + */ > + file_ser =3D file_set->files; > + for (i =3D 0; i < file_set->count; i++) { > + struct liveupdate_file_handler *fh; > + bool handler_found =3D false; > + struct luo_file *luo_file; > + > + luo_list_for_each_private(fh, &luo_file_handler_list, list) { > + if (!strcmp(fh->compatible, file_ser[i].compatible)) { > + handler_found =3D true; > + break; > + } > + } > + > + if (!handler_found) { > + pr_warn("No registered handler for compatible '%s'\n", > + file_ser[i].compatible); > + return -ENOENT; > + } > + > + luo_file =3D kzalloc(sizeof(*luo_file), GFP_KERNEL); > + if (!luo_file) > + return -ENOMEM; > + > + luo_file->fh =3D fh; > + luo_file->file =3D NULL; > + luo_file->serialized_data =3D file_ser[i].data; > + luo_file->token =3D file_ser[i].token; > + luo_file->retrieved =3D false; > + mutex_init(&luo_file->mutex); > + list_add_tail(&luo_file->list, &file_set->files_list); > + } > + > + return 0; > +} > + > +void luo_file_set_init(struct luo_file_set *file_set) > +{ > + INIT_LIST_HEAD(&file_set->files_list); > +} > + > +void luo_file_set_destroy(struct luo_file_set *file_set) > +{ > + WARN_ON(file_set->count); > + WARN_ON(!list_empty(&file_set->files_list)); > +} > + > +/** > + * liveupdate_register_file_handler - Register a file handler with LUO. > + * @fh: Pointer to a caller-allocated &struct liveupdate_file_handler. > + * The caller must initialize this structure, including a unique > + * 'compatible' string and a valid 'fh' callbacks. This function adds the > + * handler to the global list of supported file handlers. > + * > + * Context: Typically called during module initialization for file types= that > + * support live update preservation. > + * > + * Return: 0 on success. Negative errno on failure. > + */ > +int liveupdate_register_file_handler(struct liveupdate_file_handler *fh) > +{ > + struct liveupdate_file_handler *fh_iter; > + int err; > + > + if (!liveupdate_enabled()) > + return -EOPNOTSUPP; > + > + /* Sanity check that all required callbacks are set */ > + if (!fh->ops->preserve || !fh->ops->unpreserve || > + !fh->ops->retrieve || !fh->ops->finish) { You are still missing a check for can_preserve() here. It is a mandatory callback and luo_preserve_file() calls it without checking for NULL. With these and Mike's comments addressed, Reviewed-by: Pratyush Yadav > + return -EINVAL; > + } > + > + /* > + * Ensure the system is quiescent (no active sessions). > + * This prevents registering new handlers while sessions are active or > + * while deserialization is in progress. > + */ > + if (!luo_session_quiesce()) > + return -EBUSY; > + > + /* Check for duplicate compatible strings */ > + luo_list_for_each_private(fh_iter, &luo_file_handler_list, list) { > + if (!strcmp(fh_iter->compatible, fh->compatible)) { > + pr_err("File handler registration failed: Compatible string '%s' alre= ady registered.\n", > + fh->compatible); > + err =3D -EEXIST; > + goto err_resume; > + } > + } > + > + /* Pin the module implementing the handler */ > + if (!try_module_get(fh->ops->owner)) { > + err =3D -EAGAIN; > + goto err_resume; > + } > + > + INIT_LIST_HEAD(&ACCESS_PRIVATE(fh, list)); > + list_add_tail(&ACCESS_PRIVATE(fh, list), &luo_file_handler_list); > + luo_session_resume(); > + > + return 0; > + > +err_resume: > + luo_session_resume(); > + return err; > +} > + > +/** > + * liveupdate_unregister_file_handler - Unregister a liveupdate file han= dler > + * @fh: The file handler to unregister > + * > + * Unregisters the file handler from the liveupdate core. This function > + * reverses the operations of liveupdate_register_file_handler(). > + * > + * It ensures safe removal by checking that: > + * No live update session is currently in progress. > + * > + * If the unregistration fails, the internal test state is reverted. > + * > + * Return: 0 Success. -EOPNOTSUPP when live update is not enabled. -EBUS= Y A live > + * update is in progress, can't quiesce live update. > + */ > +int liveupdate_unregister_file_handler(struct liveupdate_file_handler *f= h) > +{ > + if (!liveupdate_enabled()) > + return -EOPNOTSUPP; > + > + if (!luo_session_quiesce()) > + return -EBUSY; > + > + list_del(&ACCESS_PRIVATE(fh, list)); > + module_put(fh->ops->owner); > + luo_session_resume(); > + > + return 0; > +} > diff --git a/kernel/liveupdate/luo_internal.h b/kernel/liveupdate/luo_int= ernal.h > index 1292ac47eef8..c8973b543d1d 100644 > --- a/kernel/liveupdate/luo_internal.h > +++ b/kernel/liveupdate/luo_internal.h > @@ -40,6 +40,28 @@ static inline int luo_ucmd_respond(struct luo_ucmd *uc= md, > */ > #define luo_restore_fail(__fmt, ...) panic(__fmt, ##__VA_ARGS__) >=20=20 > +/* Mimics list_for_each_entry() but for private list head entries */ > +#define luo_list_for_each_private(pos, head, member) \ > + for (struct list_head *__iter =3D (head)->next; \ > + __iter !=3D (head) && \ > + ({ pos =3D container_of(__iter, typeof(*(pos)), member); 1; }); \ > + __iter =3D __iter->next) > + > +/** > + * struct luo_file_set - A set of files that belong to the same sessions. > + * @files_list: An ordered list of files associated with this session, i= t is > + * ordered by preservation time. > + * @files: The physically contiguous memory block that holds the se= rialized > + * state of files. > + * @count: A counter tracking the number of files currently stored = in the > + * @files_list for this session. > + */ > +struct luo_file_set { > + struct list_head files_list; > + struct luo_file_ser *files; > + long count; > +}; > + > /** > * struct luo_session - Represents an active or incoming Live Update ses= sion. > * @name: A unique name for this session, used for identification = and > @@ -50,6 +72,7 @@ static inline int luo_ucmd_respond(struct luo_ucmd *ucm= d, > * previous kernel) sessions. > * @retrieved: A boolean flag indicating whether this session has been > * retrieved by a consumer in the new kernel. > + * @file_set: A set of files that belong to this session. > * @mutex: protects fields in the luo_session. > */ > struct luo_session { > @@ -57,6 +80,7 @@ struct luo_session { > struct luo_session_ser *ser; > struct list_head list; > bool retrieved; > + struct luo_file_set file_set; > struct mutex mutex; > }; >=20=20 > @@ -69,4 +93,18 @@ int luo_session_deserialize(void); > bool luo_session_quiesce(void); > void luo_session_resume(void); >=20=20 > +int luo_preserve_file(struct luo_file_set *file_set, u64 token, int fd); > +void luo_file_unpreserve_files(struct luo_file_set *file_set); > +int luo_file_freeze(struct luo_file_set *file_set, > + struct luo_file_set_ser *file_set_ser); > +void luo_file_unfreeze(struct luo_file_set *file_set, > + struct luo_file_set_ser *file_set_ser); > +int luo_retrieve_file(struct luo_file_set *file_set, u64 token, > + struct file **filep); > +int luo_file_finish(struct luo_file_set *file_set); > +int luo_file_deserialize(struct luo_file_set *file_set, > + struct luo_file_set_ser *file_set_ser); > +void luo_file_set_init(struct luo_file_set *file_set); > +void luo_file_set_destroy(struct luo_file_set *file_set); > + > #endif /* _LINUX_LUO_INTERNAL_H */ --=20 Regards, Pratyush Yadav