[RFC] Modernizing Linux authentication logs (lastlog, btmp, utmp, wtmp) with SQLite

[RFC] Modernizing Linux authentication logs (lastlog, btmp, utmp, wtmp) with SQLite

[Roman Bakshansky]

Hi all,

I'd like to share a draft RFC proposing a complete overhaul of the legacy
binary logs used for authentication auditing in Linux: lastlog, btmp, utmp,
and wtmp.

These files, designed decades ago, are running into fundamental limitations:

- Y2038 problem - they use 32-bit timestamps (time_t in lastlog,
   tv_sec in utmpx). Even on 64-bit systems the fields remain 32-bit
   due to ABI constraints, so all Linux systems are affected.
- No extensibility - any new field (e.g., container ID, service name,
   source IP) requires changing fixed structures, breaking all existing
   tools that read them.
- Poor query performance - tools like last, lastb, who have to
   scan whole files linearly; with millions of records this becomes
   painfully slow.
- No atomicity - partial writes during a crash can corrupt logs.
- Concurrency bottlenecks - multiple writers (sshd, login, etc.)
   contend for the same file with coarse locking.

To address this once and for all, the RFC proposes replacing these logs
with dedicated shared libraries that use SQLite as the storage backend:

- liblastlog2 - last login time
- libbtmp2    - failed login attempts
- libutmp2    - current sessions
- libwtmp2    - login/logout history

SQLite brings:
- 64-bit time -> Y2038 solved forever.
- Indexes -> O(log N) queries instead of full scans.
- Extensible schema -> new fields can be added without breaking old tools.
- ACID and WAL mode -> atomic writes and concurrent access.
- Portability - runs on any Linux system, no systemd dependency.

The full RFC, including preliminary database schemas and API drafts,
is available in the discussion repository:

     https://github.com/bakshansky/linux-auth-logs

I'm looking for feedback on the overall direction, the proposed
interfaces, and the open questions listed in the document (e.g.,
library naming, database location, fallback options for embedded
systems). Please use GitHub Issues for comments, or reply to this
thread - I'll monitor both.

Thanks for your time and input!

Re: [RFC] Modernizing Linux authentication logs (lastlog, btmp, utmp, wtmp) with SQLite

[Adhemerval Zanella Netto]

On 12/03/26 18:01, Roman Bakshansky wrote:
> Hi all,
> 
> I'd like to share a draft RFC proposing a complete overhaul of the legacy
> binary logs used for authentication auditing in Linux: lastlog, btmp, utmp,
> and wtmp.
> 
> These files, designed decades ago, are running into fundamental limitations:
> 
> - Y2038 problem - they use 32-bit timestamps (time_t in lastlog,
>   tv_sec in utmpx). Even on 64-bit systems the fields remain 32-bit
>   due to ABI constraints, so all Linux systems are affected.
> - No extensibility - any new field (e.g., container ID, service name,
>   source IP) requires changing fixed structures, breaking all existing
>   tools that read them.
> - Poor query performance - tools like last, lastb, who have to
>   scan whole files linearly; with millions of records this becomes
>   painfully slow.
> - No atomicity - partial writes during a crash can corrupt logs.
> - Concurrency bottlenecks - multiple writers (sshd, login, etc.)
>   contend for the same file with coarse locking.
> 
> To address this once and for all, the RFC proposes replacing these logs
> with dedicated shared libraries that use SQLite as the storage backend:
> 
> - liblastlog2 - last login time
> - libbtmp2    - failed login attempts
> - libutmp2    - current sessions
> - libwtmp2    - login/logout history
> 
> SQLite brings:
> - 64-bit time -> Y2038 solved forever.
> - Indexes -> O(log N) queries instead of full scans.
> - Extensible schema -> new fields can be added without breaking old tools.
> - ACID and WAL mode -> atomic writes and concurrent access.
> - Portability - runs on any Linux system, no systemd dependency.
> 
> The full RFC, including preliminary database schemas and API drafts,
> is available in the discussion repository:
> 
>     https://github.com/bakshansky/linux-auth-logs
> 
> I'm looking for feedback on the overall direction, the proposed
> interfaces, and the open questions listed in the document (e.g.,
> library naming, database location, fallback options for embedded
> systems). Please use GitHub Issues for comments, or reply to this
> thread - I'll monitor both.
> 
> Thanks for your time and input!

From the glibc standpoint my plan is just to make the accounting database
function no-op [1] (I hopefully to get this in the next 2.44 release).

And I think Thorsten Kukuk already adapted most of the usages in current
distros [2][3] using similar strategy, along with a better systemd
integration.  I am not sure if/when distros are incorporating his work.

[1] https://patchwork.sourceware.org/project/glibc/list/?series=37271
[2] https://www.thkukuk.de/blog/Y2038_glibc_lastlog_64bit/
[3] https://www.thkukuk.de/blog/Y2038_glibc_utmp_64bit/

Re: [RFC] Modernizing Linux authentication logs (lastlog, btmp, utmp, wtmp) with SQLite

[Vincent Lefevre]

On 2026-03-13 10:59:11 -0300, Adhemerval Zanella Netto wrote:
> From the glibc standpoint my plan is just to make the accounting database
> function no-op [1] (I hopefully to get this in the next 2.44 release).
> 
> And I think Thorsten Kukuk already adapted most of the usages in current
> distros [2][3] using similar strategy, along with a better systemd
> integration.  I am not sure if/when distros are incorporating his work.
> 
> [1] https://patchwork.sourceware.org/project/glibc/list/?series=37271
> [2] https://www.thkukuk.de/blog/Y2038_glibc_lastlog_64bit/
> [3] https://www.thkukuk.de/blog/Y2038_glibc_utmp_64bit/

FYI, utmp has been reintroduced in Debian for libutempter (and thus
applications that use this library), because systemd was not working
or at least not sufficiently documented:

  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1125682

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / Pascaline project (LIP, ENS-Lyon)

Re: [RFC] Modernizing Linux authentication logs (lastlog, btmp, utmp, wtmp) with SQLite

[Roman Bakshansky]

On 3/13/26 5:45 PM, Vincent Lefevre wrote:
> FYI, utmp has been reintroduced in Debian for libutempter (and thus
> applications that use this library), because systemd was not working
> or at least not sufficiently documented:
>
>    https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1125682
Thank you for the link to the Debian bug. This example has further 
convinced me that I am on the right track and that such an RFC is indeed 
needed by the community.

Re: [RFC] Modernizing Linux authentication logs (lastlog, btmp, utmp, wtmp) with SQLite

[Roman Bakshansky]

On 3/13/26 4:59 PM, Adhemerval Zanella Netto wrote:
>  From the glibc standpoint my plan is just to make the accounting database
> function no-op [1] (I hopefully to get this in the next 2.44 release).
>
> And I think Thorsten Kukuk already adapted most of the usages in current
> distros [2][3] using similar strategy, along with a better systemd
> integration.  I am not sure if/when distros are incorporating his work.
>
> [1] https://patchwork.sourceware.org/project/glibc/list/?series=37271
> [2] https://www.thkukuk.de/blog/Y2038_glibc_lastlog_64bit/
> [3] https://www.thkukuk.de/blog/Y2038_glibc_utmp_64bit/
Thank you for the links and information.

I have been following Thorsten Kukuk's work. I know that liblastlog2 is 
already integrated into util-linux and successfully solves the Y2038 
problem for lastlog. I am also aware of his wtmpdb project. These are 
important steps in the right direction.

My RFC proposes to go further and provide a unified solution for all 
four files: lastlog, btmp, utmp, wtmp. The idea is to create a set of 
public libraries (liblastlog2, libbtmp2, libutmp2, libwtmp2) with a 
consistent C interface, built on SQLite. This would provide 64-bit 
timestamps, indexes for fast queries, ACID transactions, and schema 
extensibility without breaking backward compatibility.

Re: [RFC] Modernizing Linux authentication logs (lastlog, btmp, utmp, wtmp) with SQLite

[Adhemerval Zanella Netto]

On 13/03/26 14:48, Roman Bakshansky wrote:
> On 3/13/26 4:59 PM, Adhemerval Zanella Netto wrote:
>>  From the glibc standpoint my plan is just to make the accounting database
>> function no-op [1] (I hopefully to get this in the next 2.44 release).
>>
>> And I think Thorsten Kukuk already adapted most of the usages in current
>> distros [2][3] using similar strategy, along with a better systemd
>> integration.  I am not sure if/when distros are incorporating his work.
>>
>> [1] https://patchwork.sourceware.org/project/glibc/list/?series=37271
>> [2] https://www.thkukuk.de/blog/Y2038_glibc_lastlog_64bit/
>> [3] https://www.thkukuk.de/blog/Y2038_glibc_utmp_64bit/
> Thank you for the links and information.
> 
> I have been following Thorsten Kukuk's work. I know that liblastlog2 is already integrated into util-linux and successfully solves the Y2038 problem for lastlog. I am also aware of his wtmpdb project. These are important steps in the right direction.
> 
> My RFC proposes to go further and provide a unified solution for all four files: lastlog, btmp, utmp, wtmp. The idea is to create a set of public libraries (liblastlog2, libbtmp2, libutmp2, libwtmp2) with a consistent C interface, built on SQLite. This would provide 64-bit timestamps, indexes for fast queries, ACID transactions, and schema extensibility without breaking backward compatibility.

I think you will need to check with the systemd developers whether they are 
willing to integrate this idea into their plans (if they haven't already). I 
recall that systemd is extensive in some form.  My understanding is that 
eventually all distros, at least the ones that are systemd-based, will use 
the systemd-provided framework instead of relying on different libraries.

From the glibc standpoint, providing this was always problematic, and I 
shared Rich's view (musl creator) that the interface is insecure and legacy [1]. 
 If I were to implement something not systemd-based, I would follow the 
direction that utmps used[2]: a daemon as the only authority to handle the 
files and all clients (utmp{x}.h} uses will just query through IPC.

It means that users should not care whether how or where the database is 
placed (the _PATH_UTM and related macro should be handled as legacy and 
deprecated), and the client should only care about the record structure 
in the usual C form, along with a version (so no extra concepts on how the 
record is placed in the file).

But I wouldn't put any extra effort into a different API as you did; rather, 
I'd check how to adapt legacy programs to work better with systemd. These 
interfaces are really legacy, and I think we should move away from them.

[1] https://www.openwall.com/lists/musl/2012/03/04/4
[2] https://skarnet.org/software/utmps/

Re: [RFC] Modernizing Linux authentication logs (lastlog, btmp, utmp, wtmp) with SQLite

[Florian Weimer]

* Roman Bakshansky:

> The full RFC, including preliminary database schemas and API drafts,
> is available in the discussion repository:
>
>      https://github.com/bakshansky/linux-auth-logs

I don't understand how SQLite (without a daemon) addresses the locking
issue.  WAL mode still uses fcntl locking.