Problems understanding nftables part 2

Problems understanding nftables part 2

[Wolfgang]

Hello all,

I have asked already some questions about nftables.  While diving deeper into it, there
are arising more questions.


In my last test I have hooked rules into the 5 inet hook filter destinations
( prerouting, input, output,postrouting, forward), to watch how packets are flowing to
my rules. Now I extended that, to see packets also flowing through nat destinations,
but I have seen no packets.

1)  It looks like, that it needs at least one configured nat-rule, which gets triggered
    to see packets flowing through the kernel. It looks like, that without such an initial
    trigger, trace is either
    a) not showing packets
    b) packet flow through nat is enabled only, after a first nat rule matched
       When I have a matching rule like in example 1, I see packets not only in prerouting,
       but also in input, output and postrouting, even when the chain contains no nat
       specific rule. But: For tcp this seems to be valid only for packets with SYN-Flag set,
       others are not showing up.
    c) As soon, as I had such a trigger-packet I see however all udp-traffic from the system,
       I have not seen, before the tcp rule triggered.
    So I have the question, if there are other options to get trace through nat-hooks enabled
    without having an initial trigger?
    Unfortunately the "dnat" option, does not allow to add a "meta nftrace set 1" behind this
    specific line, so i must trace in a more general way.
2) Prerouting, postrouting and route allow for for symbolic priorities, that seems to be broken
   for
   a) input  and
   b) output
   where I need to know the corresponding value. What is the reason behind this inconsistent
   behaviour?

I would be glad, receiving answers and/or tips for further actions, I could try?

Here my tracing configuration:

>table ip nat {
>        chain prerouting_nat{
>                type nat hook prerouting priority dstnat ; policy accept;
>## 1)
>                ip saddr AA.BB.CC.DD tcp dport 443 dnat ip to BB.CC.DD.EE
>                meta nftrace set 1
>        }
>
>        chain postrouting_nat {
>                type nat hook postrouting priority srcnat ; policy accept;
>                meta nftrace set 1
>        }
>
>        chain input_nat {
>## 2a)
># broken:       type nat hook input priority srcnat ; policy accept;
>                type nat hook input priority  100 ; policy accept;
>                meta nftrace set 1
>        }
>
>        chain output_nat {
>## 2b)
># broken:       type nat hook output priority dstnat ; policy accept;
>                type nat hook output priority -100; policy accept;
>                meta nftrace set 1
>        }
>
>        chain output_route{
>                type nat hook output priority mangle ; policy accept;
>                meta nftrace set 1
>        }
>}



Regards

  Wolfgang

Re: Problems understanding nftables part 2

[Kerin Millar]

On Wed, 29 May 2024, at 4:46 PM, Wolfgang wrote:
> Hello all,
>
> I have asked already some questions about nftables.  While diving 
> deeper into it, there
> are arising more questions.
>
>
> In my last test I have hooked rules into the 5 inet hook filter 
> destinations
> ( prerouting, input, output,postrouting, forward), to watch how packets 
> are flowing to
> my rules. Now I extended that, to see packets also flowing through nat 
> destinations,
> but I have seen no packets.
>
> 1)  It looks like, that it needs at least one configured nat-rule, 
> which gets triggered
>     to see packets flowing through the kernel. It looks like, that 
> without such an initial
>     trigger, trace is either
>     a) not showing packets
>     b) packet flow through nat is enabled only, after a first nat rule 
> matched
>        When I have a matching rule like in example 1, I see packets not 
> only in prerouting,
>        but also in input, output and postrouting, even when the chain 
> contains no nat
>        specific rule. But: For tcp this seems to be valid only for 
> packets with SYN-Flag set,
>        others are not showing up.
>     c) As soon, as I had such a trigger-packet I see however all 
> udp-traffic from the system,
>        I have not seen, before the tcp rule triggered.
>     So I have the question, if there are other options to get trace 
> through nat-hooks enabled
>     without having an initial trigger?
>     Unfortunately the "dnat" option, does not allow to add a "meta 
> nftrace set 1" behind this
>     specific line, so i must trace in a more general way.

I wouldn't consider the nat hook to be an especially useful context in which to enable tracing, partly owing to its semantics.

https://wiki.nftables.org/wiki-nftables/index.php/Performing_Network_Address_Translation_(NAT)

Instead, I would recommend using the following hook for tracing packets that arrive.

type filter hook prerouting priority raw; policy accept;

And the following hook for tracing packets that are generated by your host.

type filter hook output priority raw; policy accept;

Going about it in this way should afford you the greatest degree of insight, as regards the traversal of any given packet through your ruleset.

> 2) Prerouting, postrouting and route allow for for symbolic priorities, 
> that seems to be broken
>    for
>    a) input  and
>    b) output
>    where I need to know the corresponding value. What is the reason 
> behind this inconsistent
>    behaviour?

Unlike iptables, the design of nftables is such that all Netfilter hooks must be explicitly defined by the ruleset. Consequently, it exposes some of the rougher edges of Netfilter to the user. In particular, not all hook type and priority combinations necessarily make sense in practice. This is compounded by the matter of the nft(8) man page having tended towards under-documenting such nuances, though it has gotten a little better as of the most recent release.

During the time in which I was learning nftables, I found it useful to consider iptables as a point of reference. For instance, iptables has a built-in raw table and a built-in PREROUTING chain. One may use iptables-nft to infer how its hook is set up, in a manner whereby it is rendered explicit.

# nft flush ruleset
# iptables-nft -t raw -A PREROUTING
# nft list ruleset
table ip raw {
        chain PREROUTING {
                type filter hook prerouting priority raw; policy accept;
                counter
        }
}

The wording of the TABLES section of the iptables(8) man page can thus also be useful as a point of reference. Such a chain is suitable for acting on packets that arrive before any routing decision is made, and before the conntrack table is consulted i.e. even before a nat hook is potentially attended to. These characteristics render it particularly useful for tracing.

-- 
Kerin Millar

Re: Problems understanding nftables part 2

[Kerin Millar]

(Copying the netfilter list back in ...)

On Thu, 30 May 2024, at 5:56 PM, Wolfgang wrote:
> Hello Kerin,
>
> thanks for your answer. you wrote:
>
>> I wouldn't consider the nat hook to be an especially useful context in which to enable tracing, partly owing to its semantics.
>> https://wiki.nftables.org/wiki-nftables/index.php/Performing_Network_Address_Translation_(NAT)
>> Instead, I would recommend using the following hook for tracing packets that arrive.
>> ...
>
> my current goal is to get a feeling for the whole picture, therefore I 
> have hooks in almost every
> possible location. Those are:
>       type filter hook prerouting priority filter - 1; policy accept;
>       type filter hook postrouting priority filter - 1; policy accept;
>       type filter hook input priority filter - 1; policy accept;
>       type filter hook output priority filter - 1; policy accept;
>       type filter hook forward priority filter - 1; policy accept;
>       type nat hook prerouting priority dstnat -1 ; policy accept;
>       type nat hook postrouting priority srcnat-1 ; policy accept;
>       type nat hook input priority  99 ; policy accept;
>       type nat hook output priority -101; policy accept;
>       type filter hook ingress device "eth0" priority -301; policy 
> accept;
>       type filter hook egress device "eth0" priority filter -301; 
> policy accept;
>       type filter hook ingress device "eth1" priority filter -301; 
> policy accept;
>       type filter hook egress device "eth1" priority filter -301; 
> policy accept;
>

You are describing (not altogether persuasively) why you want for so many hooks but not why you want to set nftrace to 1 for each of them. With the exception of ingress/egress, enabling nftrace for the two previously suggested hooks would have been quite sufficient.

# Pointless if solely for enabling nftrace for a 'big picture' perspective.
# Please use raw (-300) as was suggested.
type filter hook prerouting priority filter - 1; policy accept;

# Ditto. Enable it for "hook prerouting priority raw".
type filter hook input priority filter - 1; policy accept;

# Ditto. Use "hook prerouting raw" and/or "hook output priority raw".
type filter hook forward priority filter - 1; policy accept;

# Ditto.
type nat hook prerouting priority dstnat -1 ; policy accept;

... and so on.

> I can see incoming packets in ingress/egress and I can see, that 
> packets from local processes
> talking via tcp, are leaving postrouting and reentering the system via 
> prerouting, here with a new
> trace id. I currently can't explain their way, as neither the diagram on
> https://wiki.nftables.org/wiki-nftables/index.php/Netfilter_hooks nor 
> the otherwise excellent
> article here: 
> https://thermalcircle.de/doku.php?id=blog:linux:nftables_packet_flow_netfilter_hooks_detail
> is speaking about internal communication.
>
> Another lesson I learned, that four hooks don't allow symbolic priority 
> names: nat input,
> nat output, ingress and egress.
>
> And what makes me still wonder, is the fact, that nat hooks start 
> reporting packets, after one
> packet matched a nat rule. After that I see e.g. all udp-traffic from 
> my host (only a tcp nat
> matched), which I have not seen before.  From the matching connection I 
> have however seen only those
> packets, having the SYN-flag set.
>
>
>> Unlike iptables, the design of nftables is such that all Netfilter hooks must be explicitly
>> defined by the ruleset. Consequently, it exposes some of the rougher edges of Netfilter to the
>> user. In particular, not all hook type and priority combinations necessarily make sense in
>> practice. This is compounded by the matter of the nft(8) man page having tended towards
>> under-documenting such nuances, though it has gotten a little better as of the most recent release.
>
> I think, that I have fully understood this fact. That is the reason, 
> why I wish to setup a tracing
> template, which covers explicitly all possible hooks, so I can adapt 
> that later, to diagnose
> whatever needs to be checked.

It's not clear that you understand that the value of nftrace doesn't quietly reset itself to 0 between hooks, however.

>
>> During the time in which I was learning nftables, I found it useful to consider iptables as a
>> point of reference. For instance, iptables has a built-in raw table and a built-in PREROUTING
>> chain. One may use iptables-nft to infer how its hook is set up, in a manner whereby it is rendered explicit.
>
> That is one thing, I strictly try to avoid.  I have seen, that I can do 
> a lot of stuff with
> iptables-translate, but I will explicitly not migrate old 
> iptables-stuff, which comes in my case
> already from migrations from old ipchains and ipfwadm times. So my test 
> system has no iptables stuff
> installed.
> So I like very much the feature, that I can just delete a complete 
> table with all chains and rules
> included in just one line.  Iptables-translate generates still a whole 
> bunch of delete statements.
> I see here a big advantage, as I can put all stuff for a single service 
> in one table, as long, as
> they belong to the same family (inet, netdev, bridge, arp).  So I can 
> manage things really
> separated, not messing up the whole concatenating monster.

You are free to do as you please. The point is that it's the same old Netfilter hooking system underneath. The hook types and priority levels associated with the built-in iptables tables and chains were chosen by its developers with good reason. Whether you choose to believe it or not, there is much that can be learned from this. For example, a prerouting hook with a priority level of -300 is useful in nftables for precisely the same reason that it is useful in iptables, the difference being that the iptables man page does a better job of explaining why it is useful, albeit at a more abstract level. Indeed, the nftables man page didn't even try to explain why that particular choice of priority level has the effect that it does until recently.

>
> So I just can summarize my currently most important open questions:
>
> 1) How is inet traffic flowing from application to application? Where 
> is the hidden way from
> postrouting to prerouting? It must somehow leave netfilter, as packets 
> are reappearing with a new
> trace id?

A packet is transmitted over the loopback interface then a packet is received at the loopback interface. Locally generated packets are routed over the loopback interface for all locally owned destination addresses, not only 127.0.0.0/8 and ::1. Otherwise, the fact that the loopback interface is involved isn't of any particular significance.

>
> 2) Why nat traces need a trigger through a succesful nat connection, to 
> start working? Is my
> observation right, that tcp-wise only SYN-packets pass the nat-hooks?

The previously offered wiki link attempts to explain this, though the wording could be improved. The following two bullet points have been lifted from the article in question.

- The first packet of a flow is used to look up for a matching rule which sets up the NAT binding for this flow. This also manipulates this first packet accordingly.

- No rule lookup happens for follow up packets in the flow: the NAT engine uses the NAT binding information already set up by the first packet to perform the packet manipulation.

The "information" in question is stored by the conntrack table, whose contents can be inspected with - and monitored by - the conntrack(8) utility from conntrack-tools. The overwhelming majority of Linux distributions activate the conntrack subsystem at the point that any ruleset is loaded containing at least one rule pertaining to conntrack state, thereby causing for the applicable kernel modules to be dynamically loaded. Typically, that would be a NAT rule or a rule that tries to match on ctstate. One activated, the conntrack state machine continues to perform its duties until such time as the kernel modules are unloaded, even in the case of an empty ruleset.

A TCP packet with the SYN flag set is something that might create a new entry in the conntrack table. As such, it is to be expected that it may be intercepted by a nat hook, whereas subsequent packets that the conntrack state machine can match against an existing flow won't be. In fact, even an ACK packet can create a new TCP flow, unless the value of the "net.netfilter.nf_conntrack_tcp_loose" sysctl is set to 0.

-- 
Kerin Millar

Re: Problems understanding nftables part 2

[Wolfgang]

[-- Attachment #1: Type: text/plain, Size: 4870 bytes --]

Hello Kerin,

thanks for your reply, I will try to answer your questions

> You are describing (not altogether persuasively) why you want for so many hooks but not why you
> want to set nftrace to 1 for each of them. With the exception of ingress/egress, enabling nftrace
> for the two previously suggested hooks would have been quite sufficient.

> # Pointless if solely for enabling nftrace for a 'big picture' perspective.
> # Please use raw (-300) as was suggested.
> type filter hook prerouting priority filter - 1; policy accept;

I just wan't the hooks for my understanding, and have a template, I later can take, modifying for
tracing down issues in real live.
Right now its just for understanding the system, and seeing, that those hooks are beeing called for
certain packets.
About raw priority:
When I understand https://wiki.nftables.org/wiki-nftables/index.php/Netfilter_hooks#Priority_within_hook
right, priorities have no influence that a hook is called or not. Priorities only assign the order,
in which the chains are called at this hook.
So I can have for example two chains in the same hook, one with priority -450 and one with priority
301.  So I would see e.g. in prerouting in the first hook two fragmentated packets and in the
latter hook only one defragmented packet. Or I could see in a postrouting chain a packet before and
after snat.
But my hook should see the packet in each case, please correct me, if I am wrong.

So from my understanding, priorities are getting important, as soon, as I wish to investigate in a
certain kind of problem, than the priority is very important, to understand, what I am seeing.



> It's not clear that you understand that the value of nftrace doesn't quietly reset itself to 0 between hooks, however.

Can you explain this further? Which value of nftrace? What shall reset itself?

>> 1) How is inet traffic flowing from application to application? Where
>> is the hidden way from
>> postrouting to prerouting? It must somehow leave netfilter, as packets 
>> are reappearing with a new
>> trace id?
> A packet is transmitted over the loopback interface then a packet is received at the loopback
> interface. Locally generated packets are routed over the loopback interface for all locally owned
> destination addresses, not only 127.0.0.0/8 and ::1. Otherwise, the fact that the loopback
> interface is involved isn't of any particular significance.

Thanks for that very important answer! That helps me a lot and that should be part of the diagram,
as this path is not going to ingress/egress. I try to investigate that further, as I am interested
in the whole picture. Can it be, that the "Bridge device?" decision is a tri-state decision, for
checking bridge, physical interface or loopback?
I enclose an updated diagram, where I have included
this path as an temporary one and some other findings. Please have a look at it and give me feedback.


>>
>> 2) Why nat traces need a trigger through a succesful nat connection, to 
>> start working? Is my
>> observation right, that tcp-wise only SYN-packets pass the nat-hooks?

> The previously offered wiki link attempts to explain this, though the wording could be improved.
> The following two bullet points have been lifted from the article in question.

> - The first packet of a flow is used to look up for a matching rule which sets up the NAT binding
> for this flow. This also manipulates this first packet accordingly.

> - No rule lookup happens for follow up packets in the flow: the NAT engine uses the NAT binding
> information already set up by the first packet to perform the packet manipulation.

> The "information" in question is stored by the conntrack table, whose contents can be inspected
> with - and monitored by - the conntrack(8) utility from conntrack-tools. The overwhelming majority
> of Linux distributions activate the conntrack subsystem at the point that any ruleset is loaded
> containing at least one rule pertaining to conntrack state, thereby causing for the applicable
> kernel modules to be dynamically loaded. Typically, that would be a NAT rule or a rule that tries
> to match on ctstate. One activated, the conntrack state machine continues to perform its duties
> until such time as the kernel modules are unloaded, even in the case of an empty ruleset.

> A TCP packet with the SYN flag set is something that might create a new entry in the conntrack
> table. As such, it is to be expected that it may be intercepted by a nat hook, whereas subsequent
> packets that the conntrack state machine can match against an existing flow won't be. In fact,
> even an ACK packet can create a new TCP flow, unless the value of the
> "net.netfilter.nf_conntrack_tcp_loose" sysctl is set to 0.


Thanks for that hint as well, looks like, that I have overseen that.  The picture gets more clear
every day!

Regards

  Wolfgang

[-- Attachment #2: nf-hooks-temporary-version.png --]
[-- Type: image/png, Size: 277128 bytes --]

Re: Problems understanding nftables part 2

[Kevin P. Fleming]

On Fri, May 31, 2024, at 11:11, Wolfgang wrote:
>> It's not clear that you understand that the value of nftrace doesn't quietly reset itself to 0 between hooks, however.
>
> Can you explain this further? Which value of nftrace? What shall reset itself?

It appears you misunderstood the statement.

The point being made here is that once 'nftrace' has been set to '1' on a packet, it will stay '1' for the entire lifetime of that packet through the netfilter subsystem. It will go back to zero, or any other value, and there is no need to set it to '1' again in a later hook or chain. If you set it to '1' in a very early part of the netfilter lifecycle, you'll be able to use 'nft monitor' to see the handling of that packet through the remainder of the lifecycle.

Thus, if you are setting it to '1' in 'ingress', for example, there is no need have rules to set it to '1' in any other tables or hooks or chains (unless there are different conditions applied on those rules).

Re: Problems understanding nftables part 2

[Wolfgang]

[-- Attachment #1: Type: text/plain, Size: 122 bytes --]

Hello
  just sending and updated diagram, as the previous one had one big mistake with a wrong route!
regards

  Wolfgang

[-- Attachment #2: nf-hooks-temporary-version-2.png --]
[-- Type: image/png, Size: 288027 bytes --]

Re: Problems understanding nftables part 2

[Kerin Millar]

Hi Wolfgang,

On Fri, 31 May 2024, at 4:11 PM, Wolfgang wrote:
> Hello Kerin,
>
> thanks for your reply, I will try to answer your questions
>
>> You are describing (not altogether persuasively) why you want for so many hooks but not why you
>> want to set nftrace to 1 for each of them. With the exception of ingress/egress, enabling nftrace
>> for the two previously suggested hooks would have been quite sufficient.
>
>> # Pointless if solely for enabling nftrace for a 'big picture' perspective.
>> # Please use raw (-300) as was suggested.
>> type filter hook prerouting priority filter - 1; policy accept;
>
> I just wan't the hooks for my understanding, and have a template, I 
> later can take, modifying for
> tracing down issues in real live.
> Right now its just for understanding the system, and seeing, that those 
> hooks are beeing called for
> certain packets.
> About raw priority:
> When I understand 
> https://wiki.nftables.org/wiki-nftables/index.php/Netfilter_hooks#Priority_within_hook
> right, priorities have no influence that a hook is called or not. 
> Priorities only assign the order,
> in which the chains are called at this hook.
> So I can have for example two chains in the same hook, one with 
> priority -450 and one with priority
> 301.  So I would see e.g. in prerouting in the first hook two 
> fragmentated packets and in the
> latter hook only one defragmented packet. Or I could see in a 
> postrouting chain a packet before and
> after snat.
> But my hook should see the packet in each case, please correct me, if I 
> am wrong.
>
> So from my understanding, priorities are getting important, as soon, as 
> I wish to investigate in a
> certain kind of problem, than the priority is very important, to 
> understand, what I am seeing.
>
>
>
>> It's not clear that you understand that the value of nftrace doesn't quietly reset itself to 0 between hooks, however.
>
> Can you explain this further? Which value of nftrace? What shall reset itself?

What I mean is that, once the nftrace attribute is set to 1 for a given packet, it stays that way until the packet is given its final verdict. So, if you hook at a sufficiently early juncture, it normally takes no more than two hooks by which to set the nftrace attribute to 1 and be able to trace the path of a packet through a ruleset very comprehensively. Of course, you can create as many hooks as you wish. However, if the resulting chains are to contain nothing but a single rule to set nftrace to 1 (again) then there is no need; it will only serve to make the trace output more noisy.

>
>>> 1) How is inet traffic flowing from application to application? Where
>>> is the hidden way from
>>> postrouting to prerouting? It must somehow leave netfilter, as packets 
>>> are reappearing with a new
>>> trace id?
>> A packet is transmitted over the loopback interface then a packet is received at the loopback
>> interface. Locally generated packets are routed over the loopback interface for all locally owned
>> destination addresses, not only 127.0.0.0/8 and ::1. Otherwise, the fact that the loopback
>> interface is involved isn't of any particular significance.
>
> Thanks for that very important answer! That helps me a lot and that 
> should be part of the diagram,
> as this path is not going to ingress/egress. I try to investigate that 
> further, as I am interested
> in the whole picture. Can it be, that the "Bridge device?" decision is 
> a tri-state decision, for
> checking bridge, physical interface or loopback?
> I enclose an updated diagram, where I have included
> this path as an temporary one and some other findings. Please have a 
> look at it and give me feedback.

The path does go through ingress/egress. That the interface isn't a physically tangible one doesn't matter. For that reason, I don't think that it requires any special accommodation by the diagram. For locally generated packets, "lo" may be selected by the routing decision. Meanwhile, packets arriving at "lo" are handled similarly to any other interface.

>
>
>>>
>>> 2) Why nat traces need a trigger through a succesful nat connection, to 
>>> start working? Is my
>>> observation right, that tcp-wise only SYN-packets pass the nat-hooks?
>
>> The previously offered wiki link attempts to explain this, though the wording could be improved.
>> The following two bullet points have been lifted from the article in question.
>
>> - The first packet of a flow is used to look up for a matching rule which sets up the NAT binding
>> for this flow. This also manipulates this first packet accordingly.
>
>> - No rule lookup happens for follow up packets in the flow: the NAT engine uses the NAT binding
>> information already set up by the first packet to perform the packet manipulation.
>
>> The "information" in question is stored by the conntrack table, whose contents can be inspected
>> with - and monitored by - the conntrack(8) utility from conntrack-tools. The overwhelming majority
>> of Linux distributions activate the conntrack subsystem at the point that any ruleset is loaded
>> containing at least one rule pertaining to conntrack state, thereby causing for the applicable
>> kernel modules to be dynamically loaded. Typically, that would be a NAT rule or a rule that tries
>> to match on ctstate. One activated, the conntrack state machine continues to perform its duties
>> until such time as the kernel modules are unloaded, even in the case of an empty ruleset.
>
>> A TCP packet with the SYN flag set is something that might create a new entry in the conntrack
>> table. As such, it is to be expected that it may be intercepted by a nat hook, whereas subsequent
>> packets that the conntrack state machine can match against an existing flow won't be. In fact,
>> even an ACK packet can create a new TCP flow, unless the value of the
>> "net.netfilter.nf_conntrack_tcp_loose" sysctl is set to 0.
>
>
> Thanks for that hint as well, looks like, that I have overseen that.  
> The picture gets more clear
> every day!

-- 
Kerin Millar

Re: Problems understanding nftables part 2

[Kerin Millar]

On Fri, 31 May 2024, at 4:34 PM, Wolfgang wrote:
> Hello
>   just sending and updated diagram, as the previous one had one big 
> mistake with a wrong route!
> regards

Regarding the query about the routing decision having one output, it is there to indicate that a routing decision really does take place. Since routing is not the responsibility of Netfilter, there is no decision to be made by Netfilter at all. Nevertheless, it is practically useful for the diagram to show that it happens; in particular, that output hooks are effected afterwards.

-- 
Kerin Millar

Re: Problems understanding nftables part 2

[Wolfgang]

Hello Kerin,


>>
>>> It's not clear that you understand that the value of nftrace doesn't quietly reset itself to 0 between hooks, however.
>>
>> Can you explain this further? Which value of nftrace? What shall reset itself?

> What I mean is that, once the nftrace attribute is set to 1 for a given packet, it stays that way
> until the packet is given its final verdict. So, if you hook at a sufficiently early juncture, it
> normally takes no more than two hooks by which to set the nftrace attribute to 1 and be able to
> trace the path of a packet through a ruleset very comprehensively. Of course, you can create as
> many hooks as you wish. However, if the resulting chains are to contain nothing but a single rule
> to set nftrace to 1 (again) then there is no need; it will only serve to make the trace output more noisy.

Ok, so here we have the same understanding. I really don't care actually about noisy rules, as my
test rules are only containing constraints to single connections, i create with socat, netcat etc.
This is just a tool, to see, whats going on and getting a deeper understanding.

> The path does go through ingress/egress. That the interface isn't a physically tangible one
> doesn't matter. For that reason, I don't think that it requires any special accommodation by the
> diagram. For locally generated packets, "lo" may be selected by the routing decision. Meanwhile,
> packets arriving at "lo" are handled similarly to any other interface.

For me right now its really important, as I missed in that diagram the way, internal communication
flows. I had one problem, I researched, where this explanation helped me a lot.
And as this kind of diagrams are used by so many people, with different intention, what they search,
it is -by my opinion- a very important piece.
It was for me not clear, that local traffic between two applications listening on eth0 ip addresses,
is in reality handled over lo.

In my roadmap is now to investigate tun/tap and other stuff, I assume, that this is also flowing
through lo.

Regards

  Wolfgang