Re: [Fwd: Re: EVM: Permission denied with overlayfs]

[Mimi Zohar <zohar@linux.ibm.com>]

On Wed, 2018-12-19 at 22:47 +0200, Amir Goldstein wrote:
> On Wed, Dec 19, 2018 at 8:51 PM Mimi Zohar <zohar@linux.ibm.com> wrote:
> >
> > On Wed, 2018-12-19 at 18:35 +0200, Amir Goldstein wrote:
> > > On Wed, Dec 19, 2018 at 5:47 PM Mimi Zohar <zohar@linux.ibm.com> wrote:
> > > >
> > > > Hi Ignaz, Amir,
> > > >
> > > > CONFIG_OVERLAY_FS_METACOPY was introduce in linux-4.19.  Do either of
> > > > you by have time to bisect this and find the commit that introduced
> > > > the regression?
> > > >
> > >
> > > Hi Mimi,
> > >
> > > v4.19 has a big change that removes many VFS hacks in favor of
> > > overlayfs stacked file operations.
> > >
> > > The major implication for VFS code is that file_inode(file) is now the overlayfs
> > > inode and not the real inode. Therefore, file_dentry(file) is also the overlayfs
> > > dentry and not the real dentry.
> > >
> > > I am not sure how that change impacts EVM ?
> > > FWIW, d_backing_inode(dentry) was never anything more than d_inode(dentry).
> > >
> > > Can you please try to describe in more details for someone who has no clue what
> > > EVM does how exactly the v4.19 change is manifested in the EVM use case.
> > >
> > > AFAIKT, evm_calc_hmac_or_hash() would get the overlayfs dentry both in
> > > v4.18 and v4.19 and therefore d_backing_inode(dentry) should be the
> > > overlayfs inode in both kernels (?).
> > >
> > > The value of overlayfs inode i_ino can be identical to i_ino of the real inode
> > > under some conditions, but far from always and the value of overlayfs inode
> > > i_generation is almost guaranteed to not match that of the real inode.
> > >
> > > Ignaz, can you add some more debug prints to shed some light on what
> > > exactly has changed, between the two kernels?
> > > If the calculated hashes do not match in two different execution paths,
> > > please provide two sample stack traces that see different i_ino, so we can
> > > examine them.
> >
> > Attached is the output from the script below.  With the posted kernel
> > patch, these are the dmesg messages (d_backing_inode, d_real_inode,
> > dentry->d_name.name):
> >
> > [ 8973.189712] evm: ino: 61217 1841346 61217 oo.text
> > [ 8973.202776] evm: ino: 60375 1836705 60375 ll.text
> > [ 8973.203683] evm: ino: 61217 1841346 61217 oo.text
> > [ 8973.204713] evm: ino: 60380 1839227 60380 uu.text
> >
> 
> The patch I sent can iron these changes out, but it won't iron out
> the i_generation change.

The patch didn't seem to help.

> 
> > script:
> > -------
> > #!/bin/bash
> > # A small example for reproduction (on a system with IMA appraisal):
> > set -x
> > cd /home/mimi/tmp/
> > OVERLAYFS_TEST_DIR=overlay
> > mkdir $OVERLAYFS_TEST_DIR
> > mkdir "${OVERLAYFS_TEST_DIR}/lower"
> > mkdir "${OVERLAYFS_TEST_DIR}/upper"
> > mkdir "${OVERLAYFS_TEST_DIR}/work"
> > mkdir "${OVERLAYFS_TEST_DIR}/merged"
> > mount -t overlay overlay -o lowerdir="${OVERLAYFS_TEST_DIR}/lower",upperdir="${OVERLAYFS_TEST_DIR}/upper",workdir="${OVERLAYFS_TEST_DIR}/work" "${OVERLAYFS_TEST_DIR}/merged"
> >
> > echo lower > overlay/lower/ll.text
> > echo upper > overlay/upper/uu.text
> > echo overlay > overlay/merged/oo.text
> > set +x
> > find overlay -type f -exec stat {} \;
> > echo =====
> > find overlay -type f -exec cat {} \;
> >
> > =====
> > cat: overlay/merged/ll.text: Permission denied
> > cat: overlay/merged/oo.text: Permission denied
> > cat: overlay/merged/uu.text: Permission denied
> > overlay
> > upper
> > lower
> 
> What I am still missing is:
> 1. What does the call stack look like when calculating the hash on oo.text?

*** cat file on lowerdir (ino = 258)


[  428.114646] evm: ino: 258
[  428.114659] CPU: 2 PID: 3907 Comm: cat Not tainted 4.20.0-rc2+ #1287
[  428.114663] Hardware name: LENOVO 20BTS1NJ00/20BTS1NJ00, BIOS N14ET48W (1.26 ) 06/11/2018
[  428.114666] Call Trace:
[  428.114683]  dump_stack+0x46/0x5b
[  428.114691]  hmac_add_misc+0x171/0x180
[  428.114698]  evm_calc_hmac_or_hash+0x1c9/0x280
[  428.114704]  evm_verify_hmac+0x11f/0x2b0
[  428.114710]  ? evm_protected_xattr+0x6c/0x90
[  428.114717]  ima_appraise_measurement+0x83/0x510
[  428.114725]  process_measurement+0x646/0x6f0
[  428.114782]  ? btrfs_lookup_xattr+0x6a/0xb0 [btrfs]
[  428.114832]  ? btrfs_getxattr+0xa2/0x130 [btrfs]
[  428.114841]  ? _cond_resched+0x16/0x40
[  428.114847]  ? selinux_file_open+0xa8/0xc0
[  428.114855]  ima_file_check+0x4a/0x60
[  428.114864]  path_openat+0x5fb/0x12d0
[  428.114873]  do_filp_open+0x8a/0xf0
[  428.114880]  ? _copy_to_user+0x22/0x30
[  428.114889]  ? audit_alloc_name+0xa2/0xd0
[  428.114895]  ? path_get+0x11/0x30
[  428.114901]  ? do_sys_open+0x124/0x1f0
[  428.114906]  do_sys_open+0x124/0x1f0
[  428.114914]  do_syscall_64+0x4f/0x160
[  428.114922]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  428.114928] RIP: 0033:0x7f44a270df70
[  428.114934] Code: 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 83 3d 99 e7 2c 00 00 75 10 b8 02 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 31 c3 48 83 ec 08 e8 9e df 01 00 48 89 04 24
[  428.114939] RSP: 002b:00007ffdc6a76828 EFLAGS: 00000246 ORIG_RAX: 0000000000000002
[  428.114944] RAX: ffffffffffffffda RBX: 0000000000020000 RCX: 00007f44a270df70
[  428.114948] RDX: fffffffffffe0400 RSI: 0000000000000000 RDI: 00007ffdc6a7859d
[  428.114951] RBP: 0000000000000000 R08: 0000000000000410 R09: 0000000000000000
[  428.114955] R10: 000055e41dbe7010 R11: 0000000000000246 R12: 0000000000001000
[  428.114958] R13: 00007ffdc6a76a90 R14: 0000000000000000

> 2. What does the call stack look like when failing to verify the hash
> on oo.text?

*** Cat of same file on merged dir fails (ino != 258)
cat: overlay/merged/abc.text: Permission denied

[  476.770869] evm: ino: 38593 258 38593 abc.text
[  476.770876] evm: ino: 38593
[  476.770883] CPU: 3 PID: 3928 Comm: cat Not tainted 4.20.0-rc2+ #1287
[  476.770887] Hardware name: LENOVO 20BTS1NJ00/20BTS1NJ00, BIOS N14ET48W (1.26 ) 06/11/2018
[  476.770890] Call Trace:
[  476.770906]  dump_stack+0x46/0x5b
[  476.770913]  hmac_add_misc+0x171/0x180
[  476.770920]  evm_calc_hmac_or_hash+0x1c9/0x280
[  476.770927]  evm_verify_hmac+0x11f/0x2b0
[  476.770933]  ? evm_protected_xattr+0x6c/0x90
[  476.770940]  ima_appraise_measurement+0x83/0x510
[  476.770948]  process_measurement+0x646/0x6f0
[  476.770955]  ? selinux_file_open+0xa8/0xc0
[  476.770961]  ? do_dentry_open+0x25c/0x340
[  476.770966]  ? open_with_fake_path+0x48/0x70
[  476.770974]  ? ovl_open_realfile+0x56/0xe0
[  476.770981]  ima_file_check+0x4a/0x60
[  476.770989]  path_openat+0x5fb/0x12d0
[  476.770998]  do_filp_open+0x8a/0xf0
[  476.771005]  ? _copy_to_user+0x22/0x30
[  476.771014]  ? audit_alloc_name+0xa2/0xd0
[  476.771020]  ? path_get+0x11/0x30
[  476.771026]  ? do_sys_open+0x124/0x1f0
[  476.771030]  do_sys_open+0x124/0x1f0
[  476.771038]  do_syscall_64+0x4f/0x160
[  476.771046]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  476.771052] RIP: 0033:0x7f493d6fcf70
[  476.771058] Code: 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 83 3d 99 e7 2c 00 00 75 10 b8 02 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 31 c3 48 83 ec 08 e8 9e df 01 00 48 89 04 24
[  476.771063] RSP: 002b:00007ffcdb2be628 EFLAGS: 00000246 ORIG_RAX: 0000000000000002
[  476.771068] RAX: ffffffffffffffda RBX: 0000000000020000 RCX: 00007f493d6fcf70
[  476.771072] RDX: fffffffffffe0400 RSI: 0000000000000000 RDI: 00007ffcdb2bf59c
[  476.771075] RBP: 0000000000000000 R08: 0000000000000410 R09: 0000000000000000
[  476.771078] R10: 0000562d9c5a8010 R11: 0000000000000246 R12: 0000000000001000
[  476.771081] R13: 00007ffcdb2be890 R14: 0000000000000000 R15: 0000000000000000

> 3. If the same tests does pass pre v4.19, please provide call stacks from that
> kernel as well.

>From a kernel prior to v4.19 works, because the i_ino is the same as
the original one used when calculating the HMAC.

**** cat file on merged directory

[  106.762046] evm: ino: 257
[  106.762052] CPU: 3 PID: 3766 Comm: pool Not tainted 4.18.20+ #73
[  106.762054] Hardware name: LENOVO 20BTS1NJ00/20BTS1NJ00, BIOS N14ET48W (1.26 ) 06/11/2018
[  106.762055] Call Trace:
[  106.762062]  dump_stack+0x5c/0x7b
[  106.762067]  hmac_add_misc+0x171/0x180
[  106.762070]  evm_calc_hmac_or_hash+0x1d7/0x290
[  106.762073]  evm_verify_hmac+0x120/0x2b0
[  106.762076]  ? evm_protected_xattr+0x6c/0x90
[  106.762079]  ima_appraise_measurement+0x7b/0x480
[  106.762081]  process_measurement+0x629/0x6c0
[  106.762109]  ? btrfs_lookup_xattr+0x6a/0xb0 [btrfs]
[  106.762113]  ? _cond_resched+0x16/0x40
[  106.762117]  ima_file_check+0x51/0x60
[  106.762121]  path_openat+0x5c0/0x13e0
[  106.762124]  ? __alloc_pages_nodemask+0x10f/0x290
[  106.762127]  do_filp_open+0x8a/0xf0
[  106.762131]  ? audit_alloc_name+0x7a/0xd0
[  106.762133]  ? path_get+0x11/0x30
[  106.762136]  ? do_sys_open+0x124/0x1f0
[  106.762138]  do_sys_open+0x124/0x1f0
[  106.762141]  do_syscall_64+0x55/0x170
[  106.762144]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  106.762148] RIP: 0033:0x7f853a2624fd
[  106.762149] Code: ae 20 00 00 75 10 b8 02 00 00 00 0f 05 48 3d 01 f0 ff ff 73 31 c3 48 83 ec 08 e8 ee f4 ff ff 48 89 04 24 b8 02 00 00 00 0f 05 <48> 8b 3c 24 48 89 c2 e8 37 f5 ff ff 48 89 d0 48 83 c4 08 48 3d 01
[  106.762181] RSP: 002b:00007f8533ffdaf0 EFLAGS: 00000293 ORIG_RAX: 0000000000000002
[  106.762183] RAX: ffffffffffffffda RBX: 00007f852c003150 RCX: 00007f853a2624fd
[  106.762184] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 00007f852c01a310
[  106.762186] RBP: 00007f852c01ad00 R08: 0000000000000080 R09: 0000000000000000
[  106.762187] R10: 000f4f6f9681e7e0 R11: 0000000000000293 R12: 000000000000492a
[  106.762189] R13: 00007f852c01a310 R14: 00000000ffffffff R15: 00007f852c01ad00