From mboxrd@z Thu Jan 1 00:00:00 1970 From: Kangkook Jee Subject: Re: Early processes (daemons) do not report audit events Date: Fri, 11 Sep 2015 16:17:56 -0400 Message-ID: <43B84A86-63BA-4573-9F12-5510E5E404DD@gmail.com> References: <1E6087D5-E362-4795-BD8D-1374831498E4@gmail.com> <087AE25B-79B6-4E7A-BE4B-901BE95FB81B@gmail.com> <20150911095027.GO8140@madcap2.tricolour.ca> <8C676625-C8EF-4BDD-B5AA-4C2CC50B2BB0@gmail.com> <20150911162438.GQ8140@madcap2.tricolour.ca> Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Content-Type: multipart/mixed; boundary="===============2188333698036769264==" Return-path: In-Reply-To: <20150911162438.GQ8140@madcap2.tricolour.ca> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: linux-audit-bounces@redhat.com Errors-To: linux-audit-bounces@redhat.com To: Richard Guy Briggs Cc: linux-audit@redhat.com List-Id: linux-audit@redhat.com --===============2188333698036769264== Content-Type: multipart/alternative; boundary="Apple-Mail=_82D7CD03-EAB8-4829-8B52-520FB4792792" --Apple-Mail=_82D7CD03-EAB8-4829-8B52-520FB4792792 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi Richard, I also did the same experiment for the latest distributions of Fedora = core and Debian and here=E2=80=99s the results. Fedora-22 (64-bit, 4.0.4-301.fc22.x86_64): Problem reproduced. Debian-8 (64-bit, 3.16.0-4-amd64): Problem reproduced Btw, Burn Alting (burn@swtf.dyndns.org) suggested me to append audit=3D1 = to kernel flag. I added the option to boot-loader (grub) and problem = went away.=20 Regards, Kangkook > On Sep 11, 2015, at 12:24 PM, Richard Guy Briggs = wrote: >=20 > On 15/09/11, Kangkook Jee wrote: >> =46rom the previous reply, I think I misunderstood your question = regarding kernel command line.=20 >> Here=E2=80=99s "cat /proc/cmdline=E2=80=9D results for distributions = that I=E2=80=99ve experimented.=20 >>=20 >> Ubuntu 14.04 (64-bit):=20 >> BOOT_IMAGE=3D/boot/vmlinuz-3.13.0-58-generic = root=3DUUID=3D7505f862-ce46-49e5-9d1c-e4e307844889 ro text quiet splash = vt.handoff=3D7 >>=20 >> Ubuntu 12.04 (64-bit):=20 >> BOOT_IMAGE=3D/boot/vmlinuz-3.13.0-32-generic = root=3DUUID=3D5be789be-9b0c-463e-bd18-42bfa79fb24c ro quiet splash >>=20 >> CentOS 7 (64-bit):=20 >> BOOT_IMAGE=3D/vmlinuz-3.10.0-229.el7.x86_64 = root=3D/dev/mapper/centos-root ro rd.lvm.lv=3Dcentos/root = rd.lvm.lv=3Dcentos/swap crashkernel=3Dauto rhgb quiet LANG=3Den_US.UTF-8 >>=20 >> CentOS 6 (64-bit):=20 >> ro root=3DUUID=3Da7d44560-adcc-4000-9584-8b9fcf2afd74 rd_NO_LUKS = rd_NO_LVM LANG=3Den_US.UTF-8 rd_NO_MD SYSFONT=3Dlatarcyrheb-sun16 = crashkernel=3D129M@0M KEYBOARDTYPE=3Dpc KEYTABLE=3Dus rd_NO_DM rhgb = quiet >>=20 >> I don=E2=80=99t see any audit=3D entries from all examples = above.=20 >=20 > Yes, this is what I was seeking from you. And you are correct, none = of > them have audit=3D1 as I was hoping from at least CentOS. There is a > chance that the CentOS kernel was compiled with audit=3D1 hardcoded, = but I > think that is a pretty small chance... >=20 > I'll have to look at this closer... But any Debian and Fedora data > points that you can provide would certainly be useful. >=20 >> /Kangkook >>=20 >>> On Sep 11, 2015, at 5:50 AM, Richard Guy Briggs = wrote: >>>=20 >>> On 15/09/10, Kangkook Jee wrote: >>>> Hi all, >>>>=20 >>>> I debugged a bit further to identify distributions that are = affected by the issue. >>>> I repeated the same experiment with sshd from 3 more distributions. >>>>=20 >>>> CentOS Linux release 7.1.1503 (64-bit, 3.10.0-229.el7.x86_64): = Problem NOT reproduced >>>> CentOS release 6.6 (64-bit, 2.6.32-504.el6.x86_64): Problem NOT = reproduced >>>> Ubuntu 12.04.5 LTS (64-bit, 3.13.0-32-generic): Problem reproduced >>>=20 >>> For each of these examples, what is the value of the kernel command = line >>> "audit=3D" if it is even present? It is possible that the = CentOS >>> examples include "audit=3D1" while Ubuntu omits the line. "cat >>> /proc/cmdline" should tell you the answer. >>>=20 >>>> After all, Ubuntu family are affected by the issue and I could = confirm >>>> that results are inconsistent across two different distribution >>>> families.=20 >>>=20 >>> I would be curious what your results are with a recent Debian and = with a >>> recent Fedora. >>>=20 >>>> If you can let us know how can we workaround the issue, it will be = a great help. >>>>=20 >>>> Regards, Kangkook >>>>=20 >>>>=20 >>>>> On Sep 9, 2015, at 11:50 PM, Kangkook Jee = wrote: >>>>>=20 >>>>> Dear all, >>>>>=20 >>>>> We are developing custom user space audit agent to gather system = wide system >>>>> call trace. While experimenting with various programs, we found = out that >>>>> processes (daemons) that started early (along with the system = bootstrapping) do >>>>> not report any audit events at all. These processes typically fall = into PID >>>>> range of less than 2000. Here=E2=80=99s how I reproduced the = symptom with sshd daemon. >>>>>=20 >>>>> 1. Reboot the system >>>>>=20 >>>>> 2. Add and enable audit events >>>>> # /sbin/auditctl -a exit,always -F arch=3Db64 -S clone -S close = -S creat -S dup >>>>> -S dup2 -S dup3 -S execve -S exit -S exit_group -S fork -S = open -S openat=20 >>>>> -S unlink -S unlinkat -S vfork -S 288 -S accept -S bind -S = connect=20 >>>>> -S listen -S socket -S socketpair >>>>> # /sbin/auditctl -e1 -b 102400 >>>>>=20 >>>>> 3. Connect to the system via ssh >>>>> Audit messages generated only from child processes and none are = seen from >>>>> the original daemon. >>>>>=20 >>>>> 4. Restart sshd=20 >>>>> # restart ssh >>>>>=20 >>>>> 5. Connect again to the system via ssh >>>>> Now, we see audit messages from both parent and child processes. >>>>>=20 >>>>> I did the experiment from Ubuntu 14.04.2 LTS distribution (64-bit, = kernel >>>>> version 3.13.0-58-generic). >>>>>=20 >>>>> I first wonder whether this is intended behavior of audit = framework or >>>>> not. If it is intended, I also want to know how can we configure = auditd >>>>> differently to capture system calls from all processes.=20 >>>>>=20 >>>>> Thanks a lot for your help in advance! >>>>>=20 >>>>> Regards, Kangkook >>>>>=20 >>>>=20 >>>=20 >>>> -- >>>> Linux-audit mailing list >>>> Linux-audit@redhat.com = > >>>> https://www.redhat.com/mailman/listinfo/linux-audit = = > >>>=20 >>>=20 >>> - RGB >>>=20 >>> -- >>> Richard Guy Briggs = >> >>> Senior Software Engineer, Kernel Security, AMER ENG Base Operating = Systems, Red Hat >>> Remote, Ottawa, Canada >>> Voice: +1.647.777.2635, Internal: (81) 32635, Alt: = +1.613.693.0684x3545 >>=20 >=20 > - RGB >=20 > -- > Richard Guy Briggs > > Senior Software Engineer, Kernel Security, AMER ENG Base Operating = Systems, Red Hat > Remote, Ottawa, Canada > Voice: +1.647.777.2635, Internal: (81) 32635, Alt: = +1.613.693.0684x3545 --Apple-Mail=_82D7CD03-EAB8-4829-8B52-520FB4792792 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Hi Richard,

I also did the same experiment for the latest distributions = of Fedora core and Debian and here=E2=80=99s the results.

Fedora-22 (64-bit, = 4.0.4-301.fc22.x86_64): Problem reproduced.
Debian-8 = (64-bit, 3.16.0-4-amd64): Problem reproduced

Btw, Burn Alting (burn@swtf.dyndns.org) = suggested me to append audit=3D1 to kernel flag. I = added the option to boot-loader (grub) and problem went = away. 

Regards, Kangkook


On Sep 11, 2015, at 12:24 PM, = Richard Guy Briggs <rgb@redhat.com> wrote:

On 15/09/11, Kangkook Jee wrote:
=46rom the previous = reply, I think I misunderstood your question regarding kernel command = line. 
Here=E2=80=99s "cat /proc/cmdline=E2=80=9D results for = distributions that I=E2=80=99ve experimented. 

Ubuntu 14.04 (64-bit): 
   BOOT_IMAGE=3D/boot/vmlinuz-3.13.0-58-generic = root=3DUUID=3D7505f862-ce46-49e5-9d1c-e4e307844889 ro text quiet splash = vt.handoff=3D7

Ubuntu 12.04 (64-bit): 
   BOOT_IMAGE=3D/boot/vmlinuz-3.13.0-32-generic = root=3DUUID=3D5be789be-9b0c-463e-bd18-42bfa79fb24c ro quiet splash

CentOS 7 (64-bit): 
   BOOT_IMAGE=3D/vmlinuz-3.10.0-229.el7.x86_64 = root=3D/dev/mapper/centos-root ro rd.lvm.lv=3Dcentos/root = rd.lvm.lv=3Dcentos/swap crashkernel=3Dauto rhgb quiet = LANG=3Den_US.UTF-8

CentOS 6 (64-bit): 
   ro = root=3DUUID=3Da7d44560-adcc-4000-9584-8b9fcf2afd74 rd_NO_LUKS rd_NO_LVM = LANG=3Den_US.UTF-8 rd_NO_MD SYSFONT=3Dlatarcyrheb-sun16 = crashkernel=3D129M@0M  KEYBOARDTYPE=3Dpc KEYTABLE=3Dus rd_NO_DM = rhgb quiet

I don=E2=80=99t see any = audit=3D<value> entries from all examples above. 

Yes, this is what I was = seeking from you.  And you are correct, none of
them have audit=3D1 as I was hoping from at = least CentOS.  There is a
chance that the CentOS kernel was compiled with = audit=3D1 hardcoded, but I
think that is a pretty = small chance...

I'll have to look at this = closer...  But any Debian and Fedora data
points that you can provide would certainly be = useful.

/Kangkook

On Sep 11, 2015, at 5:50 AM, Richard Guy Briggs = <rgb@redhat.com> = wrote:

On 15/09/10, Kangkook Jee wrote:
Hi all,

I debugged a bit further to identify distributions that are = affected by the issue.
I repeated the same experiment with = sshd from 3 more distributions.

CentOS = Linux release 7.1.1503 (64-bit, 3.10.0-229.el7.x86_64): Problem NOT = reproduced
CentOS release 6.6 (64-bit, = 2.6.32-504.el6.x86_64): Problem NOT reproduced
Ubuntu = 12.04.5 LTS (64-bit, 3.13.0-32-generic): Problem reproduced

For each of these examples, what = is the value of the kernel command line
"audit=3D<value>" if it is even present?  It is = possible that the CentOS
examples include "audit=3D1" = while Ubuntu omits the line.  "cat
/proc/cmdline" = should tell you the answer.

After all, Ubuntu family are affected by the = issue and I could confirm
that results are inconsistent = across two different distribution
families. 

I would be curious what your = results are with a recent Debian and with a
recent = Fedora.

If you can let us know how can we workaround the issue, it = will be a great help.

Regards, Kangkook


On Sep 9, 2015, at 11:50 PM, Kangkook Jee <aixer77@gmail.com> = wrote:

Dear all,

We are developing custom user space audit agent to gather = system wide system
call trace. While experimenting with = various programs, we found out that
processes (daemons) = that started early (along with the system bootstrapping) do
not report any audit events at all. These processes typically = fall into PID
range of less than 2000. Here=E2=80=99s how = I reproduced the symptom with sshd daemon.

1.= Reboot the system

2. Add and enable audit = events
 # /sbin/auditctl -a exit,always -F arch=3Db64 = -S clone -S close -S creat -S dup
        -S dup2 -S = dup3 -S execve -S exit -S exit_group -S fork -S open -S openat 
        -S unlink -S = unlinkat -S vfork -S 288 -S accept -S bind -S connect 
        -S listen -S = socket -S socketpair
 # /sbin/auditctl -e1 -b = 102400

3. Connect to the system via ssh
  Audit messages generated only from child = processes and none are seen from
  the original = daemon.

4. Restart sshd 
  # = restart ssh

5. Connect again to the system = via ssh
 Now, we see audit messages from both parent = and child processes.

I did the experiment = from Ubuntu 14.04.2 LTS distribution (64-bit, kernel
version= 3.13.0-58-generic).

I first wonder whether = this is intended behavior of audit framework or
not. If it = is intended, I also want to know how can we configure auditd
differently to capture system calls from all processes. 

Thanks a lot for your help in advance!

Regards, Kangkook


--
Linux-audit mailing list
Linux-audit@redhat.com <mailto:Linux-audit@redhat.com>
https://www.redhat.com/mailman/listinfo/linux-audit <https://www.redhat.com/mailman/listinfo/linux-audit>


- RGB

--
Richard Guy Briggs <rbriggs@redhat.com <mailto:rbriggs@redhat.com>>
Senior = Software Engineer, Kernel Security, AMER ENG Base Operating Systems, Red = Hat
Remote, Ottawa, Canada
Voice: = +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545


- RGB

--
Richard Guy Briggs <rbriggs@redhat.com>
Senior Software Engineer, = Kernel Security, AMER ENG Base Operating Systems, Red Hat
Remote, Ottawa, Canada
Voice: +1.647.777.2635, Internal: (81) 32635, = Alt: +1.613.693.0684x3545

= --Apple-Mail=_82D7CD03-EAB8-4829-8B52-520FB4792792-- --===============2188333698036769264== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline --===============2188333698036769264==--