From mboxrd@z Thu Jan  1 00:00:00 1970
From: Eric Wong <normalperson@yhbt.net>
Subject: Re: epoll with ONESHOT possibly fails to deliver events
Date: Thu, 20 Dec 2012 21:32:10 +0000
Message-ID: <20121220213210.GA22655@dcvr.yhbt.net>
References: <BC680BE9-F6EB-4278-819C-C552009CA884@yale.edu>
 <AF8622CA-CF9D-4F57-B4EB-FE088A752D80@yale.edu>
 <20121213093242.GA25436@dcvr.yhbt.net>
 <2F8919C4-395F-4029-8D4F-AD16A3815FC5@yale.edu>
 <9BE5F9A7-4D5B-4848-A0BA-55004240B6E0@yale.edu>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Cc: viro@zeniv.linux.org.uk, linux-fsdevel@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	"junchang.wang@yale.edu Wang" <junchang.wang@yale.edu>
To: Andreas Voellmy <andreas.voellmy@yale.edu>
Return-path: <linux-fsdevel-owner@vger.kernel.org>
Received: from dcvr.yhbt.net ([64.71.152.64]:48604 "EHLO dcvr.yhbt.net"
	rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
	id S1750983Ab2LTVcL (ORCPT <rfc822;linux-fsdevel@vger.kernel.org>);
	Thu, 20 Dec 2012 16:32:11 -0500
Content-Disposition: inline
In-Reply-To: <9BE5F9A7-4D5B-4848-A0BA-55004240B6E0@yale.edu>
Sender: linux-fsdevel-owner@vger.kernel.org
List-ID: <linux-fsdevel.vger.kernel.org>

Andreas Voellmy <andreas.voellmy@yale.edu> wrote:
> I wrote a C program that behaves similar to my original program and
> triggers the bug. The bug only arises when I use enough cores and
> threads (about 16). The program is here:
> https://github.com/AndreasVoellmy/epollbug/blob/master/epollbug.c

I finally took a closer look at your code.   I think your socketCheck()
thread is draining socket and causing the normal threads to miss
events.

Use the FIONREAD ioctl() instead to get unread bytes instead of recv().
If you want to recv() without draining the socket, you can also use
the MSG_PEEK flag.