Request for comments on dejittering app

[John Sigler <linux.kernel@free.fr>]

Hello,

I'd like to describe what I'm trying to do, so people can comment.

Basically I'm just transfering data between two systems over a network 
(possibly the Internet, but not necessarily).

(Systems A and B are synchronized by NTP.)

System A receives data from an input device on the PCI bus. This input 
device provides data at constant bit rate R, meaning if the driver 
requests B bytes, the DMA will take approximately B/R seconds.

I wrote a 'send' (user space) app on system A which requests 1316 bytes 
from the input device, adds an RTP header, and sends this data in a UDP 
datagram to system B. The RTP header provides, notably, a sequence 
number and a timestamp.

The datagrams travel over several networks, a few get lost, some arrive 
at system B, not necessarily in the order they were sent, and definitely 
not every B/R seconds.

I wrote a 'recv' (user space) app on system B whose job is to receive 
the packets from system A and re-send them (either to an output device 
on the PCI bus, or to another system on the LAN) after having removed 
the network jitter.

Here's my implementation:

Note: Packet P{i} contains a timestamp T{i}

buffer N packets
grab the current (absolute, CLOCK_REALTIME) time H{0}
send P0

for i=1 to infinity
{
   recv() a few packets (e.g. 3 at most)
   compute H{i} = H{i-1} + (T{i} - T{i-1})
   sleep until H{i}
   send packet P_i
}

The packets were timestamped on system A,
thus for every i, T{i} - T{i-1} = ~B/R
Therefore, most of the network jitter will be removed.

Any comments so far?

Now all this seems nice in theory, but I'd also like to make it work in 
practice! (And I'm having some problems.)

R is typically 38 Mbit/s => B/R is typically 277 µs
Obviously, I'm going to need high-resolution timers if I want to sleep 
for so small an interval.

As far as I understand, each packet received will generate one IRQ, and 
each write to the PCI output device will generate one IRQ. This means 
3600 IRQs per second from the NIC, and 3600 IRQs per second from the PCI 
device. Is that reasonable? (Considering a 1267 MHz P3 with no IO-APIC.)

It is easy to reduce the number of IRQs from the PCI device by grouping 
several packets for a single write. It might be worthwhile.

One problem I have is that the PCI device's driver blocks until the 
device has acknowledged the data, and the write operation sometimes 
blocks for 200, 300, even 400 µs (I have not been able to tell why).

I might not need -rt, if I'm willing to handle several packets every 
time I wake up?

buffer N packets
grab the current (absolute) time H{0}
send P0

for i=1 to infinity
{
   sleep for at least X ms
   recv() some packets (how many? as much as I can?)
   grab the current time Hnow
   while H{i} < Hnow send P{i}
}

(Rough pseudo-code.)

I'm eager to hear anyone's comments and suggestions.

Regards.