[linux-dvb] How to convert MPEG-TS to MPEG-PS on the fly?

[linux-dvb] How to convert MPEG-TS to MPEG-PS on the fly?

[Josef Wolf]

Hello,

I'd like to convert live mpeg-ts streams from DVB-S on the fly into
a mpeg-ps stream.  I know that (for example)

  mencoder -oac copy -ovc copy -of mpeg -quiet infile -o outfile.mpg

can (almost) do what I want.  But I want to do it on the fly for
multiple streams in parallel and without any temporary files.  So I
decided to roll my own implementation.

I started demuxing the TS stream:

- first, I find the PIDs I am interested in (e.g. pids 110,120,121
  for the german ZDF channel on Astra).

- then, I extract all the PES packets from those streams, leaving
  the PES headers intact.  That is, I just snip off the four bytes
  TS header and throw away the adaptation field.

  Since the PES_packet_length for the video stream is zero, I assume
  I need to collect data until the payload_unit_start_indicator
  indicates the start of the next PES packet (ignoring TS header
  and adaptation_field, of course).
  

So, now I have (with the ZDF example) three streams of PES packets:

  -> video packets coming from PID 110
  -> first audio stream coming from PID 120
  -> second audio stream coming from PID 121

Both, mplayer and vlc play the audio streams fine.  But the video
stream gives lots of artefacts (blocks) with vlc.  mplayer completely
refuses to play it with the following message:

  Playing zdf.test.
  libavformat file format detected.
  Seek failed
  LAVF: no audio or video headers found - broken file?

Why that?  Maybe I should strip off the PES headers also to get a
valid ES?  Or is PES not suitable for playback at all?  Or maybe I
should strip off packets with specific Stream_id?  Here is the
sequence of stream_id's contained in the stream:

  jw@dvb1:~$ dvbsnoop -s pes -if zdf.test|grep Stream_id|head -40
  Stream_id: 224 (0xe0)  [= ITU-T Rec. H.262 | ISO/IEC 13818-2 or ISO/IEC 11172-2 video stream]
  Stream_id: 0 (0x00)  [= picture_start_code]
  Stream_id: 181 (0xb5)  [= extension_start_code]
  Stream_id: 1 (0x01)  [= slice_start_code]
  Stream_id: 2 (0x02)  [= slice_start_code]
  [ consecutive lines deleted ]
  Stream_id: 34 (0x22)  [= slice_start_code]
  Stream_id: 35 (0x23)  [= slice_start_code]
  [ here the list of stream ids start over again and repeats ]

Maybe I should discard some of those to get a proper PES?


The next step would be to multiplex the three streams into a PS.
For this, I will have to insert the system_clock_reference_base(SCR).
Is this available somewhere in the PES headers?  Or can I reuse
(recalculate?) the program_clock_reference_base (PCR) from the
adaptation_field?


_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] How to convert MPEG-TS to MPEG-PS on the fly?

[Steven Toth]

Josef Wolf wrote:
> Hello,
> 
> I'd like to convert live mpeg-ts streams from DVB-S on the fly into
> a mpeg-ps stream.  I know that (for example)
> 
>   mencoder -oac copy -ovc copy -of mpeg -quiet infile -o outfile.mpg

Can't you use named pipes with mencoder and direct the flow into your 
target process?

- Steve



_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] How to convert MPEG-TS to MPEG-PS on the fly?

[Josef Wolf]

On Wed, Aug 20, 2008 at 05:17:18PM -0400, Steven Toth wrote:
> Josef Wolf wrote:
> >Hello,
> >
> >I'd like to convert live mpeg-ts streams from DVB-S on the fly into
> >a mpeg-ps stream.  I know that (for example)
> >
> >  mencoder -oac copy -ovc copy -of mpeg -quiet infile -o outfile.mpg
> 
> Can't you use named pipes with mencoder and direct the flow into your 
> target process?

In principle, yes.  But there is a big drawback to such a solution:
the pipes (and demuxing/muxing in a different process) will introduce
lots of context switches.  Since I want to convert four full
transponders at the same time (about 25 channels), this will certainly
kill my 450MHz PII machine.  Let alone the 25 additional mencoder
processes all running in parallel.


_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] How to convert MPEG-TS to MPEG-PS on the fly?

[barry bouwsma]

--- On Wed, 8/20/08, Josef Wolf <jw@raven.inka.de> wrote:

> > >I'd like to convert live mpeg-ts streams from DVB-S on the fly into
> > >a mpeg-ps stream.  I know that (for example)
> 
> In principle, yes.  But there is a big drawback to such a solution:
> the pipes (and demuxing/muxing in a different process) will introduce
> lots of context switches.  Since I want to convert four full
> transponders at the same time (about 25 channels), this will certainly
> kill my 450MHz PII machine.  Let alone the 25 additional mencoder
> processes all running in parallel.

Can I ask for more details?  As I'm using a 200MHz and similar
machines for full- and partial-TS work from 4 DVB cards, I have
some concerns, that may or may not be a problem.

What sort of cards are you using -- internal PCI or external USB?
When I'm handling a high bandwidth (BBC-HD) program on my internal
PCI card, not even a full transport stream, I start to feel the
CPU pinch, which will be far worse for USB streams.  Given about
36Mbit/sec per transponder, you'll be schlepping quite a bit of
data, which may give you concern.  Keep an eye on idle time.

Of course, my machine is only an MMX Pentium, and only 32MB RAM,
so will by far reach its capacity well before yours; mine seems to
max out with a 15Mbit/sec HD stream (internal PCI), a full 16Mbit/
sec transport stream via USB of DVB-T, and two filtered USB1 partial
radio streams, doing nothing but writing files to internal disks.
With less than this I've got adequate headroom.

Are you intending to use the PSen in real-time like it seems you
describe, or will you/can you be recording for later use?

It sounds like you may, given your example of ZDF, be streaming
the oeffis from 10744 (arte & Co), 11836 (ARD & Co), ZDF, and some
Dritte programs at 12110.  These are all now higher quality streams
than the commercial private channels which you may also be using
to reach your 25 channels, with video bandwidths usually around 5
to 8Mbit/sec per stream.

If I'm not mistaken, your program stream should include the video PID
data, plus an audio PID (only one, I'll assume the primary mp2 audio,
though you may choose the AC3 where present) from each channel, so no
worry about second/alternative audio, teletext, or additional program
tables sent in the full stream.

So, for example, assuming you want all the channels from the ARD
transponder, you need PIDs 101+102 (ARD), 201+202 (BR), 301+302 (hr),
601+602 (WDR), (701+702 BR-alpha if not taken from elsewhere), and
801+802 (SWR-BW), with x01 the video channel, and x02 the simple
mp2 audio.  Likewise for the other transponders of interest.

You should be able to use `dvbstream' specifying only those needed
PIDs and either write a file directly for later processing, or pipe
for immediate streaming, saving a bit of bandwidth from data you'd
not be using (teletext and such) -- provided you don't need to use
software demuxing of a full TS with your cards.

Take a look at hacking a program called `superdemux' if I recall,
which you should be able to use to split the stream into its
PID pairs -- I use it hacked to write multiple radio streams into
separate files.  Alternatively, a hack based on a teletext program
may help you separate the streams.

The program `ts2ps', part of the dvb-mpegtools suite, or something
similar from those programs, can be used to repack the data into
PS, and should be a lot more lightweight than mencoder.

Timing data is partially within each PID, so you should be able to
get a usable PS from just the two PIDs.  At least, that's how I
used to record programs, until learning that the additional PIDs
could be helpful.

Given the amount of data you'll be handling on your 450MHz machine,
you may see lost packets and thus corruption at full load, so test
by working your way from a single functioning transponder up to the
full workload.

I don't have a turnkey solution, but those are the building blocks
I'd be looking at to try to do what I think you want to do; however,
with plenty of source hacking.  Of course, if you're not doing exactly
or approximately what I've guessed, then my advice may need tweaking.


Hope this is somewhat useful
barry bouwsma


      


_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] How to convert MPEG-TS to MPEG-PS on the fly?

[Josef Wolf]

Thank you for the extensive answer, Barry!

On Thu, Aug 21, 2008 at 05:10:10AM -0700, barry bouwsma wrote:
> --- On Wed, 8/20/08, Josef Wolf <jw@raven.inka.de> wrote:
> 
> > > >I'd like to convert live mpeg-ts streams from DVB-S on the fly into
> > > >a mpeg-ps stream.  I know that (for example)
> > 
> > In principle, yes.  But there is a big drawback to such a solution:
> > the pipes (and demuxing/muxing in a different process) will introduce
> > lots of context switches.  Since I want to convert four full
> > transponders at the same time (about 25 channels), this will certainly
> > kill my 450MHz PII machine.  Let alone the 25 additional mencoder
> > processes all running in parallel.
> 
> Can I ask for more details?  As I'm using a 200MHz and similar
> machines for full- and partial-TS work from 4 DVB cards, I have
> some concerns, that may or may not be a problem.
> 
> What sort of cards are you using -- internal PCI or external USB?

Technotrends internal PCI budget cards.

> When I'm handling a high bandwidth (BBC-HD) program on my internal
> PCI card, not even a full transport stream, I start to feel the
> CPU pinch, which will be far worse for USB streams.

I am not interested in HD (yet).  But surely this will change at
some point in time.

> Given about
> 36Mbit/sec per transponder, you'll be schlepping quite a bit of
> data, which may give you concern.  Keep an eye on idle time.

Grabbing 18 TV TS streams from 3 transponders gives 60% idle at the
moment. (my fourth card has died and I have not bought a replacement
yet.  AFAIK they have stopped manufacturing the cards :-(( )

> Of course, my machine is only an MMX Pentium, and only 32MB RAM,
> so will by far reach its capacity well before yours; mine seems to
> max out with a 15Mbit/sec HD stream (internal PCI), a full 16Mbit/
> sec transport stream via USB of DVB-T, and two filtered USB1 partial
> radio streams, doing nothing but writing files to internal disks.

Watch out for a catch when writing to internal (ext3) disks: When
the commit-interval is reached and the journal is flushed, write(2)
blocks for a significant time.  You risk buffer overruns on the
incoming TS if you are reading in the same thread.  I had this problem
a long time ago when I did my first experiments with DVB drivers.

> Are you intending to use the PSen in real-time like it seems you
> describe, or will you/can you be recording for later use?

Both.  But the recording would probably be by grabbing the already
converted real-time stream via

  wget http://dvb.local:1234/zdf.ps

or something.  Decoupling recording from demuxing saves me from the
above mentioned catch.  In addition, recording can be done on every
host in my network.  I could even roll a script based on LWP to get
the start/end time of the recording correct.

> It sounds like you may, given your example of ZDF, be streaming
> the oeffis from 10744 (arte & Co), 11836 (ARD & Co), ZDF, and some
> Dritte programs at 12110.

11836 + 11954 + 12188 + 12545.  Unfortunately, they have moved arte
from 12188 and a fifth card is not supported by the drivers :-(

> If I'm not mistaken, your program stream should include the video PID
> data, plus an audio PID (only one, I'll assume the primary mp2 audio,
> though you may choose the AC3 where present) from each channel, so no
> worry about second/alternative audio, teletext, or additional program
> tables sent in the full stream.

No, I want to get all the streams so I can select language on the client
(vlc or something).

> The program `ts2ps', part of the dvb-mpegtools suite, or something
> similar from those programs, can be used to repack the data into
> PS, and should be a lot more lightweight than mencoder.

This would still need the pipes.  Introducing pipes would introduce
significant context switching since pipes are (AFAIR) only 8kbytes.
So, assuming 500kbytes/sec, I would get 240 context switches per
second for every program.  This gives a total of 6000 context switches
every second.  You need _really_ big iron to cope with this.

> Timing data is partially within each PID, so you should be able to
> get a usable PS from just the two PIDs.

Yes, timing is within the PID.  But there are lots of times there:
 - PCR, OPCR, DTS_next_AU from the adaptation field
 - PTS, DTS, ESCR_base, ECSR_extension, ES_rate from the PES header
 - there's the possibility that the PCR is carried in a different
   PID (indicated by the PCR_PID field in the PMT)

Which one do I have to use to create the PS header?  I guess I have to
use the PTS from the PES, but I fail to deduce this from the iso-13818-1.

> Given the amount of data you'll be handling on your 450MHz machine,
> you may see lost packets and thus corruption at full load, so test
> by working your way from a single functioning transponder up to the
> full workload.

I don't have lost packets, but still artefacts in the video.  Looks like
the additional stream_id's on the video-PID disturbs vlc's decoder.

_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] How to convert MPEG-TS to MPEG-PS on the fly?

[Kevin Sheehan]

Josef,

Barry was right on the money with the ts2ps suggestion below.  It's part
of the libdvb package.  You don't have to use the dvb-mpegtools app, you
can just use the lib in yours - no pipes, etc.

rgs

> Thank you for the extensive answer, Barry!
>
> On Thu, Aug 21, 2008 at 05:10:10AM -0700, barry bouwsma wrote:
>> --- On Wed, 8/20/08, Josef Wolf <jw@raven.inka.de> wrote:
>>
>> > > >I'd like to convert live mpeg-ts streams from DVB-S on the fly into
>> > > >a mpeg-ps stream.  I know that (for example)
>> >
>> > In principle, yes.  But there is a big drawback to such a solution:
>> > the pipes (and demuxing/muxing in a different process) will introduce
>> > lots of context switches.  Since I want to convert four full
>> > transponders at the same time (about 25 channels), this will certainly
>> > kill my 450MHz PII machine.  Let alone the 25 additional mencoder
>> > processes all running in parallel.
>>
>> Can I ask for more details?  As I'm using a 200MHz and similar
>> machines for full- and partial-TS work from 4 DVB cards, I have
>> some concerns, that may or may not be a problem.
>>
>> What sort of cards are you using -- internal PCI or external USB?
>
> Technotrends internal PCI budget cards.
>
>> When I'm handling a high bandwidth (BBC-HD) program on my internal
>> PCI card, not even a full transport stream, I start to feel the
>> CPU pinch, which will be far worse for USB streams.
>
> I am not interested in HD (yet).  But surely this will change at
> some point in time.
>
>> Given about
>> 36Mbit/sec per transponder, you'll be schlepping quite a bit of
>> data, which may give you concern.  Keep an eye on idle time.
>
> Grabbing 18 TV TS streams from 3 transponders gives 60% idle at the
> moment. (my fourth card has died and I have not bought a replacement
> yet.  AFAIK they have stopped manufacturing the cards :-(( )
>
>> Of course, my machine is only an MMX Pentium, and only 32MB RAM,
>> so will by far reach its capacity well before yours; mine seems to
>> max out with a 15Mbit/sec HD stream (internal PCI), a full 16Mbit/
>> sec transport stream via USB of DVB-T, and two filtered USB1 partial
>> radio streams, doing nothing but writing files to internal disks.
>
> Watch out for a catch when writing to internal (ext3) disks: When
> the commit-interval is reached and the journal is flushed, write(2)
> blocks for a significant time.  You risk buffer overruns on the
> incoming TS if you are reading in the same thread.  I had this problem
> a long time ago when I did my first experiments with DVB drivers.
>
>> Are you intending to use the PSen in real-time like it seems you
>> describe, or will you/can you be recording for later use?
>
> Both.  But the recording would probably be by grabbing the already
> converted real-time stream via
>
>   wget http://dvb.local:1234/zdf.ps
>
> or something.  Decoupling recording from demuxing saves me from the
> above mentioned catch.  In addition, recording can be done on every
> host in my network.  I could even roll a script based on LWP to get
> the start/end time of the recording correct.
>
>> It sounds like you may, given your example of ZDF, be streaming
>> the oeffis from 10744 (arte & Co), 11836 (ARD & Co), ZDF, and some
>> Dritte programs at 12110.
>
> 11836 + 11954 + 12188 + 12545.  Unfortunately, they have moved arte
> from 12188 and a fifth card is not supported by the drivers :-(
>
>> If I'm not mistaken, your program stream should include the video PID
>> data, plus an audio PID (only one, I'll assume the primary mp2 audio,
>> though you may choose the AC3 where present) from each channel, so no
>> worry about second/alternative audio, teletext, or additional program
>> tables sent in the full stream.
>
> No, I want to get all the streams so I can select language on the client
> (vlc or something).
>
>> The program `ts2ps', part of the dvb-mpegtools suite, or something
>> similar from those programs, can be used to repack the data into
>> PS, and should be a lot more lightweight than mencoder.
>
> This would still need the pipes.  Introducing pipes would introduce
> significant context switching since pipes are (AFAIR) only 8kbytes.
> So, assuming 500kbytes/sec, I would get 240 context switches per
> second for every program.  This gives a total of 6000 context switches
> every second.  You need _really_ big iron to cope with this.
>
>> Timing data is partially within each PID, so you should be able to
>> get a usable PS from just the two PIDs.
>
> Yes, timing is within the PID.  But there are lots of times there:
>  - PCR, OPCR, DTS_next_AU from the adaptation field
>  - PTS, DTS, ESCR_base, ECSR_extension, ES_rate from the PES header
>  - there's the possibility that the PCR is carried in a different
>    PID (indicated by the PCR_PID field in the PMT)
>
> Which one do I have to use to create the PS header?  I guess I have to
> use the PTS from the PES, but I fail to deduce this from the iso-13818-1.
>
>> Given the amount of data you'll be handling on your 450MHz machine,
>> you may see lost packets and thus corruption at full load, so test
>> by working your way from a single functioning transponder up to the
>> full workload.
>
> I don't have lost packets, but still artefacts in the video.  Looks like
> the additional stream_id's on the video-PID disturbs vlc's decoder.
>
> _______________________________________________
> linux-dvb mailing list
> linux-dvb@linuxtv.org
> http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb
>


_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] How to convert MPEG-TS to MPEG-PS on the fly?

[barry bouwsma]

--- On Fri, 8/22/08, Kevin Sheehan <linux-dvb@ephedrine.net> wrote:

> Barry was right on the money with the ts2ps suggestion below.  It's part
> of the libdvb package.  You don't have to use the dvb-mpegtools app, you
> can just use the lib in yours - no pipes, etc.

Actually, I either need to decide to finish waking up, or get more
sleep before trying to think, but I did learn something new from
this conversation...

My understanding of a PS stream (redundant, yeah) was based on how
ts2ps created one, with just two PIDs (0xe0 and 0xc0), which I then
assumed was the norm.

But in the case of ZDF from the original post, as well as ARD, BR,
Sat1, and so on, Josef wants to make the multiple available audio
streams from the TS (primary mp2 audio, second narrative audio/
second lang, plus AC3) within the PS for the client to select.

I'd imagine a bit of hacking is needed to ts2ps to allow one to add
more than a single audio PID to the PS, but it should be possible.

But, if there's a need for, say, the DVB subtitles from ZDF, or the
teletext (for teletext subtitles, amongst others), or any other
auxiliary data within the stream, if I understand right, one would
need to stick to writing a partial transport stream, which then
could be filtered to PS as needed.

Of course, I could be wrong, as I'm not too familiar with libdvb...



sleeeeeeep...
barry bouwsma


      


_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] How to convert MPEG-TS to MPEG-PS on the fly?

[Josef Wolf]

On Thu, Aug 21, 2008 at 10:15:37PM -0700, barry bouwsma wrote:
> --- On Fri, 8/22/08, Kevin Sheehan <linux-dvb@ephedrine.net> wrote:
> 
> > Barry was right on the money with the ts2ps suggestion below.  It's part
> > of the libdvb package.  You don't have to use the dvb-mpegtools app, you
> > can just use the lib in yours - no pipes, etc.
> 
> My understanding of a PS stream (redundant, yeah) was based on how
> ts2ps created one, with just two PIDs (0xe0 and 0xc0), which I then
> assumed was the norm.
> 
> But in the case of ZDF from the original post, as well as ARD, BR,
> Sat1, and so on, Josef wants to make the multiple available audio
> streams from the TS (primary mp2 audio, second narrative audio/
> second lang, plus AC3) within the PS for the client to select.
> 
> I'd imagine a bit of hacking is needed to ts2ps to allow one to add
> more than a single audio PID to the PS, but it should be possible.
> 
> But, if there's a need for, say, the DVB subtitles from ZDF, or the
> teletext (for teletext subtitles, amongst others), or any other
> auxiliary data within the stream, if I understand right, one would
> need to stick to writing a partial transport stream, which then
> could be filtered to PS as needed.

In addition to audio, video, program_stream_map, private_stream-X
and program_stream_directory, Table 2-19 in iso-13818-1 defines 9
more stream types:

 - ECM_stream
 - EMM_stream
 - 13818-6_DSMCC_stream
 - ISO/IEC_13522_stream
 - ITU-T Reac. H.222.1 type A
 - ITU-T Reac. H.222.1 type B
 - ITU-T Reac. H.222.1 type C
 - ITU-T Reac. H.222.1 type D
 - ITU-T Reac. H.222.1 type E

I have not looked into the details, but I guess teletext and subtitles
might be included in this list.

_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] How to convert MPEG-TS to MPEG-PS on the fly?

[Josef Wolf]

On Fri, Aug 22, 2008 at 11:07:47AM +1000, Kevin Sheehan wrote:

> Barry was right on the money with the ts2ps suggestion below.  It's part
> of the libdvb package.  You don't have to use the dvb-mpegtools app, you
> can just use the lib in yours - no pipes, etc.

I know.  But I still consider ts2ps to be too heavy for my application.
It goes and parses all the PES contents, which eats much CPU.

But at least, ts2pes was very helpful in analyzing the differences in
the stream between what I created and what ts2pes created.  Finally, I
have found the problem:

I appears that PES_packet_length==0 is allowed in TS _only_.  While
unpacking, the long packet (I have seen up to 100 kbytes) extracted 
from the TS needs to be split up into smaller ones.  ps2pes splits
into pieces with PES_packet_length==0x7fa and prepends an empty PES
header (with only the length specification) to each of the new packets.

Finally, I can generate a PES stream with one video and multiple
audio streams which is played pretty fine by mplayer and vlc.

Now I need to add the PS pack header (ts2ps adds 14 bytes) and the
system header (ts2ps adds 18 bytes).  For this, I have two more
questions:

1. Is the system_header required at all?  Table 2-33 in iso-13818-1
   seems to make it optional.

2. For creating the PS pack header, the only missing information is
   the program_mux_rate.  How is this value calculated?  How accurate
   has this value to be?  If the stream bit rate changes (maybe because
   of changes in resolution), do I need to adopt this value and generate
   a MPEG_program_end_code and add a new pack header with the new
   value?  Or will the player stop when it detects MPEG_program_end_code?

_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] How to convert MPEG-TS to MPEG-PS on the fly?

[Nick Ludlam]

On 22 Aug 2008, at 15:44, Josef Wolf wrote:

> On Fri, Aug 22, 2008 at 11:07:47AM +1000, Kevin Sheehan wrote:
>
>> Barry was right on the money with the ts2ps suggestion below.  It's  
>> part
>> of the libdvb package.  You don't have to use the dvb-mpegtools  
>> app, you
>> can just use the lib in yours - no pipes, etc.
>
> I know.  But I still consider ts2ps to be too heavy for my  
> application.
> It goes and parses all the PES contents, which eats much CPU.


I might be slightly out in terms of your requirements, but I've had  
success using
the 'replex' tool ( http://www.metzlerbros.org/dvb/ ) to do on-the-fly  
stream
conversions of a recorded TS into a PS with a specified video and  
audio PID.

The command I wrapped in a popen call from a script was:

	/opt/local/bin/replex -o /dev/stdout -i TS -v %s -a %s -t MPEG2 /dev/ 
stdin

Where the two %s's were replaced with the desired audio PID and video  
PID.
Then I wrote to, and read from the pipes accordingly.

The only issue I had with this was that it was prone to breaking  
unless the
TS stream had little or no corruption. The CPU usage for replex was
considerably lower than using mencoder or ffmpeg, although I don't know
how difficult it would be to get replex to insert multiple audio  
streams into
the resulting PS.

Nick

--
Nick Ludlam
nick@recoil.org





_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] How to convert MPEG-TS to MPEG-PS on the fly?

[Josef Wolf]

On Fri, Aug 22, 2008 at 04:44:56PM +0100, Nick Ludlam wrote:
> On 22 Aug 2008, at 15:44, Josef Wolf wrote:
>>On Fri, Aug 22, 2008 at 11:07:47AM +1000, Kevin Sheehan wrote:
>>
>>>Barry was right on the money with the ts2ps suggestion below.  It's  
>>>part of the libdvb package.  You don't have to use the dvb-mpegtools  
>>>app, you can just use the lib in yours - no pipes, etc.

I have now changed my application to create exactly what ts2ps creates.
Now I notice that neither mplayer nor vlc play the stream created by
ts2ps.

Vlc gives tons of error messages like this:

  [00000365] main video output warning: vout warning: early picture skipped (47722279483)
  [00000359] main audio output warning: received buffer in the future (47722179597)

no audio and black video :-(

If I remove the PS pack header and the PS system header (stream-id 0xba
and 0xbb) then both play the stream, but no STB plays it :-(

Next, I tried:

> The command I wrapped in a popen call from a script was:
> 
> 	/opt/local/bin/replex -o /dev/stdout -i TS -v %s -a %s -t MPEG2 
> 	/dev/ stdin

With this command, I get hopping video with VLC.

It seems to be a mess.  None of the programs seem to produce proper
streams.  Only mencoder seems to generate a proper stream.  But AFAICS,
mencoder completely decodes the stream and re-encodes it again, eating
up all the CPU.

Any more ideas how to do the conversion?

BTW: Can anybody recommend a good book on the topic?

_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] How to convert MPEG-TS to MPEG-PS on the fly?

[Nico Sabbi]

Il giorno lun, 25/08/2008 alle 21.02 +0200, Josef Wolf ha scritto:
> On Fri, Aug 22, 2008 at 04:44:56PM +0100, Nick Ludlam wrote:
> > On 22 Aug 2008, at 15:44, Josef Wolf wrote:
> >>On Fri, Aug 22, 2008 at 11:07:47AM +1000, Kevin Sheehan wrote:
> >>
> >>>Barry was right on the money with the ts2ps suggestion below.  It's  
> >>>part of the libdvb package.  You don't have to use the dvb-mpegtools  
> >>>app, you can just use the lib in yours - no pipes, etc.


dvb-mpegtools, like mplex, demuxes the TS and loses all informations
about timestamps => very likely the output will be out of sync.

> 
> I have now changed my application to create exactly what ts2ps creates.
> Now I notice that neither mplayer nor vlc play the stream created by
> ts2ps.


I'v eseen ts2ps make a real mess in countless occasions, especially 
with broken TSs.

> 
> Vlc gives tons of error messages like this:
> 
>   [00000365] main video output warning: vout warning: early picture skipped (47722279483)
>   [00000359] main audio output warning: received buffer in the future (47722179597)
> 
> no audio and black video :-(
> 
> If I remove the PS pack header and the PS system header (stream-id 0xba
> and 0xbb) then both play the stream, but no STB plays it :-(


You must have corrupted some header in your application :)

> 
> Next, I tried:
> 
> > The command I wrapped in a popen call from a script was:
> > 
> > 	/opt/local/bin/replex -o /dev/stdout -i TS -v %s -a %s -t MPEG2 
> > 	/dev/ stdin
> 
> With this command, I get hopping video with VLC.


replex never worked with at least 60% of the TSs I fed it with,
that left me no other choise than to fix mencoder's muxer, as long
as something as horribly crippled and broken like mencoder can be
fixed (that may not work if you give it 100% correct timestamps
for every single frame)

> 
> It seems to be a mess.  None of the programs seem to produce proper
> streams.  Only mencoder seems to generate a proper stream.  But AFAICS,
> mencoder completely decodes the stream and re-encodes it again, eating
> up all the CPU.

it doesn't decode: it just demuxes, although parsing audio and video
frame headers is probably eating up a good % of your cpu.
If you want to make it lighter either
- use ffmpeg, it can be surely used to generate many PS from a single TS
  although its command line is really challenging
- use -demuxer lavf and fix libmpdemux/muxer_mpeg.c so that frames are
  not analyzed within the muxer itself, but just copied verbatim in
  teh corresponding mpeg_frame_t[] array


> 
> Any more ideas how to do the conversion?
> 
> BTW: Can anybody recommend a good book on the topic?
> 

none that I know of, but muxing mpeg-ps is more trivial than it looks
like when reading the specs


_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] How to convert MPEG-TS to MPEG-PS on the fly?

[barry bouwsma]

--- On Tue, 8/26/08, Nico Sabbi <nicola.sabbi@poste.it> wrote:

> dvb-mpegtools, like mplex, demuxes the TS and loses all informations
> about timestamps => very likely the output will be out of sync.

I wondered about this, as I had used `something' to convert a
TS to a file that I could play on a standalone DVD-player with
card/USB slots.

It turns out I used `replex'; unfortunately, the source material
wasn't something for which timestamps were too critical -- stop-
motion animation from Nick Park/Aardman Animations, broadcast
with a typical fraction of a second offset betweem audio and
video PTSen in real-time:

-rwxr-xr-x 1 root root 188704772 2007-05-25 16:34 /selinux/misc/Shaun_the_Sheep-Shape_Up_With_Shaun-replex-mpeg2.mpg
-rwxr-xr-x 1 root root 193982464 2007-05-25 17:11 /selinux/misc/Shaun_the_Sheep-Bathtime-replex-dvd.mpg

In these files, I no longer see any useable timestamps (those I
do see start around 0 and work up), and as noted, if there were
to be loss of sync, I'd be unlikely to notice with this source.


For laughs, I now converted a short BBC-Four TS I had recorded as
a test with `ts2ps', and there, the PTS/DTS are present in the PS
and match those seen in the TS...

     ==> system_clock_reference_base: 859961626 (0x3341f91a)  [= 90 kHz-Timestamp: 2:39:15.1291]
         ==> PTS: 5154932522 (0x13342072a)  [= 90 kHz-Timestamp: 15:54:37.0280]
         ==> DTS: 5154921721 (0x13341dcf9)  [= 90 kHz-Timestamp: 15:54:36.9080]
         ==> PTS: 5154925321 (0x13341eb09)  [= 90 kHz-Timestamp: 15:54:36.9480]
         ==> PTS: 5154928921 (0x13341f919)  [= 90 kHz-Timestamp: 15:54:36.9880]
     ==> system_clock_reference_base: 859937348 (0x33419a44)  [= 90 kHz-Timestamp: 2:39:14.8594]
         ==> PTS: 5154908244 (0x13341a854)  [= 90 kHz-Timestamp: 15:54:36.7582]
         ==> PTS: 5154943322 (0x13342315a)  [= 90 kHz-Timestamp: 15:54:37.1480]
         ==> DTS: 5154932521 (0x133420729)  [= 90 kHz-Timestamp: 15:54:37.0280]
         ==> PTS: 5154936121 (0x133421539)  [= 90 kHz-Timestamp: 15:54:37.0680]
         ==> PTS: 5154939721 (0x133422349)  [= 90 kHz-Timestamp: 15:54:37.1080]
         ==> PTS: 5154954122 (0x133425b8a)  [= 90 kHz-Timestamp: 15:54:37.2680]

If I had lots of time to kill, I'd separate the audio stream PTSen
from the video (they have a couple tenths-of-second offset relative
to each other)...


barry bouwsma


      


_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] How to convert MPEG-TS to MPEG-PS on the fly?

[Josef Wolf]

On Tue, Aug 26, 2008 at 05:00:18AM -0700, barry bouwsma wrote:

> For laughs, I now converted a short BBC-Four TS I had recorded as
> a test with `ts2ps', and there, the PTS/DTS are present in the PS
> and match those seen in the TS...
> 
>      ==> system_clock_reference_base: 859961626 (0x3341f91a)  [= 90 kHz-Timestamp: 2:39:15.1291]

This is a PS pack header, right?

>          ==> PTS: 5154932522 (0x13342072a)  [= 90 kHz-Timestamp: 15:54:37.0280]
>          ==> DTS: 5154921721 (0x13341dcf9)  [= 90 kHz-Timestamp: 15:54:36.9080]
>          ==> PTS: 5154925321 (0x13341eb09)  [= 90 kHz-Timestamp: 15:54:36.9480]
>          ==> PTS: 5154928921 (0x13341f919)  [= 90 kHz-Timestamp: 15:54:36.9880]

Are those PES headers from audio or from video?  Noticed the hop here?

>      ==> system_clock_reference_base: 859937348 (0x33419a44)  [= 90 kHz-Timestamp: 2:39:14.8594]

PS pack header again? Hop backwards from previous pack header?

>          ==> PTS: 5154908244 (0x13341a854)  [= 90 kHz-Timestamp: 15:54:36.7582]
>          ==> PTS: 5154943322 (0x13342315a)  [= 90 kHz-Timestamp: 15:54:37.1480]
>          ==> DTS: 5154932521 (0x133420729)  [= 90 kHz-Timestamp: 15:54:37.0280]
>          ==> PTS: 5154936121 (0x133421539)  [= 90 kHz-Timestamp: 15:54:37.0680]
>          ==> PTS: 5154939721 (0x133422349)  [= 90 kHz-Timestamp: 15:54:37.1080]
>          ==> PTS: 5154954122 (0x133425b8a)  [= 90 kHz-Timestamp: 15:54:37.2680]

Again hops.  Have you tried to play this stream with vlc?

BTW: what is the DTS good for?  Isn't PTS the relevant time for playbacK?
     What difference does it make when a frame was decoded as long as it
     is presented at the correct time?

     And what is the SCRB good for?  I am totally confused by all those times.

_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] How to convert MPEG-TS to MPEG-PS on the fly?

[barry bouwsma]

--- On Tue, 8/26/08, Josef Wolf <jw@raven.inka.de> wrote:

> >      ==> system_clock_reference_base: 859961626
> (0x3341f91a)  [= 90 kHz-Timestamp: 2:39:15.1291]
> 
> This is a PS pack header, right?

Correct:

Packet_start_code_prefix: 0x000001
Stream_id: 186 (0xba)  [= MPEG_pack_start (PS)]
Pack_header:
    fixed '01': 1 (0x01)
    system_clock_reference_base:
        bit[32..30]: 0 (0x00)
        marker_bit: 1 (0x01)
        bit[29..15]: 26243 (0x6683)
        marker_bit: 1 (0x01)
        bit[14..0]: 31002 (0x791a)
        marker_bit: 1 (0x01)
         ==> system_clock_reference_base: 859961626 (0x3341f91a)  [= 90 kHz-Timestamp: 2:39:15.1291]


> >          ==> PTS: 5154932522 (0x13342072a)  [= 90
> kHz-Timestamp: 15:54:37.0280]
> >          ==> DTS: 5154921721 (0x13341dcf9)  [= 90
> kHz-Timestamp: 15:54:36.9080]
> >          ==> PTS: 5154925321 (0x13341eb09)  [= 90
> kHz-Timestamp: 15:54:36.9480]
> >          ==> PTS: 5154928921 (0x13341f919)  [= 90
> kHz-Timestamp: 15:54:36.9880]
> 
> Are those PES headers from audio or from video?  Noticed
> the hop here?

Both, and yes.  I vaguely recall some years ago using something
which gave more info about the timestamps, but I cannot for the
life of me remember what, or how.  For the above, I simply `grep'ed
on `Timestamp' and lost the context, sorry.

Anyway, let me give more detail from this PS file, using dvbsnoop,
to put the times I quoted into context.  My point was that I was
seeing the very same timestamps as in the original TS file, while
with `replex', these timestamps were lost.

Whether `replex' preserved the relative offset between video and
audio, I'd have to either create a new stream and compare, or find
the original from which I converted to the replexed files.

Back to the output from `ts2ps':


Stream_id: 224 (0xe0)  [= ITU-T Rec. H.262 | ISO/IEC 13818-2 or ISO/IEC 11172-2
video stream]
             ==> PTS: 5154932522 (0x13342072a)  [= 90 kHz-Timestamp: 15:54:37.0280]
             ==> DTS: 5154921721 (0x13341dcf9)  [= 90 kHz-Timestamp: 15:54:36.9080]

Stream_id: 224 (0xe0)  [= ITU-T Rec. H.262 | ISO/IEC 13818-2 or ISO/IEC 11172-2
video stream]
             ==> PTS: 5154925321 (0x13341eb09)  [= 90 kHz-Timestamp: 15:54:36.9480]

Stream_id: 224 (0xe0)  [= ITU-T Rec. H.262 | ISO/IEC 13818-2 or ISO/IEC 11172-2
video stream]
             ==> PTS: 5154928921 (0x13341f919)  [= 90 kHz-Timestamp: 15:54:36.9880]


Stream_id: 186 (0xba)  [= MPEG_pack_start (PS)]
         ==> system_clock_reference_base: 859937348 (0x33419a44)  [= 90 kHz-Timestamp: 2:39:14.8594]

Stream_id: 192 (0xc0)  [= ISO/IEC 13818-3 or ISO/IEC 11172-3 audio stream]
             ==> PTS: 5154908244 (0x13341a854)  [= 90 kHz-Timestamp: 15:54:36.7582]


Stream_id: 224 (0xe0)  [= ITU-T Rec. H.262 | ISO/IEC 13818-2 or ISO/IEC 11172-2
video stream]
             ==> PTS: 5154943322 (0x13342315a)  [= 90 kHz-Timestamp: 15:54:37.1480]
    DTS:
             ==> DTS: 5154932521 (0x133420729)  [= 90 kHz-Timestamp: 15:54:37.0280]

Stream_id: 224 (0xe0)  [= ITU-T Rec. H.262 | ISO/IEC 13818-2 or ISO/IEC 11172-2
video stream]
             ==> PTS: 5154936121 (0x133421539)  [= 90 kHz-Timestamp: 15:54:37.0680]


Odd, if in those packets where both PTS and DTS appear, I were
to take what is given for DTS, it fits into the monotonically
increasing +0,0400sec timestamps of PTS elsewhere.

The audio stream has about 1/4 second offset from the video,
as is well-known for such broadcasts, and matches the offset
I see that `mplayer' needs to apply to sync the two.


Now you have me curious, so I'm `dvbsnoop'ing the TS from which
I `ts2ps'ed the above values.  PIDs 5300 and 5301 are resp. the
video, and audio, from which the PS was made.

PID: 5300 (0x14b4)  [= ]
             ==> program_clock_reference: 1546464923192 (0x168107e0e38)  [= PCR-Timestamp: 15:54:36.478636]

PID: 5300 (0x14b4)  [= ]
             ==> program_clock_reference: 1546465965095 (0x168108df427)  [= PCR-Timestamp: 15:54:36.517225]

PID: 5300 (0x14b4)  [= ]
             ==> program_clock_reference: 1546467003390 (0x168109dcbfe)  [= PCR-Timestamp: 15:54:36.555681]
and on it goes.

Now, getting the payload from the relevant PIDs:

    TS sub-decoding (69 packet(s) stored for PID 0x14b4):
    =====================================================
    TS contains PES/PS stream...
        Stream_id: 224 (0xe0)  [= ITU-T Rec. H.262 | ISO/IEC 13818-2 or ISO/IEC
11172-2 video stream]
                   ==> PTS: 5154907321 (0x13341a4b9)  [= 90 kHz-Timestamp: 15:54:36.7480]

    TS sub-decoding (145 packet(s) stored for PID 0x14b4):
    =====================================================
    TS contains PES/PS stream...
        Stream_id: 224 (0xe0)  [= ITU-T Rec. H.262 | ISO/IEC 13818-2 or ISO/IEC
11172-2 video stream]
                   ==> PTS: 5154921722 (0x13341dcfa)  [= 90 kHz-Timestamp: 15:54:36.9080]
                   ==> DTS: 5154910921 (0x13341b2c9)  [= 90 kHz-Timestamp: 15:54:36.7880]

Somewhat further in the stream, we then reach the values given
above for my converted PS file:

    TS sub-decoding (290 packet(s) stored for PID 0x14b4):
    =====================================================
    TS contains PES/PS stream...
        Stream_id: 224 (0xe0)  [= ITU-T Rec. H.262 | ISO/IEC 13818-2 or ISO/IEC
11172-2 video stream]
                   ==> PTS: 5154932522 (0x13342072a)  [= 90 kHz-Timestamp: 15:54:37.0280]
                   ==> DTS: 5154921721 (0x13341dcf9)  [= 90 kHz-Timestamp: 15:54:36.9080]

    TS sub-decoding (68 packet(s) stored for PID 0x14b4):
    =====================================================
    TS contains PES/PS stream...
        Stream_id: 224 (0xe0)  [= ITU-T Rec. H.262 | ISO/IEC 13818-2 or ISO/IEC
11172-2 video stream]
                   ==> PTS: 5154925321 (0x13341eb09)  [= 90 kHz-Timestamp: 15:54:36.9480]


That matches precisely the values in the PS file, including the
discrepancy between PTS and DTS that I find curious.


> >      ==> system_clock_reference_base: 859937348
> (0x33419a44)  [= 90 kHz-Timestamp: 2:39:14.8594]
> 
> PS pack header again? Hop backwards from previous pack
> header?

Appears with context above, just before the PTS of the audio PID.
As noted, there's an offset in realtime between the video and audio
timestamps.

> >          ==> PTS: 5154908244 (0x13341a854)  [= 90
> kHz-Timestamp: 15:54:36.7582]

And that's the audio PID.  The video follows.

> >          ==> PTS: 5154943322 (0x13342315a)  [= 90
> kHz-Timestamp: 15:54:37.1480]
> >          ==> DTS: 5154932521 (0x133420729)  [= 90
> kHz-Timestamp: 15:54:37.0280]
> >          ==> PTS: 5154936121 (0x133421539)  [= 90
> kHz-Timestamp: 15:54:37.0680]
> >          ==> PTS: 5154939721 (0x133422349)  [= 90
> kHz-Timestamp: 15:54:37.1080]
> >          ==> PTS: 5154954122 (0x133425b8a)  [= 90
> kHz-Timestamp: 15:54:37.2680]
> 
> Again hops.  Have you tried to play this stream with vlc?

Not yet.  Normally I use `mplayer', with the problem that none of
my machines are fast enough to playback such a stream without me
needing to `-framedrop', and thus, the A-V offset tends to jump
around noticeably, and video/audio sync never quite reaches the
point where any lack of sync in the source would be noticeable.

I'll give it a try (means firing up X and learning options to
vlc to make it play reasonably well on a machine at 100% load)



> BTW: what is the DTS good for?  Isn't PTS the relevant
> time for playbacK?

I'll let someone who actually knows answer, rather than speculating
in ignorance.  I also thought PTS was all to be concerned with for
non-realtime playback, so that the additional wealth of timestamps
weren't too important for a stream written for later playback, but
honestly, I have no clue.


>      And what is the SCRB good for?  I am totally confused
> by all those times.

I'm with you.  I would guess many of the timestamps are used so that
realtime decoders will tend to present the decoded streams in sync
with one another, avoiding echo effects when unsynced devices in the
same room with different sized buffering and different delays in the
decoding process receive the same signal, though in reality, I've
observed devices that are noticeably out-of-sync on the same channel.



Now here's what gets interesting.  I'm accustomed to seeing on the
order of 1/4 second, more or less depending on station, offset in
real time between audio and video timestamps in TSen.

But now that I look more closely at the Shaun data, converted from
BBC One which should have timing data like the BBC Four data above,
I see something unexpected:

Stream_id: 186 (0xba)  [= MPEG_pack_start (PS)]
         ==> system_clock_reference_base: 0 (0x00000000)  [= 90 kHz-Timestamp: 0:00:00.0000]

Stream_id: 186 (0xba)  [= MPEG_pack_start (PS)]
         ==> system_clock_reference_base: 146 (0x00000092)  [= 90 kHz-Timestamp: 0:00:00.0016]

Stream_id: 192 (0xc0)  [= ISO/IEC 13818-3 or ISO/IEC 11172-3 audio stream]
             ==> PTS: 16200 (0x00003f48)  [= 90 kHz-Timestamp: 0:00:00.1800]

Stream_id: 224 (0xe0)  [= ITU-T Rec. H.262 | ISO/IEC 13818-2 or ISO/IEC 11172-2
video stream]
             ==> PTS: 23400 (0x00005b68)  [= 90 kHz-Timestamp: 0:00:00.2600]
             ==> DTS: 12600 (0x00003138)  [= 90 kHz-Timestamp: 0:00:00.1400]

and plenty of system_clock_reference_base

Stream_id: 192 (0xc0)  [= ISO/IEC 13818-3 or ISO/IEC 11172-3 audio stream]
             ==> PTS: 22680 (0x00005898)  [= 90 kHz-Timestamp: 0:00:00.2520]

Stream_id: 224 (0xe0)  [= ITU-T Rec. H.262 | ISO/IEC 13818-2 or ISO/IEC 11172-2
video stream]
             ==> PTS: 66600 (0x00010428)  [= 90 kHz-Timestamp: 0:00:00.7400]
             ==> DTS: 55800 (0x0000d9f8)  [= 90 kHz-Timestamp: 0:00:00.6200]

Stream_id: 192 (0xc0)  [= ISO/IEC 13818-3 or ISO/IEC 11172-3 audio stream]
             ==> PTS: 29160 (0x000071e8)  [= 90 kHz-Timestamp: 0:00:00.3240]

Stream_id: 192 (0xc0)  [= ISO/IEC 13818-3 or ISO/IEC 11172-3 audio stream]
             ==> PTS: 33480 (0x000082c8)  [= 90 kHz-Timestamp: 0:00:00.3720]

Stream_id: 192 (0xc0)  [= ISO/IEC 13818-3 or ISO/IEC 11172-3 audio stream]
             ==> PTS: 39960 (0x00009c18)  [= 90 kHz-Timestamp: 0:00:00.4440]

Stream_id: 192 (0xc0)  [= ISO/IEC 13818-3 or ISO/IEC 11172-3 audio stream]
             ==> PTS: 46440 (0x0000b568)  [= 90 kHz-Timestamp: 0:00:00.5160]

Stream_id: 192 (0xc0)  [= ISO/IEC 13818-3 or ISO/IEC 11172-3 audio stream]
             ==> PTS: 50760 (0x0000c648)  [= 90 kHz-Timestamp: 0:00:00.5640]

snippage...

         ==> system_clock_reference_base: 51950 (0x0000caee)  [= 90 kHz-Timestamp: 0:00:00.5772]

Stream_id: 224 (0xe0)  [= ITU-T Rec. H.262 | ISO/IEC 13818-2 or ISO/IEC 11172-2
video stream]
             ==> PTS: 109800 (0x0001ace8)  [= 90 kHz-Timestamp: 0:00:01.2200]
             ==> DTS: 99000 (0x000182b8)  [= 90 kHz-Timestamp: 0:00:01.1000]

Stream_id: 192 (0xc0)  [= ISO/IEC 13818-3 or ISO/IEC 11172-3 audio stream]
             ==> PTS: 63720 (0x0000f8e8)  [= 90 kHz-Timestamp: 0:00:00.7080]


Things I find interesting:
The PTS (DTS?) offsets between the video and audio are closer to
half a second.  I'd really have to dig to find where I archived
the original TS recording.

At the start, first audio then video packets with nearly the same
PTS appear, in spite of the source being different, before the
values approach that expected, meaning there's not a one-to-one
transfer of info from the source to the PS, by some manner.

Video timestamps here only appear as combined PTS/DTS; there are
plenty of SCRB timestamps between everything.

That `replex' file was created with a `dvd' option; the file where
I tried the `mpeg2' option is similarly interesting, with a few
scattered PTS-only video timestamps, while most of the combined
PTS-DTS packets have identical values for both, and which do not
match the PTS-only timestamps (without pasting, I see PTS-only
timestamp at 1,58sec, followed later by PTS+DTS timestamps 1,50
sec, and later audio PTS at 1,07sec, and SCRB at 1,00sec).

It's all very confusing, you see.  Both files played on the DVD
player, as I remember, without obvious problems, though I can't
remember if there was strangeness at the start -- as I was aware
of the realtime offset between audio and video within the file, I
wondered how the DVD player would cope.

And now, I'm wondering just how `replex' is arriving at these
timestamps from the TS file.


Anyway, it's not all that relevant, no?  I'm just muttering to
myself out loud, yes?  I'm not slowly going insane, right?

I'll be off in my corner, rocking back and forth

Yours &c, &c,
barry bouwsma


      


_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] How to convert MPEG-TS to MPEG-PS on the fly?

[Josef Wolf]

On Wed, Aug 27, 2008 at 12:26:00AM -0700, barry bouwsma wrote:
[ ... ]

Thanks for your patience and your efforts, Barry!

Inspired by your detailed description, I decided to hack together an
quick-n-dirty script to analyze dvbsnoop's timing output.  I attach
the script at the end of this mail.  First, I'd like to present some
of the results.  Given a file containing TS (z.ts), I ran those commands:

  $ rm z.mpg
  $ ffmpeg -f dvd -acodec copy -vcodec copy -i  z.ts  z.mpg
  $  ./parsesnoop z.mpg   20
   pkt-nr   id   len        PTS           DTS          ESCR         SCR
  00000001 0xba 0x000c                                           14:05:04.5660
  00000005 0xba 0x000c                                           14:05:04.7942
  00000007 0xe0 0x0800 22:35:16.0619               13:44:30.5997
  00000008 0xe0 0x0800 26:05:52.7719 19:59:24.0657
  00000009 0xe0 0x0800 25:29:04.5040               11:43:55.6654
  00000010 0xe0 0x0800                             25:09:57.5575  4:39:52.5434
  00000011 0xe0 0x0800                                            1:42:31.9742
  00000012 0xe0 0x0800 11:05:40.9000  6:14:46.3895 14:34:14.3564
  00000014 0xe0 0x0800 26:22:35.1851  5:58:32.8467  9:35:17.3297
  00000015 0xe0 0x0800  0:27:29.7839 23:55:23.4698
  00000016 0xe0 0x0800  0:31:07.6702
  00000017 0xe0 0x0800                             25:25:02.1821
  00000018 0xe0 0x0800                                           22:08:07.7717
  00000019 0xe0 0x0800                             25:42:03.2501
  00000020 0xe0 0x0800  4:08:35.6544 11:21:31.4357 23:12:35.3037
  00000022 0xe0 0x0800                             25:34:24.0965
  00000023 0xe0 0x0800                             12:24:51.4790
  00000024 0xe0 0x0800                             16:35:12.6386  9:34:01.5197
  00000025 0xe0 0x0800  1:17:22.1240 20:05:57.9732
  00000026 0xe0 0x0800                             23:02:50.3471 13:38:26.3393
  00000027 0xe0 0x0800 24:19:14.5942  7:04:21.1562

All of those times look pretty much wired to me.  There seem to be no
consistency at all.  Just some random numbers.  But this stream plays
very good on all the players I tried.

Then I tried:

  $ libdvb-0.5.5.1/dvb-mpegtools/ts2ps 120 110 <z.ts >z.ts2ps
  $ ./parsesnoop z.ts2ps 20
   pkt-nr   id   len        PTS           DTS          ESCR         SCR
  00000001 0xba 0x000e                                            4:42:57.2304
  00000003 0xe0 0x0800 17:58:19.1292 17:58:19.0092
  00000033 0xe0 0x0800 17:58:19.0492
  00000042 0xe0 0x0800 17:58:19.0892
  00000052 0xe0 0x0800 17:58:19.2492 17:58:19.1292
  00000055 0xba 0x000e                                            4:42:56.9789
  00000056 0xc0 0x0800 17:58:18.8777
  00000079 0xe0 0x0800 17:58:19.1692
  00000091 0xe0 0x0800 17:58:19.2092
  00000100 0xe0 0x0800 17:58:19.3692 17:58:19.2492
  00000121 0xe0 0x0800 17:58:19.2892
  00000122 0xba 0x000e                                            4:42:57.1469
  00000123 0xc0 0x0800 17:58:19.0457
  00000132 0xe0 0x0800 17:58:19.3292
  00000141 0xe0 0x0800 17:58:19.4892 17:58:19.3692
  00000161 0xe0 0x0800 17:58:19.4092
  00000174 0xe0 0x0800 17:58:19.4492
  00000183 0xe0 0x0800 17:58:19.6092 17:58:19.4892
  00000188 0xba 0x000e                                            4:42:57.3149
  00000189 0xc0 0x0800 17:58:19.2137
  00000204 0xe0 0x0800 17:58:19.5292
  $

This looks much more consistent.  The only strange thing is that ESCR
is missing.  But all the players I tried (except mplayer) fail to play
this stream in one way or another.

Please note that both streams were created from identical input.  I can
see ts2ps timestamps in the original stream.  But I can fine none of
the ffpeg timestamps in the original stream.  Strange enough, the
ts2ps stream (the one that preserves original timestamps) does not play
on any player (except mplayer) and the stream with fantasy-timestamps
plays virtually anywhere.

> And now, I'm wondering just how `replex' is arriving at these
> timestamps from the TS file.

Same question for ffmpeg...

> Anyway, it's not all that relevant, no?  I'm just muttering to
> myself out loud, yes?  I'm not slowly going insane, right?
> I'll be off in my corner, rocking back and forth

Oh, you start feeling like me ;-)

Please, can somebody recommend a _good_ book explaining all this mess?

Here's the script that I used to generate the output listed above:

#! /usr/bin/perl

use strict;
use warnings;

my ($infile, $packets) = (shift, shift);

my $dvbsnoop = "dvbsnoop-1.4.50/src/dvbsnoop -s ps -ph 0 -nohexdumpbuffer";
open (my $snoop, "$dvbsnoop -if '$infile'|") or die;

print " pkt-nr   id   len        PTS           DTS          ESCR         SCR\n";
my ($nr, $len, $sid, %ts);
while (my $line = <$snoop>) {
    $line =~ s/system_clock_reference_base/SCR_base/;

    $nr=$1, $len=$2 if $line =~ /^PS-Packet: (\d+).*Length: \d+ \(([0-9a-fx]+)\)/;
    $sid=$1 if $line =~ /^Stream_id: \d+ \(([0-9a-fx]+)\)/;
    $ts{$1}=$2 if $line =~ /(PTS|DTS|E?SCR).*Timestamp: ([\d\:\.]+)/;
    
    next unless $line =~ /=========/;
    next unless keys %ts;

    print join (" ", $nr, $sid, $len,
                map {
                    sprintf ("%13s", exists $ts{$_} ? $ts{$_} : "");
                } qw(PTS DTS ESCR SCR)), "\n";

    $sid=undef;
    %ts=();

    last if --$packets<0;
}

_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] How to convert MPEG-TS to MPEG-PS on the fly?

[Johannes Stezenbach]

On Thu, Aug 28, 2008 at 12:00:20AM +0200, Josef Wolf wrote:
> 
> Inspired by your detailed description, I decided to hack together an
> quick-n-dirty script to analyze dvbsnoop's timing output.  I attach
> the script at the end of this mail.  First, I'd like to present some
> of the results.  Given a file containing TS (z.ts), I ran those commands:
> 
>   $ rm z.mpg
>   $ ffmpeg -f dvd -acodec copy -vcodec copy -i  z.ts  z.mpg
>   $  ./parsesnoop z.mpg   20
>    pkt-nr   id   len        PTS           DTS          ESCR         SCR
>   00000001 0xba 0x000c                                           14:05:04.5660
>   00000005 0xba 0x000c                                           14:05:04.7942
>   00000007 0xe0 0x0800 22:35:16.0619               13:44:30.5997
>   00000008 0xe0 0x0800 26:05:52.7719 19:59:24.0657
...
> All of those times look pretty much wired to me.  There seem to be no
> consistency at all.  Just some random numbers.  But this stream plays
> very good on all the players I tried.
> 
> Then I tried:
> 
>   $ libdvb-0.5.5.1/dvb-mpegtools/ts2ps 120 110 <z.ts >z.ts2ps
>   $ ./parsesnoop z.ts2ps 20
>    pkt-nr   id   len        PTS           DTS          ESCR         SCR
>   00000001 0xba 0x000e                                            4:42:57.2304
>   00000003 0xe0 0x0800 17:58:19.1292 17:58:19.0092
>   00000033 0xe0 0x0800 17:58:19.0492
>   00000042 0xe0 0x0800 17:58:19.0892
>   00000052 0xe0 0x0800 17:58:19.2492 17:58:19.1292
...
> This looks much more consistent.  The only strange thing is that ESCR
> is missing.  But all the players I tried (except mplayer) fail to play
> this stream in one way or another.

The ESCR isn't normally used, I think it means the PES streams don't have
a common time base. The SCR in the second case doesn't match the
DTS/PTS values. A software player would probably ignore it anyway
and sync A/V via PTS only, but a hardware deocder might decide to
play asynchronously. Are you sure dvbsnoop output is correct?

Have you tried iso13818ps from http://www.scara.com/~schirmer/o/mplex13818/ ?
(linked from http://linuxtv.org/projects.php , BTW)

You mentioned you are discarding the adaptation fields. This means
you don't handle timebase discontinuities. Try to feed the original
TS to iso13818ps (not your filtered one), it should handle it correctly.

ISO-13818-4 (MPEG2 conformance testing) explains a bit how the
decoder model works. Look for document "ISO/IEC JTC1/SC29/WG11 N0804".
(I'm assuming you already have "ITU-T Recommendation H.222.0" aka ISO-13818-1)


HTH,
Johannes

_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] How to convert MPEG-TS to MPEG-PS on the fly?

[Josef Wolf]

Thanks for your Hints, Johannes!

On Thu, Aug 28, 2008 at 04:40:50PM +0200, Johannes Stezenbach wrote:
> The ESCR isn't normally used, I think it means the PES streams don't have
> a common time base. The SCR in the second case doesn't match the
> DTS/PTS values. A software player would probably ignore it anyway
> and sync A/V via PTS only, but a hardware deocder might decide to
> play asynchronously.

SCR don't match in the first case (which is played fine) also.

> Are you sure dvbsnoop output is correct?

I have not checked.  But I don't expect more bugs in snoop than in
the other components, since simply parse+display headers is trivial
compared to demux+parse+reshuffle+generate_additional_headers+mux

> Have you tried iso13818ps from http://www.scara.com/~schirmer/o/mplex13818/ ?
> (linked from http://linuxtv.org/projects.php , BTW)

Thanks for the link.  Description looks promising.  But neither mplayer
nor vlc plays the output created by

  mplex13818-1.1.1/iso13818ps --ts z.ts >z.iso.ps

Mplayer gives no audio and 8x8 (or 16x16?) squares which keep changing
colors.  vlc gives black video and no audio.

The output of my parsing script looks like this:

./parsesnoop z.iso.ps 20
 pkt-nr   id   len        PTS           DTS          ESCR         SCR
00000001 0xba 0x000e                                            0:00:01.9250
00000004 0xe0 0x000e 18:48:10.2516
00000043 0xba 0x000e                                            0:00:01.9250
00000044 0xc1 0x090e 18:48:10.0074
00000045 0xba 0x000e                                            0:00:01.9250
00000046 0xc0 0x150e 18:48:10.0561
00000047 0xba 0x000e                                            0:00:01.9630
00000048 0xe0 0x000e 18:48:10.2916
00000087 0xba 0x000e                                            0:00:02.0390
00000088 0xe0 0x0014 18:48:10.4516 18:48:10.3316
00000127 0xba 0x000e                                            0:00:02.0390
00000128 0xe0 0x000e 18:48:10.3716
00000167 0xba 0x000e                                            0:00:02.0770
00000168 0xe0 0x000e 18:48:10.4116
00000207 0xba 0x000e                                            0:00:02.0790
00000208 0xc1 0x090e 18:48:10.1514
00000209 0xba 0x000e                                            0:00:02.1150
00000210 0xe0 0x0014 18:48:10.5716 18:48:10.4516
00000249 0xba 0x000e                                            0:00:02.1150
00000250 0xc0 0x150e 18:48:10.2241
00000251 0xba 0x000e                                            0:00:02.1530

Here's another interesting point:  Yesterday, I had a TS capture which,
when converted to PS by ffmpeg or ts2ps, played fine on both, mplayer
and vlc.  iso13818ps did not produce a usable output.  I always do
captures for this test from the same broadcast (german ZDF channel on
Astra).  This means that at least for ts2ps, the result depends on the
time of the capture.  Maybe changed bitrate or resolution or something.

> You mentioned you are discarding the adaptation fields. This means
> you don't handle timebase discontinuities.

I have stopped talking about my program some days ago.  I don't see
any point in adopting my program to a broken stream.  Thus, before
I continue on my program, I want to make sure I have a proper reference.

I am talking about tools like ts2ps and iso13818ps, which don't produce
a usable stream. I don't think the output of those programs should be
affected by the question whether I ignore adaptation or not.

> Try to feed the original
> TS to iso13818ps (not your filtered one), it should handle it correctly.

I _am_ feeding the original TS (including adaptation-only packets).  As
you can see in my previous mail, I am capturing the TS into a file and
feed this file as input to any of the test candidates.  Whether I ignore
adaptation should not affect the other candidates in any way.

> ISO-13818-4 (MPEG2 conformance testing) explains a bit how the
> decoder model works. Look for document "ISO/IEC JTC1/SC29/WG11 N0804".

Thanks for the hint.  Looks like that's the only way to get any
clarification in this mess.

> (I'm assuming you already have "ITU-T Recommendation H.222.0" aka
> ISO-13818-1)

Yes, I have read it countless times.  But I must admit that there are
many points unclear to me.

_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] How to convert MPEG-TS to MPEG-PS on the fly?

[Johannes Stezenbach]

On Thu, Aug 28, 2008 at 09:34:05PM +0200, Josef Wolf wrote:
> On Thu, Aug 28, 2008 at 04:40:50PM +0200, Johannes Stezenbach wrote:
> 
> > Have you tried iso13818ps from http://www.scara.com/~schirmer/o/mplex13818/ ?
> 
> Thanks for the link.  Description looks promising.  But neither mplayer
> nor vlc plays the output created by
> 
>   mplex13818-1.1.1/iso13818ps --ts z.ts >z.iso.ps

I think the idea is to select _one_ of the services from the TS
by giving the service_id to iso13818ps. And maybe even select
a subset of the PES streams by giving their ids, too. But I have
to admit I never used iso13818ps. At Convergence we used iso13818ts
so that one is know to work.

> Mplayer gives no audio and 8x8 (or 16x16?) squares which keep changing
> colors.  vlc gives black video and no audio.
> 
> The output of my parsing script looks like this:

seems screwed up

> > Try to feed the original
> > TS to iso13818ps (not your filtered one), it should handle it correctly.
> 
> I _am_ feeding the original TS (including adaptation-only packets).  As
> you can see in my previous mail, I am capturing the TS into a file and
> feed this file as input to any of the test candidates.  Whether I ignore
> adaptation should not affect the other candidates in any way.

How do you capture the TS, and does it play OK in vlc?

Johannes

_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] How to convert MPEG-TS to MPEG-PS on the fly?

[Josef Wolf]

On Thu, Aug 28, 2008 at 11:09:56PM +0200, Johannes Stezenbach wrote:
> On Thu, Aug 28, 2008 at 09:34:05PM +0200, Josef Wolf wrote:
> > On Thu, Aug 28, 2008 at 04:40:50PM +0200, Johannes Stezenbach wrote:
> > 
> > > Have you tried iso13818ps from http://www.scara.com/~schirmer/o/mplex13818/ ?
> > 
> > Thanks for the link.  Description looks promising.  But neither mplayer
> > nor vlc plays the output created by
> > 
> >   mplex13818-1.1.1/iso13818ps --ts z.ts >z.iso.ps
> 
> I think the idea is to select _one_ of the services from the TS
> by giving the service_id to iso13818ps.

The TS contains only one service.

> And maybe even select a subset of the PES streams by giving their
> ids, too.

Hmm, I don't see an option to specify multiple streams (one audio
and one video).  Apart from that, IMHO it should be possible to
have multiple audio streams in a single PS.

> > Mplayer gives no audio and 8x8 (or 16x16?) squares which keep changing
> > colors.  vlc gives black video and no audio.
> > 
> > The output of my parsing script looks like this:
> 
> seems screwed up

:-)

> > > Try to feed the original
> > > TS to iso13818ps (not your filtered one), it should handle it correctly.
> > 
> > I _am_ feeding the original TS (including adaptation-only packets).  As
> > you can see in my previous mail, I am capturing the TS into a file and
> > feed this file as input to any of the test candidates.  Whether I ignore
> > adaptation should not affect the other candidates in any way.
> 
> How do you capture the TS,

I have written my own beast to demux a complete transponder and split
it into separate transport streams (every service gets its stream).
The PIDs which are mentioned in the PMT for the given service are passed
unmodified.  In addition, new PAT/PMT are created (stripped down just to
the given service).

> and does it play OK in vlc?

Both, mplayer and vlc play those streams with no problems.

Do you think the TS may be screwed?  I pass the a/v-PIDs which are
mentioned in PMT unmodified.  PAT is trivial.  PMT reflects the
contents from the original PMT (reconstructed just to keep
continuity_counter consistent).  I don't think anything can be
screwed here.

_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] How to convert MPEG-TS to MPEG-PS on the fly?

[Josef Wolf]

On Fri, Aug 29, 2008 at 12:48:56AM +0200, Josef Wolf wrote:
> On Thu, Aug 28, 2008 at 11:09:56PM +0200, Johannes Stezenbach wrote:
> > How do you capture the TS,
> 
> I have written my own beast to demux a complete transponder and split
> it into separate transport streams (every service gets its stream).
> The PIDs which are mentioned in the PMT for the given service are passed
> unmodified.  In addition, new PAT/PMT are created (stripped down just to
> the given service).

One thing I make different is that I filter out all sections with
current_next_indicator==0 and never generate such sections.  But
since PAT never changes and PMT can change only on event boundaries,
this should not affect streams with length of a couple of seconds.

> > and does it play OK in vlc?
> 
> Both, mplayer and vlc play those streams with no problems.
> 
> Do you think the TS may be screwed?  I pass the a/v-PIDs which are
> mentioned in PMT unmodified.  PAT is trivial.  PMT reflects the
> contents from the original PMT (reconstructed just to keep
> continuity_counter consistent).  I don't think anything can be
> screwed here.
> 
> _______________________________________________
> linux-dvb mailing list
> linux-dvb@linuxtv.org
> http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] PTS/DTS clarification (Was: How to convert MPEG-TS to MPEG-PS on the fly?)

[Josef Wolf]

On Wed, Aug 27, 2008 at 12:45:19AM +0200, Josef Wolf wrote:

> BTW: what is the DTS good for?  Isn't PTS the relevant time for playbacK?
>      What difference does it make when a frame was decoded as long as it
>      is presented at the correct time?
> 
>      And what is the SCRB good for?  I am totally confused by all those
>      times.

I have found a good reading on

  http://www.tek.com/Measurement/programs/mpeg_fundamentals/

This reading is much more comprehensive than the iso-13818-1.  Page 47
explains PTS/DTS.  In a nutshell, DTS is needed because of bidirectional
video encoding.  As an example, pictures can be presented in order IBBP,
but for decoding the order would be IPBB because the B pictures depend
on the I and P pictures.  Since decoders can decode only one picture at
a time, DTS is used to signal that decoding have to be done in a
different order than presentation.

So now my understanding is that for determining packet order in the PS,
DTS has to be used if it exists.  If no DTS exists, then PTS is to be
used.

Guess, my understanding is still wrong.  But I need a starting point
from which I can remove errors step by step ;-)


_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] How to convert MPEG-TS to MPEG-PS on the fly?

[Josef Wolf]

On Tue, Aug 26, 2008 at 08:49:08AM +0200, Nico Sabbi wrote:

Thanks for your answer, Nico!

> > If I remove the PS pack header and the PS system header (stream-id 0xba
> > and 0xbb) then both play the stream, but no STB plays it :-(
> You must have corrupted some header in your application :)

I have trimmed my application to create exactly the same output as
ts2ps does.  But you are correct: I should have checked ts2ps output
before adopting :-/

> replex never worked with at least 60% of the TSs I fed it with,
> that left me no other choise than to fix mencoder's muxer, as long
> as something as horribly crippled and broken like mencoder can be
> fixed (that may not work if you give it 100% correct timestamps
> for every single frame)

I agree that mencoder really needs a indent run.  But at least it
produces output that can be played on a wide variety of players/devices.

> > It seems to be a mess.  None of the programs seem to produce proper
> > streams.  Only mencoder seems to generate a proper stream.  But AFAICS,
> > mencoder completely decodes the stream and re-encodes it again, eating
> > up all the CPU.
> 
> it doesn't decode: it just demuxes, although parsing audio and video
> frame headers is probably eating up a good % of your cpu.
> If you want to make it lighter either
> - use ffmpeg, it can be surely used to generate many PS from a single TS
>   although its command line is really challenging

The problem is that I have multiple TS which are to be remuxed into
multiple PS (one-to-one).

> - use -demuxer lavf and fix libmpdemux/muxer_mpeg.c so that frames are
>   not analyzed within the muxer itself, but just copied verbatim in
>   teh corresponding mpeg_frame_t[] array

Hmm, I am not really sure I understand what you are trying to say to me.

> > Any more ideas how to do the conversion?
> > 
> > BTW: Can anybody recommend a good book on the topic?
> 
> none that I know of, but muxing mpeg-ps is more trivial than it looks
> like when reading the specs

Ugh, I am not really sure about this, given that many applications fail
badly on this.

_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] How to convert MPEG-TS to MPEG-PS on the fly?

[Josef Wolf]

On Wed, Aug 20, 2008 at 11:10:06PM +0200, Josef Wolf wrote:

>   jw@dvb1:~$ dvbsnoop -s pes -if zdf.test|grep Stream_id|head -40
>   Stream_id: 224 (0xe0)  [= ITU-T Rec. H.262 | ISO/IEC 13818-2 or ISO/IEC 11172-2 video stream]
>   Stream_id: 0 (0x00)  [= picture_start_code]
>   Stream_id: 181 (0xb5)  [= extension_start_code]
>   Stream_id: 1 (0x01)  [= slice_start_code]
>   Stream_id: 2 (0x02)  [= slice_start_code]
>   [ consecutive lines deleted ]
>   Stream_id: 34 (0x22)  [= slice_start_code]
>   Stream_id: 35 (0x23)  [= slice_start_code]
>   [ here the list of stream ids start over again and repeats ]

Table 2-18 in iso-13818-1 don't list any stream_id's below 0xBC.
Anybody knows what those stream_id's 0x00..0x23 and 0xB5 are for
and whether they could be the reason for the artefacts?

> Maybe I should discard some of those to get a proper PES?

Hmm, guess I should give it a try.

_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] How to convert MPEG-TS to MPEG-PS on the fly?

[Josef Wolf]

On Thu, Aug 21, 2008 at 09:17:58PM +0200, Josef Wolf wrote:
> On Wed, Aug 20, 2008 at 11:10:06PM +0200, Josef Wolf wrote:
> 
> >   jw@dvb1:~$ dvbsnoop -s pes -if zdf.test|grep Stream_id|head -40
> >   Stream_id: 224 (0xe0)  [= ITU-T Rec. H.262 | ISO/IEC 13818-2 or ISO/IEC 11172-2 video stream]
> >   Stream_id: 0 (0x00)  [= picture_start_code]
> >   Stream_id: 181 (0xb5)  [= extension_start_code]
> >   Stream_id: 1 (0x01)  [= slice_start_code]
> >   Stream_id: 2 (0x02)  [= slice_start_code]
> >   [ consecutive lines deleted ]
> >   Stream_id: 34 (0x22)  [= slice_start_code]
> >   Stream_id: 35 (0x23)  [= slice_start_code]
> >   [ here the list of stream ids start over again and repeats ]
> 
> Table 2-18 in iso-13818-1 don't list any stream_id's below 0xBC.
> Anybody knows what those stream_id's 0x00..0x23 and 0xB5 are for
> and whether they could be the reason for the artefacts?

The more I look at this PES stream the more confused I get:  The
stream_id 0xe0 seems to transport PTS and DTS _only_.  Everything
else seems to be contained in PES packets with those unknown
stream_id's.  Here is what I see:

At byte position 0x0:
0x0000+0x0000: 00 00 01 e0 00 00 88 80  05 25 ea ad 04 69 00 00
0x0000+0x0010: 01 00 02 9f ff fb b8 00  00 01 b5 85 55 53 9c 00

stream_id:            0xe0
PES_packet_length:       0 (unlimited)
PES_priority:            1
PTS_DTS_flags:          10
PES_header_data_length:  5 (so the header ends on position 0x0d)
PTS:                     25 ea ad 04 69 (bit fiddling not done yet)

at position 0x0e comes the next packet with stream_id==0x00
So this video PES packet don't contain any payload at all.


The next packet with stream_id==0xe0 is on position 0x2c83:

0x2c83+0x0000: 00 00 01 e0 00 00 88 c0  0b 35 ea ad 74 e9 15 ea
0x2c83+0x0010: ad 20 89 ff 00 00 01 b3  2d 02 40 33 24 9f 23 81

stream_id:             0xe0
PES_packet_length:        0 (unlimited)
PES_priority:             1
PTS_DTS_flags:           11
PES_header_data_length: 0x0b (so the header ends on position 0x13)
PTS:                     35 ea ad 74 e9 (bit fiddling not done yet)
DTS:                     15 ea ad 20 89 (bit fiddling not done yet)
payload:               0xff (ough, only one byte padding in payload?)

at position 0x14 comes the next packet with stream_id==0xb3
So this video packet don't contain any payload at all.

And so it goes on and on.  All the payload seems to be contained
in those unknown packets.  Packets with stream_id==0xe0 don't
seem to contain any payload at all.

Obviously, I must have misunderstood something very badly.  But
I simply can't figure out what.  I bet there is some sort of stuffing
or escaping that I should do when removing the TS layer?

Any ideas how those unknown stream_id's come to carry all the
ES contents?


_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

[linux-dvb] RE : How to convert MPEG-TS to MPEG-PS on the fly?

[Thierry Lelegard]

On Thu, Aug 21, 2008 at 09:17:58PM +0200, Josef Wolf wrote:

>The more I look at this PES stream the more confused I get:  The
>stream_id 0xe0 seems to transport PTS and DTS _only_.  Everything
>else seems to be contained in PES packets with those unknown
>stream_id's.  Here is what I see:

As mentioned in my previous post, the "stream ids" below B9
are ISO 13818-2 "start codes".

Let me try to clarify. A PES header starts with 00 00 01 xx
where xx is in the range B9-FF. A PES payload is a list of
elements. Each element starts with 00 00 01 xx where xx is
in the range 00-B8. The pattern 00 00 01 is never found
anywhere else in an MPEG-2 video stream.

When you analyze a TS, PES headers are "naturally" located
since they start at the beginning of a TS payload with TS
header containing a "payload start unit indicator".

After demuxing PES packets, you get a suite of elements.
Each element starts with 00 00 01 xx. When xx is in the range
B9-FF, this is the start of a PES packet, following the PES
header syntax as specified in ISO 13818-1 and potentially 
containing PTS and DTS. When xx is in the range 00-B8, this
is NOT the start of a PES header, but the start of a video
element inside the payload of the current PES packet. The
syntax of the following bytes depends on the start code xx
and is defined by ISO 13818-2, not -1.

Does this help?
-Thierry


_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] RE : How to convert MPEG-TS to MPEG-PS on the fly?

[Josef Wolf]

On Fri, Aug 22, 2008 at 10:10:09AM +0200, Thierry Lelegard wrote:
> On Thu, Aug 21, 2008 at 09:17:58PM +0200, Josef Wolf wrote:
> 
> >The more I look at this PES stream the more confused I get:  The
> >stream_id 0xe0 seems to transport PTS and DTS _only_.  Everything
> >else seems to be contained in PES packets with those unknown
> >stream_id's.  Here is what I see:
> 
> As mentioned in my previous post, the "stream ids" below B9
> are ISO 13818-2 "start codes".

Thanks for the explanation, Thierry!

I now have tried to convert via ffmpeg.  The command I used was

  ffmpeg -vcodec copy -acodec copy z.ps.mp2 -i z.ts

This creates pack headers which don't seem to fit to the syntax
defined in table 2-33 of iso-13818-1:

  00 00 01 ba 21 00 01 00  01 a1 9f 0d 00 00 01 bb

This looks strange to me.  According to the syntax, the pack
header should be 14 bytes.  Maybe I used the wrong options and
created  a mpeg1 stream?

_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] RE : How to convert MPEG-TS to MPEG-PS on the fly?

[Nico Sabbi]

Il giorno lun, 25/08/2008 alle 21.55 +0200, Josef Wolf ha scritto:
> On Fri, Aug 22, 2008 at 10:10:09AM +0200, Thierry Lelegard wrote:
> > On Thu, Aug 21, 2008 at 09:17:58PM +0200, Josef Wolf wrote:
> > 
> > >The more I look at this PES stream the more confused I get:  The
> > >stream_id 0xe0 seems to transport PTS and DTS _only_.  Everything
> > >else seems to be contained in PES packets with those unknown
> > >stream_id's.  Here is what I see:
> > 
> > As mentioned in my previous post, the "stream ids" below B9
> > are ISO 13818-2 "start codes".
> 
> Thanks for the explanation, Thierry!
> 
> I now have tried to convert via ffmpeg.  The command I used was
> 
>   ffmpeg -vcodec copy -acodec copy z.ps.mp2 -i z.ts


give the output a .dvd extension

> 
> This creates pack headers which don't seem to fit to the syntax
> defined in table 2-33 of iso-13818-1:
> 
>   00 00 01 ba 21 00 01 00  01 a1 9f 0d 00 00 01 bb
> 
> This looks strange to me.  According to the syntax, the pack
> header should be 14 bytes.  Maybe I used the wrong options and
> created  a mpeg1 stream?
> 



_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

Re: [linux-dvb] RE : How to convert MPEG-TS to MPEG-PS on the fly?

[Josef Wolf]

On Tue, Aug 26, 2008 at 08:50:13AM +0200, Nico Sabbi wrote:

> > I now have tried to convert via ffmpeg.  The command I used was
> >   ffmpeg -vcodec copy -acodec copy z.ps.mp2 -i z.ts
> 
> give the output a .dvd extension

Thanks for the hint, Nico!  Though the ffmpeg FAQ explicitly states
that file types should _not_ be recognized by extensions, this did
the trick.


_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb

[linux-dvb] RE : How to convert MPEG-TS to MPEG-PS on the fly?

[Thierry Lelegard]

On Wed, Aug 20, 2008 at 11:10:06PM +0200, Josef Wolf wrote:

>>   jw@dvb1:~$ dvbsnoop -s pes -if zdf.test|grep Stream_id|head -40
>>   Stream_id: 224 (0xe0)  [= ITU-T Rec. H.262 | ISO/IEC 13818-2 or ISO/IEC 11172-2 video stream]
>>   Stream_id: 0 (0x00)  [= picture_start_code]
>>   Stream_id: 181 (0xb5)  [= extension_start_code]
>>   Stream_id: 1 (0x01)  [= slice_start_code]
>>   Stream_id: 2 (0x02)  [= slice_start_code]
>>   [ consecutive lines deleted ]
>>   Stream_id: 34 (0x22)  [= slice_start_code]
>>   Stream_id: 35 (0x23)  [= slice_start_code]
>>   [ here the list of stream ids start over again and repeats ]
>
> Table 2-18 in iso-13818-1 don't list any stream_id's below 0xBC.
> Anybody knows what those stream_id's 0x00..0x23 and 0xB5 are for
> and whether they could be the reason for the artefacts?

They are defined ISO-13818-2 (MPEG-2 video). They are "start codes"
for PES payload elements. Stream id's (in PES headers, not
payloads) are a subset of start codes and are named "system start
codes". You won't find other start codes in PES headers, only in
PES payloads.

[Quoted from ISO-13818-2]

Table 6-1  Start code values
name                 start code value (hexadecimal)
picture_start_code   00
slice_start_code     01 through AF
reserved             B0
reserved             B1
user_data_start_code B2
sequence_header_code B3
sequence_error_code  B4
extension_start_code B5
reserved             B6
sequence_end_code    B7
group_start_code     B8
system start codes   B9 through FF

NOTE - system start codes are defined in Part 1 of this specification

[End quote]

-Thierry


_______________________________________________
linux-dvb mailing list
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb