* Battery life difference @ 2008-07-05 13:17 Felipe Balbi 2008-07-05 13:36 ` Theodore Tso 2008-07-05 15:23 ` Arjan van de Ven 0 siblings, 2 replies; 15+ messages in thread From: Felipe Balbi @ 2008-07-05 13:17 UTC (permalink / raw) To: Kernel development list, linux-acpi Hello all, I bought two x61s for testing purposes. One of them is running windows XP and the other one is running clean ubuntu 8.04 with kernel 2.6.24-19. The problem is that on windows that battery lasts around 3h30min and with linux 1h30min. 2 hours is a huge difference. Here's the scenario: . x61s with windows xp wifi on but not connected to any network, bluetooth on, display is in the less brighten state. . x61s with ubuntu 8.04 all radios off (modules removed and hw switch off), display is in the less brighten state. powertop runned and I did what it suggested me to do. Also did some other stuff from thinkwiki on how to reduce power consumption. Well, what could still be using the system power so much ? Are we missing anything on the acpi implementation or is it related to acpi integration to bus and device drivers ? Anything I could help testing and maybe even coding, I started reading the acpi specs and some notes on the linux-acpi website. One thing that called my attention is that D2 and D3 states would require context save/restore opperations (not always true in D2) and I can't see such feature in almost any drivers. We're starting to plan it for omap drivers though due to omap3's pm feature been able to reach off-states, but that's another point and has nothing to do with x86. Anyways, do we reach D2/D3 states anywhere ? I mean without context save/restore we can't reach D3 and almost no D2 as well. Any comments ? -- Best Regards, Felipe Balbi felipebalbi@users.sourceforge.net ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Battery life difference 2008-07-05 13:17 Battery life difference Felipe Balbi @ 2008-07-05 13:36 ` Theodore Tso 2008-07-05 14:00 ` Felipe Balbi 2008-07-05 15:23 ` Arjan van de Ven 1 sibling, 1 reply; 15+ messages in thread From: Theodore Tso @ 2008-07-05 13:36 UTC (permalink / raw) To: Felipe Balbi; +Cc: Kernel development list, linux-acpi On Sat, Jul 05, 2008 at 04:17:01PM +0300, Felipe Balbi wrote: > > . x61s with windows xp > wifi on but not connected to any network, bluetooth on, > display is in the less brighten state. > > . x61s with ubuntu 8.04 > all radios off (modules removed and hw switch off), > display is in the less brighten state. > powertop runned and I did what it suggested me > to do. Also did some other stuff from thinkwiki on > how to reduce power consumption. You didn't say which battery you are using; I'm guessing you are using the 4 cell slim-line battery, which is rated at 28.8 Watt-hours? How many Watts is Powertop reporting that you are using? I'm down to around 8-9 watts, and around 9.5 watts in "airplane mail reading mail" (wireless disabled, USB modules unloaded, screen brightness turned way down, laptop_mode enabled, all mail messages pulled into memory, and reading/deleting mail using mutt, a character-mode based mail reader --- if you use Evolution, your energy usage will definitely be higher :-). > Well, what could still be using the system power so > much ? See http://thunk.org/tytso/blog/2007/10/29/tip-o-the-hat-wag-o-the-finger-linux-power-savings-for-laptop-users/ - Ted ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Battery life difference 2008-07-05 13:36 ` Theodore Tso @ 2008-07-05 14:00 ` Felipe Balbi 2008-07-09 20:29 ` Pavel Machek 0 siblings, 1 reply; 15+ messages in thread From: Felipe Balbi @ 2008-07-05 14:00 UTC (permalink / raw) To: Theodore Tso, Felipe Balbi, Kernel development list, linux-acpi Hi, On Sat, Jul 5, 2008 at 4:36 PM, Theodore Tso <tytso@mit.edu> wrote: > You didn't say which battery you are using; I'm guessing you are using > the 4 cell slim-line battery, which is rated at 28.8 Watt-hours? yeah, the normal slim-battery. > How many Watts is Powertop reporting that you are using? I'm down to eheh, it's only 15.5w :-p > around 8-9 watts, and around 9.5 watts in "airplane mail reading mail" > (wireless disabled, USB modules unloaded, screen brightness turned way > down, laptop_mode enabled, all mail messages pulled into memory, and > reading/deleting mail using mutt, a character-mode based mail reader > --- if you use Evolution, your energy usage will definitely be higher :-). no crappy evolution, I use fluxbox and mutt. This ubuntu machine is for testing purposes only :-p Anyways, it's bad that we have to manually unload several modules, specially usb. I'm sure windows reaches 9 watts consumption without breaking functionality. And that's the big thing. I think that's something we should be aiming. Dontcha think? > See http://thunk.org/tytso/blog/2007/10/29/tip-o-the-hat-wag-o-the-finger-linux-power-savings-for-laptop-users/ Good article, but still. On the end user point of view, it's difficult to do rmmod/insmod operations. Recently I sent a patch to the bluez team adding a checkbox to bluetooth-properties that would send the SetMode("off") dbus message (which is the same as running hciconfig hci0 down). That helps a bit, but still the driver is not been used and keeps generating wakeup events. Why if it's not been used ? Maybe this is something we should start thinking about, right ? -- Best Regards, Felipe Balbi felipebalbi@users.sourceforge.net ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Battery life difference 2008-07-05 14:00 ` Felipe Balbi @ 2008-07-09 20:29 ` Pavel Machek 2008-07-10 5:16 ` Tino Keitel 0 siblings, 1 reply; 15+ messages in thread From: Pavel Machek @ 2008-07-09 20:29 UTC (permalink / raw) To: Felipe Balbi; +Cc: Theodore Tso, Kernel development list, linux-acpi Hi! > > You didn't say which battery you are using; I'm guessing you are using > > the 4 cell slim-line battery, which is rated at 28.8 Watt-hours? > > yeah, the normal slim-battery. > > > How many Watts is Powertop reporting that you are using? I'm down to > > eheh, it's only 15.5w :-p > > > around 8-9 watts, and around 9.5 watts in "airplane mail reading mail" > > (wireless disabled, USB modules unloaded, screen brightness turned way > > down, laptop_mode enabled, all mail messages pulled into memory, and > > reading/deleting mail using mutt, a character-mode based mail reader > > --- if you use Evolution, your energy usage will definitely be higher :-). > > no crappy evolution, I use fluxbox and mutt. This ubuntu machine is > for testing purposes only :-p > > Anyways, it's bad that we have to manually unload several modules, > specially usb. I'm sure windows reaches 9 watts consumption without > breaking functionality. And that's the big thing. I think that's something > we should be aiming. Dontcha think? Can you test Ted's suggestion to see if we have nailed the problem, first? 9W vs. 15W does not explain 1h30 vs 3h30 difference, afaict. (Of course, we are working on usb autosuspend. Try 2.6.26). -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Battery life difference 2008-07-09 20:29 ` Pavel Machek @ 2008-07-10 5:16 ` Tino Keitel 0 siblings, 0 replies; 15+ messages in thread From: Tino Keitel @ 2008-07-10 5:16 UTC (permalink / raw) To: Pavel Machek Cc: Felipe Balbi, Theodore Tso, Kernel development list, linux-acpi On Wed, Jul 09, 2008 at 22:29:20 +0200, Pavel Machek wrote: [...] > > Anyways, it's bad that we have to manually unload several modules, > > specially usb. I'm sure windows reaches 9 watts consumption without > > breaking functionality. And that's the big thing. I think that's something > > we should be aiming. Dontcha think? > > Can you test Ted's suggestion to see if we have nailed the problem, > first? > > 9W vs. 15W does not explain 1h30 vs 3h30 difference, afaict. > > (Of course, we are working on usb autosuspend. Try 2.6.26). FWIW, with dimmed display, hard disk spun down, USB autosuspend enabled for all devices (I have to enable it from userspace for the fingerprint reader), and some other tricks [*], I get slightly below 9W on my X61s with uhci_hcd and ehci_hcd loaded and active WLAN, using kernel 2.6.25.9. I also tried 2.6.26-rc to see if PCIE ASPM can save some power, but I didn't see a difference after I set it to "powersave". It looks like the BIOS already set sane values. Regards, Tino [*]: SATA link power management, HDA sound power saving, wireless LAN power saving, Ethernet speed set to 100 MBit/s, some other modules like yenta_socket unloaded ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Battery life difference 2008-07-05 13:17 Battery life difference Felipe Balbi 2008-07-05 13:36 ` Theodore Tso @ 2008-07-05 15:23 ` Arjan van de Ven 2008-07-05 15:44 ` Felipe Balbi 1 sibling, 1 reply; 15+ messages in thread From: Arjan van de Ven @ 2008-07-05 15:23 UTC (permalink / raw) To: Felipe Balbi; +Cc: Kernel development list, linux-acpi On Sat, 5 Jul 2008 16:17:01 +0300 "Felipe Balbi" <felipebalbi@users.sourceforge.net> wrote: > Hello all, > > I bought two x61s for testing purposes. One of them is running > windows XP and the other one is running clean ubuntu 8.04 with kernel > 2.6.24-19. The problem is that on windows that battery lasts around > 3h30min and with linux 1h30min. > > 2 hours is a huge difference. Here's the scenario: > > . x61s with windows xp > wifi on but not connected to any network, bluetooth on, > display is in the less brighten state. > > . x61s with ubuntu 8.04 > all radios off (modules removed and hw switch off), Which wireless is this? Same for graphics > > Anyways, do we reach D2/D3 states anywhere ? I mean without > context save/restore we can't reach D3 and almost no D2 as well. these don't give you all that much nowadays > > Any comments ? > There's a few things you can do to help stuff along; make sure you're using AHCI. Also take a look at http://kerneltrap.org/mailarchive/linux-kernel/2008/5/8/1771024 because these patches save like 0.75W in our lab. To be honest, I'm surprised at your results; in our testing (with a different distro though) we don't see such a difference and Linux is doing rather well. -- If you want to reach me at my work email, use arjan@linux.intel.com For development, discussion and tips for power savings, visit http://www.lesswatts.org ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Battery life difference 2008-07-05 15:23 ` Arjan van de Ven @ 2008-07-05 15:44 ` Felipe Balbi 2008-07-05 15:54 ` Arjan van de Ven 2008-07-06 17:21 ` Tino Keitel 0 siblings, 2 replies; 15+ messages in thread From: Felipe Balbi @ 2008-07-05 15:44 UTC (permalink / raw) To: Arjan van de Ven; +Cc: Kernel development list, linux-acpi Hi, On Sat, Jul 5, 2008 at 6:23 PM, Arjan van de Ven <arjan@infradead.org> wrote: > Which wireless is this? Same for graphics intel iwl4965 and intel mobile bm965/GL960 (graphics) > these don't give you all that much nowadays hmm... interesting :-) where should i look for more info ? Any pointers would be nice > There's a few things you can do to help stuff along; make sure you're > using AHCI. not loaded. Thanks > Also take a look at > http://kerneltrap.org/mailarchive/linux-kernel/2008/5/8/1771024 because > these patches save like 0.75W in our lab. cool, is it merged upstream ? I'll build current git kernel and reply again with new linux results. It could be that ubuntu 2.6.24 is broken somehow... > To be honest, I'm surprised at your results; in our testing (with a > different distro though) we don't see such a difference and Linux is > doing rather well. can you get powertop results in a 2.6.24 distro kernel and current git kernel ? -- Best Regards, Felipe Balbi felipebalbi@users.sourceforge.net ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Battery life difference 2008-07-05 15:44 ` Felipe Balbi @ 2008-07-05 15:54 ` Arjan van de Ven 2008-07-05 17:41 ` Felipe Balbi 2008-07-06 17:21 ` Tino Keitel 1 sibling, 1 reply; 15+ messages in thread From: Arjan van de Ven @ 2008-07-05 15:54 UTC (permalink / raw) To: Felipe Balbi; +Cc: Kernel development list, linux-acpi On Sat, 5 Jul 2008 18:44:21 +0300 "Felipe Balbi" <felipebalbi@users.sourceforge.net> wrote: > hmm... interesting :-) > where should i look for more info ? Any pointers would be nice > > > There's a few things you can do to help stuff along; make sure > > you're using AHCI. > > not loaded. Thanks you may also need to toggle your bios to enable AHCI btw > > > Also take a look at > > http://kerneltrap.org/mailarchive/linux-kernel/2008/5/8/1771024 > > because these patches save like 0.75W in our lab. > > cool, is it merged upstream ? > I'll build current git kernel and reply again with new linux results. > It could be that ubuntu 2.6.24 is broken somehow... > > > To be honest, I'm surprised at your results; in our testing (with a > > different distro though) we don't see such a difference and Linux is > > doing rather well. > > can you get powertop results in a 2.6.24 distro kernel and current git > kernel ? most of my measurement machines are in the office ;( Anyway, also make sure you use the 0.11 version of powertop; we added some newer stuff into that version... > -- If you want to reach me at my work email, use arjan@linux.intel.com For development, discussion and tips for power savings, visit http://www.lesswatts.org ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Battery life difference 2008-07-05 15:54 ` Arjan van de Ven @ 2008-07-05 17:41 ` Felipe Balbi 2008-07-05 19:34 ` Arjan van de Ven 2008-07-09 20:32 ` Pavel Machek 0 siblings, 2 replies; 15+ messages in thread From: Felipe Balbi @ 2008-07-05 17:41 UTC (permalink / raw) To: Arjan van de Ven; +Cc: Kernel development list, linux-acpi Hi, On Sat, Jul 5, 2008 at 6:54 PM, Arjan van de Ven <arjan@infradead.org> wrote: > you may also need to toggle your bios to enable AHCI btw > >> >> > Also take a look at >> > http://kerneltrap.org/mailarchive/linux-kernel/2008/5/8/1771024 >> > because these patches save like 0.75W in our lab. >> >> cool, is it merged upstream ? >> I'll build current git kernel and reply again with new linux results. >> It could be that ubuntu 2.6.24 is broken somehow... >> >> > To be honest, I'm surprised at your results; in our testing (with a >> > different distro though) we don't see such a difference and Linux is >> > doing rather well. >> >> can you get powertop results in a 2.6.24 distro kernel and current git >> kernel ? > > most of my measurement machines are in the office ;( > > Anyway, also make sure you use the 0.11 version of powertop; we added > some newer stuff into that version... Cool, now I'm using current linux 2.6.26-rc8 from git b8a0b6ccf2ba2519ace65d782b41ee91bf3c3778 Now I'm reaching 9.8w consumption :-) Really good improvements. btw, any plans to move some of the powertop features to acpid ? I mean, the wireless pm for iwl4965 could be there, couldn't it ? I'll anyways clone linux-acpi tree and try to test recent patches as much as I can, although omap is really eating my time. thanks again Arjan. -- Best Regards, Felipe Balbi felipebalbi@users.sourceforge.net ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Battery life difference 2008-07-05 17:41 ` Felipe Balbi @ 2008-07-05 19:34 ` Arjan van de Ven 2008-07-05 19:44 ` Felipe Balbi 2008-07-09 20:32 ` Pavel Machek 1 sibling, 1 reply; 15+ messages in thread From: Arjan van de Ven @ 2008-07-05 19:34 UTC (permalink / raw) To: Felipe Balbi; +Cc: Kernel development list, linux-acpi On Sat, 5 Jul 2008 20:41:07 +0300 "Felipe Balbi" <felipebalbi@users.sourceforge.net> wrote: > > Cool, now I'm using current linux 2.6.26-rc8 from git > b8a0b6ccf2ba2519ace65d782b41ee91bf3c3778 > > Now I'm reaching 9.8w consumption :-) > > Really good improvements. cool > > btw, any plans to move some of the powertop features to acpid ? > I mean, the wireless pm for iwl4965 could be there, couldn't it ? actually.. it needs to/is moving into the driver. This sort of decision needs to just happen locally in the driver; the idea that you can make power decisions (rather than policy decisions) on a global level is just no longer true (if it ever was). Actually I thought this was already in the 2.6.26-rc kernel series driver... -- If you want to reach me at my work email, use arjan@linux.intel.com For development, discussion and tips for power savings, visit http://www.lesswatts.org ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Battery life difference 2008-07-05 19:34 ` Arjan van de Ven @ 2008-07-05 19:44 ` Felipe Balbi 0 siblings, 0 replies; 15+ messages in thread From: Felipe Balbi @ 2008-07-05 19:44 UTC (permalink / raw) To: Arjan van de Ven; +Cc: Kernel development list, linux-acpi Hi, On Sat, Jul 5, 2008 at 10:34 PM, Arjan van de Ven <arjan@infradead.org> wrote: > actually.. it needs to/is moving into the driver. This sort of decision > needs to just happen locally in the driver; the idea that you can make > power decisions (rather than policy decisions) on a global level is > just no longer true (if it ever was). Got your point. Just like in omap, we're putting everything in the driver itself. > Actually I thought this was already in the 2.6.26-rc kernel series > driver... At least powertop still complains about it. Maybe it's queued for 2.6.27 ? btw, do you know if any company is selling x-series power adapters to be attached to controlled power supplies or would I have to cut one of my power supply cords (not willing to do that :-p)? -- Best Regards, Felipe Balbi felipebalbi@users.sourceforge.net ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Battery life difference 2008-07-05 17:41 ` Felipe Balbi 2008-07-05 19:34 ` Arjan van de Ven @ 2008-07-09 20:32 ` Pavel Machek 1 sibling, 0 replies; 15+ messages in thread From: Pavel Machek @ 2008-07-09 20:32 UTC (permalink / raw) To: Felipe Balbi; +Cc: Arjan van de Ven, Kernel development list, linux-acpi On Sat 2008-07-05 20:41:07, Felipe Balbi wrote: > Hi, > > On Sat, Jul 5, 2008 at 6:54 PM, Arjan van de Ven <arjan@infradead.org> wrote: > > you may also need to toggle your bios to enable AHCI btw > > > >> > >> > Also take a look at > >> > http://kerneltrap.org/mailarchive/linux-kernel/2008/5/8/1771024 > >> > because these patches save like 0.75W in our lab. > >> > >> cool, is it merged upstream ? > >> I'll build current git kernel and reply again with new linux results. > >> It could be that ubuntu 2.6.24 is broken somehow... > >> > >> > To be honest, I'm surprised at your results; in our testing (with a > >> > different distro though) we don't see such a difference and Linux is > >> > doing rather well. > >> > >> can you get powertop results in a 2.6.24 distro kernel and current git > >> kernel ? > > > > most of my measurement machines are in the office ;( > > > > Anyway, also make sure you use the 0.11 version of powertop; we added > > some newer stuff into that version... > > Cool, now I'm using current linux 2.6.26-rc8 from git > b8a0b6ccf2ba2519ace65d782b41ee91bf3c3778 > > Now I'm reaching 9.8w consumption :-) Cool... how low can Windows go? -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Battery life difference 2008-07-05 15:44 ` Felipe Balbi 2008-07-05 15:54 ` Arjan van de Ven @ 2008-07-06 17:21 ` Tino Keitel 2008-07-07 9:48 ` Filippo Zangheri 1 sibling, 1 reply; 15+ messages in thread From: Tino Keitel @ 2008-07-06 17:21 UTC (permalink / raw) To: Felipe Balbi; +Cc: Kernel development list, linux-acpi On Sat, Jul 05, 2008 at 18:44:21 +0300, Felipe Balbi wrote: > Hi, > > On Sat, Jul 5, 2008 at 6:23 PM, Arjan van de Ven <arjan@infradead.org> wrote: [...] > > Also take a look at > > http://kerneltrap.org/mailarchive/linux-kernel/2008/5/8/1771024 because > > these patches save like 0.75W in our lab. > > cool, is it merged upstream ? If I got this right, at least Linux 2.6.25 allows to control this via "/sys/class/scsi_host/host*/link_power_management_policy". Setting it to "min_power" does the trick. This is also covered by powertop 1.10 and later. Regards, Tino ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Battery life difference 2008-07-06 17:21 ` Tino Keitel @ 2008-07-07 9:48 ` Filippo Zangheri 2008-07-07 21:37 ` Tino Keitel 0 siblings, 1 reply; 15+ messages in thread From: Filippo Zangheri @ 2008-07-07 9:48 UTC (permalink / raw) To: Felipe Balbi, Kernel development list, linux-acpi [-- Attachment #1: Type: text/plain, Size: 840 bytes --] On 06/07/2008 19:21, Tino Keitel wrote: > On Sat, Jul 05, 2008 at 18:44:21 +0300, Felipe Balbi wrote: >> Hi, >> >> On Sat, Jul 5, 2008 at 6:23 PM, Arjan van de Ven <arjan@infradead.org> wrote: > > [...] > >>> Also take a look at >>> http://kerneltrap.org/mailarchive/linux-kernel/2008/5/8/1771024 because >>> these patches save like 0.75W in our lab. >> cool, is it merged upstream ? > > If I got this right, at least Linux 2.6.25 allows to control this via > "/sys/class/scsi_host/host*/link_power_management_policy". Setting it > to "min_power" does the trick. This is also covered by powertop 1.10 > and later. I cannot see this on 2.6.25.9. Is there a particular configuration option you must enable, or it just didn't make it to 2.6.25? Thanks. Regards. -- Filippo Zangheri GPG key ID: 0x6C1F2F2F [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Battery life difference 2008-07-07 9:48 ` Filippo Zangheri @ 2008-07-07 21:37 ` Tino Keitel 0 siblings, 0 replies; 15+ messages in thread From: Tino Keitel @ 2008-07-07 21:37 UTC (permalink / raw) To: Kernel development list On Mon, Jul 07, 2008 at 11:48:42 +0200, Filippo Zangheri wrote: > On 06/07/2008 19:21, Tino Keitel wrote: > > On Sat, Jul 05, 2008 at 18:44:21 +0300, Felipe Balbi wrote: > >> Hi, > >> > >> On Sat, Jul 5, 2008 at 6:23 PM, Arjan van de Ven <arjan@infradead.org> wrote: > > > > [...] > > > >>> Also take a look at > >>> http://kerneltrap.org/mailarchive/linux-kernel/2008/5/8/1771024 because > >>> these patches save like 0.75W in our lab. > >> cool, is it merged upstream ? > > > > If I got this right, at least Linux 2.6.25 allows to control this via > > "/sys/class/scsi_host/host*/link_power_management_policy". Setting it > > to "min_power" does the trick. This is also covered by powertop 1.10 > > and later. > > I cannot see this on 2.6.25.9. Is there a particular configuration option > you must enable, or it just didn't make it to 2.6.25? All I can say is that I see it with 2.6.25.9 on a Thinkpad X61s with SATA set to AHCI. See commit ca77329fb713b7fea6a307068e0dd0248e7aa640 for details. Regards, Tino ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2008-07-10 5:16 UTC | newest] Thread overview: 15+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2008-07-05 13:17 Battery life difference Felipe Balbi 2008-07-05 13:36 ` Theodore Tso 2008-07-05 14:00 ` Felipe Balbi 2008-07-09 20:29 ` Pavel Machek 2008-07-10 5:16 ` Tino Keitel 2008-07-05 15:23 ` Arjan van de Ven 2008-07-05 15:44 ` Felipe Balbi 2008-07-05 15:54 ` Arjan van de Ven 2008-07-05 17:41 ` Felipe Balbi 2008-07-05 19:34 ` Arjan van de Ven 2008-07-05 19:44 ` Felipe Balbi 2008-07-09 20:32 ` Pavel Machek 2008-07-06 17:21 ` Tino Keitel 2008-07-07 9:48 ` Filippo Zangheri 2008-07-07 21:37 ` Tino Keitel
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox